From larry at hastings.org  Tue Feb  2 05:07:24 2010
From: larry at hastings.org (Larry Hastings)
Date: Mon, 01 Feb 2010 20:07:24 -0800
Subject: [Python-ideas] Merge .pyc into .pyo--store multiple code objects in
	one file?
Message-ID: <4B67A4FC.2090404@hastings.org>



A .pyc file is made up of these three elements:
    4 byte magic number
    4 byte timestamp
    marshaled code object
A .pyo file is the same except the code object has been optimized.

I ask you: why gunk up the filesystem with two files when one would do?  
I propose we change the pyc file so it can contain multiple code 
objects.  Or, indeed, multiple arbitrary objects.  The .pyc file could 
become as a general-purpose cache for data relevant to the .py file.  
For example, perhaps the Unladen Swallow guys could cache post-JITted 
code.  Or the wordcode-based interpreter could cache its wordcode 
equivalent.

Lots of implementation choices suggest themselves.  Here's the cheapest 
approach that seems suitable:
    4 byte magic number
    4 byte timestamp
    marshaled array of pairs of ints, alternating "id of object"
        with "relative offset of object from the end of this marshaled 
array"
    marshaled code object 1
    marshaled code object 2
    ...

If you look for your cached object inside and it's not there, you 
compute your object then rewrite the file adding your id and offset to 
the end of the array.  (Your offset will always be the former size of 
the file.)  If the timestamp has changed, blow away all objects and 
start over with an empty array.

If it'd be too disruptive to change .pyc/.pyo files this way, would 
switching to a new file extension be better?

I suspect this probably isn't actually a good idea,


/larry/


From larry at hastings.org  Tue Feb  2 05:12:30 2010
From: larry at hastings.org (Larry Hastings)
Date: Mon, 01 Feb 2010 20:12:30 -0800
Subject: [Python-ideas] Merge .pyc into .pyo--store multiple code
	objects in one file?
In-Reply-To: <4B67A4FC.2090404@hastings.org>
References: <4B67A4FC.2090404@hastings.org>
Message-ID: <4B67A62E.9070101@hastings.org>



I'm a dope--this idea has already been batted around and discarded, 
resulting in the PYD directory PEP.  Sorry I missed it when it happened.

Ignore me,


/larry/



From solipsis at pitrou.net  Tue Feb  2 16:16:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 2 Feb 2010 15:16:41 +0000 (UTC)
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
References: <4B67A4FC.2090404@hastings.org>
Message-ID: <loom.20100202T161113-710@post.gmane.org>

Larry Hastings <larry at ...> writes:
> 
> I ask you: why gunk up the filesystem with two files when one would do?  
> I propose we change the pyc file so it can contain multiple code 
> objects.

I think we should dump the lie about "optimized" bytecode when the only
optimization is that we strip some docstrings, disable asserts and set __debug__
to False.

There should be only one possible bytecode file (XXX.pyc), and we could provide
a "strip" tool (and/or corresponding function in the compileall module) for
people for whom minimizing bytecode file size is important.

Also, it would be interesting to know who bothers to use "python -O" (or "-OO").
I know I never use it.


Antoine.




From markroddy at gmail.com  Tue Feb  2 17:38:53 2010
From: markroddy at gmail.com (Mark Roddy)
Date: Tue, 2 Feb 2010 11:38:53 -0500
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <loom.20100202T161113-710@post.gmane.org>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org>
Message-ID: <7930f1201002020838n450da8c8le624020abc08afc2@mail.gmail.com>

My team uses the optimized flag when building 'binaries' via py2exe
before distributing.  In the end it really is security through
obscurity, but something is always better than nothing.

On a different note, I don't think asserts belong in production.  I
fully concede that there is no real consensus either way on this
belief so I think it's important to have the option so people can run
in the mode which they prefer.

-Mark



On Tue, Feb 2, 2010 at 10:16 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Larry Hastings <larry at ...> writes:
>>
>> I ask you: why gunk up the filesystem with two files when one would do?
>> I propose we change the pyc file so it can contain multiple code
>> objects.
>
> I think we should dump the lie about "optimized" bytecode when the only
> optimization is that we strip some docstrings, disable asserts and set __debug__
> to False.
>
> There should be only one possible bytecode file (XXX.pyc), and we could provide
> a "strip" tool (and/or corresponding function in the compileall module) for
> people for whom minimizing bytecode file size is important.
>
> Also, it would be interesting to know who bothers to use "python -O" (or "-OO").
> I know I never use it.
>
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From brett at python.org  Tue Feb  2 21:49:07 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Feb 2010 12:49:07 -0800
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <loom.20100202T161113-710@post.gmane.org>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org>
Message-ID: <bbaeab101002021249j2faf21eo48465a30084f1d3@mail.gmail.com>

On Tue, Feb 2, 2010 at 07:16, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Larry Hastings <larry at ...> writes:
>>
>> I ask you: why gunk up the filesystem with two files when one would do?
>> I propose we change the pyc file so it can contain multiple code
>> objects.
>
> I think we should dump the lie about "optimized" bytecode when the only
> optimization is that we strip some docstrings, disable asserts and set __debug__
> to False.

I think the hope has always been that the peepholer would be extended
to do some tweaks that would only be reasonable under a -O flag.
Obviously this has not happened and who knows if it ever will.

But if PEP 3147 catches on this should become less of an issue.

>
> There should be only one possible bytecode file (XXX.pyc), and we could provide
> a "strip" tool (and/or corresponding function in the compileall module) for
> people for whom minimizing bytecode file size is important.

Would also require a flag for distutils for when you are installing a
package that is for production compared to debug use you byte-compile
to the level you want. But I think the compileall/strip/distutils
solution would be enough to cover all major cases.

>
> Also, it would be interesting to know who bothers to use "python -O" (or "-OO").
> I know I never use it.

Do any other languages do it this way with separate files? Or do they
tend toward not even having the option and the few that do use a strip
tool? I honestly can't think of any languages off the top of my head
where the -O flag is even actively considered by everyone beyond
C/C++.

-Brett


>
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From ncoghlan at gmail.com  Tue Feb  2 21:58:22 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 03 Feb 2010 06:58:22 +1000
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <loom.20100202T161113-710@post.gmane.org>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org>
Message-ID: <4B6891EE.5090303@gmail.com>

Antoine Pitrou wrote:
> Larry Hastings <larry at ...> writes:
>> I ask you: why gunk up the filesystem with two files when one would do?  
>> I propose we change the pyc file so it can contain multiple code 
>> objects.
> 
> I think we should dump the lie about "optimized" bytecode when the only
> optimization is that we strip some docstrings, disable asserts and set __debug__
> to False.

The fact that asserts and if __debug__ blocks get skipped under -O
strikes me as more than adequate reason to consider optimised and
non-optimised bytecode as different things. Sure, we don't mess with
things as extensively as C/C++ do with their more aggressive
optimisation settings, but completely skipping all assert statements and
some if statements isn't something we can enable by default (and can't
necessarily do with an after-the-fact stripping tool, since the removal
of entire statements can affect the construction of the symbol table).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Tue Feb  2 22:05:22 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 02 Feb 2010 22:05:22 +0100
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <4B6891EE.5090303@gmail.com>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org> <4B6891EE.5090303@gmail.com>
Message-ID: <1265144722.3343.22.camel@localhost>

Le mercredi 03 f?vrier 2010 ? 06:58 +1000, Nick Coghlan a ?crit :
> 
> The fact that asserts and if __debug__ blocks get skipped under -O
> strikes me as more than adequate reason to consider optimised and
> non-optimised bytecode as different things.

Well, do people rely on this nowadays?
With unit testing becoming very common (and recommended) practice, I'm
not sure what sprinkling asserts in debug mode really brings.





From curt at hagenlocher.org  Tue Feb  2 22:22:55 2010
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Tue, 2 Feb 2010 13:22:55 -0800
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <1265144722.3343.22.camel@localhost>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org> <4B6891EE.5090303@gmail.com>
	<1265144722.3343.22.camel@localhost>
Message-ID: <d2155e361002021322q1ff8ff0cnbfbfe9afc220dfaa@mail.gmail.com>

On Tue, Feb 2, 2010 at 1:05 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

>
> Well, do people rely on this nowadays?
> With unit testing becoming very common (and recommended) practice, I'm
> not sure what sprinkling asserts in debug mode really brings.


I like asserts. They're the best kind of comment -- the kind that are
actually checked by the computer for accuracy.

--
Curt Hagenlocher
curt at hagenlocher.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100202/1de5dd1f/attachment.html>

From collinw at gmail.com  Tue Feb  2 22:35:14 2010
From: collinw at gmail.com (Collin Winter)
Date: Tue, 2 Feb 2010 13:35:14 -0800
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <bbaeab101002021249j2faf21eo48465a30084f1d3@mail.gmail.com>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org>
	<bbaeab101002021249j2faf21eo48465a30084f1d3@mail.gmail.com>
Message-ID: <43aa6ff71002021335mf8313fbgab8bc92c3b204251@mail.gmail.com>

On Tue, Feb 2, 2010 at 12:49 PM, Brett Cannon <brett at python.org> wrote:
> On Tue, Feb 2, 2010 at 07:16, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Larry Hastings <larry at ...> writes:
>>>
>>> I ask you: why gunk up the filesystem with two files when one would do?
>>> I propose we change the pyc file so it can contain multiple code
>>> objects.
>>
>> I think we should dump the lie about "optimized" bytecode when the only
>> optimization is that we strip some docstrings, disable asserts and set __debug__
>> to False.
>
> I think the hope has always been that the peepholer would be extended
> to do some tweaks that would only be reasonable under a -O flag.
> Obviously this has not happened and who knows if it ever will.

Unladen Swallow has a number of optimizations in mind that tweak
corner cases of Python semantics, which we'd like to hide behind a
compiler flag so that you have to explicitly ask for them. We haven't
yet implemented these optimizations, since they will likely be
controversial and require discussion. Feel free to remove the -O flag
in the meantime; it can be added back later.

Collin Winter


From brett at python.org  Tue Feb  2 22:42:18 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 2 Feb 2010 13:42:18 -0800
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <1265144722.3343.22.camel@localhost>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org> 
	<4B6891EE.5090303@gmail.com> <1265144722.3343.22.camel@localhost>
Message-ID: <bbaeab101002021342x24727bf9u7f70d1bab2da8fdc@mail.gmail.com>

On Tue, Feb 2, 2010 at 13:05, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mercredi 03 f?vrier 2010 ? 06:58 +1000, Nick Coghlan a ?crit :
>>
>> The fact that asserts and if __debug__ blocks get skipped under -O
>> strikes me as more than adequate reason to consider optimised and
>> non-optimised bytecode as different things.
>
> Well, do people rely on this nowadays?
> With unit testing becoming very common (and recommended) practice, I'm
> not sure what sprinkling asserts in debug mode really brings.

We might not rely on them, but I am sure there are those who prefer
them on top of TDD (or instead of). Removing them would be somewhat
forcing a programming methodology on users which I think we should
avoid.

-Brett

>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From floris.bruynooghe at gmail.com  Wed Feb  3 00:10:30 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Tue, 2 Feb 2010 23:10:30 +0000
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <loom.20100202T161113-710@post.gmane.org>
Message-ID: <20100202231030.GB4830@laurie.devork>

n Feb 2, 3:16=A0pm, Antoine Pitrou <solip... at pitrou.net> wrote:
> Larry Hastings <larry at ...> writes:
>
> > I ask you: why gunk up the filesystem with two files when one would do
> > I propose we change the pyc file so it can contain multiple code
> > objects.
>
> I think we should dump the lie about "optimized" bytecode when the
> only optimization is that we strip some docstrings, disable asserts
> and set __debug__ to False.
>
> There should be only one possible bytecode file (XXX.pyc), and we
> could provide a "strip" tool (and/or corresponding function in the
> compileall module) for people for whom minimizing bytecode file size
> is important.
>
> Also, it would be interesting to know who bothers to use "python -O"
> (or "-OO").  I know I never use it.

We run -O in production so that __debug__ is set to False.  But I'd
much rather have only .pyc files and a strip tool + a flag (maybe
still -O) to switch between __debug__ being True or False (extra bonus
points for the strip tool being able to strip out "if __debug__:"
blocks too).

The reason I'd rather have only .pyc files is that now if you only
have .pyo files and don't start with -O you don't find the modules.
This is inconsistent as well: inside a zipfile the .pyo files are used
both when using -O and when not using that option.  Only an
inconvenience but still, it's something a customer doens't know if I
quickly ask the output of some "python -c 'something'" and forget to
tell them to use -O.

Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From rrr at ronadam.com  Wed Feb  3 01:11:04 2010
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 02 Feb 2010 18:11:04 -0600
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <loom.20100202T161113-710@post.gmane.org>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org>
Message-ID: <4B68BF18.5040903@ronadam.com>



Antoine Pitrou wrote:
> Larry Hastings <larry at ...> writes:
>> I ask you: why gunk up the filesystem with two files when one would do?  
>> I propose we change the pyc file so it can contain multiple code 
>> objects.
> 
> I think we should dump the lie about "optimized" bytecode when the only
> optimization is that we strip some docstrings, disable asserts and set __debug__
> to False.
> 
> There should be only one possible bytecode file (XXX.pyc), and we could provide
> a "strip" tool (and/or corresponding function in the compileall module) for
> people for whom minimizing bytecode file size is important.
> 
> Also, it would be interesting to know who bothers to use "python -O" (or "-OO").
> I know I never use it.

If there was an easy way to get info from python on the status and location
of the bytecode files it uses, it might not be so bad.

> python -stat module.py
   # python version
   # list of bytecode files, type, location

Then having .pyc also be used for optimized byte code could work.

On another note, it always seemed a bit backwards to me that the -OO wasn't
the default output and if it were, we would have options to run in debug
mode which would include 'docstrings', 'if debug ...' statements, and
'assert ...'  statements, for testing purposes.  <shrug>

Ron










From ncoghlan at gmail.com  Wed Feb  3 13:51:56 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 03 Feb 2010 22:51:56 +1000
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <bbaeab101002021342x24727bf9u7f70d1bab2da8fdc@mail.gmail.com>
References: <4B67A4FC.2090404@hastings.org>	<loom.20100202T161113-710@post.gmane.org>
	<4B6891EE.5090303@gmail.com> <1265144722.3343.22.camel@localhost>
	<bbaeab101002021342x24727bf9u7f70d1bab2da8fdc@mail.gmail.com>
Message-ID: <4B69716C.2010308@gmail.com>

Brett Cannon wrote:
> On Tue, Feb 2, 2010 at 13:05, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Le mercredi 03 f?vrier 2010 ? 06:58 +1000, Nick Coghlan a ?crit :
>>> The fact that asserts and if __debug__ blocks get skipped under -O
>>> strikes me as more than adequate reason to consider optimised and
>>> non-optimised bytecode as different things.
>> Well, do people rely on this nowadays?
>> With unit testing becoming very common (and recommended) practice, I'm
>> not sure what sprinkling asserts in debug mode really brings.
> 
> We might not rely on them, but I am sure there are those who prefer
> them on top of TDD (or instead of). Removing them would be somewhat
> forcing a programming methodology on users which I think we should
> avoid.

As Curt said, they're a great form of executable comment, unit tests or
no unit tests.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From dickinsm at gmail.com  Wed Feb  3 15:46:58 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 3 Feb 2010 14:46:58 +0000
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <bbaeab101002021342x24727bf9u7f70d1bab2da8fdc@mail.gmail.com>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org> <4B6891EE.5090303@gmail.com>
	<1265144722.3343.22.camel@localhost>
	<bbaeab101002021342x24727bf9u7f70d1bab2da8fdc@mail.gmail.com>
Message-ID: <5c6f2a5d1002030646y431b5e66q255d6d4bc310ec2c@mail.gmail.com>

On Tue, Feb 2, 2010 at 9:42 PM, Brett Cannon <brett at python.org> wrote:
> On Tue, Feb 2, 2010 at 13:05, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Le mercredi 03 f?vrier 2010 ? 06:58 +1000, Nick Coghlan a ?crit :
>>>
>>> The fact that asserts and if __debug__ blocks get skipped under -O
>>> strikes me as more than adequate reason to consider optimised and
>>> non-optimised bytecode as different things.
>>
>> Well, do people rely on this nowadays?
>> With unit testing becoming very common (and recommended) practice, I'm
>> not sure what sprinkling asserts in debug mode really brings.
>
> We might not rely on them, but I am sure there are those who prefer
> them on top of TDD (or instead of).  [...]

FWIW, I'm one of 'those':  I find there are assert uses that can't be
covered as neatly or directly with unit tests.  They're great for
documenting loop invariants in complicated algorithms, for example:

def xgcd(b, c):
    """Extended Euclidean algorithm: return integers g, x and y such that
    b*x + c*y == g and g is a greatest common divisor of b and c."""
    g1, g2 = b, c
    x1, x2 = 1, 0
    y1, y2 = 0, 1
    while g2:
        assert x1 * b + y1 * c == g1 and x2 * b + y2 * c == g2
        q = g1 // g2
        g1, g2 = g2, g1 - q*g2
        x1, x2 = x2, x1 - q*x2
        y1, y2 = y2, y1 - q*y2
    return g1, x1, y1

(Not that the above counts as a complicated algorithm, of course.)

Mark


From mal at egenix.com  Thu Feb  4 01:46:17 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 04 Feb 2010 01:46:17 +0100
Subject: [Python-ideas] API for VM branches (was: PEP 3146)
In-Reply-To: <3c8293b61002021053r469ee84bk77bc32a916d83fce@mail.gmail.com>
References: <3c8293b61002021053r469ee84bk77bc32a916d83fce@mail.gmail.com>
Message-ID: <4B6A18D9.4010100@egenix.com>

Collin Winter wrote:
> On Tue, Feb 2, 2010 at 2:02 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Collin Winter wrote:
> [snip]
>>> If such a restrictive plugin-based scheme had been available when we
>>> began Unladen Swallow, I do not doubt that we would have ignored it
>>> entirely. I do not like the idea of artificially tying the hands of
>>> people trying to make CPython faster. I do not see any part of Unladen
>>> Swallow that would have been made easier by such a scheme. If
>>> anything, it would have made our project more difficult.
>>
>> I don't think that it has to be restrictive - much to the contrary,
>> it would provide a consistent API to those CPython internals and
>> also clarify the separation between the various parts. Something
>> which currently does not exist in CPython.
> 
> We do not need an API to CPython's internals: we are not interfacing
> with them, we are replacing and augmenting them.

Right, and that's the idea behind making the VM pluggable: you
define the boundaries and interfaces of the components you want
to replace and then have Python redirect all calls to your
implementation by linking against your VM instead of the default
one.

>> Note that it may be easier for you (and others) to just take
>> CPython and patch it as necessary. However, this doesn't relieve
>> you from the needed maintenance - which, I presume, is one of the
>> reasons why you are suggesting to merge U-S back into CPython ;-)
> 
> That is incorrect. In the year we have been working on Unladen
> Swallow, we have only updated our vendor branch of CPython 2.6 once,
> going from 2.6.1 to 2.6.4. We have occasionally cherrypicked patches
> from the 2.6 maintenance branch to fix specific problems. The
> maintenance required by upstream CPython changes has been effectively
> zero.

Patch level releases usually don't include any disruptive
changes to the code, so I'm not sure whether your experience
with going from 2.6.1 to 2.6.4 provides a good example of
what maintaining compatibility with Python's main line of
development really means in terms of effort.

There are a few things to consider:

 * if you want to stay compatible with Python, you have to
   provide updated releases of your version for every
   supported Python version (including all patch level releases)

 * you need to build releases with more or less the same
   frequency as CPython

 * you need to closely follow Python trunk development if
   want to avoid lagging behind Python's releases by a few
   months

and all this for a long period of time.

Otherwise, I don't see how you could convince users to
use your branch instead of the Python mainline version.

> We are seeking to merge with CPython for three reasons: 1) verify that
> python-dev is interested in this project, and that we are not wasting
> our time; 2) expose the codebase to a wider, more heterogenous testing
> environment; 3) accelerate development by having more hands on the
> code. Upstream maintenance requirements have had zero impact on our
> planning.

I'm sure that python-dev is interested in your project.

I'm not sure whether it would be fair to go with your approach
instead of the other existing branches, such as Stackless or
wpython.

How easy would it be for those projects to continue supporting
Python if U-S were to be merged into main line Python ?

> In any case, I'll be interested in reading your PEP that outlines how
> the plugin interface should work, which systems will be pluggable, and
> exactly how Unladen Swallow, WPython and Stackless would benefit.
> Let's move further discussion of this to python-ideas until there's
> something more concrete here. The py3k-jit branch will live long
> enough that we could update it to work with a plugin system, assuming
> it is demonstrated to be beneficial.

Perhaps I'll write a PEP outlining the idea in more detail.
However, without support from developers of such VMs, I don't
see much reason in trying to design something that will not
eventually get used.

I have a feeling that I'm not getting the point across:
by making it easier for VM developers to hook into the CPython
system, we'd all benefit.

VM developers could rely on a consistent interface to develop against,
python-dev would know where non-mainline VMs hook into the system
and users could choose the VM that best fits their needs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 03 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From gerald.britton at gmail.com  Fri Feb  5 20:39:46 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 5 Feb 2010 14:39:46 -0500
Subject: [Python-ideas] clear() method for lists
Message-ID: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>

In the list archives, this thread

    http://mail.python.org/pipermail/python-ideas/2009-April/003897.html

discusses adding a clear() method to list objects, to complement those
available for sets and dictionaries.  Later in the thread:

    http://mail.python.org/pipermail/python-ideas/2009-April/003933.html

Christian Heimes provided a patch to do it and R. H. commented that
all it would take is Guido's blessing.

So, I'm wondering, can we do this?  What are the steps needed to ask
this work to be blessed?

-- 
Gerald Britton


From g.brandl at gmx.net  Fri Feb  5 21:19:16 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 05 Feb 2010 21:19:16 +0100
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <4B68BF18.5040903@ronadam.com>
References: <4B67A4FC.2090404@hastings.org>	<loom.20100202T161113-710@post.gmane.org>
	<4B68BF18.5040903@ronadam.com>
Message-ID: <hkhuhj$p04$1@ger.gmane.org>

Am 03.02.2010 01:11, schrieb Ron Adam:
> 
> 
> Antoine Pitrou wrote:
>> Larry Hastings <larry at ...> writes:
>>> I ask you: why gunk up the filesystem with two files when one would do?  
>>> I propose we change the pyc file so it can contain multiple code 
>>> objects.
>> 
>> I think we should dump the lie about "optimized" bytecode when the only
>> optimization is that we strip some docstrings, disable asserts and set __debug__
>> to False.
>> 
>> There should be only one possible bytecode file (XXX.pyc), and we could provide
>> a "strip" tool (and/or corresponding function in the compileall module) for
>> people for whom minimizing bytecode file size is important.
>> 
>> Also, it would be interesting to know who bothers to use "python -O" (or "-OO").
>> I know I never use it.
> 
> If there was an easy way to get info from python on the status and location
> of the bytecode files it uses, it might not be so bad.
> 
>> python -stat module.py
>    # python version
>    # list of bytecode files, type, location
> 
> Then having .pyc also be used for optimized byte code could work.
> 
> On another note, it always seemed a bit backwards to me that the -OO wasn't
> the default output and if it were, we would have options to run in debug
> mode which would include 'docstrings', 'if debug ...' statements, and
> 'assert ...'  statements, for testing purposes.  <shrug>

Removal of docstrings should be separate from the rest, though.  Quite a few
programs would stop working properly without docstrings, with the docstring
being used as a kind of easily introspectable annotation to a class or function.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From guido at python.org  Fri Feb  5 21:26:05 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 5 Feb 2010 12:26:05 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
Message-ID: <ca471dc21002051226n203cde6bt4bfefb30f6446829@mail.gmail.com>

Sure.

On Fri, Feb 5, 2010 at 11:39 AM, Gerald Britton
<gerald.britton at gmail.com> wrote:
> In the list archives, this thread
>
> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
>
> discusses adding a clear() method to list objects, to complement those
> available for sets and dictionaries. ?Later in the thread:
>
> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
>
> Christian Heimes provided a patch to do it and R. H. commented that
> all it would take is Guido's blessing.
>
> So, I'm wondering, can we do this? ?What are the steps needed to ask
> this work to be blessed?
>
> --
> Gerald Britton
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (python.org/~guido)


From gerald.britton at gmail.com  Fri Feb  5 21:32:15 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 5 Feb 2010 15:32:15 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <ca471dc21002051226n203cde6bt4bfefb30f6446829@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
	<ca471dc21002051226n203cde6bt4bfefb30f6446829@mail.gmail.com>
Message-ID: <5d1a32001002051232u33436228lc768941f1597f2ed@mail.gmail.com>

Wow!  Fast!  So, since I'm not a dev and don't know the ropes, what is
the next step to get the method included?

On Fri, Feb 5, 2010 at 3:26 PM, Guido van Rossum <guido at python.org> wrote:
> Sure.
>
> On Fri, Feb 5, 2010 at 11:39 AM, Gerald Britton
> <gerald.britton at gmail.com> wrote:
>> In the list archives, this thread
>>
>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
>>
>> discusses adding a clear() method to list objects, to complement those
>> available for sets and dictionaries. ?Later in the thread:
>>
>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
>>
>> Christian Heimes provided a patch to do it and R. H. commented that
>> all it would take is Guido's blessing.
>>
>> So, I'm wondering, can we do this? ?What are the steps needed to ask
>> this work to be blessed?
>>
>> --
>> Gerald Britton
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>



-- 
Gerald Britton


From guido at python.org  Fri Feb  5 21:46:47 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 5 Feb 2010 12:46:47 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002051232u33436228lc768941f1597f2ed@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com> 
	<ca471dc21002051226n203cde6bt4bfefb30f6446829@mail.gmail.com> 
	<5d1a32001002051232u33436228lc768941f1597f2ed@mail.gmail.com>
Message-ID: <ca471dc21002051246j1b367140o928f1b2a152c48c3@mail.gmail.com>

Ping Christian. :-)

On Fri, Feb 5, 2010 at 12:32 PM, Gerald Britton
<gerald.britton at gmail.com> wrote:
> Wow! ?Fast! ?So, since I'm not a dev and don't know the ropes, what is
> the next step to get the method included?
>
> On Fri, Feb 5, 2010 at 3:26 PM, Guido van Rossum <guido at python.org> wrote:
>> Sure.
>>
>> On Fri, Feb 5, 2010 at 11:39 AM, Gerald Britton
>> <gerald.britton at gmail.com> wrote:
>>> In the list archives, this thread
>>>
>>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
>>>
>>> discusses adding a clear() method to list objects, to complement those
>>> available for sets and dictionaries. ?Later in the thread:
>>>
>>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
>>>
>>> Christian Heimes provided a patch to do it and R. H. commented that
>>> all it would take is Guido's blessing.
>>>
>>> So, I'm wondering, can we do this? ?What are the steps needed to ask
>>> this work to be blessed?
>>>
>>> --
>>> Gerald Britton
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> http://mail.python.org/mailman/listinfo/python-ideas
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>
>
> --
> Gerald Britton
>



-- 
--Guido van Rossum (python.org/~guido)


From gerald.britton at gmail.com  Fri Feb  5 21:50:57 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 5 Feb 2010 15:50:57 -0500
Subject: [Python-ideas] clear() method for lists - Pinging Christian
	Heimes
Message-ID: <5d1a32001002051250r7826ed1ifd22559b831d7f2b@mail.gmail.com>

On Fri, Feb 5, 2010 at 3:46 PM, Guido van Rossum <guido at python.org> wrote:
> Ping Christian. :-)
>
> On Fri, Feb 5, 2010 at 12:32 PM, Gerald Britton
> <gerald.britton at gmail.com> wrote:
>> Wow! ?Fast! ?So, since I'm not a dev and don't know the ropes, what is
>> the next step to get the method included?
>>
>> On Fri, Feb 5, 2010 at 3:26 PM, Guido van Rossum <guido at python.org> wrote:
>>> Sure.
>>>
>>> On Fri, Feb 5, 2010 at 11:39 AM, Gerald Britton
>>> <gerald.britton at gmail.com> wrote:
>>>> In the list archives, this thread
>>>>
>>>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
>>>>
>>>> discusses adding a clear() method to list objects, to complement those
>>>> available for sets and dictionaries. ?Later in the thread:
>>>>
>>>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
>>>>
>>>> Christian Heimes provided a patch to do it and R. H. commented that
>>>> all it would take is Guido's blessing.
>>>>
>>>> So, I'm wondering, can we do this? ?What are the steps needed to ask
>>>> this work to be blessed?
>>>>
>>>> --
>>>> Gerald Britton
>>>> _______________________________________________
>>>> Python-ideas mailing list
>>>> Python-ideas at python.org
>>>> http://mail.python.org/mailman/listinfo/python-ideas
>>>>
>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (python.org/~guido)
>>>
>>
>>
>>
>> --
>> Gerald Britton
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>

Pinging Christian!  (Sorry I don't have your direct email address)

-- 
Gerald Britton


From raymond.hettinger at gmail.com  Fri Feb  5 22:09:42 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Fri, 5 Feb 2010 13:09:42 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
Message-ID: <554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>

ISTM, the language moratorium would preclude this change.
Also, since it doesn't actually add any new functionality,
it is more of a documentation issue, teaching basic python idioms:

    del s[:]
or
    s[:] = []

Raymond

On Feb 5, 2010, at 11:39 AM, Gerald Britton wrote:

> In the list archives, this thread
> 
>    http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
> 
> discusses adding a clear() method to list objects, to complement those
> available for sets and dictionaries.  Later in the thread:
> 
>    http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
> 
> Christian Heimes provided a patch to do it and R. H. commented that
> all it would take is Guido's blessing.
> 
> So, I'm wondering, can we do this?  What are the steps needed to ask
> this work to be blessed?
> 
> -- 
> Gerald Britton
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From guido at python.org  Fri Feb  5 22:16:17 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 5 Feb 2010 13:16:17 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com> 
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
Message-ID: <ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com>

On Fri, Feb 5, 2010 at 1:09 PM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
> ISTM, the language moratorium would preclude this change.

Oops, you're right. Scratch the go-ahead. :-(

While PEP 3003 allows case-by-case exemptions for adding methods to
built-in objects, I think this isn't one of those cases, since the
work-around sure is simple enough, and has been in the language for 20
years now. ;-)

> Also, since it doesn't actually add any new functionality,
> it is more of a documentation issue, teaching basic python idioms:
>
> ? ?del s[:]
> or
> ? ?s[:] = []

Though it's a common-enough special case that it shouldn't need slice
manipulations, which are a bit of an acquired taste. I probably would
have added this if I'd anticipated dict.clear(), which wasn't always
part of the language.

--Guido

> Raymond
>
> On Feb 5, 2010, at 11:39 AM, Gerald Britton wrote:
>
>> In the list archives, this thread
>>
>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
>>
>> discusses adding a clear() method to list objects, to complement those
>> available for sets and dictionaries. ?Later in the thread:
>>
>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
>>
>> Christian Heimes provided a patch to do it and R. H. commented that
>> all it would take is Guido's blessing.
>>
>> So, I'm wondering, can we do this? ?What are the steps needed to ask
>> this work to be blessed?
>>
>> --
>> Gerald Britton
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (python.org/~guido)


From robert.kern at gmail.com  Fri Feb  5 22:16:54 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 05 Feb 2010 15:16:54 -0600
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
Message-ID: <hki1s6$3cm$1@ger.gmane.org>

On 2010-02-05 15:09 PM, Raymond Hettinger wrote:
> ISTM, the language moratorium would preclude this change.

"""
Case-by-Case Exemptions

New methods on built-ins
   The case for adding a method to a built-in object can be made.
"""

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From lists at cheimes.de  Fri Feb  5 22:20:32 2010
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 05 Feb 2010 22:20:32 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <ca471dc21002051246j1b367140o928f1b2a152c48c3@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
	<ca471dc21002051226n203cde6bt4bfefb30f6446829@mail.gmail.com>
	<5d1a32001002051232u33436228lc768941f1597f2ed@mail.gmail.com>
	<ca471dc21002051246j1b367140o928f1b2a152c48c3@mail.gmail.com>
Message-ID: <hki22v$4r3$1@ger.gmane.org>

Guido van Rossum schrieb:
> Ping Christian. :-)

Pong ;)

I have to concur with Raymond. IMHO there is no real need for a
list.clear() method. Do we want to extend the API for the sake of a
common way to clear a mutable container?

Christian

PS: The patch was merely a finger exercise.



From gerald.britton at gmail.com  Fri Feb  5 22:21:36 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 5 Feb 2010 16:21:36 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
	<ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com>
Message-ID: <5d1a32001002051321v79784172gd4f5cc7965190928@mail.gmail.com>

On Fri, Feb 5, 2010 at 4:16 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Feb 5, 2010 at 1:09 PM, Raymond Hettinger
> <raymond.hettinger at gmail.com> wrote:
>> ISTM, the language moratorium would preclude this change.
>
> Oops, you're right. Scratch the go-ahead. :-(

So does this mean I should bring it up again for the next round?

>
> While PEP 3003 allows case-by-case exemptions for adding methods to
> built-in objects, I think this isn't one of those cases, since the
> work-around sure is simple enough, and has been in the language for 20
> years now. ;-)
>
>> Also, since it doesn't actually add any new functionality,
>> it is more of a documentation issue, teaching basic python idioms:
>>
>> ? ?del s[:]
>> or
>> ? ?s[:] = []
>
> Though it's a common-enough special case that it shouldn't need slice
> manipulations, which are a bit of an acquired taste. I probably would
> have added this if I'd anticipated dict.clear(), which wasn't always
> part of the language.
>
> --Guido
>
>> Raymond
>>
>> On Feb 5, 2010, at 11:39 AM, Gerald Britton wrote:
>>
>>> In the list archives, this thread
>>>
>>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
>>>
>>> discusses adding a clear() method to list objects, to complement those
>>> available for sets and dictionaries. ?Later in the thread:
>>>
>>> ? ?http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
>>>
>>> Christian Heimes provided a patch to do it and R. H. commented that
>>> all it would take is Guido's blessing.
>>>
>>> So, I'm wondering, can we do this? ?What are the steps needed to ask
>>> this work to be blessed?
>>>
>>> --
>>> Gerald Britton
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>



-- 
Gerald Britton


From brett at python.org  Fri Feb  5 22:55:43 2010
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Feb 2010 13:55:43 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002051321v79784172gd4f5cc7965190928@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com> 
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
	<ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com> 
	<5d1a32001002051321v79784172gd4f5cc7965190928@mail.gmail.com>
Message-ID: <bbaeab101002051355y359253b5w9e14c701defdf0b4@mail.gmail.com>

On Fri, Feb 5, 2010 at 13:21, Gerald Britton <gerald.britton at gmail.com>wrote:

> On Fri, Feb 5, 2010 at 4:16 PM, Guido van Rossum <guido at python.org> wrote:
> > On Fri, Feb 5, 2010 at 1:09 PM, Raymond Hettinger
> > <raymond.hettinger at gmail.com> wrote:
> >> ISTM, the language moratorium would preclude this change.
> >
> > Oops, you're right. Scratch the go-ahead. :-(
>
> So does this mean I should bring it up again for the next round?
>
>
Yep.

Which reminds me, how do we want to handle ideas that come up during hte
moratorium? Do we simply want to force them to be brought up again when 3.3
development opens up, or should we settle them now and require they have a
PEP or bug marked for 3.3? The former risks a thundering herd of idea but it
does make sure only serious ideas get brought up as someone has to remember
to mention it again in at least a year.

-Brett



> >
> > While PEP 3003 allows case-by-case exemptions for adding methods to
> > built-in objects, I think this isn't one of those cases, since the
> > work-around sure is simple enough, and has been in the language for 20
> > years now. ;-)
> >
> >> Also, since it doesn't actually add any new functionality,
> >> it is more of a documentation issue, teaching basic python idioms:
> >>
> >>    del s[:]
> >> or
> >>    s[:] = []
> >
> > Though it's a common-enough special case that it shouldn't need slice
> > manipulations, which are a bit of an acquired taste. I probably would
> > have added this if I'd anticipated dict.clear(), which wasn't always
> > part of the language.
> >
> > --Guido
> >
> >> Raymond
> >>
> >> On Feb 5, 2010, at 11:39 AM, Gerald Britton wrote:
> >>
> >>> In the list archives, this thread
> >>>
> >>>
> http://mail.python.org/pipermail/python-ideas/2009-April/003897.html
> >>>
> >>> discusses adding a clear() method to list objects, to complement those
> >>> available for sets and dictionaries.  Later in the thread:
> >>>
> >>>
> http://mail.python.org/pipermail/python-ideas/2009-April/003933.html
> >>>
> >>> Christian Heimes provided a patch to do it and R. H. commented that
> >>> all it would take is Guido's blessing.
> >>>
> >>> So, I'm wondering, can we do this?  What are the steps needed to ask
> >>> this work to be blessed?
> >>>
> >>> --
> >>> Gerald Britton
> >>> _______________________________________________
> >>> Python-ideas mailing list
> >>> Python-ideas at python.org
> >>> http://mail.python.org/mailman/listinfo/python-ideas
> >>
> >> _______________________________________________
> >> Python-ideas mailing list
> >> Python-ideas at python.org
> >> http://mail.python.org/mailman/listinfo/python-ideas
> >>
> >
> >
> >
> > --
> > --Guido van Rossum (python.org/~guido)
> >
>
>
>
> --
> Gerald Britton
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100205/3f50fc4b/attachment.html>

From guido at python.org  Fri Feb  5 22:59:32 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 5 Feb 2010 13:59:32 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <bbaeab101002051355y359253b5w9e14c701defdf0b4@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com> 
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
	<ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com> 
	<5d1a32001002051321v79784172gd4f5cc7965190928@mail.gmail.com> 
	<bbaeab101002051355y359253b5w9e14c701defdf0b4@mail.gmail.com>
Message-ID: <ca471dc21002051359g26d0d8bta290d8874c789563@mail.gmail.com>

On Fri, Feb 5, 2010 at 1:55 PM, Brett Cannon <brett at python.org> wrote:
> Which reminds me, how do we want to handle ideas that come up during hte
> moratorium? Do we simply want to force them to be brought up again when 3.3
> development opens up, or should we settle them now and require they have a
> PEP or bug marked for 3.3? The former risks a thundering herd of idea but it
> does make sure only serious ideas get brought up as someone has to remember
> to mention it again in at least a year.

I don't think we need to settle on a single process for this. If
someone cares enough to write a PEP or file a bug let them do it. If
they don't but remember when the moratorium ends, we can't stop them.

I suppose a new tracker category for this would be fine to add. If
someone actually goes through the efforts of writing a PEP falling in
this category we can add a new PEP status too.

-- 
--Guido van Rossum (python.org/~guido)


From brett at python.org  Fri Feb  5 23:06:05 2010
From: brett at python.org (Brett Cannon)
Date: Fri, 5 Feb 2010 14:06:05 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <ca471dc21002051359g26d0d8bta290d8874c789563@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com> 
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>
	<ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com> 
	<5d1a32001002051321v79784172gd4f5cc7965190928@mail.gmail.com> 
	<bbaeab101002051355y359253b5w9e14c701defdf0b4@mail.gmail.com> 
	<ca471dc21002051359g26d0d8bta290d8874c789563@mail.gmail.com>
Message-ID: <bbaeab101002051406s134158a9od2b62cd49f5eab80@mail.gmail.com>

On Fri, Feb 5, 2010 at 13:59, Guido van Rossum <guido at python.org> wrote:

> On Fri, Feb 5, 2010 at 1:55 PM, Brett Cannon <brett at python.org> wrote:
> > Which reminds me, how do we want to handle ideas that come up during hte
> > moratorium? Do we simply want to force them to be brought up again when
> 3.3
> > development opens up, or should we settle them now and require they have
> a
> > PEP or bug marked for 3.3? The former risks a thundering herd of idea but
> it
> > does make sure only serious ideas get brought up as someone has to
> remember
> > to mention it again in at least a year.
>
> I don't think we need to settle on a single process for this. If
> someone cares enough to write a PEP or file a bug let them do it. If
> they don't but remember when the moratorium ends, we can't stop them.
>
> I suppose a new tracker category for this would be fine to add. If
> someone actually goes through the efforts of writing a PEP falling in
> this category we can add a new PEP status too.


For issues it's probably enough to either mark them against Python 3.3 or
add a Moratorium keyword and simply not set their version in case the
moratorium goes past 3.3. As for PEPs, we can mark their targeted version as
'Moratorium' or something.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100205/f13b9c62/attachment.html>

From g.brandl at gmx.net  Sat Feb  6 15:52:57 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 06 Feb 2010 15:52:57 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <bbaeab101002051406s134158a9od2b62cd49f5eab80@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>	<ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com>
	<5d1a32001002051321v79784172gd4f5cc7965190928@mail.gmail.com>
	<bbaeab101002051355y359253b5w9e14c701defdf0b4@mail.gmail.com>
	<ca471dc21002051359g26d0d8bta290d8874c789563@mail.gmail.com>
	<bbaeab101002051406s134158a9od2b62cd49f5eab80@mail.gmail.com>
Message-ID: <hkjvpt$82b$1@ger.gmane.org>

Am 05.02.2010 23:06, schrieb Brett Cannon:
> 
> 
> On Fri, Feb 5, 2010 at 13:59, Guido van Rossum <guido at python.org
> <mailto:guido at python.org>> wrote:
> 
>     On Fri, Feb 5, 2010 at 1:55 PM, Brett Cannon
>     <brett at python.org
>     <mailto:brett at python.org>> wrote:
>     > Which reminds me, how do we want to handle ideas that come up
>     during hte
>     > moratorium? Do we simply want to force them to be brought up again
>     when 3.3
>     > development opens up, or should we settle them now and require
>     they have a
>     > PEP or bug marked for 3.3? The former risks a thundering herd of
>     idea but it
>     > does make sure only serious ideas get brought up as someone has to
>     remember
>     > to mention it again in at least a year.
> 
>     I don't think we need to settle on a single process for this. If
>     someone cares enough to write a PEP or file a bug let them do it. If
>     they don't but remember when the moratorium ends, we can't stop them.
> 
>     I suppose a new tracker category for this would be fine to add. If
>     someone actually goes through the efforts of writing a PEP falling in
>     this category we can add a new PEP status too.
> 
> 
> For issues it's probably enough to either mark them against Python 3.3
> or add a Moratorium keyword and simply not set their version in case the
> moratorium goes past 3.3. As for PEPs, we can mark their targeted
> version as 'Moratorium' or something.

+1 -- I've created http://bugs.python.org/keyword10 for the eventuality.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From ziade.tarek at gmail.com  Sat Feb  6 18:54:41 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 6 Feb 2010 18:54:41 +0100
Subject: [Python-ideas] A common configuration file standard for the stdlib
	+ configparser (or replacer) new API
Message-ID: <94bdd2611002060954m1030feffxca341899f9c27fa4@mail.gmail.com>

Hi,

I am working on a proposal that will add the need to have a new .cfg
file to read for a stdlib package.
This is an existing need already for distutils (with distutils.cfg).

I have a proposal: let's create a mechanism that allows a stdlib
package to store values in a central
configuration file that can be overriden by a per-user configuration file.

This file would be located in the "data" path returned by the install
scheme (see sysconfig.py for details)

so roughly : in sys.prefix for the global one, and in ~/.local/ for
the local per-user one.

Then each package would look in the local one and "update" the global
one with it if found

The files would be a configparser ini-like file, with one mandatory
section that is the name of the package or
module in the stdlib the filename

For example, if "multiprocessing" needs a configuration file, it will
be called "multiprocessing.cfg"
and have at least one section:

    [multiprocessing]
    foo = 1

Last, the configparser (or replacer) would provide a new API called
"get_config(package_name)" that would
read the configuration and return a config object packages can work with.

A nice add-on would be to make it possible to configure the paths, so
os packagers could
drive where the cfg file should be located on their systems

Does it sounds like a good idea ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From tjreedy at udel.edu  Sat Feb  6 19:50:53 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 06 Feb 2010 13:50:53 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <bbaeab101002051355y359253b5w9e14c701defdf0b4@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
	<554BC9E9-EFB8-4C35-A2AE-6E0A8D8201AA@gmail.com>	<ca471dc21002051316g30759c4er2de9b6568ece711f@mail.gmail.com>
	<5d1a32001002051321v79784172gd4f5cc7965190928@mail.gmail.com>
	<bbaeab101002051355y359253b5w9e14c701defdf0b4@mail.gmail.com>
Message-ID: <hkkdm9$hec$1@ger.gmane.org>

On 2/5/2010 4:55 PM, Brett Cannon wrote:

> Which reminds me, how do we want to handle ideas that come up during hte
> moratorium? Do we simply want to force them to be brought up again when
> 3.3 development opens up, or should we settle them now and require they
> have a PEP or bug marked for 3.3?

It was already suggested/agreed that future versions (3.3, 3.4) be added 
to the tracker so that people add ticklers such as 'add xyx warning for 
3.3'. This should be a feature request for 3.3.



From solipsis at pitrou.net  Sat Feb  6 22:55:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 6 Feb 2010 21:55:31 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?A_common_configuration_file_standard_for?=
	=?utf-8?q?_the_stdlib=09+_configparser_=28or_replacer=29_new_API?=
References: <94bdd2611002060954m1030feffxca341899f9c27fa4@mail.gmail.com>
Message-ID: <loom.20100206T224923-247@post.gmane.org>

Tarek Ziad? <ziade.tarek at ...> writes:
> 
> This file would be located in the "data" path returned by the install
> scheme (see sysconfig.py for details)
> 
> so roughly : in sys.prefix for the global one, and in ~/.local/ for
> the local per-user one.

I think it has already been discussed that it's not really the right place for
configuration files.  sys.prefix is wrong for that, as well as ~/.local.

If Python needs to have a notion of a standard path for config files, it should
be something separate from the "data" path.

> The files would be a configparser ini-like file, with one mandatory
> section that is the name of the package or
> module in the stdlib the filename

Why such a constraint? Isn't the name of the file sufficient?


Antoine.




From merwok at netwok.org  Sat Feb  6 23:46:34 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sat, 06 Feb 2010 23:46:34 +0100
Subject: [Python-ideas] A common configuration file standard for the
 stdlib + configparser (or replacer) new API
Message-ID: <4B6DF14A.6040807@netwok.org>

Hello everyone

Two cents: The Basedir Spec, used by free software desktop environments
like Xfce, LXDE, Rox, GNOME or KDE, proposes a scheme that would fit the
bill:

http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html

I proposed following the spec in #7175 too.

There exists a Python module to retrieve these dirs, but it?s poorly
written. I?ve been meaning to get in touch with the maintainers and
propose improvements for a while but haven?t had time yet.

A very nice thing in my opinion would be to integrate config file and
command line parsing. Logilab and others did that. Countless projects
would benefit.

Kind regards





From ziade.tarek at gmail.com  Sat Feb  6 23:55:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 6 Feb 2010 23:55:18 +0100
Subject: [Python-ideas] A common configuration file standard for the
	stdlib + configparser (or replacer) new API
In-Reply-To: <loom.20100206T224923-247@post.gmane.org>
References: <94bdd2611002060954m1030feffxca341899f9c27fa4@mail.gmail.com>
	<loom.20100206T224923-247@post.gmane.org>
Message-ID: <94bdd2611002061455m6a2855c5t33f7fe656353f848@mail.gmail.com>

On Sat, Feb 6, 2010 at 10:55 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Tarek Ziad? <ziade.tarek at ...> writes:
>>
>> This file would be located in the "data" path returned by the install
>> scheme (see sysconfig.py for details)
>>
>> so roughly : in sys.prefix for the global one, and in ~/.local/ for
>> the local per-user one.
>
> I think it has already been discussed that it's not really the right place for
> configuration files. ?sys.prefix is wrong for that, as well as ~/.local.

I am not discussing the right place for those file (e.g. if a better
place exists, this will have to be dealt in sysconfig). I am just
giving an example of a local/global place that we would use with
Python *today* if this file
is considered as a 'data' file.

IOW, we could always add a path in Python install schemes for defining
the right place for configuration
file, and then use it in the idea I am presenting.  (and I think its a
good idea)

>
> If Python needs to have a notion of a standard path for config files, it should
> be something separate from the "data" path.
>
>> The files would be a configparser ini-like file, with one mandatory
>> section that is the name of the package or
>> module in the stdlib the filename
>
> Why such a constraint? Isn't the name of the file sufficient?

Sure

Tarek


From ziade.tarek at gmail.com  Sun Feb  7 00:18:48 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 7 Feb 2010 00:18:48 +0100
Subject: [Python-ideas] A common configuration file standard for the
	stdlib + configparser (or replacer) new API
In-Reply-To: <94bdd2611002061455m6a2855c5t33f7fe656353f848@mail.gmail.com>
References: <94bdd2611002060954m1030feffxca341899f9c27fa4@mail.gmail.com>
	<loom.20100206T224923-247@post.gmane.org>
	<94bdd2611002061455m6a2855c5t33f7fe656353f848@mail.gmail.com>
Message-ID: <94bdd2611002061518t75f4e347k196e581de9412ad7@mail.gmail.com>

On Sat, Feb 6, 2010 at 11:55 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
[..]
> IOW, we could always add a path in Python install schemes for defining
> the right place for configuration
> file, and then use it in the idea I am presenting. ?(and I think its a
> good idea)

-> the idea of adding a particular path for configuration files ;)


From guido at python.org  Sun Feb  7 01:30:55 2010
From: guido at python.org (Guido van Rossum)
Date: Sat, 6 Feb 2010 16:30:55 -0800
Subject: [Python-ideas] [Python-Dev] Proposal for the getpass module
In-Reply-To: <94bdd2611002061608q109e6502kd65cc5df44c71646@mail.gmail.com>
References: <94bdd2611002061608q109e6502kd65cc5df44c71646@mail.gmail.com>
Message-ID: <ca471dc21002061630y4da86d66m9dd2bca1a85c6203@mail.gmail.com>

[redirecting to python-ideas]

On Sat, Feb 6, 2010 at 4:08 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> I would like to propose a small change in the getpass module so it's
> able to get passwords from keyrings (like KWallet, Keychain, etc)
>
> The idea is to provide a getpass.cfg configuration file where people
> can provide the name of a function to use when getpass is called.
> Then third-party projects can implement this function. For example the
> Python Keyring library.[1] could be installed and configured to be
> used by people that wants getpass calls to be handled by this tool.
>
> That's a backward compatible change, and it avoids adding any new
> module in the stdlib. Plus, it offers a greatly improved getpass
> module with no risks for the stdlib stability : it becomes a reference
> implementation with an interface for third-party implementers.
>
> A prototype is here : http://bitbucket.org/tarek/getpass/ (work in
> progress but you can get the idea)
>
> [1] http://pypi.python.org/pypi/keyring

Don't you usually have to tell the keyring which password to retrieve?
The signature for getpass() doesn't have enough info for that.

I'm not sure that not adding a new module to the stdlib is a
sufficient reason to foist the new semantics on an old function.

-- 
--Guido van Rossum (python.org/~guido)


From ziade.tarek at gmail.com  Sun Feb  7 01:42:51 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 7 Feb 2010 01:42:51 +0100
Subject: [Python-ideas] [Python-Dev] Proposal for the getpass module
In-Reply-To: <ca471dc21002061630y4da86d66m9dd2bca1a85c6203@mail.gmail.com>
References: <94bdd2611002061608q109e6502kd65cc5df44c71646@mail.gmail.com>
	<ca471dc21002061630y4da86d66m9dd2bca1a85c6203@mail.gmail.com>
Message-ID: <94bdd2611002061642s388f6aecr282a67c755eefec9@mail.gmail.com>

On Sun, Feb 7, 2010 at 1:30 AM, Guido van Rossum <guido at python.org> wrote:
> [redirecting to python-ideas]
> [..]
>
> Don't you usually have to tell the keyring which password to retrieve?
> The signature for getpass() doesn't have enough info for that.

I've slightly changed the signature to add two new options:

- service : the name of the service the password is associated to
(defaults to 'Python')
- username: the name of the user (defaults to the value returned by
getpass.getuser() )

These options are not used by the existing functions, but make it
possible for a keyring API
to handle the request properly by getting the password stored for
service+username

>
> I'm not sure that not adding a new module to the stdlib is a
> sufficient reason to foist the new semantics on an old function.

I've done it this way so the "Distutils calls getpass" use case would
work transparently, but maybe it's a bad
idea to add a hook like that after all. Maybe a new API for this would
be better, with a dummy implementation
in the stdlib and a way to configure an external tool to be used.

Tarek
-- 
Tarek Ziad? | http://ziade.org


From guido at python.org  Sun Feb  7 02:27:21 2010
From: guido at python.org (Guido van Rossum)
Date: Sat, 6 Feb 2010 17:27:21 -0800
Subject: [Python-ideas] [Python-Dev] Proposal for the getpass module
In-Reply-To: <94bdd2611002061642s388f6aecr282a67c755eefec9@mail.gmail.com>
References: <94bdd2611002061608q109e6502kd65cc5df44c71646@mail.gmail.com> 
	<ca471dc21002061630y4da86d66m9dd2bca1a85c6203@mail.gmail.com> 
	<94bdd2611002061642s388f6aecr282a67c755eefec9@mail.gmail.com>
Message-ID: <ca471dc21002061727h7847d311iacd7cb024ef8f37f@mail.gmail.com>

On Sat, Feb 6, 2010 at 4:42 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Sun, Feb 7, 2010 at 1:30 AM, Guido van Rossum <guido at python.org> wrote:
>> [redirecting to python-ideas]
>> [..]
>>
>> Don't you usually have to tell the keyring which password to retrieve?
>> The signature for getpass() doesn't have enough info for that.
>
> I've slightly changed the signature to add two new options:
>
> - service : the name of the service the password is associated to
> (defaults to 'Python')
> - username: the name of the user (defaults to the value returned by
> getpass.getuser() )
>
> These options are not used by the existing functions, but make it
> possible for a keyring API
> to handle the request properly by getting the password stored for
> service+username

You really need to think of this as adding a new API whose default
behavior is to call the original getpass() function. I don't see any
advantage to giving this function the same name as the classic
getpass() -- the default of getting a password named "Python" is
unlikely to be useful.

(If you disagree, please provide a real use case.)

>> I'm not sure that not adding a new module to the stdlib is a
>> sufficient reason to foist the new semantics on an old function.
>
> I've done it this way so the "Distutils calls getpass" use case would
> work transparently, but maybe it's a bad
> idea to add a hook like that after all. Maybe a new API for this would
> be better, with a dummy implementation
> in the stdlib and a way to configure an external tool to be used.

What's this use case? What password does distutils need to ask for?

Maybe the new API should just be a utility in distutils?

-- 
--Guido van Rossum (python.org/~guido)


From ziade.tarek at gmail.com  Sun Feb  7 08:45:10 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 7 Feb 2010 08:45:10 +0100
Subject: [Python-ideas] [Python-Dev] Proposal for the getpass module
In-Reply-To: <ca471dc21002061727h7847d311iacd7cb024ef8f37f@mail.gmail.com>
References: <94bdd2611002061608q109e6502kd65cc5df44c71646@mail.gmail.com>
	<ca471dc21002061630y4da86d66m9dd2bca1a85c6203@mail.gmail.com>
	<94bdd2611002061642s388f6aecr282a67c755eefec9@mail.gmail.com>
	<ca471dc21002061727h7847d311iacd7cb024ef8f37f@mail.gmail.com>
Message-ID: <94bdd2611002062345w6518564cqd0f5680abed5a323@mail.gmail.com>

On Sun, Feb 7, 2010 at 2:27 AM, Guido van Rossum <guido at python.org> wrote:
[..]
>
> You really need to think of this as adding a new API whose default
> behavior is to call the original getpass() function. I don't see any
> advantage to giving this function the same name as the classic
> getpass() -- the default of getting a password named "Python" is
> unlikely to be useful.
>
> (If you disagree, please provide a real use case.)

I agree. Make sense now

>
>>> I'm not sure that not adding a new module to the stdlib is a
>>> sufficient reason to foist the new semantics on an old function.
>>
>> I've done it this way so the "Distutils calls getpass" use case would
>> work transparently, but maybe it's a bad
>> idea to add a hook like that after all. Maybe a new API for this would
>> be better, with a dummy implementation
>> in the stdlib and a way to configure an external tool to be used.
>
> What's this use case? What password does distutils need to ask for?

When you register or upload a package to PyPI, you have two choices.
Either you store
your password in clear text in your pypirc file, either you type it at
the prompt, through a getpass()
call. The use case is to be able to store it in a safe place rather
than in clear in a config file.

> Maybe the new API should just be a utility in distutils?

Sounds like the solution. Although some people might use it to
interact with a Keyring without necessarily
being in the context of using Distutils, that's why I was targeting a
module oustide distutils in the stdlib.

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From ziade.tarek at gmail.com  Sun Feb  7 08:46:47 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 7 Feb 2010 08:46:47 +0100
Subject: [Python-ideas] [Python-Dev] Proposal for the getpass module
In-Reply-To: <319e029f1002062317u3425fc1et497c4de7ee463846@mail.gmail.com>
References: <94bdd2611002061608q109e6502kd65cc5df44c71646@mail.gmail.com>
	<ca471dc21002061630y4da86d66m9dd2bca1a85c6203@mail.gmail.com>
	<319e029f1002062317u3425fc1et497c4de7ee463846@mail.gmail.com>
Message-ID: <94bdd2611002062346u167e0aedv5e5dcdf02bcc298@mail.gmail.com>

2010/2/7 Lennart Regebro <regebro at gmail.com>:
[..]
>
> Having getpass calling the keyring module seems backwards to me. It
> should rather be the keyring module that calls getpass, if needed.

I think you are right, the getpass module should be considered as one
way to get a password.

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From dreamingforward at gmail.com  Mon Feb  8 02:14:36 2010
From: dreamingforward at gmail.com (average)
Date: Sun, 7 Feb 2010 19:14:36 -0600
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com>
Message-ID: <913f9f571002071714o38ee7baeobaa4c6a0cdbfcdd@mail.gmail.com>

On Fri, Feb 5, 2010 at 1:39 PM, Gerald Britton <gerald.britton at gmail.com> wrote:
> In the list archives, this thread
> discusses adding a clear() method to list objects, to complement those
> available for sets and dictionaries. ?Later in the thread:
> Christian Heimes provided a patch to do it and R. H. commented that
> all it would take is Guido's blessing.
>
> So, I'm wondering, can we do this? ?What are the steps needed to ask
> this work to be blessed?

In the abstract it seems like such a method should be part of the
Container ABC.  Since the idea of a container would imply a method to
clear its contents.

mark


From guido at python.org  Mon Feb  8 03:10:35 2010
From: guido at python.org (Guido van Rossum)
Date: Sun, 7 Feb 2010 18:10:35 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <913f9f571002071714o38ee7baeobaa4c6a0cdbfcdd@mail.gmail.com>
References: <5d1a32001002051139i7e4b7b25q8b66d1e2cd901047@mail.gmail.com> 
	<913f9f571002071714o38ee7baeobaa4c6a0cdbfcdd@mail.gmail.com>
Message-ID: <ca471dc21002071810m1ea41e83l1c90382708fd2d24@mail.gmail.com>

On Sun, Feb 7, 2010 at 5:14 PM, average <dreamingforward at gmail.com> wrote:
> On Fri, Feb 5, 2010 at 1:39 PM, Gerald Britton <gerald.britton at gmail.com> wrote:
>> In the list archives, this thread
>> discusses adding a clear() method to list objects, to complement those
>> available for sets and dictionaries. ?Later in the thread:
>> Christian Heimes provided a patch to do it and R. H. commented that
>> all it would take is Guido's blessing.
>>
>> So, I'm wondering, can we do this? ?What are the steps needed to ask
>> this work to be blessed?
>
> In the abstract it seems like such a method should be part of the
> Container ABC. ?Since the idea of a container would imply a method to
> clear its contents.

Have you actually seen Python's Container ABC?

-- 
--Guido van Rossum (python.org/~guido)


From weasley_wx at qq.com  Wed Feb 10 11:39:45 2010
From: weasley_wx at qq.com (=?ISO-8859-1?B?d3h5YXJ2?=)
Date: Wed, 10 Feb 2010 18:39:45 +0800
Subject: [Python-ideas] clear() method for lists
Message-ID: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>

what about another method clone() (or copy())? i think this maybe useful either.


------------------ Original ------------------
From:  "average"<dreamingforward at gmail.com>;
Date:  Mon, Feb 8, 2010 09:14 AM
To:  "Gerald Britton"<gerald.britton at gmail.com>; 
Cc:  "Python-Ideas"<python-ideas at python.org>; 
Subject:  Re: [Python-ideas] clear() method for lists

On Fri, Feb 5, 2010 at 1:39 PM, Gerald Britton <gerald.britton at gmail.com> wrote:
> In the list archives, this thread
> discusses adding a clear() method to list objects, to complement those
> available for sets and dictionaries.  Later in the thread:
> Christian Heimes provided a patch to do it and R. H. commented that
> all it would take is Guido's blessing.
>
> So, I'm wondering, can we do this?  What are the steps needed to ask
> this work to be blessed?

In the abstract it seems like such a method should be part of the
Container ABC.  Since the idea of a container would imply a method to
clear its contents.

mark
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
http://mail.python.org/mailman/listinfo/python-ideas

From grosser.meister.morti at gmx.net  Wed Feb 10 15:12:41 2010
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Wed, 10 Feb 2010 15:12:41 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
Message-ID: <4B72BED9.6050404@gmx.net>

On 02/10/2010 11:39 AM, wxyarv wrote:
> what about another method clone() (or copy())? i think this maybe useful either.
>

l1 = [1, 2, 3]
l2 = l1[:]
l3 = list(l1)


From gerald.britton at gmail.com  Wed Feb 10 15:54:22 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Wed, 10 Feb 2010 09:54:22 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <4B72BED9.6050404@gmx.net>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
Message-ID: <5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>

On Wed, Feb 10, 2010 at 9:12 AM, Mathias Panzenb?ck
<grosser.meister.morti at gmx.net> wrote:
> On 02/10/2010 11:39 AM, wxyarv wrote:
>>
>> what about another method clone() (or copy())? i think this maybe useful
>> either.
>>
>
> l1 = [1, 2, 3]
> l2 = l1[:]
> l3 = list(l1)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>

Yes, I plan to ask for copy() as well, when the bug tracker opens up
for 3.3, 3.4, etc.  The issue is not, "Is there already a way to do
this?"  but rather, "Can we have consistent interfaces in the sequence
types and collections where possible and appropriate?"

dict() and set() already support both clear() and copy() methods.
Previous posters have pointed to the disconnect and showed the problem
of having to test if a given iterable supports the clear() method
before calling it, in functions that take any iterable.

Also, for what it's worth:

s1 = set()
s2 = s1.copy()

is faster than

s1 = set()
s2 = set(s1)

(and also for dict()) probably because the first is
specifically-written for the copy operation whereas the second
actually iterates over s1, one item at a time.  (At least I think
that's what's going on).  I suppose that a list().copy() method might
also be faster than the other two approaches to copy a list.

Lastly, for completeness,  I suppose copy() might be appropriate for
both tuple and deque as well.

-- 
Gerald Britton


From raymond.hettinger at gmail.com  Wed Feb 10 18:10:58 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Wed, 10 Feb 2010 09:10:58 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
Message-ID: <388DBD0C-E725-4EC5-B2C3-CA7C090C2E22@gmail.com>


On Feb 10, 2010, at 6:54 AM, Gerald Britton wrote:
> Yes, I plan to ask for copy() as well, when the bug tracker opens up
> for 3.3, 3.4, etc.  The issue is not, "Is there already a way to do
> this?"  but rather, "Can we have consistent interfaces in the sequence
> types and collections where possible and appropriate?"

Use the copy module.

> dict() and set() already support both clear() and copy() methods.
> Previous posters have pointed to the disconnect and showed the problem
> of having to test if a given iterable supports the clear() method
> before calling it, in functions that take any iterable.
> 
> Also, for what it's worth:
> 
> s1 = set()
> s2 = s1.copy()
> 
> is faster than
> 
> s1 = set()
> s2 = set(s1)

I question your timing skills.  Both call the same routine to do the work
of copying entries:

   set_copy() calls make_new_set() which calls set_update_internal()
   set_init() calls set_update_internal()

If there is any difference at all, it is the constant overhead of passing
an argument to set(), not the implementation itself.  The actual
set building work is the same.

> (and also for dict()) probably because the first is
> specifically-written for the copy operation whereas the second
> actually iterates over s1, one item at a time.  (At least I think
> that's what's going on).  I suppose that a list().copy() method might
> also be faster than the other two approaches to copy a list.

You need to read some code, learn about ref counts, etc.
There's more to list copying than a memcpy().
If list.copy() were added, it would use that same underlying
code as list(s) and s[:].  There would be no speed-up.

> Lastly, for completeness,  I suppose copy() might be appropriate for
> both tuple and deque as well.

Tuples?  Really?
An immutable collection is its own copy.


Raymond



From gerald.britton at gmail.com  Wed Feb 10 18:20:41 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Wed, 10 Feb 2010 12:20:41 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <388DBD0C-E725-4EC5-B2C3-CA7C090C2E22@gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<388DBD0C-E725-4EC5-B2C3-CA7C090C2E22@gmail.com>
Message-ID: <5d1a32001002100920o40debd70m3d7194c298f554dc@mail.gmail.com>

On Wed, Feb 10, 2010 at 12:10 PM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
>
> On Feb 10, 2010, at 6:54 AM, Gerald Britton wrote:
>> Yes, I plan to ask for copy() as well, when the bug tracker opens up
>> for 3.3, 3.4, etc. ?The issue is not, "Is there already a way to do
>> this?" ?but rather, "Can we have consistent interfaces in the sequence
>> types and collections where possible and appropriate?"
>
> Use the copy module.
>
>> dict() and set() already support both clear() and copy() methods.
>> Previous posters have pointed to the disconnect and showed the problem
>> of having to test if a given iterable supports the clear() method
>> before calling it, in functions that take any iterable.
>>
>> Also, for what it's worth:
>>
>> s1 = set()
>> s2 = s1.copy()
>>
>> is faster than
>>
>> s1 = set()
>> s2 = set(s1)
>
> I question your timing skills. ?Both call the same routine to do the work
> of copying entries:
>
> ? set_copy() calls make_new_set() which calls set_update_internal()
> ? set_init() calls set_update_internal()
>
> If there is any difference at all, it is the constant overhead of passing
> an argument to set(), not the implementation itself. ?The actual
> set building work is the same.
>
>> (and also for dict()) probably because the first is
>> specifically-written for the copy operation whereas the second
>> actually iterates over s1, one item at a time. ?(At least I think
>> that's what's going on). ?I suppose that a list().copy() method might
>> also be faster than the other two approaches to copy a list.
>
> You need to read some code, learn about ref counts, etc.
> There's more to list copying than a memcpy().
> If list.copy() were added, it would use that same underlying
> code as list(s) and s[:]. ?There would be no speed-up.
>
>> Lastly, for completeness, ?I suppose copy() might be appropriate for
>> both tuple and deque as well.
>
> Tuples? ?Really?
> An immutable collection is its own copy.
>
>
> Raymond
>
>

Thanks for the feedback.  I also question my timing skills (and most
other skills that I think I have).  That's what's good about bouncing
ideas around here.  Silly ones get shot down, and rightly so!

-- 
Gerald Britton


From greg.ewing at canterbury.ac.nz  Thu Feb 11 01:02:39 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 11 Feb 2010 13:02:39 +1300
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <4B72BED9.6050404@gmx.net>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
Message-ID: <4B73491F.4000409@canterbury.ac.nz>

Mathias Panzenb?ck wrote:
> On 02/10/2010 11:39 AM, wxyarv wrote:
>> what about another method clone() (or copy())?

Last time copying lists was discussed, I seem to remember
there were considered to be too many ways of doing it
already, so I can't see another one being added.

-- 
Greg


From simon at brunningonline.net  Thu Feb 11 09:08:50 2010
From: simon at brunningonline.net (Simon Brunning)
Date: Thu, 11 Feb 2010 08:08:50 +0000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
Message-ID: <8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>

2010/2/10 Gerald Britton <gerald.britton at gmail.com>:
> Lastly, for completeness, ?I suppose copy() might be appropriate for
> both tuple and deque as well.

Why would you want to copy a tuple?

-- 
Cheers,
Simon B.


From ncoghlan at gmail.com  Thu Feb 11 12:57:13 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 11 Feb 2010 21:57:13 +1000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <4B73491F.4000409@canterbury.ac.nz>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>
	<4B73491F.4000409@canterbury.ac.nz>
Message-ID: <4B73F099.3010204@gmail.com>

Greg Ewing wrote:
> Mathias Panzenb?ck wrote:
>> On 02/10/2010 11:39 AM, wxyarv wrote:
>>> what about another method clone() (or copy())?
> 
> Last time copying lists was discussed, I seem to remember
> there were considered to be too many ways of doing it
> already, so I can't see another one being added.

With the obvious way of getting a consistent interface already being to
use copy.copy.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From gerald.britton at gmail.com  Thu Feb 11 15:51:42 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Thu, 11 Feb 2010 09:51:42 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
Message-ID: <5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>

On Thu, Feb 11, 2010 at 3:08 AM, Simon Brunning
<simon at brunningonline.net> wrote:
> 2010/2/10 Gerald Britton <gerald.britton at gmail.com>:
>> Lastly, for completeness, ?I suppose copy() might be appropriate for
>> both tuple and deque as well.
>
> Why would you want to copy a tuple?

Say you had a problem where you started with a basic tuple, then
needed to add items to it to produce some result.  Now suppose you
want to do that repeatedly.  You don't want to disturb the basic
tuple, so you make a copy of it before extending it.

e.g.

>>> country = ("US",)
>>> country_state = tuple(country)+("NY",)
>>> country_state_city = tuple(country_state) + ("NY",)
>>> country
('US',)
>>> country_state
('US', 'NY')
>>> country_state_city
('US', 'NY', 'NY')

if tuple() had a copy() method, I could write:

country_state = country.copy() + ("NY",)

etc.

Not that this is necessarily "better" in some way. I'm just thinking
about consistency across the built-in types.  If dict() and set() have
copy(), why not list() and tuple()?

On the other hand, if the consensus is _not_ to add the copy() method
to lists and tuples, why not deprecate the method in sets and dicts
and encourage folks to use the copy module or just use "newdict =
dict(olddict)" and "newset = set(oldset)" to build a new dictionary or
set from an existing one?


>
> --
> Cheers,
> Simon B.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton


From simon at brunningonline.net  Thu Feb 11 16:00:30 2010
From: simon at brunningonline.net (Simon Brunning)
Date: Thu, 11 Feb 2010 15:00:30 +0000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
Message-ID: <8c7f10c61002110700t7345da25s12935b3370b78c19@mail.gmail.com>

On 11 February 2010 14:51, Gerald Britton <gerald.britton at gmail.com> wrote:
> Say you had a problem where you started with a basic tuple, then
> needed to add items to it to produce some result.

Bzzzz! Tuples are immutable - you can't add items to them.

-- 
Cheers,
Simon B.


From p.f.moore at gmail.com  Thu Feb 11 16:01:34 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Feb 2010 15:01:34 +0000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
Message-ID: <79990c6b1002110701i361bce3mf1ef2d5bd74f3e73@mail.gmail.com>

On 11 February 2010 14:51, Gerald Britton <gerald.britton at gmail.com> wrote:
> Say you had a problem where you started with a basic tuple, then
> needed to add items to it to produce some result. ?Now suppose you
> want to do that repeatedly. ?You don't want to disturb the basic
> tuple, so you make a copy of it before extending it.
>
> e.g.
>
>>>> country = ("US",)
>>>> country_state = tuple(country)+("NY",)
>>>> country_state_city = tuple(country_state) + ("NY",)
>>>> country
> ('US',)
>>>> country_state
> ('US', 'NY')
>>>> country_state_city
> ('US', 'NY', 'NY')
>
> if tuple() had a copy() method, I could write:
>
> country_state = country.copy() + ("NY",)
>
> etc.

You do know that tuples are immutable, don't you?

Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> country = ("US",)
>>> country_state = country+("NY",)
>>> country_state_city = country_state + ("NY",)
>>> country
('US',)
>>> country_state
('US', 'NY')
>>> country_state_city
('US', 'NY', 'NY')

It sounds like you could do with reading the Python documentation a
bit more closely before proposing changes...

Paul


From andreengels at gmail.com  Thu Feb 11 16:11:20 2010
From: andreengels at gmail.com (Andre Engels)
Date: Thu, 11 Feb 2010 16:11:20 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net> 
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com> 
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com> 
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
Message-ID: <6faf39c91002110711u579ab987v7754810807be7a63@mail.gmail.com>

On Thu, Feb 11, 2010 at 3:51 PM, Gerald Britton
<gerald.britton at gmail.com> wrote:

>>>> country = ("US",)
>>>> country_state = tuple(country)+("NY",)
>>>> country_state_city = tuple(country_state) + ("NY",)
>>>> country
> ('US',)
>>>> country_state
> ('US', 'NY')
>>>> country_state_city
> ('US', 'NY', 'NY')
>
> if tuple() had a copy() method, I could write:
>
> country_state = country.copy() + ("NY",)

Note that for a tuple T

tuple(T) == T

So you can already write:

country_state = country + ("NY",)

and it will already have exactly the same effect that tuple(country)
or your proposed country.copy() would have.

-- 
Andr? Engels, andreengels at gmail.com


From phd at phd.pp.ru  Thu Feb 11 15:57:24 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Thu, 11 Feb 2010 17:57:24 +0300
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
Message-ID: <20100211145724.GA2894@phd.pp.ru>

On Thu, Feb 11, 2010 at 09:51:42AM -0500, Gerald Britton wrote:
> Say you had a problem where you started with a basic tuple, then
> needed to add items to it to produce some result.  Now suppose you
> want to do that repeatedly.  You don't want to disturb the basic
> tuple

   You can never "disturb" a tuple - it's a read-only object.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From andreengels at gmail.com  Thu Feb 11 16:12:49 2010
From: andreengels at gmail.com (Andre Engels)
Date: Thu, 11 Feb 2010 16:12:49 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <6faf39c91002110711u579ab987v7754810807be7a63@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net> 
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com> 
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com> 
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com> 
	<6faf39c91002110711u579ab987v7754810807be7a63@mail.gmail.com>
Message-ID: <6faf39c91002110712h641f3f3fh6e479a99c3774a95@mail.gmail.com>

On Thu, Feb 11, 2010 at 4:11 PM, Andre Engels <andreengels at gmail.com> wrote:

> Note that for a tuple T
>
> tuple(T) == T

Of course what I actually meant was:

tuple(T) is T


-- 
Andr? Engels, andreengels at gmail.com


From matt.horizon5 at gmail.com  Thu Feb 11 16:21:27 2010
From: matt.horizon5 at gmail.com (Matthew Russell)
Date: Thu, 11 Feb 2010 15:21:27 +0000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <20100211145724.GA2894@phd.pp.ru>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net> 
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com> 
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com> 
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com> 
	<20100211145724.GA2894@phd.pp.ru>
Message-ID: <3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>

this seems to work in python 2.x and python3.1, although I suspect it's a
bug.

>>> t = (1, 2)
>>> t += (3,)
>>> t
(1, 2, 3)


On 11 February 2010 14:57, Oleg Broytman <phd at phd.pp.ru> wrote:

> On Thu, Feb 11, 2010 at 09:51:42AM -0500, Gerald Britton wrote:
> > Say you had a problem where you started with a basic tuple, then
> > needed to add items to it to produce some result.  Now suppose you
> > want to do that repeatedly.  You don't want to disturb the basic
> > tuple
>
>    You can never "disturb" a tuple - it's a read-only object.
>
> Oleg.
> --
>     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
>           Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Cheers,
Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100211/daead258/attachment.html>

From grosser.meister.morti at gmx.net  Thu Feb 11 16:34:28 2010
From: grosser.meister.morti at gmx.net (=?UTF-8?B?TWF0aGlhcyBQYW56ZW5iw7Zjaw==?=)
Date: Thu, 11 Feb 2010 16:34:28 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
Message-ID: <4B742384.8020100@gmx.net>

On 02/11/2010 04:21 PM, Matthew Russell wrote:
> this seems to work in python 2.x and python3.1, although I suspect it's
> a bug.
>
>  >>> t = (1, 2)
>  >>> t += (3,)
>  >>> t
> (1, 2, 3)
>

I don't see any bug. "x += y" is equal to "x = x + y".

 >>> t=(1,2)
 >>> t2=t
 >>> t+=(3,)
 >>> t,t2
((1, 2, 3), (1, 2))


From phd at phd.pp.ru  Thu Feb 11 16:35:23 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Thu, 11 Feb 2010 18:35:23 +0300
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
Message-ID: <20100211153523.GA6959@phd.pp.ru>

On Thu, Feb 11, 2010 at 03:21:27PM +0000, Matthew Russell wrote:
> this seems to work in python 2.x and python3.1, although I suspect it's a
> bug.
> 
> >>> t = (1, 2)
> >>> t += (3,)
> >>> t
> (1, 2, 3)

   It's not a bug. += is not obliged to increase (extend) objects in place.
In case of read-only objects += creates a new extended object and returns
it:

>>> a = 2
>>> a += 1
>>> print a
3

   You don't suppose that 2 magically became 3, do you? Instead += replaces
an integer object pointed to by a with a different integer object.
   The same is true for tuples. The original tuple of len 2 was replaced by
a completely new tuple of len 3. If you hold a reference to the original
tuple you can find it's still intact:

>>> a = (1, 2)
>>> b = a # b *is not* a copy, b holds a reference *to the same tuple*
>>> a += (3,) # Now points to a different tuple
>>> a
(1, 2, 3)
>>> b # But the original tuple is still the same
(1, 2)

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From tlesher at gmail.com  Thu Feb 11 16:38:22 2010
From: tlesher at gmail.com (Tim Lesher)
Date: Thu, 11 Feb 2010 10:38:22 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
Message-ID: <9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>

On Thu, Feb 11, 2010 at 10:21, Matthew Russell <matt.horizon5 at gmail.com>wrote:

> this seems to work in python 2.x and python3.1, although I suspect it's a
> bug.
>
> >>> t = (1, 2)
> >>> t += (3,)
> >>> t
> (1, 2, 3)
>
> The object "t" references at the end isn't the same one that it references
at the beginning.  Note the difference between lists and tuples here:

>>> a = [1,2]
>>> id(a)
11274840
>>> a += [3,]
>>> id(a)
11274840

a is a list; augmented assignment mutates it, but it's still the same
object.

>>> b = (1,2)
>>> id(b)
13902872
>>> b += (3,)
>>> id(b)
13915800
>>>

b is a tuple; augmented assignment creates a new object and re-binds "b" to
it.

-- 
Tim Lesher <tlesher at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100211/19d165da/attachment.html>

From gerald.britton at gmail.com  Thu Feb 11 18:35:08 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Thu, 11 Feb 2010 12:35:08 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
Message-ID: <5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>

On Thu, Feb 11, 2010 at 10:38 AM, Tim Lesher <tlesher at gmail.com> wrote:
>
>
> On Thu, Feb 11, 2010 at 10:21, Matthew Russell <matt.horizon5 at gmail.com>
> wrote:
>>
>> this seems to work in python 2.x and python3.1, although I suspect it's a
>> bug.
>> >>> t = (1, 2)
>> >>> t += (3,)
>> >>> t
>> (1, 2, 3)
>>
> The object "t" references at the end isn't the same one that it references
> at the beginning. ?Note the difference between lists and tuples here:
>>>> a = [1,2]
>>>> id(a)
> 11274840
>>>> a += [3,]
>>>> id(a)
> 11274840
> a is a list; augmented assignment mutates it, but it's still the same
> object.
>>>> b = (1,2)
>>>> id(b)
> 13902872
>>>> b += (3,)
>>>> id(b)
> 13915800
>>>>
>
> b is a tuple; augmented assignment creates a new object and re-binds "b" to
> it.
> --
> Tim Lesher <tlesher at gmail.com>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>

Thanks all for helping me understand this better.  The subtly above is
something I missed.  I searched the doc for a description of it but
couldn't readily find it.  Tim's simple one-line statement and the
example above does it very nicely.

Switching gears for a moment, what is the feeling regarding the copy()
methods for dictionaries and sets?  Are they truly redundant?  Should
they be deprecated?  Should users be encouraged to use the copy module
or just  use "newdict = dict(olddict)" and "newset = set(oldset)" to
build a new dictionary or set from an existing one?

-- 
Gerald Britton


From grosser.meister.morti at gmx.net  Thu Feb 11 19:19:49 2010
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Thu, 11 Feb 2010 19:19:49 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>	<20100211145724.GA2894@phd.pp.ru>	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
Message-ID: <4B744A45.6030300@gmx.net>

On 02/11/2010 06:35 PM, Gerald Britton wrote:
 >
 > Switching gears for a moment, what is the feeling regarding the copy()
 > methods for dictionaries and sets?  Are they truly redundant?  Should
 > they be deprecated?  Should users be encouraged to use the copy module
 > or just  use "newdict = dict(olddict)" and "newset = set(oldset)" to
 > build a new dictionary or set from an existing one?

I don't know what one *should* do but I never used .copy() but always 
dict(olddict) or set(oldset).

	-panzi


From tjreedy at udel.edu  Thu Feb 11 21:36:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 11 Feb 2010 15:36:34 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>	<20100211145724.GA2894@phd.pp.ru>	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
Message-ID: <hl1pog$kg0$1@ger.gmane.org>

On 2/11/2010 12:35 PM, Gerald Britton wrote:
> On Thu, Feb 11, 2010 at 10:38 AM, Tim Lesher<tlesher at gmail.com>  wrote:

> Switching gears for a moment, what is the feeling regarding the copy()
> methods for dictionaries and sets?  Are they truly redundant?  Should
> they be deprecated?  Should users be encouraged to use the copy module
> or just  use "newdict = dict(olddict)" and "newset = set(oldset)" to
> build a new dictionary or set from an existing one?

I did not even know that they exist and do not know why they exist.
In my opinion, set(x) should special case s being a set/frozenset, and 
maybe even a dict, and so whatever set.copy does now. Ditto for dict.


Terry Jan Reedy



From turnbull at sk.tsukuba.ac.jp  Fri Feb 12 03:54:07 2010
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 12 Feb 2010 11:54:07 +0900
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
Message-ID: <87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>

Gerald Britton writes:

 > Thanks all for helping me understand this better.  The subtly above is
 > something I missed.  I searched the doc for a description of it but
 > couldn't readily find it.  Tim's simple one-line statement and the
 > example above does it very nicely.

It's in the language reference.  It is only two lines (the definition
of "immutable" and the description of assignment semantics), so easy
to miss. :-)  There probably is some discussion in the tutorial.

 > Switching gears for a moment, what is the feeling regarding the copy()
 > methods for dictionaries and sets?  Are they truly redundant?  Should
 > they be deprecated?  Should users be encouraged to use the copy module
 > or just  use "newdict = dict(olddict)" and "newset = set(oldset)" to
 > build a new dictionary or set from an existing one?

I think they are redundant.  new = type(old) should be the standard
idiom for an efficient shallow copy.  If that doesn't serve your
application's needs, use the copy module.  The responsibility for
discrimination is the application programmer's.  Superficially this
might seem to violate TOOWTDI, but actually, not.  Shallow copies and
deep copies are two very different "Its", and have to be decided by
the app author in any case.

I don't see what .copy can add.

.clear is another matter, in terms of semantics.  However, the same
effect can be achieve at the cost of indirection and extra garbage:

class DictWithClear(object):
    def __init__(self):
        self.clear()

    def clear(self):
        d = {}

    # Implement other dict methods here.

This is obviously wasteful if all you want to do is add .clear to a
"bare" dictionary.  However, in many cases the dictionary is an
attribute of a larger structure already and the only direct reference
to the dictionary is from that structure.  Then clearing by replacing
the obsolete dictionary with a fresh empty one is hardly less
efficient than clearing the obsolete contents.

There are other arguments *for* the .clear method (eg, it would be a
possibly useful optimization if instead of a class with a dictionary
attribute, the class inherited from the dictionary).



From george.sakkis at gmail.com  Fri Feb 12 08:26:15 2010
From: george.sakkis at gmail.com (George Sakkis)
Date: Fri, 12 Feb 2010 08:26:15 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>

Speaking of new potential list methods, how about list.get(index,
default=None) ala dict.get ? I'm sure this has must have come up at
some point but can't find it ATM.

George


From ncoghlan at gmail.com  Fri Feb 12 10:58:09 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 12 Feb 2010 19:58:09 +1000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>	<20100211145724.GA2894@phd.pp.ru>	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>
Message-ID: <4B752631.5050800@gmail.com>

George Sakkis wrote:
> Speaking of new potential list methods, how about list.get(index,
> default=None) ala dict.get ? I'm sure this has must have come up at
> some point but can't find it ATM.

I believe it runs afoul of the moratorium, but a getitem() builtin might
be a better idea (since it would then work for any class that implements
__getitem__).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From masklinn at masklinn.net  Fri Feb 12 11:05:08 2010
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 12 Feb 2010 11:05:08 +0100
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <4B752631.5050800@gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>	<20100211145724.GA2894@phd.pp.ru>	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>
	<4B752631.5050800@gmail.com>
Message-ID: <3547E294-6673-450F-AAD1-6CF7FB54B24E@masklinn.net>

On 12 Feb 2010, at 10:58 , Nick Coghlan wrote:
> 
> George Sakkis wrote:
>> Speaking of new potential list methods, how about list.get(index,
>> default=None) ala dict.get ? I'm sure this has must have come up at
>> some point but can't find it ATM.
> 
> I believe it runs afoul of the moratorium, but a getitem() builtin might
> be a better idea (since it would then work for any class that implements
> __getitem__).

Maybe just extending operator.itemgetter with a "default" kwarg? Wouldn't run afoul the moratorium, and would be quite a nice extension to itemgetter.

Though I'm not sure it's a very good idea for lists. Semantically, lists are to be iterated, not really to be indexed.



From ncoghlan at gmail.com  Fri Feb 12 11:22:54 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 12 Feb 2010 20:22:54 +1000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <3547E294-6673-450F-AAD1-6CF7FB54B24E@masklinn.net>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>	<20100211145724.GA2894@phd.pp.ru>	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>
	<4B752631.5050800@gmail.com>
	<3547E294-6673-450F-AAD1-6CF7FB54B24E@masklinn.net>
Message-ID: <4B752BFE.1080101@gmail.com>

Masklinn wrote:
> On 12 Feb 2010, at 10:58 , Nick Coghlan wrote:
>> George Sakkis wrote:
>>> Speaking of new potential list methods, how about list.get(index,
>>>  default=None) ala dict.get ? I'm sure this has must have come up
>>> at some point but can't find it ATM.
>> I believe it runs afoul of the moratorium, but a getitem() builtin
>> might be a better idea (since it would then work for any class that
>> implements __getitem__).
> 
> Maybe just extending operator.itemgetter with a "default" kwarg?
> Wouldn't run afoul the moratorium, and would be quite a nice
> extension to itemgetter.

Yeah, a kw-only arg for itemgetter and attrgetter could definitely work.
It would be somewhat clunky to use though.

> Though I'm not sure it's a very good idea for lists. Semantically,
> lists are to be iterated, not really to be indexed.

Using short lists as record sets happens all the time (especially with
things like str.split and other parsing operations that build up their
results incrementally).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From gerald.britton at gmail.com  Fri Feb 12 14:50:03 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 12 Feb 2010 08:50:03 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <5d1a32001002120550w4e37d700g93c29524ccf67d32@mail.gmail.com>

Thanks Tim.  The dict and set types _do_ have clear() methods, but not
the list() type.  I first ran into this sometime ago when a question
was posted about it.  It intrigued me because I saw what I thought was
a gap.  Basically I like things to be consistent.  I was also
wondering about garbage collection.  If I have a humongous list, e.g.
and "clear" it with:

mylist = []

does the old content not need to be garbage collected?  Might it not
continue to occupy its memory for a while?  OTOH do dict.clear() and
set.clear() immediately free their memory or does it just get queued
for garbage collection?

On Thu, Feb 11, 2010 at 9:54 PM, Stephen J. Turnbull
<turnbull at sk.tsukuba.ac.jp> wrote:
> Gerald Britton writes:
>
> ?> Thanks all for helping me understand this better. ?The subtly above is
> ?> something I missed. ?I searched the doc for a description of it but
> ?> couldn't readily find it. ?Tim's simple one-line statement and the
> ?> example above does it very nicely.
>
> It's in the language reference. ?It is only two lines (the definition
> of "immutable" and the description of assignment semantics), so easy
> to miss. :-) ?There probably is some discussion in the tutorial.
>
> ?> Switching gears for a moment, what is the feeling regarding the copy()
> ?> methods for dictionaries and sets? ?Are they truly redundant? ?Should
> ?> they be deprecated? ?Should users be encouraged to use the copy module
> ?> or just ?use "newdict = dict(olddict)" and "newset = set(oldset)" to
> ?> build a new dictionary or set from an existing one?
>
> I think they are redundant. ?new = type(old) should be the standard
> idiom for an efficient shallow copy. ?If that doesn't serve your
> application's needs, use the copy module. ?The responsibility for
> discrimination is the application programmer's. ?Superficially this
> might seem to violate TOOWTDI, but actually, not. ?Shallow copies and
> deep copies are two very different "Its", and have to be decided by
> the app author in any case.
>
> I don't see what .copy can add.
>
> .clear is another matter, in terms of semantics. ?However, the same
> effect can be achieve at the cost of indirection and extra garbage:
>
> class DictWithClear(object):
> ? ?def __init__(self):
> ? ? ? ?self.clear()
>
> ? ?def clear(self):
> ? ? ? ?d = {}
>
> ? ?# Implement other dict methods here.
>
> This is obviously wasteful if all you want to do is add .clear to a
> "bare" dictionary. ?However, in many cases the dictionary is an
> attribute of a larger structure already and the only direct reference
> to the dictionary is from that structure. ?Then clearing by replacing
> the obsolete dictionary with a fresh empty one is hardly less
> efficient than clearing the obsolete contents.
>
> There are other arguments *for* the .clear method (eg, it would be a
> possibly useful optimization if instead of a class with a dictionary
> attribute, the class inherited from the dictionary).
>
>



-- 
Gerald Britton


From anfedorov at gmail.com  Fri Feb 12 16:29:30 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Fri, 12 Feb 2010 10:29:30 -0500
Subject: [Python-ideas] itertools has starmap but no starreduce?
Message-ID: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com>

Is that right, or am I missing something obvious?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100212/ca9ed986/attachment.html>

From stephen at xemacs.org  Fri Feb 12 16:44:54 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 13 Feb 2010 00:44:54 +0900
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <5d1a32001002120550w4e37d700g93c29524ccf67d32@mail.gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<5d1a32001002120550w4e37d700g93c29524ccf67d32@mail.gmail.com>
Message-ID: <87bpfuwj9l.fsf@uwakimon.sk.tsukuba.ac.jp>

Gerald Britton writes:

 > I was also wondering about garbage collection.  If I have a
 > humongous list, e.g.  and "clear" it with:
 > 
 > mylist = []
 > 
 > does the old content not need to be garbage collected?

  thislist = generate_me_a_humongous_list()
  thatlist = thislist
  thatlist = []

Definitely no garbage collection.

The starting point of using garbage collection is that in general you
don't know *locally* whether something is reachable or not.  So you
need to do a global analysis.

 > OTOH do dict.clear() and set.clear() immediately free their memory
 > or does it just get queued for garbage collection?

This is covered in the manuals, but the gist is that every Python
object knows how many other objects are pointing to it (called a
refcount).  When an object's refcount drops to zero, it gets collected
(immediately, IIRC).  However ...

  thislist = []
  thatlist = [thislist]
  thislist.append(thatlist)

and you have a reference cycle.  These cycles are also collected, but
this requires more effort, and so it is done only occasionally.



From ncoghlan at gmail.com  Fri Feb 12 16:43:07 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 13 Feb 2010 01:43:07 +1000
Subject: [Python-ideas] itertools has starmap but no starreduce?
In-Reply-To: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com>
References: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com>
Message-ID: <4B75770B.6070002@gmail.com>

Andrey Fedorov wrote:
> Is that right, or am I missing something obvious?

How would starreduce differ from normal reduction?

The difference between map and starmap is that the former consumes the
iterable arguments immediately and produces a list, the latter produces
an iterator (and lazily consumes its input as values are retrieved from
the iterator).

There is no similar distinction to be made in the case of reduce: the
only relevant semantic is to consume the entire iterable and produce the
final output value.

If you're suggesting a new kind of iterator that provides a progress
report at each step through the source iterable, that isn't really
reduction anymore, so the starreduce name would be misleading.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Fri Feb 12 16:45:16 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 13 Feb 2010 01:45:16 +1000
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <87bpfuwj9l.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>	<4B72BED9.6050404@gmx.net>	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>	<20100211145724.GA2894@phd.pp.ru>	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>	<5d1a32001002120550w4e37d700g93c29524ccf67d32@mail.gmail.com>
	<87bpfuwj9l.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4B75778C.8040808@gmail.com>

Stephen J. Turnbull wrote:
> This is covered in the manuals, but the gist is that every Python
> object knows how many other objects are pointing to it (called a
> refcount).  When an object's refcount drops to zero, it gets collected
> (immediately, IIRC).

This description applies for CPython (the one from python.org), since
that uses refcounting with cyclic garbage collection. Other Python
implementations work differently (e.g. Jython and IronPython rely on the
garbage collector in their underlying VMs)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From gerald.britton at gmail.com  Fri Feb 12 17:32:56 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 12 Feb 2010 11:32:56 -0500
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <4B75778C.8040808@gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<5d1a32001002120550w4e37d700g93c29524ccf67d32@mail.gmail.com>
	<87bpfuwj9l.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4B75778C.8040808@gmail.com>
Message-ID: <5d1a32001002120832o40490b8dpf14833f9c90bb234@mail.gmail.com>

Thanks everyone.  Good to know!

On Fri, Feb 12, 2010 at 10:45 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Stephen J. Turnbull wrote:
>> This is covered in the manuals, but the gist is that every Python
>> object knows how many other objects are pointing to it (called a
>> refcount). ?When an object's refcount drops to zero, it gets collected
>> (immediately, IIRC).
>
> This description applies for CPython (the one from python.org), since
> that uses refcounting with cyclic garbage collection. Other Python
> implementations work differently (e.g. Jython and IronPython rely on the
> garbage collector in their underlying VMs)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
>



-- 
Gerald Britton


From algorias at gmail.com  Fri Feb 12 17:54:34 2010
From: algorias at gmail.com (Vitor Bosshard)
Date: Fri, 12 Feb 2010 12:54:34 -0400
Subject: [Python-ideas] itertools has starmap but no starreduce?
In-Reply-To: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com>
References: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com>
Message-ID: <2987c46d1002120854r3f3c3ar581c26ad9de92695@mail.gmail.com>

2010/2/12 Andrey Fedorov <anfedorov at gmail.com>:
> Is that right, or am I missing something obvious?


The semantics are non-obvious, for one. I assume you mean that the
following lines would be equivalent:

starreduce(f, iterable, initial)
reduce(lambda x,y: f(x, *y), iterable, initial)


This is useful for the types of reduction where the result of the
function is one type, and the iterable is a sequence of args to that
function. For example:

reduce(lambda x,y: str.replace(x, *y), (('a', 'aa'), ('b', 'bb',)), some_string)

could be written as

starreduce(str.replace, (('a', 'aa'), ('b', 'bb',)), some_string)

It seems like a very narrow use-case, unless you mean something else.

Vitor


From anfedorov at gmail.com  Fri Feb 12 19:59:08 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Fri, 12 Feb 2010 13:59:08 -0500
Subject: [Python-ideas] itertools has starmap but no starreduce?
In-Reply-To: <2987c46d1002120854r3f3c3ar581c26ad9de92695@mail.gmail.com>
References: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com> 
	<2987c46d1002120854r3f3c3ar581c26ad9de92695@mail.gmail.com>
Message-ID: <7659cab31002121059r4bcfe11fnd812a826e3ca3c84@mail.gmail.com>

>
> On Fri, Feb 12, 2010 at 11:54 AM, Vitor Bosshard <algorias at gmail.com>
>  wrote:

I assume you mean that the following lines would be equivalent:


> starreduce(f, iterable, initial)

reduce(lambda x,y: f(x, *y), iterable, initial)


Precisely.

It seems like a very narrow use-case [...]


Agreed, that example is. I also use it for high-level functional stuff -
something like Haskell's monads or Java's "chain-of-responsibility". Like:

    output = reduce(lambda x, f: f(x), chain_of_processors, input)

So, given a list [f,g,h], this would be h(g(f(input))). The star-case is
when input is a n-touple, and each function works on `n' arguments. But now
that I wrote it out without a for loop, it's obviously trivial to add the
star in the right place. My mistake! Thanks for helping me catch it :)

Cheers,
Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100212/317ac645/attachment.html>

From anfedorov at gmail.com  Fri Feb 12 19:59:45 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Fri, 12 Feb 2010 13:59:45 -0500
Subject: [Python-ideas] itertools has starmap but no starreduce?
In-Reply-To: <7659cab31002121059r4bcfe11fnd812a826e3ca3c84@mail.gmail.com>
References: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com> 
	<2987c46d1002120854r3f3c3ar581c26ad9de92695@mail.gmail.com> 
	<7659cab31002121059r4bcfe11fnd812a826e3ca3c84@mail.gmail.com>
Message-ID: <7659cab31002121059o40f61ed4h622a11a8e0def08c@mail.gmail.com>

Aside: there is no built-in for "lambda x, f: f(x)" somewhere, is there?

On Fri, Feb 12, 2010 at 1:59 PM, Andrey Fedorov <anfedorov at gmail.com> wrote:

> On Fri, Feb 12, 2010 at 11:54 AM, Vitor Bosshard <algorias at gmail.com>
>>  wrote:
>
> I assume you mean that the following lines would be equivalent:
>
>
>> starreduce(f, iterable, initial)
>
> reduce(lambda x,y: f(x, *y), iterable, initial)
>
>
> Precisely.
>
> It seems like a very narrow use-case [...]
>
>
> Agreed, that example is. I also use it for high-level functional stuff -
> something like Haskell's monads or Java's "chain-of-responsibility". Like:
>
>     output = reduce(lambda x, f: f(x), chain_of_processors, input)
>
> So, given a list [f,g,h], this would be h(g(f(input))). The star-case is
> when input is a n-touple, and each function works on `n' arguments. But now
> that I wrote it out without a for loop, it's obviously trivial to add the
> star in the right place. My mistake! Thanks for helping me catch it :)
>
> Cheers,
> Andrey
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100212/3d3576a3/attachment.html>

From phd at phd.pp.ru  Fri Feb 12 20:19:40 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Fri, 12 Feb 2010 22:19:40 +0300
Subject: [Python-ideas] itertools has starmap but no starreduce?
In-Reply-To: <7659cab31002121059o40f61ed4h622a11a8e0def08c@mail.gmail.com>
References: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com>
	<2987c46d1002120854r3f3c3ar581c26ad9de92695@mail.gmail.com>
	<7659cab31002121059r4bcfe11fnd812a826e3ca3c84@mail.gmail.com>
	<7659cab31002121059o40f61ed4h622a11a8e0def08c@mail.gmail.com>
Message-ID: <20100212191940.GC26135@phd.pp.ru>

On Fri, Feb 12, 2010 at 01:59:45PM -0500, Andrey Fedorov wrote:
> Aside: there is no built-in for "lambda x, f: f(x)" somewhere, is there?

   apply(f, x) but why do you need it at all? Just call f(x).

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From anfedorov at gmail.com  Fri Feb 12 20:33:52 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Fri, 12 Feb 2010 14:33:52 -0500
Subject: [Python-ideas] itertools has starmap but no starreduce?
In-Reply-To: <20100212191940.GC26135@phd.pp.ru>
References: <7659cab31002120729y1198f25dscfc9e55153057cc@mail.gmail.com> 
	<2987c46d1002120854r3f3c3ar581c26ad9de92695@mail.gmail.com> 
	<7659cab31002121059r4bcfe11fnd812a826e3ca3c84@mail.gmail.com> 
	<7659cab31002121059o40f61ed4h622a11a8e0def08c@mail.gmail.com> 
	<20100212191940.GC26135@phd.pp.ru>
Message-ID: <7659cab31002121133m7bb6d399s8d208bdedfc95983@mail.gmail.com>

hah, I suppose I could do this with partial(apply, args=x), but that doesn't
seem like the best idea.

- Andrey

On Fri, Feb 12, 2010 at 2:19 PM, Oleg Broytman <phd at phd.pp.ru> wrote:

> On Fri, Feb 12, 2010 at 01:59:45PM -0500, Andrey Fedorov wrote:
> > Aside: there is no built-in for "lambda x, f: f(x)" somewhere, is there?
>
>    apply(f, x) but why do you need it at all? Just call f(x).
>
> Oleg.
> --
>     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
>           Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100212/ae513739/attachment.html>

From greg.ewing at canterbury.ac.nz  Fri Feb 12 23:25:21 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 13 Feb 2010 11:25:21 +1300
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <4B752631.5050800@gmail.com>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>
	<4B752631.5050800@gmail.com>
Message-ID: <4B75D551.3080903@canterbury.ac.nz>

Nick Coghlan wrote:

> I believe it runs afoul of the moratorium, but a getitem() builtin might
> be a better idea (since it would then work for any class that implements
> __getitem__).

Not quite the same way, though. The dict get() method knows
about the internals of the object, so it can work very
efficiently and without danger of masking bugs by catching
the wrong exception. A generic getitem() wouldn't be able
to do that.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Feb 12 23:33:33 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 13 Feb 2010 11:33:33 +1300
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <3547E294-6673-450F-AAD1-6CF7FB54B24E@masklinn.net>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>
	<4B752631.5050800@gmail.com>
	<3547E294-6673-450F-AAD1-6CF7FB54B24E@masklinn.net>
Message-ID: <4B75D73D.4020008@canterbury.ac.nz>

Masklinn wrote:

> Though I'm not sure it's a very good idea for lists. Semantically, lists
 > are to be iterated, not really to be indexed.

I don't think I agree that lists are meant primarily for
iteration. Indexing them is a perfectly legitimate and
useful thing to do.

However, I would agree that a list.get() operation with a
default seems to be a rather rare requirement. Usually
when you index a list, the index is generated by some
algorithm that guarantees it's within range.

I can't remember ever wanting a list.get() myself, and
if I ever did, I would be quite happy to write my own.

-- 
Greg


From raymond.hettinger at gmail.com  Fri Feb 12 23:35:25 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Fri, 12 Feb 2010 14:35:25 -0800
Subject: [Python-ideas] clear() method for lists
In-Reply-To: <4B75D73D.4020008@canterbury.ac.nz>
References: <tencent_6D6F4B5376EA9E296EEFD7CE@qq.com>
	<4B72BED9.6050404@gmx.net>
	<5d1a32001002100654n530b9e4fx482dcd0574d834c@mail.gmail.com>
	<8c7f10c61002110008o7b86eba4q70d972ce7436dedc@mail.gmail.com>
	<5d1a32001002110651y10b11ba0qde1fdb80f24e356b@mail.gmail.com>
	<20100211145724.GA2894@phd.pp.ru>
	<3b5110851002110721r2a6372bv29b296d3e798a20b@mail.gmail.com>
	<9613db601002110738h5e2eb5as73fee399b8eba8ac@mail.gmail.com>
	<5d1a32001002110935n143281bm38bfaa19547061ee@mail.gmail.com>
	<87sk97w4ds.fsf@uwakimon.sk.tsukuba.ac.jp>
	<91ad5bf81002112326m4f0c9f74i95b524a81ba3379@mail.gmail.com>
	<4B752631.5050800@gmail.com>
	<3547E294-6673-450F-AAD1-6CF7FB54B24E@masklinn.net>
	<4B75D73D.4020008@canterbury.ac.nz>
Message-ID: <C7B34BA7-F8CC-4D70-80C9-1CB38A67F6C9@gmail.com>


On Feb 12, 2010, at 2:33 PM, Greg Ewing wrote:

> Masklinn wrote:
> 
>> Though I'm not sure it's a very good idea for lists. Semantically, lists
> > are to be iterated, not really to be indexed.
> 
> I don't think I agree that lists are meant primarily for
> iteration. Indexing them is a perfectly legitimate and
> useful thing to do.
> 
> However, I would agree that a list.get() operation with a
> default seems to be a rather rare requirement. Usually
> when you index a list, the index is generated by some
> algorithm that guarantees it's within range.
> 
> I can't remember ever wanting a list.get() myself, and
> if I ever did, I would be quite happy to write my own.

I concur. Put me down for a -1.

Also, I think this idea was discussed once or twice here
before and it got shot down then too. 


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100212/4c9eac4c/attachment.html>

From greg at krypto.org  Mon Feb 15 21:09:07 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 15 Feb 2010 12:09:07 -0800
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <43aa6ff71002021335mf8313fbgab8bc92c3b204251@mail.gmail.com>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org> 
	<bbaeab101002021249j2faf21eo48465a30084f1d3@mail.gmail.com> 
	<43aa6ff71002021335mf8313fbgab8bc92c3b204251@mail.gmail.com>
Message-ID: <52dc1c821002151209n65426a32u3ea4c9714a64a916@mail.gmail.com>

On Tue, Feb 2, 2010 at 1:35 PM, Collin Winter <collinw at gmail.com> wrote:

> On Tue, Feb 2, 2010 at 12:49 PM, Brett Cannon <brett at python.org> wrote:
> > On Tue, Feb 2, 2010 at 07:16, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >> Larry Hastings <larry at ...> writes:
> >>>
> >>> I ask you: why gunk up the filesystem with two files when one would do?
> >>> I propose we change the pyc file so it can contain multiple code
> >>> objects.
> >>
> >> I think we should dump the lie about "optimized" bytecode when the only
> >> optimization is that we strip some docstrings, disable asserts and set
> __debug__
> >> to False.
> >
> > I think the hope has always been that the peepholer would be extended
> > to do some tweaks that would only be reasonable under a -O flag.
> > Obviously this has not happened and who knows if it ever will.
>
> Unladen Swallow has a number of optimizations in mind that tweak
> corner cases of Python semantics, which we'd like to hide behind a
> compiler flag so that you have to explicitly ask for them. We haven't
> yet implemented these optimizations, since they will likely be
> controversial and require discussion. Feel free to remove the -O flag
> in the meantime; it can be added back later.
>
> Collin Winter


Anything new will need its own flag to enable/disable anyways (if it is
insufficient to leave it to being done at runtime on a per module basis via
a sys.xxx() call) as people already rely on today's exact behavior of -O.
 We should never equate disabling assert statements (a change to the actual
program logic) with actual optimization.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100215/0375004a/attachment.html>

From zac256 at gmail.com  Mon Feb 15 23:10:36 2010
From: zac256 at gmail.com (Zac Burns)
Date: Mon, 15 Feb 2010 14:10:36 -0800
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <52dc1c821002151209n65426a32u3ea4c9714a64a916@mail.gmail.com>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org>
	<bbaeab101002021249j2faf21eo48465a30084f1d3@mail.gmail.com>
	<43aa6ff71002021335mf8313fbgab8bc92c3b204251@mail.gmail.com>
	<52dc1c821002151209n65426a32u3ea4c9714a64a916@mail.gmail.com>
Message-ID: <333edbe81002151410s39021d89wf7e2e6ed17b3c7e5@mail.gmail.com>

We use -O where I work in performance critical places. It also better than
halves the startup time, which some users are sensitive to.

--
Zachary Burns
(407)590-4814
Aim - Zac256FL
Production Engineer (Digital Overlord)
Zindagi Games
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100215/6034fd82/attachment.html>

From zac256 at gmail.com  Mon Feb 15 23:55:08 2010
From: zac256 at gmail.com (Zac Burns)
Date: Mon, 15 Feb 2010 14:55:08 -0800
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <333edbe81002151410s39021d89wf7e2e6ed17b3c7e5@mail.gmail.com>
References: <4B67A4FC.2090404@hastings.org>
	<loom.20100202T161113-710@post.gmane.org>
	<bbaeab101002021249j2faf21eo48465a30084f1d3@mail.gmail.com>
	<43aa6ff71002021335mf8313fbgab8bc92c3b204251@mail.gmail.com>
	<52dc1c821002151209n65426a32u3ea4c9714a64a916@mail.gmail.com>
	<333edbe81002151410s39021d89wf7e2e6ed17b3c7e5@mail.gmail.com>
Message-ID: <333edbe81002151455q7c3ded5dife299dcf4ee8216c@mail.gmail.com>

On Mon, Feb 15, 2010 at 2:10 PM, Zac Burns <zac256 at gmail.com> wrote:

> We use -O where I work in performance critical places. It also better than
> halves the startup time, which some users are sensitive to.
>
>
> --
> Zachary Burns
> (407)590-4814
> Aim - Zac256FL
> Production Engineer (Digital Overlord)
> Zindagi Games
>
>
On second thought, what I just said about halving the startup time isn't
true - it's probably something else that we are doing when also applying -O.
Sorry about that. It does contribute to better startup time though... just
not half.
--
Zachary Burns
(407)590-4814
Aim - Zac256FL
Production Engineer (Digital Overlord)
Zindagi Games
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100215/3e471a21/attachment.html>

From mal at egenix.com  Tue Feb 16 00:08:59 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 16 Feb 2010 00:08:59 +0100
Subject: [Python-ideas] Dump .pyo and the "optimize" flag
In-Reply-To: <333edbe81002151455q7c3ded5dife299dcf4ee8216c@mail.gmail.com>
References: <4B67A4FC.2090404@hastings.org>	<loom.20100202T161113-710@post.gmane.org>	<bbaeab101002021249j2faf21eo48465a30084f1d3@mail.gmail.com>	<43aa6ff71002021335mf8313fbgab8bc92c3b204251@mail.gmail.com>	<52dc1c821002151209n65426a32u3ea4c9714a64a916@mail.gmail.com>	<333edbe81002151410s39021d89wf7e2e6ed17b3c7e5@mail.gmail.com>
	<333edbe81002151455q7c3ded5dife299dcf4ee8216c@mail.gmail.com>
Message-ID: <4B79D40B.4080203@egenix.com>

I wonder what people have again the -O flag.

You are probably missing an important use case: when writing
larger applications you typically add lots of debug sections
to the code.

Using the -O flag, this code doesn't only get disabled,
it gets removed and that can lead to a serious optimization,
hence the name of the flag.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 15 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From anfedorov at gmail.com  Fri Feb 19 17:12:11 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Fri, 19 Feb 2010 11:12:11 -0500
Subject: [Python-ideas] Merging a generator's "next" and "send" methods?
Message-ID: <7659cab31002190812l5f6b2777u9a4807b6e3f19b0e@mail.gmail.com>

Since .next() seems to be equivalent to .send(None), why wasn't .next() just
given an optional parameter which defaults to None?

- Andrey

On Thu, Oct 1, 2009 at 1:10 PM, Simon Forman <sajmikins at gmail.com> wrote:

> On Wed, Sep 30, 2009 at 4:24 PM, Andrey Fedorov <anfedorov at gmail.com>
> wrote:
> > As far as I can tell, a generator's .next() is equivalent to .send(None).
> Is
> > this true?
>
> They are equivalent AFAIK.
>
> > If so, [why] aren't they unified in a method with a single argument which
> defaults
> > to None?
> > - Andrey
>
> next() predates send().
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100219/3d732603/attachment.html>

From guido at python.org  Fri Feb 19 17:28:06 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 19 Feb 2010 11:28:06 -0500
Subject: [Python-ideas] Merging a generator's "next" and "send" methods?
In-Reply-To: <7659cab31002190812l5f6b2777u9a4807b6e3f19b0e@mail.gmail.com>
References: <7659cab31002190812l5f6b2777u9a4807b6e3f19b0e@mail.gmail.com>
Message-ID: <ca471dc21002190828n68ab579o24e8d5f9f24b7526@mail.gmail.com>

On Fri, Feb 19, 2010 at 11:12 AM, Andrey Fedorov <anfedorov at gmail.com> wrote:
> Since .next() seems to be equivalent to .send(None), why wasn't .next() just
> given an optional parameter which defaults to None?

Because for built-in iterator types the next() method is implemented
in C and we didn't want to invalidate existing C code by giving it an
optional argument.

But why do you care?

--Guido

> - Andrey
> On Thu, Oct 1, 2009 at 1:10 PM, Simon Forman <sajmikins at gmail.com> wrote:
>>
>> On Wed, Sep 30, 2009 at 4:24 PM, Andrey Fedorov <anfedorov at gmail.com>
>> wrote:
>> > As far as I can tell, a generator's .next() is equivalent to
>> > .send(None). Is
>> > this true?
>>
>> They are equivalent AFAIK.
>>
>> > If so, [why] aren't they unified in a method with a single argument
>> > which defaults
>> > to None?
>> > - Andrey
>>
>> next() predates send().
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
--Guido van Rossum (python.org/~guido)


From masklinn at masklinn.net  Fri Feb 19 17:32:59 2010
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 19 Feb 2010 17:32:59 +0100
Subject: [Python-ideas] Merging a generator's "next" and "send" methods?
In-Reply-To: <7659cab31002190812l5f6b2777u9a4807b6e3f19b0e@mail.gmail.com>
References: <7659cab31002190812l5f6b2777u9a4807b6e3f19b0e@mail.gmail.com>
Message-ID: <B66142B5-BB63-48AA-A2FD-A93C65196312@masklinn.net>

On 19 Feb 2010, at 17:12 , Andrey Fedorov wrote:
> 
> Since .next() seems to be equivalent to .send(None), why wasn't .next() just
> given an optional parameter which defaults to None?
> 
> - Andrey
You'd have to check the ML discussions on PEP 342 (Coroutines via Enhanced Generators). The PEP explicitly states that send(None) is equivalent to next()

>   A new method for generator-iterators is proposed, called send().  It
    takes exactly one argument, which is the value that should be "sent
    in" to the generator.  Calling send(None) is exactly equivalent to
    calling a generator's next() method.  Calling send() with any other
    value is the same, except that the value produced by the generator's
    current yield expression will be different.

A new method was probably created because it would be quite unclear that `next(value)` sends that value into the generator for consumption by a yield expression. Whereas the purpose of `send(value)` looks pretty obvious.

From anfedorov at gmail.com  Fri Feb 19 20:38:59 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Fri, 19 Feb 2010 14:38:59 -0500
Subject: [Python-ideas] Merging a generator's "next" and "send" methods?
In-Reply-To: <B66142B5-BB63-48AA-A2FD-A93C65196312@masklinn.net>
References: <7659cab31002190812l5f6b2777u9a4807b6e3f19b0e@mail.gmail.com> 
	<B66142B5-BB63-48AA-A2FD-A93C65196312@masklinn.net>
Message-ID: <7659cab31002191138r387a33d2j5bddb966e889864c@mail.gmail.com>

Guido van Rossum <guido at python.org> wrote:

> Because for built-in iterator types the next() method is implemented in C
> and we didn't want to invalidate existing C code by giving it an optional
> argument.


Ah, makes sense.

But why do you care?


Curiosity more than anything.

It would be elegant if .send() had identical behavior to .next() without an
argument, instead of throwing an exception, but it's no biggie.

Cheers,
Andrey

Masklinn <masklinn at masklinn.net> wrote:

> You'd have to check the ML discussions on PEP 342 (Coroutines via Enhanced
> Generators).


Sweet, thanks for the pointer. Sorry for not searching harder.

On Fri, Feb 19, 2010 at 11:32 AM, Masklinn <masklinn at masklinn.net> wrote:

> On 19 Feb 2010, at 17:12 , Andrey Fedorov wrote:
> >
> > Since .next() seems to be equivalent to .send(None), why wasn't .next()
> just
> > given an optional parameter which defaults to None?
> >
> > - Andrey
> You'd have to check the ML discussions on PEP 342 (Coroutines via Enhanced
> Generators). The PEP explicitly states that send(None) is equivalent to
> next()
>
> >   A new method for generator-iterators is proposed, called send().  It
>    takes exactly one argument, which is the value that should be "sent
>    in" to the generator.  Calling send(None) is exactly equivalent to
>    calling a generator's next() method.  Calling send() with any other
>    value is the same, except that the value produced by the generator's
>    current yield expression will be different.
>
> A new method was probably created because it would be quite unclear that
> `next(value)` sends that value into the generator for consumption by a yield
> expression. Whereas the purpose of `send(value)` looks pretty obvious.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100219/f495d05f/attachment.html>

From masklinn at masklinn.net  Fri Feb 19 21:28:18 2010
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 19 Feb 2010 21:28:18 +0100
Subject: [Python-ideas] warnings package might provide an Option or a
	pre-filled OptionParser
Message-ID: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>

Currently, `warnings` uses the `-W` interpreter-level option for non-programmatic configuration.

While nice, this option isn't available out of the box to scripts where the interpreter is implicit (./script.py, or `script` where it's on $PATH). I believe it would be nice if `warnings` provided something one could simply drop into `optparse` to automagically get -W handling, without having to define custom option actions every time.

What do you think?

From solipsis at pitrou.net  Fri Feb 19 21:41:12 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 19 Feb 2010 20:41:12 +0000 (UTC)
Subject: [Python-ideas] warnings package might provide an Option or
	a	pre-filled OptionParser
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
Message-ID: <hlmt18$ke5$1@ger.gmane.org>

Le Fri, 19 Feb 2010 21:28:18 +0100, Masklinn a ?crit?:
> 
> While nice, this option isn't available out of the box to scripts where
> the interpreter is implicit (./script.py, or `script` where it's on
> $PATH). I believe it would be nice if `warnings` provided something one
> could simply drop into `optparse` to automagically get -W handling,
> without having to define custom option actions every time.

It makes sense. Can you propose a patch on the tracker?



From masklinn at masklinn.net  Fri Feb 19 22:16:18 2010
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 19 Feb 2010 22:16:18 +0100
Subject: [Python-ideas] warnings package might provide an Option or
	a	pre-filled OptionParser
In-Reply-To: <hlmt18$ke5$1@ger.gmane.org>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<hlmt18$ke5$1@ger.gmane.org>
Message-ID: <710EC0BE-5C7F-4EE2-BCF1-7F0C4A904570@masklinn.net>

On 19 Feb 2010, at 21:41 , Antoine Pitrou wrote:
> 
> Le Fri, 19 Feb 2010 21:28:18 +0100, Masklinn a ?crit :
>> 
>> While nice, this option isn't available out of the box to scripts where
>> the interpreter is implicit (./script.py, or `script` where it's on
>> $PATH). I believe it would be nice if `warnings` provided something one
>> could simply drop into `optparse` to automagically get -W handling,
>> without having to define custom option actions every time.
> 
> It makes sense. Can you propose a patch on the tracker?
I can try. Should it be for Python 3 trunk?

From solipsis at pitrou.net  Fri Feb 19 22:37:25 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 19 Feb 2010 21:37:25 +0000 (UTC)
Subject: [Python-ideas] warnings package might provide an Option
	or	a	pre-filled OptionParser
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<hlmt18$ke5$1@ger.gmane.org>
	<710EC0BE-5C7F-4EE2-BCF1-7F0C4A904570@masklinn.net>
Message-ID: <hln0al$q1e$1@ger.gmane.org>

Le Fri, 19 Feb 2010 22:16:18 +0100, Masklinn a ?crit?:
>>> 
>>> While nice, this option isn't available out of the box to scripts
>>> where the interpreter is implicit (./script.py, or `script` where it's
>>> on $PATH). I believe it would be nice if `warnings` provided something
>>> one could simply drop into `optparse` to automagically get -W
>>> handling, without having to define custom option actions every time.
>> 
>> It makes sense. Can you propose a patch on the tracker?
> I can try. Should it be for Python 3 trunk?

Either 2.x trunk (the SVN trunk) or 3.x trunk (the "py3k" branch), as you 
prefer.



From greg.ewing at canterbury.ac.nz  Sat Feb 20 23:19:26 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 21 Feb 2010 11:19:26 +1300
Subject: [Python-ideas] warnings package might provide an Option or a
 pre-filled OptionParser
In-Reply-To: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
Message-ID: <4B805FEE.1030900@canterbury.ac.nz>

Masklinn wrote:
> I believe it would be nice if `warnings` provided something
> one could simply drop into `optparse` to automagically get 
> -W handling

Wouldn't this be better provided as a feature of the
optparse module? Otherwise you'd be making warnings
dependent on optparse, or at least having to know
about it, which seems like the wrong direction for
the dependency to go in.

-- 
Greg


From masklinn at masklinn.net  Sun Feb 21 15:16:52 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 21 Feb 2010 15:16:52 +0100
Subject: [Python-ideas] warnings package might provide an Option or a
	pre-filled OptionParser
In-Reply-To: <4B805FEE.1030900@canterbury.ac.nz>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<4B805FEE.1030900@canterbury.ac.nz>
Message-ID: <81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>

On 20 Feb 2010, at 23:19 , Greg Ewing wrote:
> 
> Masklinn wrote:
>> I believe it would be nice if `warnings` provided something
>> one could simply drop into `optparse` to automagically get -W handling
> 
> Wouldn't this be better provided as a feature of the
> optparse module? Otherwise you'd be making warnings
> dependent on optparse, or at least having to know
> about it, which seems like the wrong direction for
> the dependency to go in.

I don't know, I'm not sure it makes much sense for optparse to start providing pre-built options for random stuff in the stdlib does it? I feel it'd be more sensible to have each module provide its own set of pre-built Options (if it applies), plus I also believe it's more sensible documentation-wise (someone reading on how warnings works would be more likely to find the Options documentation in warnings than go to optparse and get it from there).

Plus ? at least in the case of warnings ? the option would definitely make calls to warnings's "private" functions, I believe making sections of optparse depend on implementation details of other stdlib modules would be much worse than making stdlib modules optionally depend on optparse.

Finally, considering the stdlib example as an example of how to do things, would it make more sense for third-party systems/libraries to provide pre-built Options or for them to try to get their options merged into optparse?

I believe the former, though I might be wrong.

From ctb at msu.edu  Sun Feb 21 15:28:33 2010
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 21 Feb 2010 06:28:33 -0800
Subject: [Python-ideas] warnings package might provide an Option or
	a	pre-filled OptionParser
In-Reply-To: <81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<4B805FEE.1030900@canterbury.ac.nz>
	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
Message-ID: <20100221142833.GA14149@idyll.org>

On Sun, Feb 21, 2010 at 03:16:52PM +0100, Masklinn wrote:
> On 20 Feb 2010, at 23:19 , Greg Ewing wrote:
> > 
> > Masklinn wrote:
> >> I believe it would be nice if `warnings` provided something
> >> one could simply drop into `optparse` to automagically get -W handling
> > 
> > Wouldn't this be better provided as a feature of the
> > optparse module? Otherwise you'd be making warnings
> > dependent on optparse, or at least having to know
> > about it, which seems like the wrong direction for
> > the dependency to go in.
> 
> I don't know, I'm not sure it makes much sense for optparse to start providing pre-built options for random stuff in the stdlib does it? I feel it'd be more sensible to have each module provide its own set of pre-built Options (if it applies), plus I also believe it's more sensible documentation-wise (someone reading on how warnings works would be more likely to find the Options documentation in warnings than go to optparse and get it from there).

Isn't this conflating two things that don't belong together, though?

warnings is a Python developer issue.

optparse provides command line arguments for users to make use of.

Generally, when running things from the command line, I shouldn't care about
what language (or language version, or internal details of that language)
I'm using.

--titus
-- 
C. Titus Brown, ctb at msu.edu


From masklinn at masklinn.net  Sun Feb 21 15:39:42 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 21 Feb 2010 15:39:42 +0100
Subject: [Python-ideas] warnings package might provide an Option or a
	pre-filled OptionParser
In-Reply-To: <20100221142833.GA14149@idyll.org>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<4B805FEE.1030900@canterbury.ac.nz>
	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
	<20100221142833.GA14149@idyll.org>
Message-ID: <2CD33742-77F8-4E30-B0D4-E7BF15DF4AFA@masklinn.net>

On 21 Feb 2010, at 15:28 , C. Titus Brown wrote:
> 
> warnings is a Python developer issue.
> 
> optparse provides command line arguments for users to make use of.
> 
> Generally, when running things from the command line, I shouldn't care about
> what language (or language version, or internal details of that language)
> I'm using.
There are cases where the user is a python developers and might want to manipulate warnings e.g. a web framework's development server or a pluggable tool where one is developing new plugins within the tool, in both cases the startup script is often called directly, not using an explicit python call. Though I didn't quite remember to describe those use cases when I made the initial proposal. Providing the option or not would be left at to the tool creator's judgement.

From eric at trueblade.com  Sun Feb 21 15:28:33 2010
From: eric at trueblade.com (Eric Smith)
Date: Sun, 21 Feb 2010 09:28:33 -0500
Subject: [Python-ideas] warnings package might provide an Option or a
 pre-filled OptionParser
In-Reply-To: <81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>	<4B805FEE.1030900@canterbury.ac.nz>
	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
Message-ID: <4B814311.7060007@trueblade.com>

Masklinn wrote:
> On 20 Feb 2010, at 23:19 , Greg Ewing wrote:
>> Masklinn wrote:
>>> I believe it would be nice if `warnings` provided something
>>> one could simply drop into `optparse` to automagically get -W handling
>> Wouldn't this be better provided as a feature of the
>> optparse module? Otherwise you'd be making warnings
>> dependent on optparse, or at least having to know
>> about it, which seems like the wrong direction for
>> the dependency to go in.
> 
> I don't know, I'm not sure it makes much sense for optparse to start providing pre-built options for random stuff in the stdlib does it? I feel it'd be more sensible to have each module provide its own set of pre-built Options (if it applies), plus I also believe it's more sensible documentation-wise (someone reading on how warnings works would be more likely to find the Options documentation in warnings than go to optparse and get it from there).
> 
> Plus ? at least in the case of warnings ? the option would definitely make calls to warnings's "private" functions, I believe making sections of optparse depend on implementation details of other stdlib modules would be much worse than making stdlib modules optionally depend on optparse.
> 
> Finally, considering the stdlib example as an example of how to do things, would it make more sense for third-party systems/libraries to provide pre-built Options or for them to try to get their options merged into optparse?
> 
> I believe the former, though I might be wrong.

-1 on modifying either optparse or warnings. It's coupling things that 
don't need to be coupled.

Plus, we're adding argparse. Do we really want both argparse and 
optparse to be coupled to warnings?

Eric.


From ctb at msu.edu  Sun Feb 21 16:02:07 2010
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 21 Feb 2010 07:02:07 -0800
Subject: [Python-ideas] warnings package might provide an Option or
	a	pre-filled OptionParser
In-Reply-To: <2CD33742-77F8-4E30-B0D4-E7BF15DF4AFA@masklinn.net>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<4B805FEE.1030900@canterbury.ac.nz>
	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
	<20100221142833.GA14149@idyll.org>
	<2CD33742-77F8-4E30-B0D4-E7BF15DF4AFA@masklinn.net>
Message-ID: <20100221150207.GA5585@idyll.org>

On Sun, Feb 21, 2010 at 03:39:42PM +0100, Masklinn wrote:
> On 21 Feb 2010, at 15:28 , C. Titus Brown wrote:
> > warnings is a Python developer issue.
> > 
> > optparse provides command line arguments for users to make use of.
> > 
> > Generally, when running things from the command line, I shouldn't care about
> > what language (or language version, or internal details of that language)
> > I'm using.
>
> There are cases where the user is a python developers and might want to manipulate warnings e.g. a web framework's development server or a pluggable tool where one is developing new plugins within the tool, in both cases the startup script is often called directly, not using an explicit python call. Though I didn't quite remember to describe those use cases when I made the initial proposal. Providing the option or not would be left at to the tool creator's judgement.

OK, I wouldn't be planning on using it, but I can see that it might be useful
to some.

I like Greg's idea of putting it into optparse over warnings because the
dependency link is more appropriate; optparse is not related to language
behavior, like warnings is.  Plus there's been some recent movement on
the idea that optparse may be deprecated (silently or otherwise...) in
future Python releases.

Whatever you decide, Antoine has said "go for it" -- make up a patch
and see what happens ;).

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu


From masklinn at masklinn.net  Sun Feb 21 16:19:24 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 21 Feb 2010 16:19:24 +0100
Subject: [Python-ideas] warnings package might provide an Option or a
	pre-filled OptionParser
In-Reply-To: <20100221150207.GA5585@idyll.org>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<4B805FEE.1030900@canterbury.ac.nz>
	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
	<20100221142833.GA14149@idyll.org>
	<2CD33742-77F8-4E30-B0D4-E7BF15DF4AFA@masklinn.net>
	<20100221150207.GA5585@idyll.org>
Message-ID: <2B426BAB-AEF5-4408-BE1A-C8D94A2DE18D@masklinn.net>

On 21 Feb 2010, at 16:02 , C. Titus Brown wrote:
> 
> I like Greg's idea of putting it into optparse over warnings because the
> dependency link is more appropriate; optparse is not related to language
> behavior, like warnings is.
Yeah but as I said above, I'm not sure it's optparse's role to provide pre-built/convenience options for other stdlib modules, especially given the chance those option would hit into the modules's implementation details.

>  Plus there's been some recent movement on
> the idea that optparse may be deprecated (silently or otherwise...) in
> future Python releases.
Well I'm starting the impl on -trunk, and if optparse is deprecated in favor of argparse, I'm sure the same facilities can be built for argparse. It's not like the patch is gigantic.

> Whatever you decide, Antoine has said "go for it" -- make up a patch
> and see what happens ;).
Will do, thanks a lot.

From ctb at msu.edu  Sun Feb 21 17:59:20 2010
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 21 Feb 2010 08:59:20 -0800
Subject: [Python-ideas] warnings package might provide an Option or
	a	pre-filled OptionParser
In-Reply-To: <2B426BAB-AEF5-4408-BE1A-C8D94A2DE18D@masklinn.net>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<4B805FEE.1030900@canterbury.ac.nz>
	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
	<20100221142833.GA14149@idyll.org>
	<2CD33742-77F8-4E30-B0D4-E7BF15DF4AFA@masklinn.net>
	<20100221150207.GA5585@idyll.org>
	<2B426BAB-AEF5-4408-BE1A-C8D94A2DE18D@masklinn.net>
Message-ID: <20100221165920.GB24387@idyll.org>

On Sun, Feb 21, 2010 at 04:19:24PM +0100, Masklinn wrote:
> On 21 Feb 2010, at 16:02 , C. Titus Brown wrote:
> > I like Greg's idea of putting it into optparse over warnings because the
> > dependency link is more appropriate; optparse is not related to language
> > behavior, like warnings is.
>
> Yeah but as I said above, I'm not sure it's optparse's role to provide pre-built/convenience options for other stdlib modules, especially given the chance those option would hit into the modules's implementation details.

...and that's exactly why I'm not enthusiastic about your idea in the first
place.  I don't see it cleanly fitting in either location, which may mean the
functionality doesn't belong in the stdlib at all.

I'm new to python-ideas, and I'm not a python-dev committer, so this is all
just my opinion.  I'm going to guess that your best bet is to provide a patch
which will provide a basis for substantive discussion.

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu


From masklinn at masklinn.net  Sun Feb 21 19:04:20 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 21 Feb 2010 19:04:20 +0100
Subject: [Python-ideas] warnings package might provide an Option or a
	pre-filled OptionParser
In-Reply-To: <20100221165920.GB24387@idyll.org>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>
	<4B805FEE.1030900@canterbury.ac.nz>
	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
	<20100221142833.GA14149@idyll.org>
	<2CD33742-77F8-4E30-B0D4-E7BF15DF4AFA@masklinn.net>
	<20100221150207.GA5585@idyll.org>
	<2B426BAB-AEF5-4408-BE1A-C8D94A2DE18D@masklinn.net>
	<20100221165920.GB24387@idyll.org>
Message-ID: <13D4C08F-482A-4C91-9EBD-156D1CB87299@masklinn.net>

On 21 Feb 2010, at 17:59 , C. Titus Brown wrote:
> I'm new to python-ideas, and I'm not a python-dev committer
Well so and neither am I so? I'm completely open to cricicism from new-to-python-ideas-and-not-python-commiters :)

> I'm going to guess that your best bet is to provide a patch
> which will provide a basis for substantive discussion.
Which is now done with issue 7976.

From ncoghlan at gmail.com  Mon Feb 22 13:29:07 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 22 Feb 2010 22:29:07 +1000
Subject: [Python-ideas] warnings package might provide an Option or a
 pre-filled OptionParser
In-Reply-To: <4B814311.7060007@trueblade.com>
References: <0EFF1580-B1CA-475F-B87A-B4D94BF792EE@masklinn.net>	<4B805FEE.1030900@canterbury.ac.nz>	<81005944-E9E3-44B6-A79E-A2CE9251A81D@masklinn.net>
	<4B814311.7060007@trueblade.com>
Message-ID: <4B827893.8080007@gmail.com>

Eric Smith wrote:
> Plus, we're adding argparse. Do we really want both argparse and
> optparse to be coupled to warnings?

I agree with Titus - with the modules being peers rather than one being
obviously higher level than the other the dependency feels wrong no
matter which way it runs. And I'm a core developer that's been hanging
around on python-ideas for a long time* ;)

That said, the idea of creating a public API in the warnings module to
expose the functionality of warnings._processoptions in a clean fashion
seems like a reasonable idea that achieves the same goal without
coupling it to a particular command line parsing approach.

Cheers,
Nick.

* Just so nobody takes that comment the wrong way, keep in mind that
things like this don't actually matter all that much in most
python-ideas discussions - whether or not you're talking sense is
usually far more important. I just couldn't resist pointing it out after
Titus's comment :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From weasley_wx at qq.com  Wed Feb 24 07:23:29 2010
From: weasley_wx at qq.com (=?ISO-8859-1?B?d3h5YXJ2?=)
Date: Wed, 24 Feb 2010 14:23:29 +0800
Subject: [Python-ideas] op-assign in function argument
Message-ID: <tencent_2D525DD02706BB6B0552A42D@qq.com>

has anybody thought the op-assign in function argument is a good idea? now we can use parameter name to specifie argument, like this:
  
 def foo(arg1): pass
  
 # call
 foo(arg1 = "bar")
  
 but, if function has default parameter, it will be inconvenience:
 def foo_with_long_defualts(arg1 = we | are | default | flags | set): pass
  
 # call
 foo_with_long_defaults(arg1 = we | are | default | flags | set | another_flags)
  
 so, why not add op-assign support in named argument?
  
 foo_with_long_defaults(arg1 |= another_flags)
  
  
 are there anybody has the same idea with me?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100224/c7d5ee19/attachment.html>

From ironfroggy at gmail.com  Wed Feb 24 14:34:24 2010
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Wed, 24 Feb 2010 08:34:24 -0500
Subject: [Python-ideas] op-assign in function argument
In-Reply-To: <tencent_2D525DD02706BB6B0552A42D@qq.com>
References: <tencent_2D525DD02706BB6B0552A42D@qq.com>
Message-ID: <76fd5acf1002240534g1f7c0ce8xa7eba9fb7dd2a836@mail.gmail.com>

On Wed, Feb 24, 2010 at 1:23 AM, wxyarv <weasley_wx at qq.com> wrote:
> has anybody thought the op-assign in function argument is a good idea? now
> we can use parameter name to specifie argument, like this:
>
> def foo(arg1): pass
>
> # call
> foo(arg1 = "bar")
>
> but, if function has default parameter, it will be inconvenience:
> def foo_with_long_defualts(arg1 = we | are | default | flags | set): pass
>
> # call
> foo_with_long_defaults(arg1 = we | are | default | flags | set |
> another_flags)
>
> so, why not add op-assign support in named argument?
>
> foo_with_long_defaults(arg1 |= another_flags)

If you wan to do this, you can just support it in the function
yourself. That is, if the function wants to take an additional_flags
parameter, it should. I don't think the complications this could add
to the call syntax are good. It becomes a form of
assignment-as-an-expression and makes the call harder to parse (by
human eyes).

> are there anybody has the same idea with me?

-1

> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy


From ncoghlan at gmail.com  Wed Feb 24 14:57:44 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 24 Feb 2010 23:57:44 +1000
Subject: [Python-ideas] op-assign in function argument
In-Reply-To: <76fd5acf1002240534g1f7c0ce8xa7eba9fb7dd2a836@mail.gmail.com>
References: <tencent_2D525DD02706BB6B0552A42D@qq.com>
	<76fd5acf1002240534g1f7c0ce8xa7eba9fb7dd2a836@mail.gmail.com>
Message-ID: <4B853058.9000200@gmail.com>

Calvin Spealman wrote:
> On Wed, Feb 24, 2010 at 1:23 AM, wxyarv <weasley_wx at qq.com> wrote:
>> has anybody thought the op-assign in function argument is a good idea? now
>> we can use parameter name to specifie argument, like this:
>>
>> def foo(arg1): pass
>>
>> # call
>> foo(arg1 = "bar")
>>
>> but, if function has default parameter, it will be inconvenience:
>> def foo_with_long_defualts(arg1 = we | are | default | flags | set): pass
>>
>> # call
>> foo_with_long_defaults(arg1 = we | are | default | flags | set |
>> another_flags)
>>
>> so, why not add op-assign support in named argument?
>>
>> foo_with_long_defaults(arg1 |= another_flags)
> 
> If you wan to do this, you can just support it in the function
> yourself. That is, if the function wants to take an additional_flags
> parameter, it should. I don't think the complications this could add
> to the call syntax are good. It becomes a form of
> assignment-as-an-expression and makes the call harder to parse (by
> human eyes).

Even better is to just give the default args a name:

FOO_DEFAULTS = we | are | default | flags | set
def foo(arg=FOO_DEFAULTS):
  ...

my_flags = FOO_DEFAULTS | other_flags
foo(arg=my_flags)

(Also -1 on this syntax idea - simply giving the default value a name is
a much simpler and more obvious approach)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From denis.spir at gmail.com  Thu Feb 25 14:20:28 2010
From: denis.spir at gmail.com (spir)
Date: Thu, 25 Feb 2010 14:20:28 +0100
Subject: [Python-ideas] filter-only list comps
Message-ID: <20100225142028.53882a92@o>

Hello,

I recently realized the following, and wonder whether there is a reason for this, which I am unable to figure out.

A list comp can both map and filter:
	[x*x for x in numbers if x%2==1]
When it only maps, we can simply get rid of the filter part:
	[x*x for x in numbers]
Meaning we are not forced to write:
	[x*x for x in numbers if True]
But the analog does not apply to filter-only list comps:
	[x in numbers if x%2==1]	# SyntaxError

Why? I find this syntax clear, actually clearer than
	[x for x in numbers if x%2==1]
that confusingly repeats the item.
Would there be any parsing issue if we let down "<expression> for" when it does nothing?

Side note: I find the core part of a comprehension be "<item> in <collection>":
	[item in collection]
would be equivalent to list(collection).
	(item in collection)
would just build an iterator on collection.
From this POV, this core can be preceded by a mapping expression and/or followed by a filtering condition.


Denis
-- 
________________________________

la vita e estrany

spir.wikidot.com



From ncoghlan at gmail.com  Thu Feb 25 14:43:25 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 25 Feb 2010 23:43:25 +1000
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <20100225142028.53882a92@o>
References: <20100225142028.53882a92@o>
Message-ID: <4B867E7D.4060101@gmail.com>

spir wrote:
> numbers if x%2==1] that confusingly repeats the item. Would there be
> any parsing issue if we let down "<expression> for" when it does
> nothing?

Yes, there's a parsing problem:

"x in numbers" is a containment test

"x in numbers if cond else whatever" is a containment test combined with
a conditional expression

"x for x in numbers" and "x for x in numbers if cond" are
genexps/comprehensions (depending on the kind of brackets you put around
them, if any)

"x in number if cond" is none of the above, but the parser can't tell it
isn't meant to be the second one and hence will choke on the missing "else":

>>> 1 in [] if True
  File "<stdin>", line 1
    1 in [] if True
                  ^
SyntaxError: invalid syntax
>>> 1 in [] if True else None
False


Cheers,
Nick.
-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Thu Feb 25 14:45:13 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 25 Feb 2010 23:45:13 +1000
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <4B867E7D.4060101@gmail.com>
References: <20100225142028.53882a92@o> <4B867E7D.4060101@gmail.com>
Message-ID: <4B867EE9.7060403@gmail.com>

Nick Coghlan wrote:
> spir wrote:
>> numbers if x%2==1] that confusingly repeats the item. Would there be
>> any parsing issue if we let down "<expression> for" when it does
>> nothing?
> 
> Yes, there's a parsing problem:

Even aside from the interpreter parsing problem, there's actually a more
important human parsing problem. Without a filter call to imply
iteration, we want to retain the "for" keyword so human readers can
easily tell there is a loop involved in the construct.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From fdrake at acm.org  Thu Feb 25 15:20:56 2010
From: fdrake at acm.org (Fred Drake)
Date: Thu, 25 Feb 2010 09:20:56 -0500
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <4B867EE9.7060403@gmail.com>
References: <20100225142028.53882a92@o> <4B867E7D.4060101@gmail.com> 
	<4B867EE9.7060403@gmail.com>
Message-ID: <9cee7ab81002250620v32ed05adk403909f8708652bf@mail.gmail.com>

On Thu, Feb 25, 2010 at 8:45 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Even aside from the interpreter parsing problem, there's actually a more
> important human parsing problem.

Another human parsing problem is this:

    x = [(k, v) in somedict.items() if condition(v)]

What should be stored in the result?  While we can decide what it
should be, it's not blindingly obvious for a reader.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller


From greg.ewing at canterbury.ac.nz  Fri Feb 26 00:40:22 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 26 Feb 2010 12:40:22 +1300
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <20100225142028.53882a92@o>
References: <20100225142028.53882a92@o>
Message-ID: <4B870A66.5020103@canterbury.ac.nz>

spir wrote:

> But the analog does not apply to filter-only list comps:
> 	[x in numbers if x%2==1]	# SyntaxError

If you were allowed to say that, you should also be
able to say

    [x in numbers]

but this already has a meaning as an ordinary list
constructor.

Requiring the word 'for' to appear in an LC ensures
that there is never any ambiguity.

However, it might be possible to phrase it another
way, e.g.

   [x from numbers if x%2==1]

-- 
Greg


From fdrake at acm.org  Fri Feb 26 00:35:17 2010
From: fdrake at acm.org (Fred Drake)
Date: Thu, 25 Feb 2010 18:35:17 -0500
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <4B870A66.5020103@canterbury.ac.nz>
References: <20100225142028.53882a92@o> <4B870A66.5020103@canterbury.ac.nz>
Message-ID: <9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>

On Thu, Feb 25, 2010 at 6:40 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> ?[x from numbers if x%2==1]

I like this, but it has the same readability issue I noted for cases
where decomposition is used:

    [(k, v) from somedict.items() if condition(v)]


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller


From greg.ewing at canterbury.ac.nz  Fri Feb 26 01:11:10 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 26 Feb 2010 13:11:10 +1300
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
References: <20100225142028.53882a92@o> <4B870A66.5020103@canterbury.ac.nz>
	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
Message-ID: <4B87119E.2060702@canterbury.ac.nz>

Fred Drake wrote:

> I like this, but it has the same readability issue I noted for cases
> where decomposition is used:
> 
>     [(k, v) from somedict.items() if condition(v)]

I don't think that's too bad if you keep in mind that

   [x from stuff]

is a shorthand for

   [x for x in stuff]

whatever x happens to be. So your example would be
equivalent to

   [(k, v) for (k, v) in somedict.items() if condition(v)]

As a bonus, the short form could probably be made more
efficient, because the tuples produced by the for-loop
could be put straight into the result list, instead of
having to re-pack k and v into a new tuple.

-- 
Greg


From stephen at xemacs.org  Fri Feb 26 04:28:05 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 26 Feb 2010 12:28:05 +0900
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <4B87119E.2060702@canterbury.ac.nz>
References: <20100225142028.53882a92@o> <4B870A66.5020103@canterbury.ac.nz>
	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
	<4B87119E.2060702@canterbury.ac.nz>
Message-ID: <87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 >    [x from stuff]
 > 
 > is a shorthand for
 > 
 >    [x for x in stuff]

Any time you say "shorthand" you're probably violating TOOWTDI. :-)

 > As a bonus, the short form could probably be made more
 > efficient, because the tuples produced by the for-loop
 > could be put straight into the result list, instead of
 > having to re-pack k and v into a new tuple.

This is a straightforward optimization that should be done anyway,
though, if it isn't done already.


From denis.spir at gmail.com  Fri Feb 26 09:43:06 2010
From: denis.spir at gmail.com (spir)
Date: Fri, 26 Feb 2010 09:43:06 +0100
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <4B870A66.5020103@canterbury.ac.nz>
References: <20100225142028.53882a92@o>
	<4B870A66.5020103@canterbury.ac.nz>
Message-ID: <20100226094306.2f22e03e@o>

On Fri, 26 Feb 2010 12:40:22 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> spir wrote:
> 
> > But the analog does not apply to filter-only list comps:
> > 	[x in numbers if x%2==1]	# SyntaxError
> 
> If you were allowed to say that, you should also be
> able to say
> 
>     [x in numbers]
> but this already has a meaning as an ordinary list
> constructor.

Yes, that's precisely the semantics it should have (consistent with my view that <item in collection> is the core of a comprehension), since there is neither a mapping, nore a filter.

> 
> Requiring the word 'for' to appear in an LC ensures
> that there is never any ambiguity.

Yes, but the issue is rather with (x in numbers) beeing a test. (--> ambiguity of "in")

> However, it might be possible to phrase it another
> way, e.g.
> 
>    [x from numbers if x%2==1]

Wow, I like that. Better term than "in", in my opinion. (and solves the ambiguity of "in", but then consistency require to change "for x in numbers", which has about -1% chances to happen, I guess ;-)

Denis

-- 
________________________________

la vita e estrany

spir.wikidot.com



From denis.spir at gmail.com  Fri Feb 26 09:50:48 2010
From: denis.spir at gmail.com (spir)
Date: Fri, 26 Feb 2010 09:50:48 +0100
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
References: <20100225142028.53882a92@o> <4B870A66.5020103@canterbury.ac.nz>
	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
Message-ID: <20100226095048.315d95ca@o>

On Thu, 25 Feb 2010 18:35:17 -0500
Fred Drake <fdrake at acm.org> wrote:

> On Thu, Feb 25, 2010 at 6:40 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > ?[x from numbers if x%2==1]
> 
> I like this, but it has the same readability issue I noted for cases
> where decomposition is used:
> 
>     [(k, v) from somedict.items() if condition(v)]

You're right on readability. On the other hand, unpacking is a common practice in python.
Can this be mis-interpreted? (I mean, is it ambiguous?)
Also note that custom iterators can generate "items" of arbitrary complexity. This point cannot be addressed by the syntax for comprehensions.

Denis
-- 
________________________________

la vita e estrany

spir.wikidot.com



From denis.spir at gmail.com  Fri Feb 26 10:04:59 2010
From: denis.spir at gmail.com (spir)
Date: Fri, 26 Feb 2010 10:04:59 +0100
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <4B87119E.2060702@canterbury.ac.nz>
References: <20100225142028.53882a92@o> <4B870A66.5020103@canterbury.ac.nz>
	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
	<4B87119E.2060702@canterbury.ac.nz>
Message-ID: <20100226100459.45b49957@o>

On Fri, 26 Feb 2010 13:11:10 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Fred Drake wrote:
> 
> > I like this, but it has the same readability issue I noted for cases
> > where decomposition is used:
> > 
> >     [(k, v) from somedict.items() if condition(v)]
> 
> I don't think that's too bad if you keep in mind that
> 
>    [x from stuff]
> 
> is a shorthand for
> 
>    [x for x in stuff]
> 
> whatever x happens to be. So your example would be
> equivalent to
> 
>    [(k, v) for (k, v) in somedict.items() if condition(v)]
> 
> As a bonus, the short form could probably be made more
> efficient, because the tuples produced by the for-loop
> could be put straight into the result list, instead of
> having to re-pack k and v into a new tuple.

(Discussing my own shorthand proposal :-)

Sure, but remains a lexical issue:

   [x in stuff] (x in stuff)
cannot be accepted because of ambiguity with membership test in the iterator case.

   [x from stuff] (x from stuff))
solves that. But then for consistency "from" should be extented to all comprehensions
   [x*x for x from stuff if x%2==1]
and, more importantly, to traversal loops:
   for x from stuff:
because the sense is analog.
"in" would remain only as membership test operator.
This is very few probable ;-)
The issue is the ambiguity of "in".

Denis

-- 
________________________________

la vita e estrany

spir.wikidot.com



From stefan_ml at behnel.de  Fri Feb 26 14:47:38 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 26 Feb 2010 14:47:38 +0100
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20100225142028.53882a92@o>
	<4B870A66.5020103@canterbury.ac.nz>	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>	<4B87119E.2060702@canterbury.ac.nz>
	<87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <hm8jds$uno$1@dough.gmane.org>

Stephen J. Turnbull, 26.02.2010 04:28:
> Greg Ewing writes:
> 
>  >    [x from stuff]
>  > 
>  > is a shorthand for
>  > 
>  >    [x for x in stuff]
> 
> Any time you say "shorthand" you're probably violating TOOWTDI. :-)

Absolutely, so I would expect this to have zero change to make it in.


>  > As a bonus, the short form could probably be made more
>  > efficient, because the tuples produced by the for-loop
>  > could be put straight into the result list, instead of
>  > having to re-pack k and v into a new tuple.
> 
> This is a straightforward optimization that should be done anyway,
> though, if it isn't done already.

That's a good idea. Note, however, that the item extracted from the
iterable is not necessarily a tuple:

    >>> l = [[1,2], [3,4]]
    >>> [ (x,y) for (x,y) in l]
    [(1, 2), (3, 4)]

So the tuple packing may or may not be required, but that can only be
decided at runtime.

Stefan



From bborcic at gmail.com  Fri Feb 26 17:09:42 2010
From: bborcic at gmail.com (Boris Borcic)
Date: Fri, 26 Feb 2010 17:09:42 +0100
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <hm8jds$uno$1@dough.gmane.org>
References: <20100225142028.53882a92@o>	<4B870A66.5020103@canterbury.ac.nz>	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>	<4B87119E.2060702@canterbury.ac.nz>	<87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<hm8jds$uno$1@dough.gmane.org>
Message-ID: <hm8rqe$vi1$1@dough.gmane.org>

Stefan Behnel wrote:
> Stephen J. Turnbull, 26.02.2010 04:28:
>> Greg Ewing writes:
>>
>>   >     [x from stuff]
>>   >
>>   >  is a shorthand for
>>   >
>>   >     [x for x in stuff]
>>
>> Any time you say "shorthand" you're probably violating TOOWTDI. :-)
>
> Absolutely, so I would expect this to have zero change to make it in.

Now that's quite a pun, or more exactly quite an ? peu pr?s. Of course if 
TOOWTDI runs, it gets characteristic of any change that TOOWTDI is violated, eg, 
the one old way, the one new way.

- Boris




From g.brandl at gmx.net  Fri Feb 26 22:32:02 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 26 Feb 2010 22:32:02 +0100
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <hm8rqe$vi1$1@dough.gmane.org>
References: <20100225142028.53882a92@o>	<4B870A66.5020103@canterbury.ac.nz>	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>	<4B87119E.2060702@canterbury.ac.nz>	<87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>	<hm8jds$uno$1@dough.gmane.org>
	<hm8rqe$vi1$1@dough.gmane.org>
Message-ID: <hm9emq$7ju$1@dough.gmane.org>

Am 26.02.2010 17:09, schrieb Boris Borcic:
> Stefan Behnel wrote:
>> Stephen J. Turnbull, 26.02.2010 04:28:
>>> Greg Ewing writes:
>>>
>>>   >     [x from stuff]
>>>   >
>>>   >  is a shorthand for
>>>   >
>>>   >     [x for x in stuff]
>>>
>>> Any time you say "shorthand" you're probably violating TOOWTDI. :-)
>>
>> Absolutely, so I would expect this to have zero change to make it in.
> 
> Now that's quite a pun, or more exactly quite an ? peu pr?s. Of course if 
> TOOWTDI runs, it gets characteristic of any change that TOOWTDI is violated, eg, 
> the one old way, the one new way.

Not really -- remember that there is "obvious" in the acronym.  We should
strive to make such changes that don't violate it.  For example, the "with"
statement introduced the obvious way to handle resources.  The "if-else"
expression introduced the obvious way to express conditionals. Etc.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From greg.ewing at canterbury.ac.nz  Fri Feb 26 23:03:27 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 27 Feb 2010 11:03:27 +1300
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20100225142028.53882a92@o> <4B870A66.5020103@canterbury.ac.nz>
	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
	<4B87119E.2060702@canterbury.ac.nz>
	<87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4B88452F.2020400@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> Greg Ewing writes:
> 
>  > the tuples produced by the for-loop
>  > could be put straight into the result list, instead of
>  > having to re-pack k and v into a new tuple.
> 
> This is a straightforward optimization that should be done anyway,
> though, if it isn't done already.

I don't think it's as straightforward as all that. You
can't do it at compile time, because even if the target
and result expression are textually identical, the
repacked item isn't necessarily the same type as the
one produced by the iteration.

-- 
Greg


From denis.spir at gmail.com  Sat Feb 27 13:31:25 2010
From: denis.spir at gmail.com (spir)
Date: Sat, 27 Feb 2010 13:31:25 +0100
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <hm9emq$7ju$1@dough.gmane.org>
References: <20100225142028.53882a92@o> <4B870A66.5020103@canterbury.ac.nz>
	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>
	<4B87119E.2060702@canterbury.ac.nz>
	<87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<hm8jds$uno$1@dough.gmane.org> <hm8rqe$vi1$1@dough.gmane.org>
	<hm9emq$7ju$1@dough.gmane.org>
Message-ID: <20100227133125.4e49f0c7@o>

On Fri, 26 Feb 2010 22:32:02 +0100
Georg Brandl <g.brandl at gmx.net> wrote:

> > Now that's quite a pun, or more exactly quite an ? peu pr?s. Of course if 
> > TOOWTDI runs, it gets characteristic of any change that TOOWTDI is violated, eg, 
> > the one old way, the one new way.  
> 
> Not really -- remember that there is "obvious" in the acronym.  We should
> strive to make such changes that don't violate it.  For example, the "with"
> statement introduced the obvious way to handle resources.  The "if-else"
> expression introduced the obvious way to express conditionals. Etc.

From that point of view, I guess the obvious way to express the absence of a mapping should not be the expression of an identity mapping (in the sense of the math identity function) ;-)
    (for) item in collection 	vs 	item for item in collection
In the same way that we don't express the absence of a filter using a True filter:
    expr for item in collection 	vs 	expr for item in collection if True

But sure, again, the ambiguity of "in" seems to be an obstacle for python to adopt the obvious way.

Denis
-- 
________________________________

la vita e estrany

spir.wikidot.com



From ncoghlan at gmail.com  Sat Feb 27 15:39:32 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 28 Feb 2010 00:39:32 +1000
Subject: [Python-ideas] filter-only list comps
In-Reply-To: <20100227133125.4e49f0c7@o>
References: <20100225142028.53882a92@o>
	<4B870A66.5020103@canterbury.ac.nz>	<9cee7ab81002251535u72a95a13nb1e92211b9e2c05f@mail.gmail.com>	<4B87119E.2060702@canterbury.ac.nz>	<87pr3sekvu.fsf@uwakimon.sk.tsukuba.ac.jp>	<hm8jds$uno$1@dough.gmane.org>
	<hm8rqe$vi1$1@dough.gmane.org>	<hm9emq$7ju$1@dough.gmane.org>
	<20100227133125.4e49f0c7@o>
Message-ID: <4B892EA4.6050402@gmail.com>

spir wrote:
> But sure, again, the ambiguity of "in" seems to be an obstacle for
> python to adopt the obvious way.

Again, the human parsing problem is actually more important than the
computer parser limitations.

The Python for loop conditions developers to think "for x in seq <do
something>" (where <do something> is written out as a colon, a newline,
some indentation and additional code).

The genexp/comprehension syntax, brings the "<do something>" to the
front as "<produce this value> for x in seq". Mapping is just the
degenerate case where the value provided *is* x, but making that
implicit doesn't make *anything* more obvious in the way that omitting
the filter clause obviously means "unfiltered".

Instead, omitting the clause makes it look like the start of a regular
for loop (assuming the for keyword is kept), so a developer is going to
go looking for the "do something" part of the loop and won't find it.

You're suggesting a change that doesn't make anything clearer (and, in
fact, will almost certainly make reading code harder since developers
will have another idiom to learn), provides a second way of writing "x
for x", and doesn't really even save the person writing the code any time.

What's the concrete benefit that justifies even the time we've spent
discussing the matter in this thread, let alone anyone actually taking
the time to code the necessary grammar changes?

Cheers,
Nick.

P.S. We haven't even mentioned the fact that this runs afoul of the
language change moratorium yet :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------