From stephen at xemacs.org  Mon Nov  1 02:05:21 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 01 Nov 2010 10:05:21 +0900
Subject: [Python-ideas] Accepting "?" as a valid character for identifiers
In-Reply-To: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
References: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
Message-ID: <87pqup4zvy.fsf@uwakimon.sk.tsukuba.ac.jp>

Andre Roberge writes:

 > In some languages (e.g. Scheme, Ruby, etc.), the question mark character (?)
 > is a valid character for identifiers.  I find that using it well can improve
 > readability of programs written in those languages.

As pointed out by others, this won't work well in Python.

There is an established idiom for this in Python.  What in Scheme
would be spelled "PREDICATE?" in Python is spelled "is_PREDICATE".

 > Given that one can legally mimic ? in Python identifiers, and given that the
 > ? symbol is not used for anything in Python, would it be possible to
 > consider allowing the use of ? as a valid character in an identifier?

As Georg points out, it is used for something.  Punctuation is
reserved for use as an operator.  That fact that it's not assigned any
specific meaning today doesn't mean that it's unused.  Of course
exceptions are always possible, but this simply doesn't have a good
enough reason to justify an exception.

In any case, this is covered by the moratorium.  By the time a Python
incorporating this suggestion could appear, your students will all
have graduated and will be working for companies, conforming to style
guides that specify use of the "is_PREDICATE" naming scheme.


From cs at zip.com.au  Mon Nov  1 04:29:29 2010
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 1 Nov 2010 14:29:29 +1100
Subject: [Python-ideas] Accepting "?" as a valid character for
	identifiers
In-Reply-To: <iakeve$6mn$1@dough.gmane.org>
References: <iakeve$6mn$1@dough.gmane.org>
Message-ID: <20101101032929.GA22908@cskk.homeip.net>

On 31Oct2010 20:10, Stefan Behnel <stefan_ml at behnel.de> wrote:
| Hmm, that reminds me. I think we should reconsider PEP 3117. There's
| still some value in it.
| http://www.python.org/dev/peps/pep-3117/

Hmm. It really needs an update: Victor Borge's pronunciation protocol
should be adopted to aid verbal discussion of code using the proposed
extensions.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

quantum-dot anti-counterfeiting
- overhead by WIRED at the Intelligent Printing conference Oct2006


From jimjjewett at gmail.com  Mon Nov  1 21:36:36 2010
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 1 Nov 2010 16:36:36 -0400
Subject: [Python-ideas] Accepting "?" as a valid character for
	identifiers
In-Reply-To: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
References: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
Message-ID: <AANLkTikB+7m3hC9SrWuaWkBShkSqrzpsmj9uuBRz2hUO@mail.gmail.com>

On 10/31/10, Andre Roberge <andre.roberge at gmail.com> wrote:
> In some languages (e.g. Scheme, Ruby, etc.), the question mark character (?)
> is a valid character for identifiers.

> ... given that the
> ? symbol is not used for anything in Python, would it be possible to
> consider allowing the use of ? as a valid character in an identifier?

One of the things that came out in the decorator discussion is that
the few unused ASCII characters are precious -- and are already in use
by various extensions.  That doesn't mean they absolutely can't be
used (@ was taken in the end), but the barrier is much higher than
"are we using it already"?

-jJ


From kristjan at ccpgames.com  Tue Nov  2 01:08:51 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 2 Nov 2010 08:08:51 +0800
Subject: [Python-ideas] time.wallclock() or other similar stuff
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>

Bringing this in from python-dev, and http://bugs.python.org/issue10278

Summary:  time.clock() means two different things on Windows and Unix.  So, if you need to measure time across a blocking call in a portable way, you need to use time.time() (this fact is not documented).  Time.time has its own problems though, such as low resolution (compared to time.clock on windows) and being susceptible to being adjusted by the user.

I proposed adding a time.wallclock() to address this issue, which would be the most suitable function for the job on any platform.  The patch is an example implementation.  Any thoughts?  And yes, I acknowledge that the time module is complicated enough as it is :)

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101102/7f10a21f/attachment.html>

From python at mrabarnett.plus.com  Tue Nov  2 02:05:52 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 02 Nov 2010 01:05:52 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
Message-ID: <4CCF63F0.10603@mrabarnett.plus.com>

On 02/11/2010 00:08, Kristj?n Valur J?nsson wrote:
> Bringing this in from python-dev, and http://bugs.python.org/issue10278
>
> Summary:  time.clock() means two different things on Windows and Unix.
> So, if you need to measure time across a blocking call in a portable
> way, you need to use time.time() (this fact is not documented).
> Time.time has its own problems though, such as low resolution (compared
> to time.clock on windows) and being susceptible to being adjusted by the
> user.
>
> I proposed adding a time.wallclock() to address this issue, which would
> be the most suitable function for the job on any platform.  The patch is
> an example implementation.  Any thoughts?  And yes, I acknowledge that
> the time module is complicated enough as it is J
>
I'm not sure about the name because the familiar wall clock changes for
daylight savings time. :-)


From steve at pearwood.info  Tue Nov  2 02:50:57 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Nov 2010 12:50:57 +1100
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
Message-ID: <4CCF6E81.6070203@pearwood.info>

Kristj?n Valur J?nsson wrote:
> Bringing this in from python-dev, and http://bugs.python.org/issue10278
> 
> Summary:  time.clock() means two different things on Windows and Unix.  So, if you need to measure time across a blocking call in a portable way, you need to use time.time() (this fact is not documented).  Time.time has its own problems though, such as low resolution (compared to time.clock on windows) and being susceptible to being adjusted by the user.
> 
> I proposed adding a time.wallclock() to address this issue, which would be the most suitable function for the job on any platform.  The patch is an example implementation.  Any thoughts?  And yes, I acknowledge that the time module is complicated enough as it is :)


I'm not particularly keen on "wall_clock" as a name. The name suggests 
it should return something like "12:37:15". I prefer a name that tells 
you what you would use the function for: this is the right function to 
use for timers, so call it "timer".

The timeit module includes:

if sys.platform == "win32":
     # On Windows, the best timer is time.clock()
     default_timer = time.clock
else:
     # On most other platforms the best timer is time.time()
     default_timer = time.time

(Aside: what about WinCE? Shouldn't it also use time.clock?)

and the timeit.Timer class takes an argument "timer" which defaults to 
default_timer, so there is precedent on the name.


+1 on a platform-dependent alias to time() or clock()
+1 on calling it "timer"
-0 on calling it "wall_clock"



-- 
Steven



From kristjan at ccpgames.com  Tue Nov  2 03:27:42 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 2 Nov 2010 10:27:42 +0800
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CCF6E81.6070203@pearwood.info>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF79@exchcn.ccp.ad.local>

Well, maybe all we need is the timeit module, then?  I didn't realize that we have default_timer there.

Otherwise, I'm not sure about "timer".  To me that word implies some sort of context to be timed, or action to be taken, but I'm +0 on it.

K

-----Original Message-----
From: python-ideas-bounces+kristjan=ccpgames.com at python.org [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Steven D'Aprano
Sent: Tuesday, November 02, 2010 9:51
To: python-ideas at python.org
Subject: Re: [Python-ideas] time.wallclock() or other similar stuff
The timeit module includes:

if sys.platform == "win32":
     # On Windows, the best timer is time.clock()
     default_timer = time.clock
else:
     # On most other platforms the best timer is time.time()
     default_timer = time.time


+1 on a platform-dependent alias to time() or clock()
+1 on calling it "timer"
-0 on calling it "wall_clock"



-- 
Steven

_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
http://mail.python.org/mailman/listinfo/python-ideas



From alexander.belopolsky at gmail.com  Tue Nov  2 03:52:07 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 1 Nov 2010 22:52:07 -0400
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CCF6E81.6070203@pearwood.info>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
Message-ID: <AANLkTikbXCx=w-AZOU3LBdBtb7G2xsLtHc2bMWuckevV@mail.gmail.com>

On Mon, Nov 1, 2010 at 9:50 PM, Steven D'Aprano <steve at pearwood.info> wrote:
..
> The timeit module includes:
>
> if sys.platform == "win32":
> ? ?# On Windows, the best timer is time.clock()
> ? ?default_timer = time.clock
> else:
> ? ?# On most other platforms the best timer is time.time()
> ? ?default_timer = time.time
>

Doesn't this mean that the requested functionality is already available as

from timeit import default_timer

?


From steve at pearwood.info  Tue Nov  2 04:38:14 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Nov 2010 14:38:14 +1100
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <AANLkTikbXCx=w-AZOU3LBdBtb7G2xsLtHc2bMWuckevV@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCF6E81.6070203@pearwood.info>
	<AANLkTikbXCx=w-AZOU3LBdBtb7G2xsLtHc2bMWuckevV@mail.gmail.com>
Message-ID: <4CCF87A6.9010708@pearwood.info>

Alexander Belopolsky wrote:
> On Mon, Nov 1, 2010 at 9:50 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> ..
>> The timeit module includes:
>>
>> if sys.platform == "win32":
>>    # On Windows, the best timer is time.clock()
>>    default_timer = time.clock
>> else:
>>    # On most other platforms the best timer is time.time()
>>    default_timer = time.time
>>
> 
> Doesn't this mean that the requested functionality is already available as
> 
> from timeit import default_timer
> 
> ?


Well, yes, it does, but:

(1) How many people would think to look in timeit for this function? 
Both you and Kristj?n were apparently surprised that it already exists, 
and I'd bet money that you weren't the only ones. timeit seems to be 
more of an application than a library (although I acknowledge the border 
between the two is fuzzy), and people don't think about using it as a 
library.

(2) The time module is the natural place to look for time-related 
functions. It would be nicer if timeit could import the default timer 
from time:

from time import timer as default_timer  # or wall_clock if you prefer

(3) The default_timer is an implementation detail of timeit. It could 
change, or disappear, or be renamed. You'll note it isn't documented:

http://docs.python.org/library/timeit.html

(4) I started off thinking that we should just document timer 
(wall_clock) as an alias to one of time() or clock(), but I changed my 
mind. That's an implementation detail that we should be free to change 
in the future. For the sake of information hiding, we shouldn't promise 
that the timer will be one of time.clock() or time.time(). Who knows, 
maybe there are (or will be someday) supported platforms where neither 
choice is appropriate, and the timer needs to be something else?



-- 
Steven



From python at mrabarnett.plus.com  Tue Nov  2 05:07:45 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 02 Nov 2010 04:07:45 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CCF87A6.9010708@pearwood.info>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCF6E81.6070203@pearwood.info>	<AANLkTikbXCx=w-AZOU3LBdBtb7G2xsLtHc2bMWuckevV@mail.gmail.com>
	<4CCF87A6.9010708@pearwood.info>
Message-ID: <4CCF8E91.3000501@mrabarnett.plus.com>

On 02/11/2010 03:38, Steven D'Aprano wrote:
> Alexander Belopolsky wrote:
>> On Mon, Nov 1, 2010 at 9:50 PM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>> ..
>>> The timeit module includes:
>>>
>>> if sys.platform == "win32":
>>> # On Windows, the best timer is time.clock()
>>> default_timer = time.clock
>>> else:
>>> # On most other platforms the best timer is time.time()
>>> default_timer = time.time
>>>
>>
>> Doesn't this mean that the requested functionality is already
>> available as
>>
>> from timeit import default_timer
>>
>> ?
>
>
> Well, yes, it does, but:
>
> (1) How many people would think to look in timeit for this function?
> Both you and Kristj?n were apparently surprised that it already exists,
> and I'd bet money that you weren't the only ones. timeit seems to be
> more of an application than a library (although I acknowledge the border
> between the two is fuzzy), and people don't think about using it as a
> library.
>
> (2) The time module is the natural place to look for time-related
> functions. It would be nicer if timeit could import the default timer
> from time:
>
> from time import timer as default_timer # or wall_clock if you prefer
>
> (3) The default_timer is an implementation detail of timeit. It could
> change, or disappear, or be renamed. You'll note it isn't documented:
>
> http://docs.python.org/library/timeit.html
>
> (4) I started off thinking that we should just document timer
> (wall_clock) as an alias to one of time() or clock(), but I changed my
> mind. That's an implementation detail that we should be free to change
> in the future. For the sake of information hiding, we shouldn't promise
> that the timer will be one of time.clock() or time.time(). Who knows,
> maybe there are (or will be someday) supported platforms where neither
> choice is appropriate, and the timer needs to be something else?
>
+1


From dreamingforward at gmail.com  Tue Nov  2 05:09:09 2010
From: dreamingforward at gmail.com (average)
Date: Mon, 1 Nov 2010 22:09:09 -0600
Subject: [Python-ideas] Accepting "?" as a valid character for
	identifiers
In-Reply-To: <iakadn$kgu$1@dough.gmane.org>
References: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
	<iakadn$kgu$1@dough.gmane.org>
Message-ID: <AANLkTimoMkUNjOBKgYgU9YOenevY2WAMcqzGJiN-iQmh@mail.gmail.com>

> Another drawback of introducing such a convention this late in the design
> of
> the language is that you can never have it applied consistently.  Changing
> the builtin and stdlib instances alone would need hundreds of compatibility
> aliases.
>
> Actually, I think this would improve the standard library dramatically.
Particularly, if it was coupled with a complete redesign/philosophy for
naming conventions with respect to verb/noun, methods/properties,
adverbs/adjectives (as modifiers to their respective parts of speech).
There are always judgment calls for some of these items, but that probably
signifies a refactoring is in order because there shouldn't probably be any
ambiguity in a part of speech.  (I don't know -- this is an interesting CS
language-design issue.  Anyone know of any work done in this area?)  Case
should also be re-considered carefully also.

In general when one comes to code that one hasn't seen or designed, without
more syntactical support to indicate semantics of all the many identifiers,
I've found myself too-easily dismissive of re-using code because of all the
work it takes to work out the meaning and the idiosyncracies of the
individuals -- it's like wearing someone else's underwear.

marcos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101101/2d1f120f/attachment.html>

From pyideas at rebertia.com  Tue Nov  2 05:26:14 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Mon, 1 Nov 2010 21:26:14 -0700
Subject: [Python-ideas] Accepting "?" as a valid character for
	identifiers
In-Reply-To: <AANLkTimoMkUNjOBKgYgU9YOenevY2WAMcqzGJiN-iQmh@mail.gmail.com>
References: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
	<iakadn$kgu$1@dough.gmane.org>
	<AANLkTimoMkUNjOBKgYgU9YOenevY2WAMcqzGJiN-iQmh@mail.gmail.com>
Message-ID: <AANLkTimFh2KhF0y+xcbrX=AkQu48qPetHQ-FFBGvYcKF@mail.gmail.com>

On Mon, Nov 1, 2010 at 9:09 PM, average <dreamingforward at gmail.com> wrote:
>> Another drawback of introducing such a convention this late in the design
>> of
>> the language is that you can never have it applied consistently. ?Changing
>> the builtin and stdlib instances alone would need hundreds of
>> compatibility
>> aliases.
>>
> Actually, I think this would improve the standard library dramatically.
> Particularly, if it was coupled with a complete redesign/philosophy for
> naming conventions with respect to verb/noun, methods/properties,
> adverbs/adjectives (as modifiers to their respective parts of speech).

That's a mighty big "if". Woe to those tasked with refactoring the std
lib to make it compliant ("batteries included" is a liability in this
case). Woe to all the users who would have to make numerous trivial,
fiddly code changes.
Hell, parts of the std lib aren't even currently PEP 8-compliant, much
less PEP "8.5"-compliant.

Perhaps in Python 4 there will be "The Great Std Lib Refactoring of
2025". :-) Until then, good luck.

Cheers,
Chris
--
*stares intently at the passing Iguana*
http://blog.rebertia.com


From ianb at colorstudy.com  Tue Nov  2 05:38:01 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 1 Nov 2010 23:38:01 -0500
Subject: [Python-ideas] Accepting "?" as a valid character for
	identifiers
In-Reply-To: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
References: <AANLkTinWr=5vuc66z_DT+W5LCPgOBW+oe7cB6f1vvmQ3@mail.gmail.com>
Message-ID: <AANLkTi=s3GfVDZWzckhzQrwsJN1=hR5pKiWkh-8e5WdT@mail.gmail.com>

General impression:

? and ! are allowed in Ruby, which like Python is a multi-paradigm
language.  Despite the availability (from the beginning) they are still
inconsistently used in Ruby.  I attribute this to convention -- Ruby, like
any language, borrows from the experience of other programmers and other
interfaces.  Python is like Ruby but with years of extra history.  We can
never apply ? and ! consistently, and without consistency they are random
notations added according to the original programmer's history (i.e., when
they came from Lisp they'll use the punctuation, and when they come from
other languages they'll use nothing).

So... nice idea, but a couple decades too late?

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101101/c1fab068/attachment.html>

From tjreedy at udel.edu  Tue Nov  2 05:47:48 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 02 Nov 2010 00:47:48 -0400
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CCF87A6.9010708@pearwood.info>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCF6E81.6070203@pearwood.info>	<AANLkTikbXCx=w-AZOU3LBdBtb7G2xsLtHc2bMWuckevV@mail.gmail.com>
	<4CCF87A6.9010708@pearwood.info>
Message-ID: <iao55d$n4d$1@dough.gmane.org>

On 11/1/2010 11:38 PM, Steven D'Aprano wrote:

> (1) How many people would think to look in timeit for this function?
> Both you and Kristj?n were apparently surprised that it already exists,
> and I'd bet money that you weren't the only ones. timeit seems to be
> more of an application than a library (although I acknowledge the border
> between the two is fuzzy), and people don't think about using it as a
> library.
>
> (2) The time module is the natural place to look for time-related
> functions. It would be nicer if timeit could import the default timer
> from time:
>
> from time import timer as default_timer # or wall_clock if you prefer
>
> (3) The default_timer is an implementation detail of timeit. It could
> change, or disappear, or be renamed. You'll note it isn't documented:
>
> http://docs.python.org/library/timeit.html
>
> (4) I started off thinking that we should just document timer
> (wall_clock) as an alias to one of time() or clock(), but I changed my
> mind. That's an implementation detail that we should be free to change
> in the future. For the sake of information hiding, we shouldn't promise
> that the timer will be one of time.clock() or time.time(). Who knows,
> maybe there are (or will be someday) supported platforms where neither
> choice is appropriate, and the timer needs to be something else?

+1 for timer in time module from me too.
Having two time functions in time of opposite ratings on different 
platforms has always (13 years or whenever there became two) seemed like 
a wart to me.

-- 
Terry Jan Reedy




From mal at egenix.com  Tue Nov  2 10:33:26 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 02 Nov 2010 10:33:26 +0100
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
Message-ID: <4CCFDAE6.5070707@egenix.com>

Kristj?n Valur J?nsson wrote:
> Bringing this in from python-dev, and http://bugs.python.org/issue10278
> 
> Summary:  time.clock() means two different things on Windows and Unix.  So, if you need to measure time across a blocking call in a portable way, you need to use time.time() (this fact is not documented).  Time.time has its own problems though, such as low resolution (compared to time.clock on windows) and being susceptible to being adjusted by the user.
>
> I proposed adding a time.wallclock() to address this issue, which would be the most suitable function for the job on any platform.  The patch is an example implementation.  Any thoughts?  And yes, I acknowledge that the time module is complicated enough as it is :)

You might want to look at the systimes.py module that comes with pybench
(see Tools/pybench). This already provides a cross-platform way of
accessing high accuracy timers for benchmarking and the like.

For obvious reasons this tries to measure run-time, though, and not
elapsed time.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From denis.spir at gmail.com  Tue Nov  2 11:18:15 2010
From: denis.spir at gmail.com (spir)
Date: Tue, 2 Nov 2010 11:18:15 +0100
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CCF87A6.9010708@pearwood.info>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
	<AANLkTikbXCx=w-AZOU3LBdBtb7G2xsLtHc2bMWuckevV@mail.gmail.com>
	<4CCF87A6.9010708@pearwood.info>
Message-ID: <20101102111815.3dbe0ef1@o>

On Tue, 02 Nov 2010 14:38:14 +1100
Steven D'Aprano <steve at pearwood.info> wrote:

> Alexander Belopolsky wrote:

> Well, yes, it does, but:
> 
> (1) How many people would think to look in timeit for this function? 
> Both you and Kristj?n were apparently surprised that it already exists, 
> and I'd bet money that you weren't the only ones. timeit seems to be 
> more of an application than a library (although I acknowledge the border 
> between the two is fuzzy), and people don't think about using it as a 
> library.
> 
> (2) The time module is the natural place to look for time-related 
> functions. It would be nicer if timeit could import the default timer 
> from time:
> 
> from time import timer as default_timer  # or wall_clock if you prefer
> 
> (3) The default_timer is an implementation detail of timeit. It could 
> change, or disappear, or be renamed. You'll note it isn't documented:
> 
> http://docs.python.org/library/timeit.html
> 
> (4) I started off thinking that we should just document timer 
> (wall_clock) as an alias to one of time() or clock(), but I changed my 
> mind. That's an implementation detail that we should be free to change 
> in the future. For the sake of information hiding, we shouldn't promise 
> that the timer will be one of time.clock() or time.time(). Who knows, 
> maybe there are (or will be someday) supported platforms where neither 
> choice is appropriate, and the timer needs to be something else?

Very good summary of the present issue. I think the proposed solution is just what we need.

Denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From solipsis at pitrou.net  Tue Nov  2 12:35:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 2 Nov 2010 12:35:47 +0100
Subject: [Python-ideas] time.wallclock() or other similar stuff
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
Message-ID: <20101102123547.230b50fc@pitrou.net>

On Tue, 02 Nov 2010 12:50:57 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> 
> +1 on a platform-dependent alias to time() or clock()
> +1 on calling it "timer"

"time.timer()" is much too close to "time.time()".

Regards

Antoine.




From flub at devork.be  Tue Nov  2 14:21:18 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Tue, 2 Nov 2010 13:21:18 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <20101102123547.230b50fc@pitrou.net>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
	<20101102123547.230b50fc@pitrou.net>
Message-ID: <AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>

On 2 November 2010 11:35, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 02 Nov 2010 12:50:57 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
>>
>> +1 on a platform-dependent alias to time() or clock()
>> +1 on calling it "timer"
>
> "time.timer()" is much too close to "time.time()".

I'd agree with this.  I'm actually +1 on the original
time.wallclock(), wall time is a common concept, so having the word
"wall" in it's api seems very natural to me.  .walltime() would
probably work as well.

Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From rob.cliffe at btinternet.com  Tue Nov  2 14:43:04 2010
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 02 Nov 2010 13:43:04 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCF6E81.6070203@pearwood.info>	<20101102123547.230b50fc@pitrou.net>
	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>
Message-ID: <4CD01568.40907@btinternet.com>



On 02/11/2010 13:21, Floris Bruynooghe wrote:
> On 2 November 2010 11:35, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>> On Tue, 02 Nov 2010 12:50:57 +1100
>> Steven D'Aprano<steve at pearwood.info>  wrote:
>>> +1 on a platform-dependent alias to time() or clock()
>>> +1 on calling it "timer"
>> "time.timer()" is much too close to "time.time()".
> I'd agree with this.  I'm actually +1 on the original
> time.wallclock(), wall time is a common concept, so having the word
> "wall" in it's api seems very natural to me.  .walltime() would
> probably work as well.
>
> Regards
> Floris
>
>
I beg to differ.
Seeing "wall clock" just confused me - my reaction was what the ..... is 
he talking about?  Even when I understood it, it didn't seem a very 
appropriate name - I would have found something like "system clock" or 
"system time" more intelligible.
Whereas "timer" is a very familiar concept (in computing, and in life in 
general).
And I think most people are familiar with time.time(), so will recognize 
that time.timer() is something different.  Though I agree it would be 
better if the names were more different.
Best wishes
Rob Cliffe


From denis.spir at gmail.com  Tue Nov  2 16:02:47 2010
From: denis.spir at gmail.com (spir)
Date: Tue, 2 Nov 2010 16:02:47 +0100
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CD01568.40907@btinternet.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
	<20101102123547.230b50fc@pitrou.net>
	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>
	<4CD01568.40907@btinternet.com>
Message-ID: <20101102160247.17a02759@o>

On Tue, 02 Nov 2010 13:43:04 +0000
Rob Cliffe <rob.cliffe at btinternet.com> wrote:

> I beg to differ.
> Seeing "wall clock" just confused me - my reaction was what the ..... is 
> he talking about?  Even when I understood it, it didn't seem a very 
> appropriate name - I would have found something like "system clock" or 
> "system time" more intelligible.
> Whereas "timer" is a very familiar concept (in computing, and in life in 
> general).
> And I think most people are familiar with time.time(), so will recognize 
> that time.timer() is something different.  Though I agree it would be 
> better if the names were more different.

I share this point of view. "wallclock" is really misleading, it simply does not tell what the object is, let's one guess something totally different; and what about its purpose? (also think at  non-english speakers).
Imo, "timer" is exactly the proper term -- the issue due to proximity with "time" is real, but far less important.

Denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From jamzo at mac.com  Tue Nov  2 16:17:07 2010
From: jamzo at mac.com (James Noble)
Date: Tue, 02 Nov 2010 11:17:07 -0400
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <20101102160247.17a02759@o>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info> <20101102123547.230b50fc@pitrou.net>
	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>
	<4CD01568.40907@btinternet.com> <20101102160247.17a02759@o>
Message-ID: <09BEB9F7-1BFC-4446-9578-DFE3AF019DB1@mac.com>


On Nov 2, 2010, at 11:02 AM, spir wrote:

> Imo, "timer" is exactly the proper term -- the issue due to proximity with "time" is real, but far less important.

How about "stopwatch"

James


From solipsis at pitrou.net  Tue Nov  2 16:35:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 2 Nov 2010 16:35:27 +0100
Subject: [Python-ideas] time.wallclock() or other similar stuff
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
	<20101102123547.230b50fc@pitrou.net>
	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>
	<4CD01568.40907@btinternet.com> <20101102160247.17a02759@o>
Message-ID: <20101102163527.2512e219@pitrou.net>

On Tue, 2 Nov 2010 16:02:47 +0100
spir <denis.spir at gmail.com> wrote:
> Imo, "timer" is exactly the proper term

I'm not sure it's the proper term. A timer is often a countdown with a
callback. See for example threading.Timer and signal.setitimer.

> the issue due to proximity with "time" is real, but far less
> important.

I think it would be too easy to use one instead of the other, and get
strange semantics (both count in seconds but with slightly different
characteristic).

Regards

Antoine.




From rob.cliffe at btinternet.com  Tue Nov  2 20:21:48 2010
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 02 Nov 2010 19:21:48 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <20101102163527.2512e219@pitrou.net>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCF6E81.6070203@pearwood.info>	<20101102123547.230b50fc@pitrou.net>	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>	<4CD01568.40907@btinternet.com>
	<20101102160247.17a02759@o> <20101102163527.2512e219@pitrou.net>
Message-ID: <4CD064CC.4070207@btinternet.com>

On 02/11/2010 15:35, Antoine Pitrou wrote:
> On Tue, 2 Nov 2010 16:02:47 +0100
> spir<denis.spir at gmail.com>  wrote:
>> Imo, "timer" is exactly the proper term
> I'm not sure it's the proper term. A timer is often a countdown with a
> callback. See for example threading.Timer and signal.setitimer.
>
>> the issue due to proximity with "time" is real, but far less
>> important.
> I think it would be too easy to use one instead of the other, and get
> strange semantics (both count in seconds but with slightly different
> characteristic).
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
Well, how about "clock"?
"time.clock()" seems quite evocative to me.
Best wishes
Rob Cliffe


From brian.curtin at gmail.com  Tue Nov  2 20:40:37 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 2 Nov 2010 14:40:37 -0500
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CD064CC.4070207@btinternet.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
	<20101102123547.230b50fc@pitrou.net>
	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>
	<4CD01568.40907@btinternet.com> <20101102160247.17a02759@o>
	<20101102163527.2512e219@pitrou.net>
	<4CD064CC.4070207@btinternet.com>
Message-ID: <AANLkTik_RrZ541Wvp3X77YciCK+h+_jpz+_xE+dp4AJg@mail.gmail.com>

On Tue, Nov 2, 2010 at 14:21, Rob Cliffe <rob.cliffe at btinternet.com> wrote:

> On 02/11/2010 15:35, Antoine Pitrou wrote:
>
>> On Tue, 2 Nov 2010 16:02:47 +0100
>> spir<denis.spir at gmail.com>  wrote:
>>
>>> Imo, "timer" is exactly the proper term
>>>
>> I'm not sure it's the proper term. A timer is often a countdown with a
>> callback. See for example threading.Timer and signal.setitimer.
>>
>>  the issue due to proximity with "time" is real, but far less
>>> important.
>>>
>> I think it would be too easy to use one instead of the other, and get
>> strange semantics (both count in seconds but with slightly different
>> characteristic).
>>
>> Regards
>>
>> Antoine.
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>>  Well, how about "clock"?
> "time.clock()" seems quite evocative to me.
> Best wishes
> Rob Cliffe


time.clock is already a part of the problem.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101102/f377ea22/attachment.html>

From jh at improva.dk  Tue Nov  2 20:42:57 2010
From: jh at improva.dk (Jacob Holm)
Date: Tue, 02 Nov 2010 20:42:57 +0100
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <09BEB9F7-1BFC-4446-9578-DFE3AF019DB1@mac.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCF6E81.6070203@pearwood.info>
	<20101102123547.230b50fc@pitrou.net>	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>	<4CD01568.40907@btinternet.com>
	<20101102160247.17a02759@o>
	<09BEB9F7-1BFC-4446-9578-DFE3AF019DB1@mac.com>
Message-ID: <4CD069C1.2090704@improva.dk>

On 2010-11-02 16:17, James Noble wrote:
> 
> On Nov 2, 2010, at 11:02 AM, spir wrote:
> 
>> Imo, "timer" is exactly the proper term -- the issue due to proximity with "time" is real, but far less important.
> 
> How about "stopwatch"

With that name, I would expect a "proper" stopwatch object with "get",
"start", "stop" and "reset" methods.  Not just an alias for time.time or
time.clock.

I wouldn't mind seeing such an object in the stdlib, but it is easy
enough to define in terms of the function that is actually proposed.
For example:


from time import timer # the new function

class stopwatch(object):

    def __init__(self, t=0.0):
        self._value = t
        self._started = None

    def get(self):
        s = self._started
        if s is not None:
            return self._value + timer() - s
        return self._value

    def reset(self, t=0.0):
        self._value = t
        if self._started is not None:
            self._started = timer()

    def is_running(self):
        return self._started is not None

    def start(self):
        if self._started is None:
            self._started = timer()

    def stop(self):
        s = self._started
        if s is not None:
            self._value += timer() - s
            self._started = None

    value = property(get, reset, reset)

    running = property(is_running, None, stop)

    @running.setter
    def running(self, value):
        if value:
            if self._started is None:
                self._started = timer()
        else:
            s = self._started
            if s is not None:
                self._value += timer() - s
                self._started = None


Oh, and +1 for the proposed function - I don't really care what it's called.

- Jacob


From guido at python.org  Tue Nov  2 21:33:09 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 2 Nov 2010 13:33:09 -0700
Subject: [Python-ideas] Possible PEP 380 tweak
In-Reply-To: <4CCCB3B6.3010209@canterbury.ac.nz>
References: <AANLkTin1FNkHLww=yyUa2bhQ8w76FuUurRsG-5a2AXQ7@mail.gmail.com>
	<4CC63065.9040507@improva.dk>
	<AANLkTi=O4i81MTD9p251uUV+8PHrOXFfPggeuyvWa7jQ@mail.gmail.com>
	<AANLkTi=1X2e_d4OywtPNd8gf73-5epThejoD6F3KmOQ6@mail.gmail.com>
	<4CC6E94F.3090702@improva.dk>
	<AANLkTi=5HOYVUgzkKzk9eF4-DtPHfUV9afUKP1MiUcj2@mail.gmail.com>
	<AANLkTinOTtZzmJR2Vmm59Up_h0Pg7DnHJacKmvS_HJ=-@mail.gmail.com>
	<4CC889F1.8010603@improva.dk>
	<AANLkTimMjjgoHDNU-v_xjnjQEPTONnRqQpskoRsdeJ5W@mail.gmail.com>
	<4CC939E5.5070700@improva.dk>
	<AANLkTimysMXEaZY0xf+6AKGfYA=V-7Cu9g8HQuC3WNRV@mail.gmail.com>
	<4CC9FC87.1040600@canterbury.ac.nz>
	<AANLkTimownK9cPWFJnQX5UnuXY9RU+xcZsFmBUX7u3zU@mail.gmail.com>
	<4CCA752D.5090904@canterbury.ac.nz>
	<AANLkTikYpO+11RX=gf6UeqEBwT+NNQPc6nQTtY=ZKUu_@mail.gmail.com>
	<4CCB8C50.2010404@canterbury.ac.nz>
	<AANLkTikr9FDhTLisoPfR+bkDZTR_KVYVk=3FBSBT5zJw@mail.gmail.com>
	<4CCBB84C.2080809@canterbury.ac.nz>
	<AANLkTi=YMM-xyFis1wiybWnNJXBJ0YxF2m=t2-H9CENx@mail.gmail.com>
	<4CCCB3B6.3010209@canterbury.ac.nz>
Message-ID: <AANLkTikQ0k8hwHMJp7UmU0O3N32AJWR_8t5n4mP=_f3A@mail.gmail.com>

On Sat, Oct 30, 2010 at 5:09 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> A different approach to fixing this is for the throwing code to keep
>> throwing EOFError until the generator stops yielding values:
>
> That's precisely what I would recommend.
>
>> This solution doesn't quite work though, because the count returned
>> will include the nodes that were yielded while the stack of generators
>> was winding down.
>>
>> My pragmatic solution for this is to change the
>> protocol so that stopping the generator means that the node yielded
>> last should not be included in the count.
>
> This whole example seems contrived to me, so it's hard to
> say whether this is a good or bad solution.

I tried to come up with an example that made non-trivial use of
yield-from. Iterating over a binary tree is about the most natural
example I could think of. The desire to interrupt the iteration in the
middle also seems natural. I agree that the addition of a return value
for the whole generator is somewhat contrived. Maybe a better value to
return would have been the maximum depth encountered?

>> I propose to
>> modify g.close() to keep throwing GeneratorExit until the generator
>> stops yielding values, and then capture the return value from
>> StopIteration if that is what was raised. The beauty is then that the
>> PEP 380 expansion can stop special-casing GeneratorExit: it just
>> treats it as every other exception.
>
> This was actually suggested during the initial round of
> discussion, and shot down -- if I remember correctly, on the
> grounds that it could result in infinite loops. But if you're
> no longer concerned about that, it's worth considering.
>
> My concern is that this would be a fairly substantial change
> to the intended semantics of close() -- it would no longer be
> a way of aborting a generator and forcing it to clean up as
> quickly as possible.
>
> But maybe you don't mind losing that functionality?

I've thought about this long and hard, and in the end I do mind losing
the special semantics of GeneratorExit. I am withdrawing my proposal.
I don't think the PEP should be weighed down with a separate method +
exception to request a return value either -- rather, a framework that
needs such functionality can create a local convention. I doubt that
the intermingling of frameworks will be such that that is much of a
burden for framework users -- if it is, we can always write a PEP to
standardize such a convention later.

With this, I think PEP 380 can be accepted pretty much as is, although
it would be nice if a part of my example made it into the PEP -- I
don't think there is anything wrong with PEPs having examples.

-- 
--Guido van Rossum (python.org/~guido)


From greg.ewing at canterbury.ac.nz  Tue Nov  2 23:15:50 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 03 Nov 2010 11:15:50 +1300
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <20101102111815.3dbe0ef1@o>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info>
	<AANLkTikbXCx=w-AZOU3LBdBtb7G2xsLtHc2bMWuckevV@mail.gmail.com>
	<4CCF87A6.9010708@pearwood.info> <20101102111815.3dbe0ef1@o>
Message-ID: <4CD08D96.5020803@canterbury.ac.nz>

If you're looking for new function names, I'd suggest a pair
of functions called real_time() and cpu_time().

-- 
Greg



From ncoghlan at gmail.com  Tue Nov  2 23:17:26 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 3 Nov 2010 08:17:26 +1000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CCFDAE6.5070707@egenix.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCFDAE6.5070707@egenix.com>
Message-ID: <AANLkTink0F5miB6x3Sichvs2bTeu3T1pU0O1gC1hriEa@mail.gmail.com>

2010/11/2 M.-A. Lemburg <mal at egenix.com>:
> You might want to look at the systimes.py module that comes with pybench
> (see Tools/pybench). This already provides a cross-platform way of
> accessing high accuracy timers for benchmarking and the like.
>
> For obvious reasons this tries to measure run-time, though, and not
> elapsed time.

I think MAL answered the naming question without even realising it:

time.elapsed_time(start_time=0)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From greg.ewing at canterbury.ac.nz  Tue Nov  2 23:18:38 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 03 Nov 2010 11:18:38 +1300
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <4CD01568.40907@btinternet.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCF6E81.6070203@pearwood.info> <20101102123547.230b50fc@pitrou.net>
	<AANLkTi=YeQ5iw6D2UvqDWh-0DNN_=fBhAxy+TJBooGpa@mail.gmail.com>
	<4CD01568.40907@btinternet.com>
Message-ID: <4CD08E3E.8070206@canterbury.ac.nz>

Rob Cliffe wrote:

> Whereas "timer" is a very familiar concept (in computing, and in life in 
> general).

However, it suggests a factory function that returns some
kind of object that performs timing, rather than simply
returning a time.

-- 
Greg



From fuzzyman at voidspace.org.uk  Wed Nov  3 03:41:04 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 3 Nov 2010 02:41:04 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <AANLkTink0F5miB6x3Sichvs2bTeu3T1pU0O1gC1hriEa@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCFDAE6.5070707@egenix.com>
	<AANLkTink0F5miB6x3Sichvs2bTeu3T1pU0O1gC1hriEa@mail.gmail.com>
Message-ID: <AANLkTi=PCQ=oWMnsRwe1dK=E6iUuoR=6o+ChX1ctTiN4@mail.gmail.com>

On 2 November 2010 22:17, Nick Coghlan <ncoghlan at gmail.com> wrote:

> 2010/11/2 M.-A. Lemburg <mal at egenix.com>:
> > You might want to look at the systimes.py module that comes with pybench
> > (see Tools/pybench). This already provides a cross-platform way of
> > accessing high accuracy timers for benchmarking and the like.
> >
> > For obvious reasons this tries to measure run-time, though, and not
> > elapsed time.
>
> I think MAL answered the naming question without even realising it:
>
> time.elapsed_time(start_time=0)
>


I'm +1 on the functionality and either elapsed_time or wallclock is fine
with me.

Perhaps the patch author should decide...

Michael



>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101103/161686b0/attachment.html>

From python at mrabarnett.plus.com  Wed Nov  3 04:04:03 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 03 Nov 2010 03:04:03 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <AANLkTi=PCQ=oWMnsRwe1dK=E6iUuoR=6o+ChX1ctTiN4@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCFDAE6.5070707@egenix.com>	<AANLkTink0F5miB6x3Sichvs2bTeu3T1pU0O1gC1hriEa@mail.gmail.com>
	<AANLkTi=PCQ=oWMnsRwe1dK=E6iUuoR=6o+ChX1ctTiN4@mail.gmail.com>
Message-ID: <4CD0D123.4000105@mrabarnett.plus.com>

On 03/11/2010 02:41, Michael Foord wrote:
>
>
> On 2 November 2010 22:17, Nick Coghlan <ncoghlan at gmail.com
> <mailto:ncoghlan at gmail.com>> wrote:
>
>     2010/11/2 M.-A. Lemburg <mal at egenix.com <mailto:mal at egenix.com>>:
>      > You might want to look at the systimes.py module that comes with
>     pybench
>      > (see Tools/pybench). This already provides a cross-platform way of
>      > accessing high accuracy timers for benchmarking and the like.
>      >
>      > For obvious reasons this tries to measure run-time, though, and not
>      > elapsed time.
>
>     I think MAL answered the naming question without even realising it:
>
>     time.elapsed_time(start_time=0)
>
>
>
> I'm +1 on the functionality and either elapsed_time or wallclock is fine
> with me.
>
> Perhaps the patch author should decide...
>
And deprive us of the chance to bikeshed? :-)


From kristjan at ccpgames.com  Wed Nov  3 06:26:27 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 3 Nov 2010 13:26:27 +0800
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>

Completely off topic.
What's with factory functions?  Why not a class?  And why are there factory functions in threading.py and not classes?
K

-----Original Message-----
From: python-ideas-bounces+kristjan=ccpgames.com at python.org [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Greg Ewing
Sent: Wednesday, November 03, 2010 6:19
To: python-ideas at python.org
Subject: Re: [Python-ideas] time.wallclock() or other similar stuff

However, it suggests a factory function that returns some kind of object that performs timing, rather than simply returning a time.




From solipsis at pitrou.net  Wed Nov  3 09:58:52 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 3 Nov 2010 09:58:52 +0100
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
References: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>
Message-ID: <20101103095852.66d57199@pitrou.net>

On Wed, 3 Nov 2010 13:26:27 +0800
Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:
> And why are there factory functions in threading.py and not classes?

I'm not sure, but it looks like a poor man's way of discouraging
subclassing.

Regards

Antoine.




From mal at egenix.com  Wed Nov  3 10:13:49 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 03 Nov 2010 10:13:49 +0100
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
In-Reply-To: <20101103095852.66d57199@pitrou.net>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>
	<20101103095852.66d57199@pitrou.net>
Message-ID: <4CD127CD.6010006@egenix.com>

Antoine Pitrou wrote:
> On Wed, 3 Nov 2010 13:26:27 +0800
> Kristj?n Valur J?nsson
> <kristjan at ccpgames.com> wrote:
>> And why are there factory functions in threading.py and not classes?
> 
> I'm not sure, but it looks like a poor man's way of discouraging
> subclassing.

You usually use factory functions if you only define an API that
can then be implemented in many ways.

With a class hierarchy you are always bound to one implementation
and moving from Python to C or providing different specifically
optimized implementations can become cumbersome.

ElementTree is another example that uses factory functions.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From kristjan at ccpgames.com  Wed Nov  3 12:14:56 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 3 Nov 2010 19:14:56 +0800
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
In-Reply-To: <4CD127CD.6010006@egenix.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>
	<20101103095852.66d57199@pitrou.net> <4CD127CD.6010006@egenix.com>
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE576201A3@exchcn.ccp.ad.local>

What is a class but a factory function?
As you explain it, providing a factory function is merely a fancy way of saying: "Look, don't subclass this class, since we might change it."
Wouldn't it be better to be explicit about it?
But even builtin classes are subclassable now, and people rely in internal features of all LIB classes at their peril, so I can't see that this matters.

Cheers,
Kristjan

-----Original Message-----
From: python-ideas-bounces+kristjan=ccpgames.com at python.org [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On Behalf Of M.-A. Lemburg
Sent: Wednesday, November 03, 2010 17:14
To: python-ideas at python.org
Subject: Re: [Python-ideas] Factory functions (was: RE: time.wallclock() or other similar stuff)

Antoine Pitrou wrote:
> On Wed, 3 Nov 2010 13:26:27 +0800
> Kristj?n Valur J?nsson
> <kristjan at ccpgames.com> wrote:
>> And why are there factory functions in threading.py and not classes?
> 
> I'm not sure, but it looks like a poor man's way of discouraging 
> subclassing.

You usually use factory functions if you only define an API that can then be implemented in many ways.

With a class hierarchy you are always bound to one implementation and moving from Python to C or providing different specifically optimized implementations can become cumbersome.



From rob.cliffe at btinternet.com  Wed Nov  3 12:28:51 2010
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Wed, 03 Nov 2010 11:28:51 +0000
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <AANLkTi=PCQ=oWMnsRwe1dK=E6iUuoR=6o+ChX1ctTiN4@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>	<4CCFDAE6.5070707@egenix.com>	<AANLkTink0F5miB6x3Sichvs2bTeu3T1pU0O1gC1hriEa@mail.gmail.com>
	<AANLkTi=PCQ=oWMnsRwe1dK=E6iUuoR=6o+ChX1ctTiN4@mail.gmail.com>
Message-ID: <4CD14773.9080203@btinternet.com>



On 03/11/2010 02:41, Michael Foord wrote:
>
>
> On 2 November 2010 22:17, Nick Coghlan <ncoghlan at gmail.com 
> <mailto:ncoghlan at gmail.com>> wrote:
>
>     2010/11/2 M.-A. Lemburg <mal at egenix.com <mailto:mal at egenix.com>>:
>     > You might want to look at the systimes.py module that comes with
>     pybench
>     > (see Tools/pybench). This already provides a cross-platform way of
>     > accessing high accuracy timers for benchmarking and the like.
>     >
>     > For obvious reasons this tries to measure run-time, though, and not
>     > elapsed time.
>
>     I think MAL answered the naming question without even realising it:
>
>     time.elapsed_time(start_time=0)
>
>
>
> I'm +1 on the functionality and either elapsed_time or wallclock is 
> fine with me.
>
> Perhaps the patch author should decide...
>
> Michael
>
elapsed_time!  Please!  Hate wallclock.
Rob Cliffe
>
>
>     Cheers,
>     Nick.
>
>     --
>     Nick Coghlan   | ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>  
>     |   Brisbane, Australia
>     _______________________________________________
>     Python-ideas mailing list
>     Python-ideas at python.org <mailto:Python-ideas at python.org>
>     http://mail.python.org/mailman/listinfo/python-ideas
>
>
>
>
> -- 
> http://www.voidspace.org.uk
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101103/55595ab1/attachment.html>

From mal at egenix.com  Wed Nov  3 12:44:41 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 03 Nov 2010 12:44:41 +0100
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE576201A3@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>
	<20101103095852.66d57199@pitrou.net> <4CD127CD.6010006@egenix.com>
	<2E034B571A5CE44E949B9FCC3B6D24EE576201A3@exchcn.ccp.ad.local>
Message-ID: <4CD14B29.4020802@egenix.com>

Kristj?n Valur J?nsson wrote:
> What is a class but a factory function?
> As you explain it, providing a factory function is merely a fancy way of saying: "Look, don't subclass this class, since we might change it."
> Wouldn't it be better to be explicit about it?

Isn't using a factory function instead of a class explicit enough ?

> But even builtin classes are subclassable now, and people rely in internal features of all LIB classes at their peril, so I can't see that this matters.

I lost you there.

The key difference is that issubclass() or isinstance() checks
are not possible with factory functions. On that's done on purpose,
since either the implementation cannot guarantee that the used classes
are singletons or it decides on the used implementation depending
on what it finds installed on the system.

> Cheers,
> Kristjan
> 
> -----Original Message-----
> From: python-ideas-bounces+kristjan=ccpgames.com at python.org [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On Behalf Of M.-A. Lemburg
> Sent: Wednesday, November 03, 2010 17:14
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Factory functions (was: RE: time.wallclock() or other similar stuff)
> 
> Antoine Pitrou wrote:
>> On Wed, 3 Nov 2010 13:26:27 +0800
>> Kristj?n Valur J?nsson
>> <kristjan at ccpgames.com> wrote:
>>> And why are there factory functions in threading.py and not classes?
>>
>> I'm not sure, but it looks like a poor man's way of discouraging 
>> subclassing.
> 
> You usually use factory functions if you only define an API that can then be implemented in many ways.
> 
> With a class hierarchy you are always bound to one implementation and moving from Python to C or providing different specifically optimized implementations can become cumbersome.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From guido at python.org  Wed Nov  3 16:08:38 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 3 Nov 2010 08:08:38 -0700
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
In-Reply-To: <20101103095852.66d57199@pitrou.net>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>
	<20101103095852.66d57199@pitrou.net>
Message-ID: <AANLkTim99hMOiiFZJWB+RXyTz1OxeEjU9f-5kpysycaZ@mail.gmail.com>

On Wed, Nov 3, 2010 at 1:58 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 3 Nov 2010 13:26:27 +0800
> Kristj?n Valur J?nsson
> <kristjan at ccpgames.com> wrote:
>> And why are there factory functions in threading.py and not classes?
>
> I'm not sure, but it looks like a poor man's way of discouraging
> subclassing.

Indeed; the intention was that it should be possible to replace the
various lock classes with built-in objects without having to worry
about continuing subclasses. While this was done before built-in
objects could be subclassed, even today I still like to prevent
subclassing the lock classes since the ability to subclass still
constrains the implementation of the built-in type somewhat. I think
it's more important to have the possibility to switch in a faster
platform-specific lock implementation. I do admit that so far this
hasn't happened...

-- 
--Guido van Rossum (python.org/~guido)


From solipsis at pitrou.net  Wed Nov  3 16:19:24 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 03 Nov 2010 16:19:24 +0100
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
In-Reply-To: <AANLkTim99hMOiiFZJWB+RXyTz1OxeEjU9f-5kpysycaZ@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>
	<20101103095852.66d57199@pitrou.net>
	<AANLkTim99hMOiiFZJWB+RXyTz1OxeEjU9f-5kpysycaZ@mail.gmail.com>
Message-ID: <1288797564.5969.12.camel@localhost.localdomain>

Le mercredi 03 novembre 2010 ? 08:08 -0700, Guido van Rossum a ?crit :
> On Wed, Nov 3, 2010 at 1:58 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Wed, 3 Nov 2010 13:26:27 +0800
> > Kristj?n Valur J?nsson
> > <kristjan at ccpgames.com> wrote:
> >> And why are there factory functions in threading.py and not classes?
> >
> > I'm not sure, but it looks like a poor man's way of discouraging
> > subclassing.
> 
> Indeed; the intention was that it should be possible to replace the
> various lock classes with built-in objects without having to worry
> about continuing subclasses. While this was done before built-in
> objects could be subclassed, even today I still like to prevent
> subclassing the lock classes since the ability to subclass still
> constrains the implementation of the built-in type somewhat. I think
> it's more important to have the possibility to switch in a faster
> platform-specific lock implementation. I do admit that so far this
> hasn't happened...

Actually, it has (with both Lock and RLock); but I don't think
subclassing would have been an issue anyway.

Regards

Antoine.




From ncoghlan at gmail.com  Wed Nov  3 23:04:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 4 Nov 2010 08:04:55 +1000
Subject: [Python-ideas] Factory functions (was: RE: time.wallclock() or
 other similar stuff)
In-Reply-To: <AANLkTim99hMOiiFZJWB+RXyTz1OxeEjU9f-5kpysycaZ@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5762010A@exchcn.ccp.ad.local>
	<20101103095852.66d57199@pitrou.net>
	<AANLkTim99hMOiiFZJWB+RXyTz1OxeEjU9f-5kpysycaZ@mail.gmail.com>
Message-ID: <AANLkTinCx8RhwUJw14X9ecPPE7QcNZp_oyNMjwH_epiJ@mail.gmail.com>

On Thu, Nov 4, 2010 at 1:08 AM, Guido van Rossum <guido at python.org> wrote:
> Indeed; the intention was that it should be possible to replace the
> various lock classes with built-in objects without having to worry
> about continuing subclasses. While this was done before built-in
> objects could be subclassed, even today I still like to prevent
> subclassing the lock classes since the ability to subclass still
> constrains the implementation of the built-in type somewhat. I think
> it's more important to have the possibility to switch in a faster
> platform-specific lock implementation. I do admit that so far this
> hasn't happened...

I think the adventures with dict() over the years show pretty clearly
the kind of care the superclass has to take to properly allow
subclassing (i.e. frequently checking the actual class before taking
shortcuts that bypass potential overrides in subclasses). It's not
impossible, obviously, but it does require extra care and attention in
the superclass. Using a factory function instead is a very clear way
to say "no, I'm not making any promises about your ability to inherit
useful functionality from this implementation". Historically, you
could have your cake and eat it too by providing a Mixin class to help
implementation of alternative approaches, as well as the original
factory function. These days, ABCs are probably a better choice for
the niche previously occupied by the Mixin classes.

The cases that mildly irritate me are the factory functions that are
*named* with CamelCase as if they were classes, just because they're
surprising when I find out the name gives the wrong impression of the
metaclass involved (i.e. a function rather than a type).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ddasilva at umd.edu  Fri Nov  5 01:21:37 2010
From: ddasilva at umd.edu (Daniel da Silva)
Date: Thu, 4 Nov 2010 20:21:37 -0400
Subject: [Python-ideas] Multi-line strings that respect indentation
Message-ID: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>

On several occasions I have run into code that will do something like the
following with a multiline string:


def some_func():
>     x, y = process_something()
>
>     val = """
> <xml>
>   <myThing>
>     <val>%s</val>
>     <otherVal>%s</otherVal>
>   </myThing>
>
</xml>
> """ % (x, y)
>
>     return val
>

To me, this is rather ugly because it messes up the indentation of
some_func(). Suppose we could have a multiline string, that when started on
a line indented four spaces, ignores the first four spaces on each line of
the literal when creating the actual string?

In this example, I will use four quotes to start such a string. I think the
syntax for this could vary though. It would be something like this:

def some_func():
>     x, y = process_something()
>
>     val = """"
>     <xml>
>       <myThing>
>         <val>%s</val>
>         <otherVal>%s</otherVal>
>       </myThing>
>     </xml>
>     """" % (x, y)
>
>     return val
>

That way, the indentation in the function would be preserved, making
everything easy to scan, and the indentation in the output would not suffer.


What do you all think?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101104/75ec2be3/attachment.html>

From guido at python.org  Fri Nov  5 01:35:45 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 4 Nov 2010 17:35:45 -0700
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
Message-ID: <AANLkTi=5AF-6YjqO02a82rLdvUGo-pTJtBVQmMDmfyuW@mail.gmail.com>

IIRC Scala has triple-quoted strings with an added feature for this.
We should ask some Scala users if they like it.

On Thu, Nov 4, 2010 at 5:21 PM, Daniel da Silva <ddasilva at umd.edu> wrote:
>
> On several occasions I have run into code that will do something like the
> following with a multiline string:
>
>
>> def some_func():
>> ??? x, y = process_something()
>>
>> ??? val = """
>> <xml>
>> ? <myThing>
>> ??? <val>%s</val>
>> ??? <otherVal>%s</otherVal>
>> ? </myThing>
>>
>> </xml>
>> """ % (x, y)
>>
>> ??? return val
>
> To me, this is rather ugly because it messes up the indentation of
> some_func(). Suppose we could have a multiline string, that when started on
> a line indented four spaces, ignores the first four spaces on each line of
> the literal when creating the actual string?
>
> In this example, I will use four quotes to start such a string. I think the
> syntax for this could vary though. It would be something like this:
>
>> def some_func():
>> ??? x, y = process_something()
>>
>> ??? val = """"
>> ??? <xml>
>> ????? <myThing>
>> ??????? <val>%s</val>
>> ??????? <otherVal>%s</otherVal>
>> ????? </myThing>
>> ??? </xml>
>> ??? """" % (x, y)
>>
>> ??? return val
>
> That way, the indentation in the function would be preserved, making
> everything easy to scan, and the indentation in the output would not suffer.
>
>
> What do you all think?
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
--Guido van Rossum (python.org/~guido)


From anfedorov at gmail.com  Fri Nov  5 02:07:00 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Thu, 4 Nov 2010 21:07:00 -0400
Subject: [Python-ideas] Remapping a table's keys
Message-ID: <AANLkTikgARiLm-yKUpnAf=UhHbUBs4cF66ZbwgCNW93x@mail.gmail.com>

Hi guys,

I often run into the problem when I need to systematically rename a bunch of
keys in a table. For example, a JSON API returns an object where I need to
map "Media Cost" to "adjusted_partner_cost", "Impressions" to "bid_count",
etc. One way of doing this is:

key_map = {
    'Media Cost': 'adjusted_partner_cost',
    'Impressions': 'bids_won',
    'Day': 'date'
}

mykeys = {}
for k, v in dict_from_elsewhere.items():
  if k in key_map:
    ret[key_map[k]] = v
  else:
    ret[k] = dict_from_elsewhere[k]


I would love to be able to instead say something like:

mykeys = defaultdict(lambda x:x, {
    'Media Cost': 'adjusted_partner_cost',
    'Impressions': 'bids_won',
    'Day': 'date'
})


But default functions don't take parameters. Ideas?

- Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101104/4eeba7bc/attachment.html>

From stephen at xemacs.org  Fri Nov  5 02:10:14 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 05 Nov 2010 10:10:14 +0900
Subject: [Python-ideas]  Multi-line strings that respect indentation
In-Reply-To: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
Message-ID: <874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel da Silva writes:

 > To me, this is rather ugly because it messes up the indentation of
 > some_func(). Suppose we could have a multiline string, that when started on
 > a line indented four spaces, ignores the first four spaces on each line of
 > the literal when creating the actual string?

We do.

from textwrap import dedent
def some_func():
    x, y = process_something()

    val = dedent("""\
    <xml>
      <myThing>
        <val>%s</val>
        <otherVal>%s</otherVal>
      </myThing>
    </xml>
    """) % (x, y)

    return val

I don't think the function call is ugly enough to fix with syntax.


From raymond.hettinger at gmail.com  Fri Nov  5 02:18:29 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Thu, 4 Nov 2010 18:18:29 -0700
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
Message-ID: <B55C03C0-35E8-436C-9759-9F1CC56B7846@gmail.com>


On Nov 4, 2010, at 5:21 PM, Daniel da Silva wrote:

> 
> On several occasions I have run into code that will do something like the following with a multiline string:
> 
> 
> def some_func():
>     x, y = process_something()
> 
>     val = """
> <xml>
>   <myThing>
>     <val>%s</val>
>     <otherVal>%s</otherVal>
>   </myThing>
> </xml>
> """ % (x, y)
> 
>     return val
> 
> To me, this is rather ugly because it messes up the indentation of some_func(). Suppose we could have a multiline string, that when started on a line indented four spaces, ignores the first four spaces on each line of the literal when creating the actual string?
> 
> In this example, I will use four quotes to start such a string. I think the syntax for this could vary though. It would be something like this:
> 
> def some_func():
>     x, y = process_something()
>     
>     val = """"
>     <xml>
>       <myThing>
>         <val>%s</val>
>         <otherVal>%s</otherVal>
>       </myThing>
>     </xml>
>     """" % (x, y)
> 
>     return val
> 
> That way, the indentation in the function would be preserved, making everything easy to scan, and the indentation in the output would not suffer.


Why not use textwrap.dedent() to remove the common, leading whitespace:

      return dedent(val)


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101104/5d6a2b98/attachment.html>

From ben+python at benfinney.id.au  Fri Nov  5 02:18:51 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Nov 2010 12:18:51 +1100
Subject: [Python-ideas] Multi-line strings that respect indentation
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
Message-ID: <878w18ftz8.fsf@benfinney.id.au>

Daniel da Silva <ddasilva at umd.edu> writes:

> On several occasions I have run into code that will do something like the
> following with a multiline string:
>
> def some_func():
> >     x, y = process_something()
> >
> >     val = """
> > <xml>
> >   <myThing>
> >     <val>%s</val>
> >     <otherVal>%s</otherVal>
> >   </myThing>
> >
> </xml>
> > """ % (x, y)
> >
> >     return val
> >
>
> To me, this is rather ugly because it messes up the indentation of
> some_func(). Suppose we could have a multiline string, that when
> started on a line indented four spaces, ignores the first four spaces
> on each line of the literal when creating the actual string?

The standard library time machine to the rescue::

    import textwrap

    def some_func():
        x, y = process_something()
    
        val = textwrap.dedent("""
            <xml>
              <myThing>
                <val>%s</val>
                <otherVal>%s</otherVal>
              </myThing>
            
            </xml>
            """) % (x, y)

        return val

I use this technique very often in my code. I'm surprised that it
doesn't have a wider share of Python programmers. Spread the knowledge!

-- 
 \      ?I don't want to live peacefully with difficult realities, and |
  `\     I see no virtue in savoring excuses for avoiding a search for |
_o__)                        real answers.? ?Paul Z. Myers, 2009-09-12 |
Ben Finney



From pyideas at rebertia.com  Fri Nov  5 02:20:20 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Thu, 4 Nov 2010 18:20:20 -0700
Subject: [Python-ideas] Remapping a table's keys
In-Reply-To: <AANLkTikgARiLm-yKUpnAf=UhHbUBs4cF66ZbwgCNW93x@mail.gmail.com>
References: <AANLkTikgARiLm-yKUpnAf=UhHbUBs4cF66ZbwgCNW93x@mail.gmail.com>
Message-ID: <AANLkTinbUrtF_mKAyndn6pPL2=cixFM0Vn5VZMXLXJZb@mail.gmail.com>

On Thu, Nov 4, 2010 at 6:07 PM, Andrey Fedorov <anfedorov at gmail.com> wrote:
> Hi guys,
> I often run into the problem when I need to systematically rename a bunch of
> keys in a table. For example, a JSON API returns an object where I need to
> map "Media Cost" to "adjusted_partner_cost", "Impressions" to "bid_count",
> etc. One way of doing this is:
>
> key_map?= {
> ?? ?'Media Cost': 'adjusted_partner_cost',
> ?? ?'Impressions': 'bids_won',
> ?? ?'Day': 'date'
> }
> mykeys = {}
> for k, v in dict_from_elsewhere.items():
> ??if k in key_map:
> ?? ?ret[key_map[k]] = v
> ??else:
> ?? ?ret[k] =?dict_from_elsewhere[k]

# Requires Python 2.7+ (dict comprehension)
ret = {key_map.get(k, k) : v for k,v in dict_from_elsewhere.items()}

Cheers,
Chris
--
http://blog.rebertia.com


From steve at pearwood.info  Fri Nov  5 02:23:09 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Nov 2010 12:23:09 +1100
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
Message-ID: <4CD35C7D.1080307@pearwood.info>

Daniel da Silva wrote:
> On several occasions I have run into code that will do something like the
> following with a multiline string:
[...]
> To me, this is rather ugly because it messes up the indentation of
> some_func(). Suppose we could have a multiline string, that when started on
> a line indented four spaces, ignores the first four spaces on each line of
> the literal when creating the actual string?
> 
> In this example, I will use four quotes to start such a string. 

Please no. Three quotes is large enough. Also, four quotes currently is 
legal: it is a triple-quoted string that begins with a quotation mark. 
You would be changing that behaviour and likely breaking code.

I don't think we need syntax for this, but if we do, I'd prefer to add a 
prefix similar to the r"" or u"" syntax. Perhaps w"" to normalise 
whitespace?

But as I said, I don't think we need syntax for this. I'd be happy if 
textwrap.dedent() became a built-in string method.

def some_func():
     x, y = process_something()
     val = """
     <xml>
       <myThing>
         <val>%s</val>
         <otherVal>%s</otherVal>
       </myThing>
     </xml>
     """.dedent() % (x, y)
     return val



-- 
Steven



From ddasilva at umd.edu  Fri Nov  5 03:08:01 2010
From: ddasilva at umd.edu (Daniel da Silva)
Date: Thu, 4 Nov 2010 22:08:01 -0400
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <4CD35C7D.1080307@pearwood.info>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<4CD35C7D.1080307@pearwood.info>
Message-ID: <AANLkTinzJudbwCbu7=2dZueXPotLRMG3xzMEL+BZXQXu@mail.gmail.com>

Thanks for all of your replies. I did not know about the dedent() function.
I will use it for now on. If anyone else desires to push this, feel free,
but I am satisfied.

Daniel

On Thu, Nov 4, 2010 at 9:23 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> Daniel da Silva wrote:
>
>> On several occasions I have run into code that will do something like the
>> following with a multiline string:
>>
> [...]
>
>  To me, this is rather ugly because it messes up the indentation of
>> some_func(). Suppose we could have a multiline string, that when started
>> on
>> a line indented four spaces, ignores the first four spaces on each line of
>> the literal when creating the actual string?
>>
>> In this example, I will use four quotes to start such a string.
>>
>
> Please no. Three quotes is large enough. Also, four quotes currently is
> legal: it is a triple-quoted string that begins with a quotation mark. You
> would be changing that behaviour and likely breaking code.
>
> I don't think we need syntax for this, but if we do, I'd prefer to add a
> prefix similar to the r"" or u"" syntax. Perhaps w"" to normalise
> whitespace?
>
> But as I said, I don't think we need syntax for this. I'd be happy if
> textwrap.dedent() became a built-in string method.
>
>
> def some_func():
>    x, y = process_something()
>    val = """
>    <xml>
>      <myThing>
>        <val>%s</val>
>        <otherVal>%s</otherVal>
>      </myThing>
>    </xml>
>    """.dedent() % (x, y)
>    return val
>
>
>
> --
> Steven
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101104/f53f7c1f/attachment.html>

From ben+python at benfinney.id.au  Fri Nov  5 03:10:41 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Nov 2010 13:10:41 +1100
Subject: [Python-ideas] Remapping a table's keys
References: <AANLkTikgARiLm-yKUpnAf=UhHbUBs4cF66ZbwgCNW93x@mail.gmail.com>
	<AANLkTinbUrtF_mKAyndn6pPL2=cixFM0Vn5VZMXLXJZb@mail.gmail.com>
Message-ID: <87zktoed0e.fsf@benfinney.id.au>

Chris Rebert <pyideas at rebertia.com> writes:

> # Requires Python 2.7+ (dict comprehension)
> ret = {key_map.get(k, k) : v for k,v in dict_from_elsewhere.items()}

If using a Python without dict comprehensions but with generator
expressions::

    ret = dict(
        (key_map.get(k, k), v)
        for (k, v) in dict_from_elsewhere.items())

-- 
 \       ?Pinky, are you pondering what I'm pondering?? ?Well, I think |
  `\     so, Brain, but do I really need two tongues?? ?_Pinky and The |
_o__)                                                           Brain_ |
Ben Finney



From denis.spir at gmail.com  Fri Nov  5 07:29:24 2010
From: denis.spir at gmail.com (spir)
Date: Fri, 5 Nov 2010 07:29:24 +0100
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
Message-ID: <20101105072924.0b099819@o>

On Thu, 4 Nov 2010 20:21:37 -0400
Daniel da Silva <ddasilva at umd.edu> wrote:

> On several occasions I have run into code that will do something like the
> following with a multiline string:
> 
> 
> def some_func():
> >     x, y = process_something()
> >
> >     val = """
> > <xml>
> >   <myThing>
> >     <val>%s</val>
> >     <otherVal>%s</otherVal>
> >   </myThing>
> >
> </xml>
> > """ % (x, y)
> >
> >     return val
> >
> 
> To me, this is rather ugly because it messes up the indentation of
> some_func(). Suppose we could have a multiline string, that when started on
> a line indented four spaces, ignores the first four spaces on each line of
> the literal when creating the actual string?
> 
> In this example, I will use four quotes to start such a string. I think the
> syntax for this could vary though. It would be something like this:
> 
> def some_func():
> >     x, y = process_something()
> >
> >     val = """"
> >     <xml>
> >       <myThing>
> >         <val>%s</val>
> >         <otherVal>%s</otherVal>
> >       </myThing>
> >     </xml>
> >     """" % (x, y)
> >
> >     return val
> >
> 
> That way, the indentation in the function would be preserved, making
> everything easy to scan, and the indentation in the output would not suffer.
> 
> 
> What do you all think?

I'm +++ for this. Even more in python because one has no choice about indentation. But I posted the same proposal some time ago (together with a note that there is no need for """...""" around multiline strings), and it had no apparent success.


Denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From denis.spir at gmail.com  Fri Nov  5 07:36:36 2010
From: denis.spir at gmail.com (spir)
Date: Fri, 5 Nov 2010 07:36:36 +0100
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20101105073636.40cb6727@o>

On Fri, 05 Nov 2010 10:10:14 +0900
"Stephen J. Turnbull" <stephen at xemacs.org> wrote:

> Daniel da Silva writes:
> 
>  > To me, this is rather ugly because it messes up the indentation of
>  > some_func(). Suppose we could have a multiline string, that when started on
>  > a line indented four spaces, ignores the first four spaces on each line of
>  > the literal when creating the actual string?
> 
> We do.
> 
> from textwrap import dedent
> def some_func():
>     x, y = process_something()
> 
>     val = dedent("""\
>     <xml>
>       <myThing>
>         <val>%s</val>
>         <otherVal>%s</otherVal>
>       </myThing>
>     </xml>
>     """) % (x, y)
> 
>     return val
> 
> I don't think the function call is ugly enough to fix with syntax.

It's just reversing the point of view. The right thing would be to have Python code correctly aligned on indentation _by default_. And possibly provide an alternative for cases (which?) where one would want indent not taken into account.

Denis

-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From denis.spir at gmail.com  Fri Nov  5 07:42:26 2010
From: denis.spir at gmail.com (spir)
Date: Fri, 5 Nov 2010 07:42:26 +0100
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <4CD35C7D.1080307@pearwood.info>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<4CD35C7D.1080307@pearwood.info>
Message-ID: <20101105074226.741734f3@o>

On Fri, 05 Nov 2010 12:23:09 +1100
Steven D'Aprano <steve at pearwood.info> wrote:

> Daniel da Silva wrote:
> > On several occasions I have run into code that will do something like the
> > following with a multiline string:
> [...]
> > To me, this is rather ugly because it messes up the indentation of
> > some_func(). Suppose we could have a multiline string, that when started on
> > a line indented four spaces, ignores the first four spaces on each line of
> > the literal when creating the actual string?
> > 
> > In this example, I will use four quotes to start such a string. 
> 
> Please no. Three quotes is large enough. Also, four quotes currently is 
> legal: it is a triple-quoted string that begins with a quotation mark. 
> You would be changing that behaviour and likely breaking code.
> 
> I don't think we need syntax for this, but if we do, I'd prefer to add a 
> prefix similar to the r"" or u"" syntax. Perhaps w"" to normalise 
> whitespace?
> 
> But as I said, I don't think we need syntax for this. I'd be happy if 
> textwrap.dedent() became a built-in string method.
> 
> def some_func():
>      x, y = process_something()
>      val = """
>      <xml>
>        <myThing>
>          <val>%s</val>
>          <otherVal>%s</otherVal>
>        </myThing>
>      </xml>
>      """.dedent() % (x, y)
>      return val

Yes, that would be a good workaround. (Does dedent() base it's behaviour on first (non-empty) line?)

Denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From taleinat at gmail.com  Fri Nov  5 09:39:23 2010
From: taleinat at gmail.com (Tal Einat)
Date: Fri, 5 Nov 2010 10:39:23 +0200
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
Message-ID: <AANLkTinOpHeXuwY8j83BK8t8i=sXcdJ1=xGq0zwq7qwc@mail.gmail.com>

Daniel da Silva wrote:
>
> On several occasions I have run into code that will do something like the
> following with a multiline string:
>
>
>> def some_func():
>> ??? x, y = process_something()
>>
>> ??? val = """
>> <xml>
>> ? <myThing>
>> ??? <val>%s</val>
>> ??? <otherVal>%s</otherVal>
>> ? </myThing>
>>
>> </xml>
>> """ % (x, y)
>>
>> ??? return val
>
> To me, this is rather ugly because it messes up the indentation of
> some_func(). Suppose we could have a multiline string, that when started on
> a line indented four spaces, ignores the first four spaces on each line of
> the literal when creating the actual string?
>
> In this example, I will use four quotes to start such a string. I think the
> syntax for this could vary though. It would be something like this:
>
>> def some_func():
>> ??? x, y = process_something()
>>
>> ??? val = """"
>> ??? <xml>
>> ????? <myThing>
>> ??????? <val>%s</val>
>> ??????? <otherVal>%s</otherVal>
>> ????? </myThing>
>> ??? </xml>
>> ??? """" % (x, y)
>>
>> ??? return val
>
> That way, the indentation in the function would be preserved, making
> everything easy to scan, and the indentation in the output would not suffer.
>
>
> What do you all think?

+1 since this would mean Python would also dedent multi-line
doc-strings automatically. I really have having doc-strings indented
according to their indentation in the code.

- Tal


From ncoghlan at gmail.com  Fri Nov  5 15:45:56 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 6 Nov 2010 00:45:56 +1000
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>

On Fri, Nov 5, 2010 at 11:10 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> ?> To me, this is rather ugly because it messes up the indentation of
> ?> some_func(). Suppose we could have a multiline string, that when started on
> ?> a line indented four spaces, ignores the first four spaces on each line of
> ?> the literal when creating the actual string?
> I don't think the function call is ugly enough to fix with syntax.

I do use the textwrap.dedent workaround myself, but I think it is
sufficiently flawed for a proper fix to be worth considering:

1. It doesn't work for docstrings (as Tal pointed out)
2. It postpones until runtime an operation that could fairly easily be
carried out at compile time instead

Note that a method on str objects fixes none of those problems, so
isn't much of a gain from this point of view. As new string prefix
would handle the task nicely, though. I personally like "d for
dedent", with all d-strings (even single-quoted ones) being implicitly
multiline as the colour for that particular bikeshed:

def some_func():
   x, y = process_something()

   val = d"\
   <xml>
     <myThing>
       <val>%s</val>
       <otherVal>%s</otherVal>
     </myThing>
   </xml>
   ") % (x, y)

   return val

I'm no more than +0 on the idea though. It strikes me as an awful lot
of effort in implementing, documenting and promoting the idea for
something that provides at best a minimal improvement in aesthetics
and performance.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From stephen at xemacs.org  Fri Nov  5 15:54:59 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 05 Nov 2010 23:54:59 +0900
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <20101105073636.40cb6727@o>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101105073636.40cb6727@o>
Message-ID: <87vd4bhlbw.fsf@uwakimon.sk.tsukuba.ac.jp>

spir writes:

 > It's just reversing the point of view. The right thing would be to
 > have Python code correctly aligned on indentation _by default_.

This is plausible but I don't necessarily agree that it's the right
thing.

If I'm writing a program which writes structured text like XML, which
is conventionally indented, I actually prefer

    print("""\
<xml>
  A really uninteresting XML document.
</xml>
""")

to

    print("""\
          <xml>
            A really uninteresting XML document.
          </xml>
          """)

For one thing, in many realistic cases it's probably actually
formatted like this:

    print("""\
<xml>
""")
    print("""\
  A really uninteresting XML document.
""")
    print("""\
</xml>
""")

and it would be a real PITA trying to get that right.



From stephen at xemacs.org  Fri Nov  5 16:00:47 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 06 Nov 2010 00:00:47 +0900
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <20101105074226.741734f3@o>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<4CD35C7D.1080307@pearwood.info> <20101105074226.741734f3@o>
Message-ID: <87tyjvhl28.fsf@uwakimon.sk.tsukuba.ac.jp>

spir writes:

 > (Does dedent() base it's behaviour on first (non-empty) line?)

No, it chooses the longest common whitespace prefix in the whole
string.

I'm not sure what it does with blank lines.  A quick test ... in
Python 2.5.5, it ignores them for the purpose of determining the
dedent.  That is, you can have any amount of whitespace on a blank
line, but it doesn't affect the dedent computed.




From tjreedy at udel.edu  Fri Nov  5 21:37:47 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 05 Nov 2010 16:37:47 -0400
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
Message-ID: <ib1pup$4ku$1@dough.gmane.org>

On 11/5/2010 10:45 AM, Nick Coghlan wrote:

> I do use the textwrap.dedent workaround myself, but I think it is
> sufficiently flawed for a proper fix to be worth considering:
>
> 1. It doesn't work for docstrings (as Tal pointed out)

This does:

def f(x):
     "Am I a docstring\n"\
     "even though I start in pieces?\n"\
     "Oh, x is a dummy param\n"
     pass
print(f.__doc__)

# prints 3 lines, but not without '\' escape at line ends

> 2. It postpones until runtime an operation that could fairly easily be
> carried out at compile time instead

Above is. Not that I like the extra fluff needed to make it work.
It makes a 'd' for dedent literal prefix look inviting.

-- 
Terry Jan Reedy



From cmjohnson.mailinglist at gmail.com  Sat Nov  6 00:48:49 2010
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Fri, 5 Nov 2010 13:48:49 -1000
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <ib1pup$4ku$1@dough.gmane.org>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
	<ib1pup$4ku$1@dough.gmane.org>
Message-ID: <AANLkTi=6oOA0YxsyKExGaBGjSoXzF6Kq4RwN1VWgeyv0@mail.gmail.com>

On Fri, Nov 5, 2010 at 10:37 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 11/5/2010 10:45 AM, Nick Coghlan wrote:
>
>> I do use the textwrap.dedent workaround myself, but I think it is
>> sufficiently flawed for a proper fix to be worth considering:
>>
>> 1. It doesn't work for docstrings (as Tal pointed out)
>
> This does:
>
> def f(x):
> ? ?"Am I a docstring\n"\
> ? ?"even though I start in pieces?\n"\
> ? ?"Oh, x is a dummy param\n"
> ? ?pass
> print(f.__doc__)
>
> # prints 3 lines, but not without '\' escape at line ends

Can the parser be changed to do automagic joining on that the same way
that it auto joins ("a"

"b")? I can't imagine any scenario where anyone would rely on starting
the line after a docstring with a string literal, since the only
plausible run time effect it might have would be to intern some
strings for later. Even "string".method() wouldn't have any effect
without something like x = in front of.

-- Carl Johnson


From eric at trueblade.com  Sat Nov  6 00:55:56 2010
From: eric at trueblade.com (Eric Smith)
Date: Fri, 05 Nov 2010 19:55:56 -0400
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTi=6oOA0YxsyKExGaBGjSoXzF6Kq4RwN1VWgeyv0@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>	<ib1pup$4ku$1@dough.gmane.org>
	<AANLkTi=6oOA0YxsyKExGaBGjSoXzF6Kq4RwN1VWgeyv0@mail.gmail.com>
Message-ID: <4CD4998C.2000904@trueblade.com>

On 11/5/10 7:48 PM, Carl M. Johnson wrote:
> On Fri, Nov 5, 2010 at 10:37 AM, Terry Reedy<tjreedy at udel.edu>  wrote:
>> On 11/5/2010 10:45 AM, Nick Coghlan wrote:
>>
>>> I do use the textwrap.dedent workaround myself, but I think it is
>>> sufficiently flawed for a proper fix to be worth considering:
>>>
>>> 1. It doesn't work for docstrings (as Tal pointed out)
>>
>> This does:
>>
>> def f(x):
>>     "Am I a docstring\n"\
>>     "even though I start in pieces?\n"\
>>     "Oh, x is a dummy param\n"
>>     pass
>> print(f.__doc__)
>>
>> # prints 3 lines, but not without '\' escape at line ends
>
> Can the parser be changed to do automagic joining on that the same way
> that it auto joins ("a"
>
> "b")? I can't imagine any scenario where anyone would rely on starting
> the line after a docstring with a string literal, since the only
> plausible run time effect it might have would be to intern some
> strings for later. Even "string".method() wouldn't have any effect
> without something like x = in front of.

 >>> def f(x):
...     ("Am I a docstring\n"
...      "even though I start in pieces?\n"
...      "Oh, x is a dummy param\n")
...     pass
...
 >>> print(f.__doc__)
Am I a docstring
even though I start in pieces?
Oh, x is a dummy param

 >>>

-- 
Eric.


From ben+python at benfinney.id.au  Sat Nov  6 02:13:32 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 06 Nov 2010 12:13:32 +1100
Subject: [Python-ideas] Multi-line strings that respect indentation
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
	<ib1pup$4ku$1@dough.gmane.org>
	<AANLkTi=6oOA0YxsyKExGaBGjSoXzF6Kq4RwN1VWgeyv0@mail.gmail.com>
Message-ID: <87hbfvdzk3.fsf@benfinney.id.au>

"Carl M. Johnson"
<cmjohnson.mailinglist at gmail.com> writes:

> Can the parser be changed to do automagic joining on that the same way
> that it auto joins ("a"
>
> "b")?

There's no need to change the parser, when what you suggest works fine
in existing Python *and* is easier to maintain without the trailing
backslashes:

    def f(x):
        ("Am I a docstring\n"
         "even though I start in pieces?\n"
         "Oh, x is a dummy param\n")
        pass
    print(f.__doc__)

But that's still crap. I think the existing situation works too: the
existing docstring processors already know to handle docstrings
according to <URL:http://www.python.org/dev/peps/pep-0257/#id20>.

Introducing more workarounds for multi-line strings, when the existing
tools appear to work fine, gets a -0.5 from me.

-- 
 \      ?Fox News gives you both sides of every story: the President?s |
  `\  side and the Vice President?s side.? ?Steven Colbert, 2006-04-29 |
_o__)                                                                  |
Ben Finney



From jared.grubb at gmail.com  Fri Nov  5 21:43:35 2010
From: jared.grubb at gmail.com (Jared Grubb)
Date: Fri, 5 Nov 2010 13:43:35 -0700
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
Message-ID: <67D19981-77AF-4890-BA1D-E437DC43D2C5@gmail.com>

I like the idea of have d""" as the marker, as I've often wanted this for writing out custom HTML or source code.

But, would the level of dedent be based on the level of indent of the code line containing the d""" marker? If so, would you want a parsing/syntax error if the string violated the indentation levels of the surrounding code? (the dedent would have to take place post-parsing, since the grammar knows nothing about INDENT tokens inside strings, so it's not truly a "parser" error).

The easiest would be to do the equivalent of dedent and take the longest common whitespace length. But, the purist in me would kinda hope that if, for example, the code was indented to column 8 but the text was all at column 12, that the resulting string would have 4 leading spaces on each line.

Jared

On 5 Nov 2010, at 07:45, Nick Coghlan wrote:

> On Fri, Nov 5, 2010 at 11:10 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>>  > To me, this is rather ugly because it messes up the indentation of
>>  > some_func(). Suppose we could have a multiline string, that when started on
>>  > a line indented four spaces, ignores the first four spaces on each line of
>>  > the literal when creating the actual string?
>> I don't think the function call is ugly enough to fix with syntax.
> 
> I do use the textwrap.dedent workaround myself, but I think it is
> sufficiently flawed for a proper fix to be worth considering:
> 
> 1. It doesn't work for docstrings (as Tal pointed out)
> 2. It postpones until runtime an operation that could fairly easily be
> carried out at compile time instead
> 
> Note that a method on str objects fixes none of those problems, so
> isn't much of a gain from this point of view. As new string prefix
> would handle the task nicely, though. I personally like "d for
> dedent", with all d-strings (even single-quoted ones) being implicitly
> multiline as the colour for that particular bikeshed:
> 
> def some_func():
>   x, y = process_something()
> 
>   val = d"\
>   <xml>
>     <myThing>
>       <val>%s</val>
>       <otherVal>%s</otherVal>
>     </myThing>
>   </xml>
>   ") % (x, y)
> 
>   return val
> 
> I'm no more than +0 on the idea though. It strikes me as an awful lot
> of effort in implementing, documenting and promoting the idea for
> something that provides at best a minimal improvement in aesthetics
> and performance.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From python at mrabarnett.plus.com  Sat Nov  6 19:35:07 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 06 Nov 2010 18:35:07 +0000
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <67D19981-77AF-4890-BA1D-E437DC43D2C5@gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
	<67D19981-77AF-4890-BA1D-E437DC43D2C5@gmail.com>
Message-ID: <4CD59FDB.4070007@mrabarnett.plus.com>

On 05/11/2010 20:43, Jared Grubb wrote:
> I like the idea of have d""" as the marker, as I've often wanted this
> for writing out custom HTML or source code.
>
> But, would the level of dedent be based on the level of indent of the
> code line containing the d""" marker? If so, would you want a
> parsing/syntax error if the string violated the indentation levels of
> the surrounding code? (the dedent would have to take place
> post-parsing, since the grammar knows nothing about INDENT tokens
> inside strings, so it's not truly a "parser" error).
>
> The easiest would be to do the equivalent of dedent and take the
> longest common whitespace length. But, the purist in me would kinda
> hope that if, for example, the code was indented to column 8 but the
> text was all at column 12, that the resulting string would have 4
> leading spaces on each line.
>
[snip]
You could also add an 'indent' method to strings:

text.indent(4) would indent each line by 4 spaces

text.indent(-4) would dedent each line by 4 spaces; an exception would
be raised if a line started with fewer than 4 spaces, unless that line
contained only spaces.

text.indent(4, '>') would indent each line by 4 * '>'.


From stephen at xemacs.org  Sun Nov  7 09:58:16 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 07 Nov 2010 17:58:16 +0900
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <67D19981-77AF-4890-BA1D-E437DC43D2C5@gmail.com>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimrgD-rcMXeBADcFPiyhQE3dur9nv1KhMgdhxhS@mail.gmail.com>
	<67D19981-77AF-4890-BA1D-E437DC43D2C5@gmail.com>
Message-ID: <87lj55h5nb.fsf@uwakimon.sk.tsukuba.ac.jp>

Jared Grubb writes:

 > The easiest would be to do the equivalent of dedent and take the
 > longest common whitespace length. But, the purist in me would kinda
 > hope that if, for example, the code was indented to column 8 but
 > the text was all at column 12, that the resulting string would have
 > 4 leading spaces on each line.

-1

You could probably come up with rules to get this right for the
parser, but how can it possibly guess exactly what any given user
means by

    def amethod (self):
        self.bmethod_has_fleece_as_white_as_snow("""\
            Mary had a little lamb
            little lamb
            little lamb
            The lamb may be a little small
            But this poem is just plain lame.""")

Is that supposed to be flush left, indented 4 spaces, indented 8
spaces, or indented 12 spaces?

I think the dedent rule (longest common whitespace prefix) makes the
most sense.


From greg at krypto.org  Sun Nov  7 19:48:22 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 7 Nov 2010 10:48:22 -0800
Subject: [Python-ideas] Multi-line strings that respect indentation
In-Reply-To: <874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=jBxAB32oCgjszUmBU-nZ4GXw=mQzBE4YOpV9Y@mail.gmail.com>
	<874obwinih.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <AANLkTi=_j7VS3G18uHQ4We5m4qi+DPh-=jG_rxW8Dx38@mail.gmail.com>

I would never use a quadruple quote for this if it were ever implemented.  A
better precedent would be a letter prefix to the quotes similar to what we
do for raw strings and bytes constants today.  m""" perhaps.

I'm -0.5 on this.

I don't think it is a necessary feature thanks to the textwrap.dedent.  But
the reason I'm not -1 is that using dedent incurs runtime cost to process
the larger string constant to generate the desired one.  The best way around
this is to declare your large string constants at the module level where you
don't need to worry about dedent at all.

-gps

On Thu, Nov 4, 2010 at 6:10 PM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> Daniel da Silva writes:
>
>  > To me, this is rather ugly because it messes up the indentation of
>  > some_func(). Suppose we could have a multiline string, that when started
> on
>  > a line indented four spaces, ignores the first four spaces on each line
> of
>  > the literal when creating the actual string?
>
> We do.
>
> from textwrap import dedent
> def some_func():
>    x, y = process_something()
>
>     val = dedent("""\
>     <xml>
>      <myThing>
>        <val>%s</val>
>        <otherVal>%s</otherVal>
>      </myThing>
>    </xml>
>    """) % (x, y)
>
>    return val
>
> I don't think the function call is ugly enough to fix with syntax.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101107/5f1359b6/attachment.html>

From debatem1 at gmail.com  Mon Nov  8 21:10:36 2010
From: debatem1 at gmail.com (geremy condra)
Date: Mon, 8 Nov 2010 15:10:36 -0500
Subject: [Python-ideas] time.wallclock() or other similar stuff
In-Reply-To: <AANLkTink0F5miB6x3Sichvs2bTeu3T1pU0O1gC1hriEa@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FF54@exchcn.ccp.ad.local>
	<4CCFDAE6.5070707@egenix.com>
	<AANLkTink0F5miB6x3Sichvs2bTeu3T1pU0O1gC1hriEa@mail.gmail.com>
Message-ID: <AANLkTi=7OTDYFmQfptgXAoHrFuPU6Jdj5rL8fDOdSmzr@mail.gmail.com>

On Tue, Nov 2, 2010 at 6:17 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 2010/11/2 M.-A. Lemburg <mal at egenix.com>:
>> You might want to look at the systimes.py module that comes with pybench
>> (see Tools/pybench). This already provides a cross-platform way of
>> accessing high accuracy timers for benchmarking and the like.
>>
>> For obvious reasons this tries to measure run-time, though, and not
>> elapsed time.
>
> I think MAL answered the naming question without even realising it:
>
> time.elapsed_time(start_time=0)

+1 from me.

Geremy Condra


From kristjan at ccpgames.com  Wed Nov 10 03:52:00 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 10 Nov 2010 10:52:00 +0800
Subject: [Python-ideas] pool threads
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>

Hi there.
Occasionally you want to do something on a worker thread from Python.  Some operating systems, like Windows, maintain a threadpool for which you can queue a user work item to execute.  I was thinking that it could be a good idea to expose such a functionality by the thread module.

With something  like:

def thread.queue_call(callable, args, kwargs):
#invoke the OS threadpool api and have callable called


Then in threading.py:
Try:
  From thread import queue_call
Except ImportError:
  Def queue_call(callable, args, kwargs):
    #default implementation
Thread(target=callable, args=args, kwargs=kwargs).start()


Having operating system support for this would free us from creating a new real thread every time such a pattern is invoked.  It would also improve the latency of such a call since a thread creation isn't free.

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101110/7eeb9cdb/attachment.html>

From guido at python.org  Wed Nov 10 05:07:51 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 9 Nov 2010 20:07:51 -0800
Subject: [Python-ideas] pool threads
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>
Message-ID: <AANLkTi=PuSON1=2PhFanS=_Cyyvn9rLNhD6d8vat5Z3b@mail.gmail.com>

Have you looked at PEP 3148?

2010/11/9 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> Hi there.
>
> Occasionally you want to do something on a worker thread from Python. ?Some
> operating systems, like Windows, maintain a threadpool for which you can
> queue a user work item to execute.? I was thinking that it could be a good
> idea to expose such a functionality by the thread module.
>
>
>
> With something ?like:
>
>
>
> def thread.queue_call(callable, args, kwargs):
>
> #invoke the OS threadpool api and have callable called
>
>
>
>
>
> Then in threading.py:
>
> Try:
>
> ? From thread import queue_call
>
> Except ImportError:
>
> ? Def queue_call(callable, args, kwargs):
>
> ??? #default implementation
>
> Thread(target=callable, args=args, kwargs=kwargs).start()
>
>
>
>
>
> Having operating system support for this would free us from creating a new
> real thread every time such a pattern is invoked.? It would also improve the
> latency of such a call since a thread creation isn?t free.
>
>
>
> K
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
--Guido van Rossum (python.org/~guido)


From kristjan at ccpgames.com  Wed Nov 10 05:14:27 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 10 Nov 2010 12:14:27 +0800
Subject: [Python-ideas] pool threads
In-Reply-To: <AANLkTi=PuSON1=2PhFanS=_Cyyvn9rLNhD6d8vat5Z3b@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>
	<AANLkTi=PuSON1=2PhFanS=_Cyyvn9rLNhD6d8vat5Z3b@mail.gmail.com>
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE57872370@exchcn.ccp.ad.local>

Ah, now that you mention it, I stumbled across this last PyCon :)
Looks nice, particularly if we can have classes such as ThreadPoolExecutor rely on OS support to do their stuff.  As such, it might be good not to make any promises about max_workers and so on.  Well, on windows one can have the OS create private thread pools to your specification, but I have found it useful to just rely on the default one...
Anyway, these were just idle thoughts.  Let's carry on...

K



-----Original Message-----
From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum
Sent: Wednesday, November 10, 2010 12:08
To: Kristj?n Valur J?nsson
Cc: python-ideas at python.org
Subject: Re: [Python-ideas] pool threads

Have you looked at PEP 3148?



From jason.orendorff at gmail.com  Wed Nov 10 05:37:23 2010
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Tue, 9 Nov 2010 22:37:23 -0600
Subject: [Python-ideas] pool threads
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>
Message-ID: <AANLkTinr6WC5GQmkgPdS9iHjrJ+r_tJiASg7t+ZDHs6z@mail.gmail.com>

2010/11/9 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> Occasionally you want to do something on a worker thread from Python. ?Some
> operating systems, like Windows, maintain a threadpool for which you can
> queue a user work item to execute.? I was thinking that it could be a good
> idea to expose such a functionality by the thread module.

multiprocessing.dummy provides a thread pool class. So another way to
implement queue_call would be:

    from multiprocessing.dummy import Pool
    _mypool = Pool()
    queue_call = _mypool.apply_async

-j


From solipsis at pitrou.net  Wed Nov 10 09:38:00 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 10 Nov 2010 09:38:00 +0100
Subject: [Python-ideas] pool threads
References: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>
	<AANLkTi=PuSON1=2PhFanS=_Cyyvn9rLNhD6d8vat5Z3b@mail.gmail.com>
	<2E034B571A5CE44E949B9FCC3B6D24EE57872370@exchcn.ccp.ad.local>
Message-ID: <20101110093800.51457a01@pitrou.net>

On Wed, 10 Nov 2010 12:14:27 +0800
Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:

> Ah, now that you mention it, I stumbled across this last PyCon :)
> Looks nice, particularly if we can have classes such as
> ThreadPoolExecutor rely on OS support to do their stuff.

What does it bring?





From solipsis at pitrou.net  Thu Nov 11 00:45:42 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 11 Nov 2010 00:45:42 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
	hierarchy (again)
Message-ID: <20101111004542.1761633e@pitrou.net>


Hello,

Georg (who manages the 3.2 release) suggested that I push PEP 3151 (*)
in the hope that it can get in before the 3.2 feature freeze.

Is there any consensus that PEP 3151 should get in 3.2, or even that it
should get in at all?  If there's some process issue that I'm missing,
I'd be glad to know about it.

(*) http://www.python.org/dev/peps/pep-3151/

Thanks

Antoine.





From mal at egenix.com  Thu Nov 11 10:09:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 11 Nov 2010 10:09:34 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <20101111004542.1761633e@pitrou.net>
References: <20101111004542.1761633e@pitrou.net>
Message-ID: <4CDBB2CE.4020004@egenix.com>

Antoine Pitrou wrote:
> 
> Hello,
> 
> Georg (who manages the 3.2 release) suggested that I push PEP 3151 (*)
> in the hope that it can get in before the 3.2 feature freeze.
> 
> Is there any consensus that PEP 3151 should get in 3.2, or even that it
> should get in at all?  If there's some process issue that I'm missing,
> I'd be glad to know about it.
> 
> (*) http://www.python.org/dev/peps/pep-3151/

In its current state, I don't think it should go in 3.2:

There are a couple of cases where you remove more error classes
which will cause existing code to catch more errors than what
the code was designed for, e.g. catching socket.error will
now also catch all IOErrors, code catching OSErrors will now
also catch IOErrors.

By making the scope of an except clause larger, you alter the
semantics of programs using the more fine-grained hierarchies
in ways which are both difficult to detect and harder to work
around.

Since user exceptions often inherit from the stdlib ones,
the situation propogates to user exceptions as well, so
even finding all the places in the code that must be
reviewed gets difficult.

OTOH, the removal of the more fine-grained classes
doesn't gain us anything, e.g. it's pretty obvious that
the os module raises OSErrors, the socket module socket.errors,
the mmap module mmap.errors and so on.

Please also have a look at http://www.python.org/dev/peps/pep-0348/
since some of the things you raise in your PEP were already discussed
and decided a while back, including whether removal of exceptions
and addition of new, more fine-grained ones

I'm +1 on adding more fine-grained error classes, but don't
see much use in removing existing fine-grained structures.

BTW: Why are you discussing the PEP on python-ideas and not
on python-dev where PEPs normally go for further discussion
after they been assigned a PEP number (see
http://www.python.org/dev/peps/pep-0001/) ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Thu Nov 11 13:03:36 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 11 Nov 2010 22:03:36 +1000
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <4CDBB2CE.4020004@egenix.com>
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
Message-ID: <AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>

On Thu, Nov 11, 2010 at 7:09 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Antoine Pitrou wrote:
>>
>> Hello,
>>
>> Georg (who manages the 3.2 release) suggested that I push PEP 3151 (*)
>> in the hope that it can get in before the 3.2 feature freeze.
>>
>> Is there any consensus that PEP 3151 should get in 3.2, or even that it
>> should get in at all? ?If there's some process issue that I'm missing,
>> I'd be glad to know about it.
>>
>> (*) http://www.python.org/dev/peps/pep-3151/
>
> In its current state, I don't think it should go in 3.2:
>
> There are a couple of cases where you remove more error classes
> which will cause existing code to catch more errors than what
> the code was designed for, e.g. catching socket.error will
> now also catch all IOErrors, code catching OSErrors will now
> also catch IOErrors.
>
> By making the scope of an except clause larger, you alter the
> semantics of programs using the more fine-grained hierarchies
> in ways which are both difficult to detect and harder to work
> around.

This issue is addressed directly in the PEP by the "Compatibility
Strategy" section. Code which checks errno will not suddenly start
catching additional exceptions due to the merger of IOError and
OSError.

However, I agree with you that the PEP would benefit from keeping
socket.error, select.error and mmap.error as separate IOError
subclasses. Existing code may legitimately be catching those in try
blocks that include other unrelated operations that may raise IOError.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From g.brandl at gmx.net  Thu Nov 11 13:40:02 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 11 Nov 2010 13:40:02 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
	hierarchy (again)
In-Reply-To: <AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
Message-ID: <ibgo9k$3jg$1@dough.gmane.org>

Am 11.11.2010 13:03, schrieb Nick Coghlan:
> On Thu, Nov 11, 2010 at 7:09 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Antoine Pitrou wrote:
>>>
>>> Hello,
>>>
>>> Georg (who manages the 3.2 release) suggested that I push PEP 3151 (*)
>>> in the hope that it can get in before the 3.2 feature freeze.
>>>
>>> Is there any consensus that PEP 3151 should get in 3.2, or even that it
>>> should get in at all?  If there's some process issue that I'm missing,
>>> I'd be glad to know about it.
>>>
>>> (*) http://www.python.org/dev/peps/pep-3151/
>>
>> In its current state, I don't think it should go in 3.2:
>>
>> There are a couple of cases where you remove more error classes
>> which will cause existing code to catch more errors than what
>> the code was designed for, e.g. catching socket.error will
>> now also catch all IOErrors, code catching OSErrors will now
>> also catch IOErrors.
>>
>> By making the scope of an except clause larger, you alter the
>> semantics of programs using the more fine-grained hierarchies
>> in ways which are both difficult to detect and harder to work
>> around.
> 
> This issue is addressed directly in the PEP by the "Compatibility
> Strategy" section. Code which checks errno will not suddenly start
> catching additional exceptions due to the merger of IOError and
> OSError.
> 
> However, I agree with you that the PEP would benefit from keeping
> socket.error, select.error and mmap.error as separate IOError
> subclasses. Existing code may legitimately be catching those in try
> blocks that include other unrelated operations that may raise IOError.

Agreed.  Let's keep the potential incompatibilities and surprises to a
minimum.  After all, behavior changes in this area will only show up
in the exceptional case, which depending on the context will not be
triggered frequently, and its test coverage is often low due to the
relative difficulty of mocking these things.

cheers,
Georg



From solipsis at pitrou.net  Thu Nov 11 14:08:30 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 11 Nov 2010 14:08:30 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
	hierarchy (again)
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
Message-ID: <20101111140830.1825a836@pitrou.net>

On Thu, 11 Nov 2010 22:03:36 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> This issue is addressed directly in the PEP by the "Compatibility
> Strategy" section. Code which checks errno will not suddenly start
> catching additional exceptions due to the merger of IOError and
> OSError.
> 
> However, I agree with you that the PEP would benefit from keeping
> socket.error, select.error and mmap.error as separate IOError
> subclasses. Existing code may legitimately be catching those in try
> blocks that include other unrelated operations that may raise IOError.

Most socket I/O errors use different errnos (socket-specific errnos such
as ECONNRESET, etc.), so merging the exception classes wouldn't change
anything for careful code which checks the errno.

As for mmap.error, I think it's a design mistake in the first place,
since mmap'ed files are not consistently different from normal files.

As for select.error, the way the select module itself is generally
inconsistent in which exceptions it raises also makes me think it is a
design or implementation bug. The fact that it doesn't even inherit
EnvinromentError reinforces that feeling.

More fundamentally, the effort to create a detailed IO exception
hierarchy is much less useful if we keep islands such as socket.error,
mmap.error, since those errors won't benefit. The PEP only achieves
maximum usefulnesss if it can leverage a single IO exception base
class.


That said, it is obvious that this PEP can raise compatibility concerns
for uncaring code (code which is already poorly written in the first
place). There's no way to reorganize the exception hierarchy without
risking such issues. Actually, even without reorganizing the exception
hierarchy, it often happens that we casually change the exception type
raised by a given function, because we think the old behaviour was
mistaken. Nobody apparently blames us for doing that (perhaps they
are not reading python-checkins carefully enough :-)).

So, as a data point, carefully merging the exception classes (all of
OSError, IOError, EnvinronmentError, WindowsError, VMSError,
socket.error, mmap.error and select.error) doesn't produce any error in
the regression test suite. You can look at the SVN branch named
"pep-3151" for that.

Regards

Antoine.




From ncoghlan at gmail.com  Thu Nov 11 14:50:36 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 11 Nov 2010 23:50:36 +1000
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <20101111140830.1825a836@pitrou.net>
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
	<20101111140830.1825a836@pitrou.net>
Message-ID: <AANLkTi=5j7u_f3aNe=0iGr8d0RZB3SNpjxxLzOCwpLo1@mail.gmail.com>

On Thu, Nov 11, 2010 at 11:08 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Most socket I/O errors use different errnos (socket-specific errnos such
> as ECONNRESET, etc.), so merging the exception classes wouldn't change
> anything for careful code which checks the errno.
>
> As for mmap.error, I think it's a design mistake in the first place,
> since mmap'ed files are not consistently different from normal files.
>
> As for select.error, the way the select module itself is generally
> inconsistent in which exceptions it raises also makes me think it is a
> design or implementation bug. The fact that it doesn't even inherit
> EnvinromentError reinforces that feeling.
>
> More fundamentally, the effort to create a detailed IO exception
> hierarchy is much less useful if we keep islands such as socket.error,
> mmap.error, since those errors won't benefit. The PEP only achieves
> maximum usefulnesss if it can leverage a single IO exception base
> class.
>
>
> That said, it is obvious that this PEP can raise compatibility concerns
> for uncaring code (code which is already poorly written in the first
> place). There's no way to reorganize the exception hierarchy without
> risking such issues. Actually, even without reorganizing the exception
> hierarchy, it often happens that we casually change the exception type
> raised by a given function, because we think the old behaviour was
> mistaken. Nobody apparently blames us for doing that (perhaps they
> are not reading python-checkins carefully enough :-)).
>
> So, as a data point, carefully merging the exception classes (all of
> OSError, IOError, EnvinronmentError, WindowsError, VMSError,
> socket.error, mmap.error and select.error) doesn't produce any error in
> the regression test suite. You can look at the SVN branch named
> "pep-3151" for that.

Perhaps the better approach would be to explicitly list out the
exceptions being merged in the compatibility section and state which
case applies:

"Code that cares should already be checking the errno attribute":
socket.error, OSError, WindowsError, VMSError

"Already cannot be reliably handled distinct from IOError due to the
way the affected modules are implemented": mmap.error, select.error

Then we can state that any exception clause which changes scope due to
the new merger into IOError was already broken and we're just slightly
changing exactly how it is broken.

That said, didn't we originally punt on this PEP for 3.2 due to the
moratorium rather than due to any lack of time to get it done?

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From solipsis at pitrou.net  Thu Nov 11 14:58:08 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 11 Nov 2010 14:58:08 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <AANLkTi=5j7u_f3aNe=0iGr8d0RZB3SNpjxxLzOCwpLo1@mail.gmail.com>
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
	<20101111140830.1825a836@pitrou.net>
	<AANLkTi=5j7u_f3aNe=0iGr8d0RZB3SNpjxxLzOCwpLo1@mail.gmail.com>
Message-ID: <1289483888.3549.19.camel@localhost.localdomain>

Le jeudi 11 novembre 2010 ? 23:50 +1000, Nick Coghlan a ?crit :
> 
> Perhaps the better approach would be to explicitly list out the
> exceptions being merged in the compatibility section and state which
> case applies:
> 
> "Code that cares should already be checking the errno attribute":
> socket.error, OSError, WindowsError, VMSError
> 
> "Already cannot be reliably handled distinct from IOError due to the
> way the affected modules are implemented": mmap.error, select.error
> 
> Then we can state that any exception clause which changes scope due to
> the new merger into IOError was already broken and we're just slightly
> changing exactly how it is broken.

Ok, I will look at that.

> That said, didn't we originally punt on this PEP for 3.2 due to the
> moratorium rather than due to any lack of time to get it done?

I myself thought that the moratorium would get in the way, but Georg
asked me if it could be implemented in time for 3.2, which is why I'm
bringing it here.
Getting a pronouncement would be good anyway.

Regards

Antoine.




From mal at egenix.com  Thu Nov 11 16:03:31 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 11 Nov 2010 16:03:31 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <20101111140830.1825a836@pitrou.net>
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
	<20101111140830.1825a836@pitrou.net>
Message-ID: <4CDC05C3.6060906@egenix.com>

Antoine Pitrou wrote:
> On Thu, 11 Nov 2010 22:03:36 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> This issue is addressed directly in the PEP by the "Compatibility
>> Strategy" section. Code which checks errno will not suddenly start
>> catching additional exceptions due to the merger of IOError and
>> OSError.
>>
>> However, I agree with you that the PEP would benefit from keeping
>> socket.error, select.error and mmap.error as separate IOError
>> subclasses. Existing code may legitimately be catching those in try
>> blocks that include other unrelated operations that may raise IOError.
> 
> Most socket I/O errors use different errnos (socket-specific errnos such
> as ECONNRESET, etc.), so merging the exception classes wouldn't change
> anything for careful code which checks the errno.
> 
> As for mmap.error, I think it's a design mistake in the first place,
> since mmap'ed files are not consistently different from normal files.
> 
> As for select.error, the way the select module itself is generally
> inconsistent in which exceptions it raises also makes me think it is a
> design or implementation bug. The fact that it doesn't even inherit
> EnvinromentError reinforces that feeling.
> 
> More fundamentally, the effort to create a detailed IO exception
> hierarchy is much less useful if we keep islands such as socket.error,
> mmap.error, since those errors won't benefit. The PEP only achieves
> maximum usefulnesss if it can leverage a single IO exception base
> class.

Why ? You can use multiple inheritance to plug the existing
class structure into a new one.

> That said, it is obvious that this PEP can raise compatibility concerns
> for uncaring code (code which is already poorly written in the first
> place). 

There's nothing wrong with code such as this:

try:
  ...
except socket.error:
  print 'Network problem'
except IOError:
  print 'File I/O problem'
except OSError:
  print 'File system problem'

You probably have a different use of try-except in mind. Those
exception handling blocks are used at various levels in an
application and the code run between try-except may well be dealing
with sockets and file I/O, but require two different sets of
problem resolution or reporting implementations.

The PEP breaks such code.

It also doesn't make a difference whether you additionally
check errno or not in the except clauses, since Python will
never branch into later except clauses if the class matches
the first found exception class.

In the above example, the except socket.error clause would
receive all errors, but it's not prepared to deal with them,
since the code was written with a different hierarchy in mind.

And indeed, "careful" code using a more coarse class hierarchy
as basis is more likely to break because of such changes than
code which works based on classes of exceptions. Simply due
to the fact that the errno value loses the additional
context information of how it came to be under the PEP
proposal.

Since this is a known fact, PEP 348 used the 2.x-3.x transition
to implement those changes. The 3.x-4.x transition would be the
next chance to apply such structural changes.

> There's no way to reorganize the exception hierarchy without
> risking such issues. Actually, even without reorganizing the exception
> hierarchy, it often happens that we casually change the exception type
> raised by a given function, because we think the old behaviour was
> mistaken. Nobody apparently blames us for doing that (perhaps they
> are not reading python-checkins carefully enough :-)).

Such changes need to be carefully documented in the NEWS file.

They may well introduce DoS security problems in applications,
e.g. by causing an application to crash with a traceback due
to a try-except not triggering because of the change.

So again, +1 on adding the new exceptions and integrating
them with the existing IOError subclasses. -1 on removing those
subclasses.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From g.brandl at gmx.net  Thu Nov 11 17:01:00 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 11 Nov 2010 17:01:00 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
	hierarchy (again)
In-Reply-To: <4CDC05C3.6060906@egenix.com>
References: <20101111004542.1761633e@pitrou.net>
	<4CDBB2CE.4020004@egenix.com>	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>	<20101111140830.1825a836@pitrou.net>
	<4CDC05C3.6060906@egenix.com>
Message-ID: <ibh42f$usu$1@dough.gmane.org>

Am 11.11.2010 16:03, schrieb M.-A. Lemburg:
> Antoine Pitrou wrote:
>> On Thu, 11 Nov 2010 22:03:36 +1000
>> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> This issue is addressed directly in the PEP by the "Compatibility
>>> Strategy" section. Code which checks errno will not suddenly start
>>> catching additional exceptions due to the merger of IOError and
>>> OSError.
>>>
>>> However, I agree with you that the PEP would benefit from keeping
>>> socket.error, select.error and mmap.error as separate IOError
>>> subclasses. Existing code may legitimately be catching those in try
>>> blocks that include other unrelated operations that may raise IOError.
>> 
>> Most socket I/O errors use different errnos (socket-specific errnos such
>> as ECONNRESET, etc.), so merging the exception classes wouldn't change
>> anything for careful code which checks the errno.
>> 
>> As for mmap.error, I think it's a design mistake in the first place,
>> since mmap'ed files are not consistently different from normal files.
>> 
>> As for select.error, the way the select module itself is generally
>> inconsistent in which exceptions it raises also makes me think it is a
>> design or implementation bug. The fact that it doesn't even inherit
>> EnvinromentError reinforces that feeling.
>> 
>> More fundamentally, the effort to create a detailed IO exception
>> hierarchy is much less useful if we keep islands such as socket.error,
>> mmap.error, since those errors won't benefit. The PEP only achieves
>> maximum usefulnesss if it can leverage a single IO exception base
>> class.
> 
> Why ? You can use multiple inheritance to plug the existing
> class structure into a new one.
> 
>> That said, it is obvious that this PEP can raise compatibility concerns
>> for uncaring code (code which is already poorly written in the first
>> place). 
> 
> There's nothing wrong with code such as this:
> 
> try:
>   ...
> except socket.error:
>   print 'Network problem'
> except IOError:
>   print 'File I/O problem'
> except OSError:
>   print 'File system problem'

And as Antoine already showed, whether a given code that does I/O raises
IOError or OSError is pretty much arbitrary.  I agree for socket.error
though; there needs to be a catch-all way to intercept network exceptions.

Georg



From solipsis at pitrou.net  Thu Nov 11 17:40:28 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 11 Nov 2010 17:40:28 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
	hierarchy (again)
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
	<20101111140830.1825a836@pitrou.net> <4CDC05C3.6060906@egenix.com>
	<ibh42f$usu$1@dough.gmane.org>
Message-ID: <20101111174028.40af55ba@pitrou.net>

On Thu, 11 Nov 2010 17:01:00 +0100
Georg Brandl <g.brandl at gmx.net> wrote:
> 
> And as Antoine already showed, whether a given code that does I/O raises
> IOError or OSError is pretty much arbitrary.  I agree for socket.error
> though; there needs to be a catch-all way to intercept network exceptions.

I can understand the desire to intercept network exceptions to print
out messages.

However, socket.error can also indicate a programming error: for
example, trying to read from an unconnected socket. So, having a
catch-all for socket.error can hide bugs in the code. The real way to
discriminate errors is by using errno, not the module raising the
error. That's why the PEP proposes errno-specific classes such as
ConnectionError.

Besides, useful feedback for the user requires printing the error
message (which is filled out by the OS and doesn't depend on Python's
exception hierarchy), and preferably which network service failed
responding; just printing "network error" is completely unhelpful.

Regards

Antoine.




From mal at egenix.com  Thu Nov 11 18:13:00 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 11 Nov 2010 18:13:00 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <ibh42f$usu$1@dough.gmane.org>
References: <20101111004542.1761633e@pitrou.net>
	<4CDBB2CE.4020004@egenix.com>	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>	<20101111140830.1825a836@pitrou.net>
	<4CDC05C3.6060906@egenix.com> <ibh42f$usu$1@dough.gmane.org>
Message-ID: <4CDC241C.9080703@egenix.com>

Georg Brandl wrote:
>>> That said, it is obvious that this PEP can raise compatibility concerns
>>> for uncaring code (code which is already poorly written in the first
>>> place). 
>>
>> There's nothing wrong with code such as this:
>>
>> try:
>>   ...
>> except socket.error:
>>   print 'Network problem'
>> except IOError:
>>   print 'File I/O problem'
>> except OSError:
>>   print 'File system problem'
> 
> And as Antoine already showed, whether a given code that does I/O raises
> IOError or OSError is pretty much arbitrary.  I agree for socket.error
> though; there needs to be a catch-all way to intercept network exceptions.

I'm not sure where Antoine got this impression. Stdlib modules
traditionally always implemented a base error class for exceptions
specific to that module and sometimes also added subclasses
for additional breakdown.

The OSError originated in the os module's os.error and was
only raised for os module exceptions. With the introduction
of OSError as a builtin, other modules started using this
exception as well, which is a bit unfortunate. I still don't
find its use arbitrary, though.

IOError, OTOH, was used for exceptions raised by C lib I/O
related functions and operations. This was never tied to a single
module, but almost always used for typical I/O operations on
files, sockets, devices, etc.

As such, both depend on the runtime environment, hence the
EnvironmentError base class.

I also find assumptions such as "However, the user is interested in the nature of the error, not in
which part of the interpreter it comes from (since the latter is obvious from reading the traceback
message or application source code)." misleading.

Of course, the user is interested in which part of the interpreter an
exception originated and yes, she does want to take action on
the exception based on the context of origin. That was the whole
point why we added per module error base classes in the first
place.

Antoine can't really be serious when expecting the user to
run the exception by the inspect module to find that context,
e.g. to determine whether it was caused by a socket, an open
file, or an attempt to ask the file system for a directory
listing. All of these can e.g. raise an error with errno
EACCESS.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Thu Nov 11 22:50:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 12 Nov 2010 07:50:39 +1000
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <4CDC05C3.6060906@egenix.com>
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
	<20101111140830.1825a836@pitrou.net> <4CDC05C3.6060906@egenix.com>
Message-ID: <AANLkTinU9t=tdbahd=XocDLMLOuw-DgewKTrcuWyjWZn@mail.gmail.com>

On Fri, Nov 12, 2010 at 1:03 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> There's nothing wrong with code such as this:
>
> try:
> ?...
> except socket.error:
> ?print 'Network problem'
> except IOError:
> ?print 'File I/O problem'
> except OSError:
> ?print 'File system problem'
>
> You probably have a different use of try-except in mind. Those
> exception handling blocks are used at various levels in an
> application and the code run between try-except may well be dealing
> with sockets and file I/O, but require two different sets of
> problem resolution or reporting implementations.
>
> The PEP breaks such code.

No, that's the kind of code we're saying is already subtly broken, and
we're just changing the way it is broken to be far more obvious
(instead of *sometimes* printing the wrong message based on exactly
what went wrong, it will instead consistently print the message
associated with whichever of IOError and OSError is listed first). The
standard library may have been consistent about IOError vs OSError at
one point in time, but that time is in the past. Any code which
doesn't catch (or ignore) them both and treat them as equivalent to
each other is either already broken (if the try block contains code
which may raise either exception) or won't be affected by the proposal
in the PEP (if the try block is guaranteed to raise only the exception
type already caught by the try block).

Trying to restore a consistent distinction between them is even more
of compatibility problem than simply giving up and merging them. I
thought the PEP already made this point, but it may not be as clear if
you don't already agree with the conclusion.

There is a slightly better case for keeping socket.error around since
we've been fairly consistent in having the networking code raise that
rather than a bare IOError. However, even there the value of the
distinction is blurring, since the other layers of the IO stack will
still raise IOError.

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From mal at egenix.com  Fri Nov 12 00:31:18 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 12 Nov 2010 00:31:18 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
 hierarchy (again)
In-Reply-To: <AANLkTinU9t=tdbahd=XocDLMLOuw-DgewKTrcuWyjWZn@mail.gmail.com>
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
	<20101111140830.1825a836@pitrou.net> <4CDC05C3.6060906@egenix.com>
	<AANLkTinU9t=tdbahd=XocDLMLOuw-DgewKTrcuWyjWZn@mail.gmail.com>
Message-ID: <4CDC7CC6.2000003@egenix.com>

Nick Coghlan wrote:
> On Fri, Nov 12, 2010 at 1:03 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> There's nothing wrong with code such as this:
>>
>> try:
>>  ...
>> except socket.error:
>>  print 'Network problem'
>> except IOError:
>>  print 'File I/O problem'
>> except OSError:
>>  print 'File system problem'
>>
>> You probably have a different use of try-except in mind. Those
>> exception handling blocks are used at various levels in an
>> application and the code run between try-except may well be dealing
>> with sockets and file I/O, but require two different sets of
>> problem resolution or reporting implementations.
>>
>> The PEP breaks such code.
> 
> No, that's the kind of code we're saying is already subtly broken, and
> we're just changing the way it is broken to be far more obvious
> (instead of *sometimes* printing the wrong message based on exactly
> what went wrong, it will instead consistently print the message
> associated with whichever of IOError and OSError is listed first). The
> standard library may have been consistent about IOError vs OSError at
> one point in time, but that time is in the past. Any code which
> doesn't catch (or ignore) them both and treat them as equivalent to
> each other is either already broken (if the try block contains code
> which may raise either exception) or won't be affected by the proposal
> in the PEP (if the try block is guaranteed to raise only the exception
> type already caught by the try block).

I don't follow you.

Example: If I run code that e.g. uses os.fork()
and get an IOError, I don't want to catch that, since this is
likely a programming error of some sort which I don't want to
mask. However, if I get an OSError (the error raised by os.fork()),
I can report this as a resource problem to higher level code
which can then take appropriate action.

Your arguing that because the stdlib has blurred the distinction
between IOError and OSError (as a result of making OSError builtin
rather than letting it live in the os module), it is OK to now
blur the distinction between error handling code that was meant
to run for issues that are related to the actually used functions
and possibly causing the error reporting to malfunction.

Antoine is arguing that simply by relying on the errno you
can write "careful" code. That's not correct. If I write
code that only checks the errno without knowing in which context
this was set, I cannot always make assumptions on what that
errno really means, e.g. EACCES, EBADF or EPERM are used by lots
of C lib and OS functions, so there's no implicit hint to
the subsystem where the error originated.

If we go down that route, we might as well merge TypeError
into ValueError as well, since the distinction between those
has been blurred long before OSError was even introduced.

But we don't do that. And the reason is that we know which
functions or methods return which types of errors and use
that knowledge to direct the flow of error handling into
the right lanes.

Besides, there are lots of functions in the os module that don't do
any I/O in the standard sense - why should those raise an IOError ?
Most of the functions deal with OS services, so the name OSError
is a lot more appropriate.

Note that the same kind of blurring is likely going to happen
for the newly suggested additional exceptions. Sooner or later
people are going to use ConnectionError for all kinds of connections,
not only socket-related ones. Same for TimeoutError and
PermissionError.

So in the end, you have to go and check the errno code again,
just to see whether the exception you're looking at really
originated from a socket or file operation which set errno,
provided .errno isn't set to the E-values mentioned in the PEP
per default... or we just merge everything back again into
IOError and leave it to the programmers to find some other
way to signal where the exception originated.

Not very helpful.

Perhaps it's indeed better to clearly draw the line between
IOError and OSError again, like you say, instead of trying to
"fix" things by further blurring information.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From solipsis at pitrou.net  Fri Nov 12 01:35:14 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 12 Nov 2010 01:35:14 +0100
Subject: [Python-ideas] PEP 3151 - Reworking the OS and IO exception
	hierarchy (again)
References: <20101111004542.1761633e@pitrou.net> <4CDBB2CE.4020004@egenix.com>
	<AANLkTinKdTnqQ_CKjmx9revYRVu3r4k6regTjk-JJ1Ms@mail.gmail.com>
	<20101111140830.1825a836@pitrou.net> <4CDC05C3.6060906@egenix.com>
	<AANLkTinU9t=tdbahd=XocDLMLOuw-DgewKTrcuWyjWZn@mail.gmail.com>
	<4CDC7CC6.2000003@egenix.com>
Message-ID: <20101112013514.599f81fb@pitrou.net>

On Fri, 12 Nov 2010 00:31:18 +0100
"M.-A. Lemburg" <mal at egenix.com> wrote:
> Nick Coghlan wrote:
> > On Fri, Nov 12, 2010 at 1:03 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> >> There's nothing wrong with code such as this:
> >>
> >> try:
> >>  ...
> >> except socket.error:
> >>  print 'Network problem'
> >> except IOError:
> >>  print 'File I/O problem'
> >> except OSError:
> >>  print 'File system problem'
> >>
> >> You probably have a different use of try-except in mind. Those
> >> exception handling blocks are used at various levels in an
> >> application and the code run between try-except may well be dealing
> >> with sockets and file I/O, but require two different sets of
> >> problem resolution or reporting implementations.
> >>
> >> The PEP breaks such code.
> > 
> > No, that's the kind of code we're saying is already subtly broken, and
> > we're just changing the way it is broken to be far more obvious
> > (instead of *sometimes* printing the wrong message based on exactly
> > what went wrong, it will instead consistently print the message
> > associated with whichever of IOError and OSError is listed first). The
> > standard library may have been consistent about IOError vs OSError at
> > one point in time, but that time is in the past. Any code which
> > doesn't catch (or ignore) them both and treat them as equivalent to
> > each other is either already broken (if the try block contains code
> > which may raise either exception) or won't be affected by the proposal
> > in the PEP (if the try block is guaranteed to raise only the exception
> > type already caught by the try block).
> 
> I don't follow you.
> 
> Example: If I run code that e.g. uses os.fork()
> and get an IOError, I don't want to catch that, since this is
> likely a programming error of some sort which I don't want to
> mask. However, if I get an OSError (the error raised by os.fork()),
> I can report this as a resource problem to higher level code
> which can then take appropriate action.

If you're writing code such as
(which I'm sure is frowned upon in the stdlib):

    try:
        ... 99 lines of code ...
        os.fork()
        ... 99 other lines of code ...
    except OSError:
        # oh, os.fork() must have failed

you're already living dangerously, and even without this PEP.
OSError is not "the error raised by os.fork()", it is the error raised
by *many* stdlib functions including os.fork().  It could be raised by
many of the other 198 lines of code in the try ... except block above.

So, the goal of the exception category is to tell you what the problem
is, not where it happened.  How you know where it happened is by having
fine-grained try ... except blocks.
(and if you're a human, you can also read the traceback)

> Antoine is arguing that simply by relying on the errno you
> can write "careful" code. That's not correct. If I write
> code that only checks the errno without knowing in which context
> this was set, I cannot always make assumptions on what that
> errno really means, e.g. EACCES, EBADF or EPERM are used by lots
> of C lib and OS functions, so there's no implicit hint to
> the subsystem where the error originated.

Checking exceptions without knowing in which context they are raised
*is* doomed to failure. Your OSError could come from a lot of places
already, so you can't do anything meaningful with it.

On the other hand, if using a fine-grained try ... except block you
know that e.g. EACCES (or PermissionError) was issued by os.rename(),
you know precisely what it means.

> If we go down that route, we might as well merge TypeError
> into ValueError as well, since the distinction between those
> has been blurred long before OSError was even introduced.

Well, no, on the contrary.

The distinction between TypeError and ValueError (and KeyError and
SyntaxError and MemoryError... etc.) is *precisely* the kind of
distinction which is advocated by this PEP, that is: a distinction
between different kinds of problems. Not a distinction based on the
location where a problem happens.

(otherwise we would have DictError, StrError, BytesError, IntError...
instead)

And, indeed, the PEP helps improve the consistency of the exception
hierarchy in that respect.

(if you prefer the other kind of consistency, you can propose a PEP to
introduce DictError, ForkError, RenameError, etc.)

> Besides, there are lots of functions in the os module that don't do
> any I/O in the standard sense - why should those raise an IOError ?
> Most of the functions deal with OS services, so the name OSError
> is a lot more appropriate.

Well, you were part of the earlier discussion where this was discussed.
I'll let you browse through the archives.

> Sooner or later
> people are going to use ConnectionError for all kinds of connections,
> not only socket-related ones.

I'm not sure what the problem with reusing exceptions is. We do it all
the time.

Regards

Antoine.




From steve at pearwood.info  Mon Nov 15 00:15:06 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 15 Nov 2010 10:15:06 +1100
Subject: [Python-ideas] stats module Was: minmax() function ...
In-Reply-To: <201010180357.59264.steve@pearwood.info>
References: <201010161111.21847.steve@pearwood.info>	<622121A3-6A51-4735-A292-9F82502BB623@gmail.com>
	<201010180357.59264.steve@pearwood.info>
Message-ID: <4CE06D7A.5080306@pearwood.info>

For anyone interested, I have put up a new release for the stats module 
suggested by Raymond.

http://pypi.python.org/pypi/stats/

The biggest change is that I'm now targeting Python 3.x code rather than 
2.x. This is my first non-trivial piece of work written for 3.x, and can 
I say, it's a pleasure to work with!




Steven D'Aprano wrote:
> On Sat, 16 Oct 2010 11:33:02 am Raymond Hettinger wrote:
>>> Are you still interested in working on it, or is this a subtle hint
>>> that somebody else should do so?
>> Hmm, perhaps this would be less subtle:
>> HEY, WHY DON'T YOU GUYS GO TO WORK ON A STATS MODULE!
> 
> 
> http://pypi.python.org/pypi/stats
> 
> It is not even close to production ready. It needs unit tests. The API 
> should be considered unstable. There's no 3.x version yet. Obviously it 
> has no real-world usage. But if anyone would like to contribute, 
> critique or criticize, I welcome feedback or assistance, or even just 
> encouragement.
>



-- 
Steven


From alexandre.conrad at gmail.com  Mon Nov 15 04:15:03 2010
From: alexandre.conrad at gmail.com (Alexandre Conrad)
Date: Sun, 14 Nov 2010 19:15:03 -0800
Subject: [Python-ideas] syntax for set
Message-ID: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>

Hi,

I am sure this has come up before and if it doesn't exist I believe
there is a good reason for it. Anyway, here's my thought: how about
having a syntax representation for sets? A mix of a dict and
list/tuple syntax would make the most sense, such as:

>>> isinstance({"foo", "bar"}, set)
True

The problem here is: how to represent an empty set as {} would
conflict with the dict repr. Then how about using a brand new
"container" syntax that wouldn't conflict with existing ones?

>>> <"foo", "bar"> == set(("foo", "bar"))
True

>>> <> == set()
True

-- 
Alex | twitter.com/alexconrad


From python at mrabarnett.plus.com  Mon Nov 15 04:23:41 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 15 Nov 2010 03:23:41 +0000
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
Message-ID: <4CE0A7BD.1070801@mrabarnett.plus.com>

On 15/11/2010 03:15, Alexandre Conrad wrote:
> Hi,
>
> I am sure this has come up before and if it doesn't exist I believe
> there is a good reason for it. Anyway, here's my thought: how about
> having a syntax representation for sets? A mix of a dict and
> list/tuple syntax would make the most sense, such as:
>
[snip]
This already exists in Python 2.7 and Python 3.


From alexandre.conrad at gmail.com  Mon Nov 15 04:39:46 2010
From: alexandre.conrad at gmail.com (Alexandre Conrad)
Date: Sun, 14 Nov 2010 19:39:46 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE0A7BD.1070801@mrabarnett.plus.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
Message-ID: <AANLkTinHZRRKAjzi=YWwH9byUSsg6NjzvJ_G56CpGdX0@mail.gmail.com>

Oh! ... That is something I missed. I'll have a look at it. Thanks.

2010/11/14 MRAB <python at mrabarnett.plus.com>:
> On 15/11/2010 03:15, Alexandre Conrad wrote:
>>
>> Hi,
>>
>> I am sure this has come up before and if it doesn't exist I believe
>> there is a good reason for it. Anyway, here's my thought: how about
>> having a syntax representation for sets? A mix of a dict and
>> list/tuple syntax would make the most sense, such as:
>>
> [snip]
> This already exists in Python 2.7 and Python 3.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Alex | twitter.com/alexconrad


From cool-rr at cool-rr.com  Mon Nov 15 16:46:43 2010
From: cool-rr at cool-rr.com (cool-RR)
Date: Mon, 15 Nov 2010 17:46:43 +0200
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
Message-ID: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>

Hello.

Recently I had the need to filter objects based on whether they're picklable
or not:

http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items

I'm not sure what's a good way to check for a specific object whether it's
picklable.

<http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items>This
led me to think: Maybe we should have an `Unpicklable` abstract base class
in the `collections` module? Then various unpicklable classes, like locks,
files or widgets, could inherit from this class to signify that they cannot
be pickled.

What do you think?


Ram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101115/af5996d3/attachment.html>

From tjreedy at udel.edu  Mon Nov 15 19:45:00 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 15 Nov 2010 13:45:00 -0500
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE0A7BD.1070801@mrabarnett.plus.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
Message-ID: <ibrv3a$pk6$1@dough.gmane.org>

On 11/14/2010 10:23 PM, MRAB wrote:
> On 15/11/2010 03:15, Alexandre Conrad wrote:
>> Hi,
>>
>> I am sure this has come up before and if it doesn't exist I believe
>> there is a good reason for it. Anyway, here's my thought: how about
>> having a syntax representation for sets? A mix of a dict and
>> list/tuple syntax would make the most sense, such as:
>>
> [snip]
> This already exists in Python 2.7 and Python 3.

 >>> {1,2,3} == set((1,2,3))
True

The *only* glitch is the lack of an empty set notation. The idea of
dict() == {:}; set()=={};
which would be the obvious choice if starting fresh,
was considered but rejected by Guido as too disruptive
and introducing another barrier between Python 2 and 3.

-- 
Terry Jan Reedy



From guido at python.org  Mon Nov 15 20:02:53 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 15 Nov 2010 11:02:53 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <ibrv3a$pk6$1@dough.gmane.org>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
Message-ID: <AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>

There is nothing obvious about {:} for the empty dictionary. :-)

On Mon, Nov 15, 2010 at 10:45 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 11/14/2010 10:23 PM, MRAB wrote:
>>
>> On 15/11/2010 03:15, Alexandre Conrad wrote:
>>>
>>> Hi,
>>>
>>> I am sure this has come up before and if it doesn't exist I believe
>>> there is a good reason for it. Anyway, here's my thought: how about
>>> having a syntax representation for sets? A mix of a dict and
>>> list/tuple syntax would make the most sense, such as:
>>>
>> [snip]
>> This already exists in Python 2.7 and Python 3.
>
>>>> {1,2,3} == set((1,2,3))
> True
>
> The *only* glitch is the lack of an empty set notation. The idea of
> dict() == {:}; set()=={};
> which would be the obvious choice if starting fresh,
> was considered but rejected by Guido as too disruptive
> and introducing another barrier between Python 2 and 3.
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (python.org/~guido)


From alexander.belopolsky at gmail.com  Mon Nov 15 20:33:44 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 15 Nov 2010 14:33:44 -0500
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
Message-ID: <AANLkTik7SnSvaG=GMFq4bwyJa9SskEXVLDVoJsk7Span@mail.gmail.com>

On Mon, Nov 15, 2010 at 2:02 PM, Guido van Rossum <guido at python.org> wrote:
> There is nothing obvious about {:} for the empty dictionary. :-)
>

The obvious notation for an empty set is ? ('\N{EMPTY SET}'), but
fortunately (!) this is not a valid identifier.


From denis.spir at gmail.com  Mon Nov 15 21:01:35 2010
From: denis.spir at gmail.com (spir)
Date: Mon, 15 Nov 2010 21:01:35 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
Message-ID: <20101115210135.7f9d25fe@o>

On Mon, 15 Nov 2010 11:02:53 -0800
Guido van Rossum <guido at python.org> wrote:

> There is nothing obvious about {:} for the empty dictionary. :-)

Anyone votes for {/} meaning ?

Denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From denis.spir at gmail.com  Mon Nov 15 21:02:29 2010
From: denis.spir at gmail.com (spir)
Date: Mon, 15 Nov 2010 21:02:29 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
Message-ID: <20101115210229.1261830f@o>

[sorry, sent unintentionally]

On Mon, 15 Nov 2010 11:02:53 -0800
Guido van Rossum <guido at python.org> wrote:

> There is nothing obvious about {:} for the empty dictionary. :-)

Anyone votes for {/} meaning empty set?

Denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From python at mrabarnett.plus.com  Mon Nov 15 21:16:07 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 15 Nov 2010 20:16:07 +0000
Subject: [Python-ideas] syntax for set
In-Reply-To: <20101115210229.1261830f@o>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<20101115210229.1261830f@o>
Message-ID: <4CE19507.2010104@mrabarnett.plus.com>

On 15/11/2010 20:02, spir wrote:
> [sorry, sent unintentionally]
>
> On Mon, 15 Nov 2010 11:02:53 -0800
> Guido van Rossum<guido at python.org>  wrote:
>
>> There is nothing obvious about {:} for the empty dictionary. :-)
>
> Anyone votes for {/} meaning empty set?
>
No, but I would vote for {,}.


From alexandre.conrad at gmail.com  Mon Nov 15 21:34:25 2010
From: alexandre.conrad at gmail.com (Alexandre Conrad)
Date: Mon, 15 Nov 2010 12:34:25 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <20101115210229.1261830f@o>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<20101115210229.1261830f@o>
Message-ID: <AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>

2010/11/15 spir <denis.spir at gmail.com>:
> [sorry, sent unintentionally]
>
> On Mon, 15 Nov 2010 11:02:53 -0800
> Guido van Rossum <guido at python.org> wrote:
>
>> There is nothing obvious about {:} for the empty dictionary. :-)

{:} for empty dict could make some sense. If that is ever accepted, it
will probably go in Python 4 as the change is so important.

> Anyone votes for {/} meaning empty set?

-1.

List, tuple and dict have their own literals. Now sets shares dict's
literals. That's why I proposed using a new container literal for
sets, such as <1, 2, 3> where an empty set would be <>. As lists and
tuples are the mutable/immutable pairs, I thought we could have
literals for frozenset as well. But I can no longer find available
opening/closing symbols on my keyboard. Darn.

I also thought about doubling braces, such as {{1, 2, 3}}, then {{}}
for an empty set would work. (and probably break some templating
engines out there ;)). But it conflicts with "a dict in a set" (which
is unhashable anyway, but that's not a valid reason).

-- 
Alex | twitter.com/alexconrad


From alexandre.conrad at gmail.com  Mon Nov 15 21:35:14 2010
From: alexandre.conrad at gmail.com (Alexandre Conrad)
Date: Mon, 15 Nov 2010 12:35:14 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE19507.2010104@mrabarnett.plus.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<20101115210229.1261830f@o> <4CE19507.2010104@mrabarnett.plus.com>
Message-ID: <AANLkTin1d34SexsL4jR-qD7F8ifmtM+z946qZgDHGC9G@mail.gmail.com>

2010/11/15 MRAB <python at mrabarnett.plus.com>:
> On 15/11/2010 20:02, spir wrote:
>> Anyone votes for {/} meaning empty set?
>>
> No, but I would vote for {,}.

I tend to like that.
-- 
Alex | twitter.com/alexconrad


From ddasilva at umd.edu  Mon Nov 15 21:37:17 2010
From: ddasilva at umd.edu (Daniel da Silva)
Date: Mon, 15 Nov 2010 15:37:17 -0500
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTin1d34SexsL4jR-qD7F8ifmtM+z946qZgDHGC9G@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<20101115210229.1261830f@o> <4CE19507.2010104@mrabarnett.plus.com>
	<AANLkTin1d34SexsL4jR-qD7F8ifmtM+z946qZgDHGC9G@mail.gmail.com>
Message-ID: <AANLkTikN0SQ6_-f2BTKowC37Z4jF_+oJ1Tnb0DA+m7UE@mail.gmail.com>

On Mon, Nov 15, 2010 at 3:35 PM, Alexandre Conrad <
alexandre.conrad at gmail.com> wrote:

> 2010/11/15 MRAB <python at mrabarnett.plus.com>:
> > On 15/11/2010 20:02, spir wrote:
> >> Anyone votes for {/} meaning empty set?
> >>
> > No, but I would vote for {,}.
>
> I tend to like that.
>

I agree. I think this would be a good syntax alternative to "set()".



> --

Alex | twitter.com/alexconrad
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101115/d40e29c3/attachment.html>

From python at mrabarnett.plus.com  Mon Nov 15 21:52:44 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 15 Nov 2010 20:52:44 +0000
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>	<20101115210229.1261830f@o>
	<AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>
Message-ID: <4CE19D9C.9090603@mrabarnett.plus.com>

On 15/11/2010 20:34, Alexandre Conrad wrote:
> 2010/11/15 spir<denis.spir at gmail.com>:
>> [sorry, sent unintentionally]
>>
>> On Mon, 15 Nov 2010 11:02:53 -0800
>> Guido van Rossum<guido at python.org>  wrote:
>>
>>> There is nothing obvious about {:} for the empty dictionary. :-)
>
> {:} for empty dict could make some sense. If that is ever accepted, it
> will probably go in Python 4 as the change is so important.
>
>> Anyone votes for {/} meaning empty set?
>
> -1.
>
> List, tuple and dict have their own literals. Now sets shares dict's
> literals. That's why I proposed using a new container literal for
> sets, such as<1, 2, 3>  where an empty set would be<>. As lists and
> tuples are the mutable/immutable pairs, I thought we could have
> literals for frozenset as well. But I can no longer find available
> opening/closing symbols on my keyboard. Darn.
>
> I also thought about doubling braces, such as {{1, 2, 3}}, then {{}}
> for an empty set would work. (and probably break some templating
> engines out there ;)). But it conflicts with "a dict in a set" (which
> is unhashable anyway, but that's not a valid reason).
>
I set is unordered, like a dict, so using {...} for both kind of makes
sense, with the entries in the dict literal consisting of pairs
separated by ":", which is why I would've liked {:}.

Let's leave <...> for some future use. (Could they have been used for
tuples, thus avoiding all the discussions about how "," builds tuples,
apart from the empty tuple? :-))


From greg.ewing at canterbury.ac.nz  Mon Nov 15 22:04:42 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 16 Nov 2010 10:04:42 +1300
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<20101115210229.1261830f@o>
	<AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>
Message-ID: <4CE1A06A.6010806@canterbury.ac.nz>

Alexandre Conrad wrote:
> That's why I proposed using a new container literal for
> sets, such as <1, 2, 3>

Please, no angle brackets. They're ugly and too easily
confused with comparison operators, at least by humans
trying to read the code.

-- 
Greg


From tjreedy at udel.edu  Mon Nov 15 22:57:23 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 15 Nov 2010 16:57:23 -0500
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
Message-ID: <ibsac1$idh$1@dough.gmane.org>

On 11/15/2010 2:02 PM, Guido van Rossum wrote:
> There is nothing obvious about {:} for the empty dictionary. :-)

'Obvious' is always relative to the beholder ;-).

Let me try again:
If I were designing a new language and had already decided on
{}, {1}, {1,2} ... for set displays and
??, {1:'one'} ... for dict displays and
on the corresponding comprehension forms as in Python,
then one 'obvious' choice *to me' for ?? would be '{:}'.
Substitute any other shifted char, preferably typed by the right hand, 
if you want.

However, the hypothetical de novo choice of something other than '{}' 
for ?? is irrelevant to the fact the Python 3 was not a new language and 
that changing the meaning of a syntactic constant like '{}' would have 
caused problems. I am more appreciative of that after seeing the 
problems people have had the the shift of meaning for unprefixed string 
constants.

-- 
Terry Jan Reedy



From greg.ewing at canterbury.ac.nz  Mon Nov 15 21:56:46 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 16 Nov 2010 09:56:46 +1300
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
Message-ID: <4CE19E8E.80109@canterbury.ac.nz>

Guido van Rossum wrote:
> There is nothing obvious about {:} for the empty dictionary. :-)

I think it has about the same level of obviousness as (x,)
for a 1-element tuple. You would have to learn it, but
after that you could easily remember which is which, because
dicts are made with colons and sets aren't.

A backwards-compatible alternative would be to use {.}
for an empty set. Think of a set as half a dict, and the
dot as half a colon. :-)

-- 
Greg


From debatem1 at gmail.com  Mon Nov 15 23:02:25 2010
From: debatem1 at gmail.com (geremy condra)
Date: Mon, 15 Nov 2010 14:02:25 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE1A06A.6010806@canterbury.ac.nz>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<20101115210229.1261830f@o>
	<AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>
	<4CE1A06A.6010806@canterbury.ac.nz>
Message-ID: <AANLkTin_RqBCy8EXsxHQ03cGd=rq9KodC9TGwqczp5xj@mail.gmail.com>

On Mon, Nov 15, 2010 at 1:04 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Alexandre Conrad wrote:
>>
>> That's why I proposed using a new container literal for
>> sets, such as <1, 2, 3>
>
> Please, no angle brackets. They're ugly and too easily
> confused with comparison operators, at least by humans
> trying to read the code.

Hmm, I find them easy to read and no harder on the eyes than {}, but
then I seldom have large numbers of < and > in my code.

Geremy Condra


From masklinn at masklinn.net  Mon Nov 15 23:09:39 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 15 Nov 2010 23:09:39 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE19D9C.9090603@mrabarnett.plus.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>	<20101115210229.1261830f@o>
	<AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>
	<4CE19D9C.9090603@mrabarnett.plus.com>
Message-ID: <9EF3781C-3F35-4AB2-8A45-6CCA727B6619@masklinn.net>

On 2010-11-15, at 21:52 , MRAB wrote:
> 
> I set is unordered, like a dict, so using {...} for both kind of makes
> sense, with the entries in the dict literal consisting of pairs
> separated by ":", which is why I would've liked {:}.

And as somebody suggested a few days/weeks ago, it leaves the door open to `[k:v]` as an ordereddict literal, as a parallel to {}/{:}.

From steve at pearwood.info  Mon Nov 15 23:52:47 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Nov 2010 09:52:47 +1100
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE19D9C.9090603@mrabarnett.plus.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>	<20101115210229.1261830f@o>	<AANLkTimvJV2+8Dyo0T_7QJpMrpKDBR1ZO-sMieQ-AfX8@mail.gmail.com>
	<4CE19D9C.9090603@mrabarnett.plus.com>
Message-ID: <4CE1B9BF.2010201@pearwood.info>

MRAB wrote:

> Let's leave <...> for some future use. (Could they have been used for
> tuples, thus avoiding all the discussions about how "," builds tuples,
> apart from the empty tuple? :-))

But comma *is* the tuple-building operator. This allows us to write:

x, y = p, q
return x, y, z

instead of being forced to including brackets. Are you suggesting that 
commas should be optional?

t = <x y z>

This will be an unfortunate source of errors unless we also drop 
implicit string concatenation (which is too useful to drop!)

t = <"a" "b" 2>  # 2-tuple, not 3

As for using < > for tuples, that leads to such horrors as:

if <a, b> < <c, d>: ...
elif <b, a> > <d, c>: ...

which is too ghastly for words.

Now, can we please stop bike-shedding over something so trivial as a way 
to write an empty set as a literal? What's wrong with just writing set()?


-- 
Steven


From rrr at ronadam.com  Tue Nov 16 00:27:08 2010
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 15 Nov 2010 17:27:08 -0600
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE19E8E.80109@canterbury.ac.nz>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<4CE19E8E.80109@canterbury.ac.nz>
Message-ID: <4CE1C1CC.9050106@ronadam.com>



On 11/15/2010 02:56 PM, Greg Ewing wrote:
> Guido van Rossum wrote:
>> There is nothing obvious about {:} for the empty dictionary. :-)
>
> I think it has about the same level of obviousness as (x,)
> for a 1-element tuple. You would have to learn it, but
> after that you could easily remember which is which, because
> dicts are made with colons and sets aren't.
>
> A backwards-compatible alternative would be to use {.}
> for an empty set. Think of a set as half a dict, and the
> dot as half a colon. :-)

"set()" is only two chars longer than any of the suggestions. It's also 
clear about what it is.  The spelling isn't the issue.

In fact, if we can optimise some common constructors, such as in the cases 
of an empty containers, we can toss out the special syntax's.


ra:~$ python3.2 -m timeit "dict()"
1000000 loops, best of 3: 0.401 usec per loop
ra:~$ python3.2 -m timeit "{}"
10000000 loops, best of 3: 0.0649 usec per loop

ra:~$ python3.2 -m timeit "list()"
1000000 loops, best of 3: 0.284 usec per loop
ra:~$ python3.2 -m timeit "[]"
10000000 loops, best of 3: 0.0724 usec per loop

ra:~$ python3.2 -m timeit "tuple()"
1000000 loops, best of 3: 0.222 usec per loop
ra:~$ python3.2 -m timeit "()"
10000000 loops, best of 3: 0.0456 usec per loop

ra at Gutsy:~$ python3.2 -m timeit "set()"
1000000 loops, best of 3: 0.234 usec per loop


Not seriously*, ;-)
    Ron


* IMO: The real value of the special syntax is that it makes it easy to 
write out nested objects in a concise way.  The missing empty set syntax 
doesn't cause much problems with that.

  line_data = {'points':[(1 , 2), (3, 4)], 'color':'blue', 'style':'dotted'}




From ben+python at benfinney.id.au  Tue Nov 16 01:16:11 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 16 Nov 2010 11:16:11 +1100
Subject: [Python-ideas] syntax for set
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<20101115210229.1261830f@o> <4CE19507.2010104@mrabarnett.plus.com>
Message-ID: <8739r2azsk.fsf@benfinney.id.au>

MRAB <python at mrabarnett.plus.com> writes:

> No, but I would vote for {,}.

?1.

The comma in the set literal syntax is used to separate items, and so
that looks like a set that explicitly *not* empty, but rather containing
(unspecified) items.

-- 
 \         ?Broken promises don't upset me. I just think, why did they |
  `\                                         believe me?? ?Jack Handey |
_o__)                                                                  |
Ben Finney



From ben+python at benfinney.id.au  Tue Nov 16 01:17:34 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 16 Nov 2010 11:17:34 +1100
Subject: [Python-ideas] syntax for set
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTinqTE=GLuS9ijEuQKL+AHBUA6GwUD9Ogwu+0_ST@mail.gmail.com>
	<ibsac1$idh$1@dough.gmane.org>
Message-ID: <87y68u9l5t.fsf@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> On 11/15/2010 2:02 PM, Guido van Rossum wrote:
> > There is nothing obvious about {:} for the empty dictionary. :-)
>
> 'Obvious' is always relative to the beholder ;-).

The proposed syntax is non-obvious if you're Dutch, perhaps? :-)

-- 
 \                         ?Sometimes I ? no, I don't.? ?Steven Wright |
  `\                                                                   |
_o__)                                                                  |
Ben Finney



From ben+python at benfinney.id.au  Tue Nov 16 01:19:04 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 16 Nov 2010 11:19:04 +1100
Subject: [Python-ideas] syntax for set
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
Message-ID: <87tyji9l3b.fsf@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> The *only* glitch is the lack of an empty set notation.

It's a very small glitch, especially because ?set()? is so few
characters. I see no pressing need to have syntactic sugar for that.

-- 
 \       ?Crime is contagious? if the government becomes a lawbreaker, |
  `\          it breeds contempt for the law.? ?Justice Louis Brandeis |
_o__)                                                                  |
Ben Finney



From dreamingforward at gmail.com  Tue Nov 16 04:16:40 2010
From: dreamingforward at gmail.com (average)
Date: Mon, 15 Nov 2010 20:16:40 -0700
Subject: [Python-ideas] syntax for set
In-Reply-To: <ibrv3a$pk6$1@dough.gmane.org>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
Message-ID: <AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>

The *only* glitch is the lack of an empty set notation. The idea of

> dict() == {:}; set()=={};
> which would be the obvious choice if starting fresh,
> was considered but rejected by Guido as too disruptive
> and introducing another barrier between Python 2 and 3.


I have to say that the *lack*of those two suggested literals are a sore
point for me (that along with the fact that dict doesn't inherit from set).
Call me a nit-picker, but (without them) they are like sand in the vast
gearbox of my mind when using python3: small but disruptive.  I would argue
that the lack of them is what is *preventing* the adoption of python3.  And
although I appreciate the kind reluctance to change the long-standing {} for
the empty dict, I hasten to mention that the universe will naturally
gravitate to that which is most elegant, so better to feel the pain at the
beginning than a long demise without them.  (I suspect there would even be a
small but measurable increase in downloads should python offer the option. )

Along those lines of simple ease-of-use issues, help() should have a verbose
option, so that doctests don't clutter up basic interface information.  And
I put another vote (from a discussion long ago) in favor of a test()
built-in to make test-driven development a core part of the language.  This
would be a quite novel and powerful step in front of any other known
language.

Just some minor thoughts from nobody in particular  .... :)

marcos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101115/3babeab7/attachment.html>

From scott+python-ideas at scottdial.com  Tue Nov 16 11:19:22 2010
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Tue, 16 Nov 2010 05:19:22 -0500
Subject: [Python-ideas] pool threads
In-Reply-To: <AANLkTi=PuSON1=2PhFanS=_Cyyvn9rLNhD6d8vat5Z3b@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE57872329@exchcn.ccp.ad.local>
	<AANLkTi=PuSON1=2PhFanS=_Cyyvn9rLNhD6d8vat5Z3b@mail.gmail.com>
Message-ID: <4CE25AAA.9010308@scottdial.com>

On 11/9/2010 11:07 PM, Guido van Rossum wrote:
> Have you looked at PEP 3148?

I have been rewriting a web interface for a cluster of servers that I
wanted to do batch queries in parallel and the first that came to mind
was to try out this futures module (I'm using the version from pypi
w/Python 2.6).

I had some difficulty getting started using the module due to a lack of
examples of all the different ways of using pools (in particular, there
is no example showing something like what Kristj?n wants -- a
set-it-and-forget-it pool). However, once I got past the learning curve,
I made some code that does what I want.

Despite the code working, it wasn't as fast as I thought it should be,
so I started profiling the code, and I noticed some craziness. Looking
closer at it, I hardly believe anyone else has used this for anything
but toy examples. For clarity, the code I am profiling is basically this:

executor = futures.ThreadPoolExecutor(len(servers))
futures_list = []
for address in servers:
    def call(address=address):
        client = None
        load = None
        try:
            client = Client(address)
            load = client.load()
        finally:
            if client:
                client.close()
        return (address, load)
    futures_list.append(executor.submit(call))

c.servers = []
for future in futures.as_completed(futures_list):
    result = future.result()
    if result:
        address, load = result
        c.servers.append({'server': address, 'load': load})

Running this yields an insane number of wait()s on an Event() object.

.../futures/_base.py:149 as_completed x36     2673.00ms
.../threading.py:391     wait         x421111 1720.10ms

Looking closer at as_completed(), the call to wait is wrong:

            waiter.event.wait(timeout)

should be:

            waiter.event.wait(wait_timeout)

However, that isn't my problem. More importantly, the Event() itself
represents the wrong synchronization primitive:

        waiter = _create_and_install_waiters(fs, FIRST_COMPLETED)

Creates an Event() that is set() when the first future in fs completes,
which then spins as_completed() as the event will stay set forever.

Obviously, I will file a bug report (w/patch) about this, but I post my
experience here because it's pretty much exemplifies why I believe even
small libraries should see use in the wild before being tossed into the
stdlib -- I'm glad 3.2 is only alpha. Might be nice if the release page
mentioned this module to encourage more use of it.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From stefan_ml at behnel.de  Tue Nov 16 12:59:31 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 16 Nov 2010 12:59:31 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
Message-ID: <ibtrn5$e8m$1@dough.gmane.org>

average, 16.11.2010 04:16:
> The *only* glitch is the lack of an empty set notation. The idea of
>
>> dict() == {:}; set()=={};
>> which would be the obvious choice if starting fresh,
>> was considered but rejected by Guido as too disruptive
>> and introducing another barrier between Python 2 and 3.
>
> I have to say that the *lack*of those two suggested literals are a sore
> point for me (that along with the fact that dict doesn't inherit from set).
> Call me a nit-picker, but (without them) they are like sand in the vast
> gearbox of my mind when using python3: small but disruptive.  I would argue
> that the lack of them is what is *preventing* the adoption of python3.

I doubt that.

Stefan



From python at mrabarnett.plus.com  Tue Nov 16 19:32:35 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 16 Nov 2010 18:32:35 +0000
Subject: [Python-ideas] syntax for set
In-Reply-To: <ibtrn5$e8m$1@dough.gmane.org>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
	<ibtrn5$e8m$1@dough.gmane.org>
Message-ID: <4CE2CE43.3040907@mrabarnett.plus.com>

On 16/11/2010 11:59, Stefan Behnel wrote:
> average, 16.11.2010 04:16:
>> The *only* glitch is the lack of an empty set notation. The idea of
>>
>>> dict() == {:}; set()=={};
>>> which would be the obvious choice if starting fresh,
>>> was considered but rejected by Guido as too disruptive
>>> and introducing another barrier between Python 2 and 3.
>>
>> I have to say that the *lack*of those two suggested literals are a sore
>> point for me (that along with the fact that dict doesn't inherit from
>> set).
>> Call me a nit-picker, but (without them) they are like sand in the vast
>> gearbox of my mind when using python3: small but disruptive. I would
>> argue
>> that the lack of them is what is *preventing* the adoption of python3.
>
> I doubt that.
>
I doubt it too.


From dreamingforward at gmail.com  Wed Nov 17 06:02:54 2010
From: dreamingforward at gmail.com (average)
Date: Tue, 16 Nov 2010 22:02:54 -0700
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CE2CE43.3040907@mrabarnett.plus.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
	<ibtrn5$e8m$1@dough.gmane.org>
	<4CE2CE43.3040907@mrabarnett.plus.com>
Message-ID: <AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>

On Tue, Nov 16, 2010 at 11:32 AM, MRAB <python at mrabarnett.plus.com> wrote:

> On 16/11/2010 11:59, Stefan Behnel wrote:
>
>> average, 16.11.2010 04:16:
>>
>>> The *only* glitch is the lack of an empty set notation. The idea of
>>>
>>>  dict() == {:}; set()=={};
>>>> which would be the obvious choice if starting fresh,
>>>> was considered but rejected by Guido as too disruptive
>>>> and introducing another barrier between Python 2 and 3.
>>>>
>>>
>>> I have to say that the *lack*of those two suggested literals are a sore
>>> point for me (that along with the fact that dict doesn't inherit from
>>> set).
>>> Call me a nit-picker, but (without them) they are like sand in the vast
>>> gearbox of my mind when using python3: small but disruptive. I would
>>> argue
>>> that the lack of them is what is *preventing* the adoption of python3.
>>>
>>
>> I doubt that.
>>
>>  I doubt it too.
>
> Haha, of course you do.  If you didn't, it would've already been done.
But don't mislead yourselves either, as members of the old guard (?), you
never get to measure the users that walk away.   And you run into the easy
trap of "false neutrals".

marcos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101116/5027c413/attachment.html>

From bruce at leapyear.org  Wed Nov 17 06:27:28 2010
From: bruce at leapyear.org (Bruce Leban)
Date: Tue, 16 Nov 2010 21:27:28 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
	<ibtrn5$e8m$1@dough.gmane.org> <4CE2CE43.3040907@mrabarnett.plus.com>
	<AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>
Message-ID: <AANLkTi=e=m52cNTRJxU_iq0T5-sVWb24hKZ-70eiXHBu@mail.gmail.com>

That would explain why no one uses C, C++, C#, Ruby, Java, Basic, Fortran,
etc. None of these languages have a simple literal for an empty set.

That's why we're all using Pascal which does have a set literal [].

--- Bruce :-)
Learn about security: http://j.mp/gruyere-security



On Tue, Nov 16, 2010 at 9:02 PM, average <dreamingforward at gmail.com> wrote:

> On Tue, Nov 16, 2010 at 11:32 AM, MRAB <python at mrabarnett.plus.com> wrote:
>
>> On 16/11/2010 11:59, Stefan Behnel wrote:
>>
>>> average, 16.11.2010 04:16:
>>>
>>>> The *only* glitch is the lack of an empty set notation. The idea of
>>>>
>>>>  dict() == {:}; set()=={};
>>>>> which would be the obvious choice if starting fresh,
>>>>> was considered but rejected by Guido as too disruptive
>>>>> and introducing another barrier between Python 2 and 3.
>>>>>
>>>>
>>>> I have to say that the *lack*of those two suggested literals are a sore
>>>> point for me (that along with the fact that dict doesn't inherit from
>>>> set).
>>>> Call me a nit-picker, but (without them) they are like sand in the vast
>>>> gearbox of my mind when using python3: small but disruptive. I would
>>>> argue
>>>> that the lack of them is what is *preventing* the adoption of python3.
>>>>
>>>
>>> I doubt that.
>>>
>>>  I doubt it too.
>>
>> Haha, of course you do.  If you didn't, it would've already been done.
> But don't mislead yourselves either, as members of the old guard (?), you
> never get to measure the users that walk away.   And you run into the easy
> trap of "false neutrals".
>
> marcos
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101116/a9e06fef/attachment.html>

From stephen at xemacs.org  Wed Nov 17 08:39:13 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 17 Nov 2010 16:39:13 +0900
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
	<ibtrn5$e8m$1@dough.gmane.org>
	<4CE2CE43.3040907@mrabarnett.plus.com>
	<AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>
Message-ID: <87pqu4csbi.fsf@uwakimon.sk.tsukuba.ac.jp>

average writes:

 > But don't mislead yourselves either, as members of the old guard (?), you
 > never get to measure the users that walk away.

Nobody does, though.  At some point a project has to go with the
information it has, and the users it has.

Anyway, the only literal that matters whose invention is (more or
less) recorded in history is "0".  All the rest are syntactic
artificial sweeteners.



From swapnil.st at gmail.com  Wed Nov 17 09:26:43 2010
From: swapnil.st at gmail.com (Swapnil Talekar)
Date: Wed, 17 Nov 2010 13:56:43 +0530
Subject: [Python-ideas] In-process interpreters
Message-ID: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>

I am working on a project where we have a very peculiar requirements for an
embedded interpreter. In my opinion these requirements cannot be extremely
rare. I am posting this message to get some feedback from people who have
worked on similar stuff or are interested in such a solution. If such a
solution would be useful to a lot people if readily available as open
source.

We need to run multiple interpreters in-process. Which means we need to
create completely independent environments for each interpreter using some
very dirty hack. Since Python inherently does not support this. Python
threads( created using python's threading module ) are not of much help
because of the GIL.
For this purpose the solution we have adopted is moving all the mutable
python globals to heap.

Nokia did a similar change to Python while porting it to S60 platform.
Reference <http://research.nokia.com/files/PythonOnAPhone.ppt>
It seems Nokia has stopped using this solution but I am not aware of how are
they going about now since I cannot think of any other solution to run
multiple in-process interpreters. Jukka, can you throw some light ?

Lua already supports this.

I would expect other projects like mod_python would also have to face
similar problem but I am not aware how they are handling it.

People who have ported Python on flat-memory model OSes such as VxWorks
surely face this problem (Since there are no processes) unless they don't
have requirement of running multiple interpreters.

The second major change we did is to reinitialize python after executing of
each script. This is because of the limitation of running it on embedded
platform hardware and limited memory constraints. Again Python inherently
does not provide support for this. (Calling Py_Initialize and
Py_Finalize repetitively is not enough. It does not completely reinitialize
python and is also known to leak memory)


Would it be worthwhile (benefit a lot of people ) if these changes be made
to standard python (eg. http://bugs.python.org/issue3329 ). Possibly a new
branch of Python which supports multiple in-process interpreters?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101117/7628d699/attachment.html>

From pyideas at rebertia.com  Wed Nov 17 09:41:32 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Wed, 17 Nov 2010 00:41:32 -0800
Subject: [Python-ideas] In-process interpreters
In-Reply-To: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
Message-ID: <AANLkTik-jcBUZqqQ4zSTAj4OnMHYC_C9_iVtP_hVif+E@mail.gmail.com>

On Wed, Nov 17, 2010 at 12:26 AM, Swapnil Talekar <swapnil.st at gmail.com> wrote:
> I am working on a project where we have a very peculiar requirements for an
> embedded interpreter. In my opinion these requirements cannot be extremely
> rare. I am posting this message to get some feedback from people who have
> worked on similar stuff or are interested in such a solution. If such a
> solution would be useful to a lot people if readily available as open
> source.
> We need to run multiple interpreters in-process. Which means we need to
> create completely independent environments for each interpreter using some
> very dirty hack. Since Python inherently does not support this. Python
> threads( created using python's threading module ) are not of much help
> because of the GIL.
> For this purpose the solution we have adopted is moving all the mutable
> python globals to heap.
> Nokia did a similar change to Python while porting it to S60 platform.
> Reference
> It seems Nokia has stopped using this solution but I am not aware of how are
> they going about now since I cannot think of any other solution to run
> multiple in-process interpreters. Jukka, can you throw some light ?
> Lua already supports this.
> I would expect other projects like mod_python would also have to face
> similar problem but I am not aware how they are handling it.

It used* "sub-interpreters" apparently:
http://modpython.org/live/current/doc-html/pyapi-interps.html
http://docs.python.org/c-api/init.html#Py_NewInterpreter

Cheers,
Chris

*mod_python stopped being actively developed in June, and was
apparently moribund since 2007.
--
http://blog.rebertia.com


From sergio at gruposinternet.com.br  Wed Nov 17 13:39:38 2010
From: sergio at gruposinternet.com.br (=?ISO-8859-1?Q?S=E9rgio?= Surkamp)
Date: Wed, 17 Nov 2010 10:39:38 -0200
Subject: [Python-ideas] In-process interpreters
In-Reply-To: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
Message-ID: <20101117103938.39199ba2@icedearth.corp.grupos.com.br>

Em Wed, 17 Nov 2010 13:56:43 +0530
Swapnil Talekar <swapnil.st at gmail.com> escreveu:

> We need to run multiple interpreters in-process. Which means we need
> to create completely independent environments for each interpreter
> using some very dirty hack. Since Python inherently does not support
> this. Python threads( created using python's threading module ) are
> not of much help because of the GIL.
> For this purpose the solution we have adopted is moving all the
> mutable python globals to heap.

What about using multiple processes? You could use pipes to exchange
information between them.

http://docs.python.org/library/subprocess.html
http://docs.python.org/library/os.html#os.fork (Unix only)

Regards,
-- 
  .:''''':.
.:'        `     S?rgio Surkamp | Gerente de Rede
::    ........   sergio at gruposinternet.com.br
`:.        .:'
  `:,   ,.:'     *Grupos Internet S.A.*
    `: :'        R. Lauro Linhares, 2123 Torre B - Sala 201
     : :         Trindade - Florian?polis - SC
     :.'
     ::          +55 48 3234-4109
     :
     '           http://www.gruposinternet.com.br


From swapnil.st at gmail.com  Wed Nov 17 15:36:47 2010
From: swapnil.st at gmail.com (Swapnil Talekar)
Date: Wed, 17 Nov 2010 20:06:47 +0530
Subject: [Python-ideas] Fwd:  In-process interpreters
In-Reply-To: <AANLkTikika6gjUHy8nHGeqEaBykrzc5nvbQM0NXqaXYQ@mail.gmail.com>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
	<20101117103938.39199ba2@icedearth.corp.grupos.com.br>
	<AANLkTikika6gjUHy8nHGeqEaBykrzc5nvbQM0NXqaXYQ@mail.gmail.com>
Message-ID: <AANLkTinPihaRKL70aHu-7JWVPB+N75zMqwxm6T=aKzZn@mail.gmail.com>

> Em Wed, 17 Nov 2010 13:56:43 +0530
> Swapnil Talekar <swapnil.st at gmail.com> escreveu:
>
> > We need to run multiple interpreters in-process. Which means we need
> > to create completely independent environments for each interpreter
> > using some very dirty hack. Since Python inherently does not support
> > this. Python threads( created using python's threading module ) are
> > not of much help because of the GIL.
> > For this purpose the solution we have adopted is moving all the
> > mutable python globals to heap.
>
> What about using multiple processes? You could use pipes to exchange
> information between them.
>

As stated in the question, this is not possible for OSes like VxWorks (which
is not rare on embedded platforms). Also there could be other constraints
like the software architecture of the given system which make  having the
embedded interpreter in-process, more useful. Exchanging messages with
separate interpreter process is slower. Also, the embedding process will
have to create and expose a whole new interface which can be used by
external processes. While if the interpreter is in-process, then its just a
matter adding the services of embedding process as C-extensions to Python.
One function call is all you need.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101117/6d2a91f4/attachment.html>

From swapnil.st at gmail.com  Wed Nov 17 15:46:20 2010
From: swapnil.st at gmail.com (Swapnil Talekar)
Date: Wed, 17 Nov 2010 20:16:20 +0530
Subject: [Python-ideas]  In-process interpreters
In-Reply-To: <AANLkTik-jcBUZqqQ4zSTAj4OnMHYC_C9_iVtP_hVif+E@mail.gmail.com>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
	<AANLkTik-jcBUZqqQ4zSTAj4OnMHYC_C9_iVtP_hVif+E@mail.gmail.com>
Message-ID: <AANLkTin_PPAr9ko4ZrAxF-j7zsbmXaRvoKjdoOJofFj-@mail.gmail.com>

On Wed, Nov 17, 2010 at 2:11 PM, Chris Rebert <pyideas at rebertia.com> wrote:

> On Wed, Nov 17, 2010 at 12:26 AM, Swapnil Talekar <swapnil.st at gmail.com>
> wrote:
> > I am working on a project where we have a very peculiar requirements for
> an
> > embedded interpreter. In my opinion these requirements cannot be
> extremely
> > rare. I am posting this message to get some feedback from people who have
> > worked on similar stuff or are interested in such a solution. If such a
> > solution would be useful to a lot people if readily available as open
> > source.
> > We need to run multiple interpreters in-process. Which means we need to
> > create completely independent environments for each interpreter using
> some
> > very dirty hack. Since Python inherently does not support this. Python
> > threads( created using python's threading module ) are not of much help
> > because of the GIL.
> > For this purpose the solution we have adopted is moving all the mutable
> > python globals to heap.
> > Nokia did a similar change to Python while porting it to S60 platform.
> > Reference
> > It seems Nokia has stopped using this solution but I am not aware of how
> are
> > they going about now since I cannot think of any other solution to run
> > multiple in-process interpreters. Jukka, can you throw some light ?
> > Lua already supports this.
> > I would expect other projects like mod_python would also have to face
> > similar problem but I am not aware how they are handling it.
>
> It used* "sub-interpreters" apparently:
> http://modpython.org/live/current/doc-html/pyapi-interps.html
> http://docs.python.org/c-api/init.html#Py_NewInterpreter


Sub-interpreters aren't attractive for the very reasons given in the Bugs
and caveats section of the above link. Moreover Sub-interpreters do not
release memory (as far as I have tested) even after calling
Py_EndInterpreter. This makes them unusable on platforms with high memory
constraint.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101117/34c6680a/attachment.html>

From kristjan at ccpgames.com  Thu Nov 18 09:06:14 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 18 Nov 2010 16:06:14 +0800
Subject: [Python-ideas] In-process interpreters
In-Reply-To: <AANLkTin_PPAr9ko4ZrAxF-j7zsbmXaRvoKjdoOJofFj-@mail.gmail.com>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
	<AANLkTik-jcBUZqqQ4zSTAj4OnMHYC_C9_iVtP_hVif+E@mail.gmail.com>
	<AANLkTin_PPAr9ko4ZrAxF-j7zsbmXaRvoKjdoOJofFj-@mail.gmail.com>
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE57872A3D@exchcn.ccp.ad.local>

This is one of the things I'd have liked to see as part of py3k:  "fixing" python for embedding in applications and defices.
It would have meant getting rid of all globals and statics and tracking them in some sort of "context" structure.  Since you'd have to change the C API, you could take the opportunity to fix some of its ugliest warts, such as inconsistent reference counting semantics between methods (making it difficult to write good smart pointer wrappers for it, as in COM.  COM has very simple, clear and universal reference counting rules).
K


From: python-ideas-bounces+kristjan=ccpgames.com at python.org [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Swapnil Talekar
Sent: Wednesday, November 17, 2010 22:46
To: Chris Rebert; python-ideas at python.org
Subject: [Python-ideas] In-process interpreters


On Wed, Nov 17, 2010 at 2:11 PM, Chris Rebert <pyideas at rebertia.com<mailto:pyideas at rebertia.com>> wrote:
On Wed, Nov 17, 2010 at 12:26 AM, Swapnil Talekar <swapnil.st<http://swapnil.st>@gmail.com<http://gmail.com>> wrote:
> I am working on a project where we have a very peculiar requirements for an
> embedded interpreter. In my opinion these requirements cannot be extremely
> rare. I am posting this message to get some feedback from people who have
> worked on similar stuff or are interested in such a solution. If such a
> solution would be useful to a lot people if readily available as open
> source.
> We need to run multiple interpreters in-process. Which means we need to
> create completely independent environments for each interpreter using some
> very dirty hack. Since Python inherently does not support this. Python
> threads( created using python's threading module ) are not of much help
> because of the GIL.
> For this purpose the solution we have adopted is moving all the mutable
> python globals to heap.
> Nokia did a similar change to Python while porting it to S60 platform.
> Reference
> It seems Nokia has stopped using this solution but I am not aware of how are
> they going about now since I cannot think of any other solution to run
> multiple in-process interpreters. Jukka, can you throw some light ?
> Lua already supports this.
> I would expect other projects like mod_python would also have to face
> similar problem but I am not aware how they are handling it.
It used* "sub-interpreters" apparently:
http://modpython.org/live/current/doc-html/pyapi-interps.html
http://docs.python.org/c-api/init.html#Py_NewInterpreter

Sub-interpreters aren't attractive for the very reasons given in the Bugs and caveats section of the above link. Moreover Sub-interpreters do not release memory (as far as I have tested) even after calling Py_EndInterpreter. This makes them unusable on platforms with high memory constraint.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101118/17722e22/attachment.html>

From tlesher at gmail.com  Thu Nov 18 10:08:04 2010
From: tlesher at gmail.com (Tim Lesher)
Date: Thu, 18 Nov 2010 04:08:04 -0500
Subject: [Python-ideas] In-process interpreters
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE57872A3D@exchcn.ccp.ad.local>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>
	<AANLkTik-jcBUZqqQ4zSTAj4OnMHYC_C9_iVtP_hVif+E@mail.gmail.com>
	<AANLkTin_PPAr9ko4ZrAxF-j7zsbmXaRvoKjdoOJofFj-@mail.gmail.com>
	<2E034B571A5CE44E949B9FCC3B6D24EE57872A3D@exchcn.ccp.ad.local>
Message-ID: <AANLkTi=eyWz=EwwtAuer+SQC+_jOZxWymN-eeFtr03PS@mail.gmail.com>

2010/11/18 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> This is one of the things I?d have liked to see as part of py3k:? ?fixing?
> python for embedding in applications and defices.

My current work project has similar issues (also an embedded Python on
embedded hardware). We've examined Py_Initialize()/Py_Finalize()
pairing, subinterpreters, and out-of-proc interpreters, and are in the
same predicament.

I believe one of the main difficulties will be doing it in a way that
doesn't penalize the 99% use case that most of the Python world
experiences (one interpreter/one process, named python).  That's been,
for example, the fatal issue with most of the efforts to eliminate the
global interpreter lock.

It doesn't matter how nice it makes life for us if it causes
performance or other headaches for the vast majority of Python users.
-- 
Tim Lesher <tlesher at gmail.com>


From ziade.tarek at gmail.com  Thu Nov 18 16:14:35 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Nov 2010 16:14:35 +0100
Subject: [Python-ideas] Detecting circular imports ?
Message-ID: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>

Hello,

This morning I tried to fix an issue for a while before I realized I
had a circular import. This issue is not obvious because you get a
cryptic error, like an AttributeError and it can tak a while before
finding out.

I don't know of this was mentioned before, or how hard it would be.
But it would be nice if Python had a specific "CircularImportError"
raised in that case, or something..
That would be a fabulous hint for developers.

Cheers
Tarek
-- 
Tarek Ziad? | http://ziade.org


From solipsis at pitrou.net  Thu Nov 18 16:22:36 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 18 Nov 2010 16:22:36 +0100
Subject: [Python-ideas] Detecting circular imports ?
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
Message-ID: <20101118162236.680ddca7@pitrou.net>

On Thu, 18 Nov 2010 16:14:35 +0100
Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> 
> I don't know of this was mentioned before, or how hard it would be.
> But it would be nice if Python had a specific "CircularImportError"
> raised in that case, or something..
> That would be a fabulous hint for developers.

It can't be an error, since circular imports are supported.
There could be a warning, but IMO it would need to be silenced by
default (which means ImportWarning isn't appropriate).

Regards

Antoine.




From ianb at colorstudy.com  Thu Nov 18 16:36:13 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 18 Nov 2010 09:36:13 -0600
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
Message-ID: <AANLkTimKrrkCpGhtaBF4Fh7J5mZK+5AuD+TyuM1cpaUZ@mail.gmail.com>

On Thu, Nov 18, 2010 at 9:14 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> This morning I tried to fix an issue for a while before I realized I
> had a circular import. This issue is not obvious because you get a
> cryptic error, like an AttributeError and it can tak a while before
> finding out.
>
> I don't know of this was mentioned before, or how hard it would be.
> But it would be nice if Python had a specific "CircularImportError"
> raised in that case, or something..
> That would be a fabulous hint for developers.
>

Maybe more specifically, a module could be marked as "loading" in some
fashion, until it's been completely executed.  Any AttributeError from a
loading module would instead raise something like CircularImportError
(subclassing AttributeError).

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101118/663ddfac/attachment.html>

From g.brandl at gmx.net  Thu Nov 18 16:41:35 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 18 Nov 2010 16:41:35 +0100
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <20101118162236.680ddca7@pitrou.net>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<20101118162236.680ddca7@pitrou.net>
Message-ID: <ic3hif$u6i$1@dough.gmane.org>

Am 18.11.2010 16:22, schrieb Antoine Pitrou:
> On Thu, 18 Nov 2010 16:14:35 +0100
> Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> 
>> I don't know of this was mentioned before, or how hard it would be.
>> But it would be nice if Python had a specific "CircularImportError"
>> raised in that case, or something..
>> That would be a fabulous hint for developers.
> 
> It can't be an error, since circular imports are supported.
> There could be a warning, but IMO it would need to be silenced by
> default (which means ImportWarning isn't appropriate).

I thought ImportWarning was silenced by default?

Anyway, I don't think that would be helpful for developers, who would
need to turn on the warnings explicitly to see them.  But to think of
that, you already need to suspect circular imports, and then it's easy
to see it from the traceback anyway.

Georg



From ziade.tarek at gmail.com  Thu Nov 18 17:00:43 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Nov 2010 17:00:43 +0100
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <ic3hif$u6i$1@dough.gmane.org>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<20101118162236.680ddca7@pitrou.net> <ic3hif$u6i$1@dough.gmane.org>
Message-ID: <AANLkTikdf67bJ-Ar6jg5ASarCWBqMzFs3ikUYfu+AWLj@mail.gmail.com>

On Thu, Nov 18, 2010 at 4:41 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 18.11.2010 16:22, schrieb Antoine Pitrou:
>> On Thu, 18 Nov 2010 16:14:35 +0100
>> Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>>
>>> I don't know of this was mentioned before, or how hard it would be.
>>> But it would be nice if Python had a specific "CircularImportError"
>>> raised in that case, or something..
>>> That would be a fabulous hint for developers.
>>
>> It can't be an error, since circular imports are supported.
>> There could be a warning, but IMO it would need to be silenced by
>> default (which means ImportWarning isn't appropriate).
>
> I thought ImportWarning was silenced by default?
>
> Anyway, I don't think that would be helpful for developers, who would
> need to turn on the warnings explicitly to see them. ?But to think of
> that, you already need to suspect circular imports, and then it's easy
> to see it from the traceback anyway.

mmm I am not talking about warnings here. As Ian said, and to be more
explicit the use case is:

  Your program crashed with an error because you have a circular
import -- that's a bug you need to fix.

How can we tell it in a more explicit fashion that an AttributeError
which is not very meaningful in this case ?


> Georg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Tarek Ziad? | http://ziade.org


From dreamingforward at gmail.com  Thu Nov 18 22:40:05 2010
From: dreamingforward at gmail.com (average)
Date: Thu, 18 Nov 2010 14:40:05 -0700
Subject: [Python-ideas] syntax for set
In-Reply-To: <87pqu4csbi.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
	<ibtrn5$e8m$1@dough.gmane.org>
	<4CE2CE43.3040907@mrabarnett.plus.com>
	<AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>
	<87pqu4csbi.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <AANLkTimbW5fMxRcHpod6NpUPZ7dMPpNwfbi2Da3ud55O@mail.gmail.com>

 > But don't mislead yourselves either, as members of the old guard (?), you

>  > never get to measure the users that walk away.
>
> Nobody does, though.  At some point a project has to go with the
> information it has, and the users it has.
>

Truly.  But the point is that the universe will inevitably evolve towards
elegance.  To only "go on with the information it has and the users it has"
ensures that it remains merely a branch off the trunk of the ideal
(consequently drawing few new users), that eventually will need either to be
*deliberately* pruned, or will wither and break off unexpectedly at some
future point.  Remember, the bulk of new, loyal, users are drawn by an ideal
(you might remember yourself being among them), not even necessarily by how
big the library is.

Just some thoughts to consider....

marcos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101118/9fe45c68/attachment.html>

From steve at pearwood.info  Thu Nov 18 23:50:53 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 19 Nov 2010 09:50:53 +1100
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTimbW5fMxRcHpod6NpUPZ7dMPpNwfbi2Da3ud55O@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>	<ibtrn5$e8m$1@dough.gmane.org>	<4CE2CE43.3040907@mrabarnett.plus.com>	<AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>	<87pqu4csbi.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimbW5fMxRcHpod6NpUPZ7dMPpNwfbi2Da3ud55O@mail.gmail.com>
Message-ID: <4CE5ADCD.5050904@pearwood.info>

average wrote:

> Truly.  But the point is that the universe will inevitably evolve towards
> elegance.  

"Inevitably"? I think the only think we can say that the universe will 
inevitably evolve towards is a state of maximum entropy. But frankly, I 
don't see the relevance to the future heat death of the universe in 
hundreds of billions of years to Python's syntax.


 > To only "go on with the information it has and the users it has"
> ensures that it remains merely a branch off the trunk of the ideal
> (consequently drawing few new users), that eventually will need either to be
> *deliberately* pruned, or will wither and break off unexpectedly at some
> future point.  Remember, the bulk of new, loyal, users are drawn by an ideal
> (you might remember yourself being among them), not even necessarily by how
> big the library is.

I beg to differ... it seems to me that users are drawn to languages for 
many reasons, and while elegance can be one of them, it is fairly low 
down the list. Why else would C++, VB and PHP be so popular, while 
Haskell and Scheme remain tiny and unpopular (although very influential)?

Besides, how do you define "elegant"? I've heard it said, in full 
seriousness, that BrainF*** is the most elegant language because it is a 
fully Turing-complete language in only six commands (plus two more for IO).

Personally, I see Python as an extremely elegant language, but that 
elegance is tempered by a very strong dose of practicality and realism. 
Gratuitously breaking backwards compatibility just so that there is a 
literal for the empty set is not what I call either practical or realistic.



-- 
Steven



From greg.ewing at canterbury.ac.nz  Fri Nov 19 00:07:27 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 19 Nov 2010 12:07:27 +1300
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <ic3hif$u6i$1@dough.gmane.org>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<20101118162236.680ddca7@pitrou.net> <ic3hif$u6i$1@dough.gmane.org>
Message-ID: <4CE5B1AF.6080502@canterbury.ac.nz>

Georg Brandl wrote:
> But to think of
> that, you already need to suspect circular imports, and then it's easy
> to see it from the traceback anyway.

Right. Whenever I've had a problem with circular imports, it
was usually fairly obvious what the cause was.

-- 
Greg


From mal at egenix.com  Fri Nov 19 00:14:00 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 19 Nov 2010 00:14:00 +0100
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <4CE5B1AF.6080502@canterbury.ac.nz>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>	<20101118162236.680ddca7@pitrou.net>
	<ic3hif$u6i$1@dough.gmane.org> <4CE5B1AF.6080502@canterbury.ac.nz>
Message-ID: <4CE5B338.1010607@egenix.com>

Greg Ewing wrote:
> Georg Brandl wrote:
>> But to think of
>> that, you already need to suspect circular imports, and then it's easy
>> to see it from the traceback anyway.
> 
> Right. Whenever I've had a problem with circular imports, it
> was usually fairly obvious what the cause was.

Same here and most cases I've had in the past actually presented
themselves as... ImportErrors :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 19 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From fuzzyman at voidspace.org.uk  Fri Nov 19 00:21:08 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 18 Nov 2010 23:21:08 +0000
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <4CE5B338.1010607@egenix.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<20101118162236.680ddca7@pitrou.net> <ic3hif$u6i$1@dough.gmane.org>
	<4CE5B1AF.6080502@canterbury.ac.nz> <4CE5B338.1010607@egenix.com>
Message-ID: <AANLkTikbQ5uVedUfr5_EWqZGUpSu79jRgoPoyK5cUTrJ@mail.gmail.com>

On 18 November 2010 23:14, M.-A. Lemburg <mal at egenix.com> wrote:

> Greg Ewing wrote:
> > Georg Brandl wrote:
> >> But to think of
> >> that, you already need to suspect circular imports, and then it's easy
> >> to see it from the traceback anyway.
> >
> > Right. Whenever I've had a problem with circular imports, it
> > was usually fairly obvious what the cause was.
>
> Same here and most cases I've had in the past actually presented
> themselves as... ImportErrors :-)
>


If you use the "from module import something" form it will be an
ImportError. If you do "import module" and then "module.something" you will
see an AttributeError. These are the ones that can be confusing.

Michael



>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Nov 19 2010)
> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101118/467e3941/attachment.html>

From ziade.tarek at gmail.com  Fri Nov 19 00:29:22 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 19 Nov 2010 00:29:22 +0100
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <AANLkTikbQ5uVedUfr5_EWqZGUpSu79jRgoPoyK5cUTrJ@mail.gmail.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<20101118162236.680ddca7@pitrou.net> <ic3hif$u6i$1@dough.gmane.org>
	<4CE5B1AF.6080502@canterbury.ac.nz> <4CE5B338.1010607@egenix.com>
	<AANLkTikbQ5uVedUfr5_EWqZGUpSu79jRgoPoyK5cUTrJ@mail.gmail.com>
Message-ID: <AANLkTinZ-RnWd67G=ze6JunwO0NMV1ahp5eEU0SV35Cz@mail.gmail.com>

On Fri, Nov 19, 2010 at 12:21 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>
>
> On 18 November 2010 23:14, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>> Greg Ewing wrote:
>> > Georg Brandl wrote:
>> >> But to think of
>> >> that, you already need to suspect circular imports, and then it's easy
>> >> to see it from the traceback anyway.
>> >
>> > Right. Whenever I've had a problem with circular imports, it
>> > was usually fairly obvious what the cause was.
>>
>> Same here and most cases I've had in the past actually presented
>> themselves as... ImportErrors :-)
>
>
> If you use the "from module import something" form it will be an
> ImportError. If you do "import module" and then "module.something" you will
> see an AttributeError. These are the ones that can be confusing.

Yes that's the case I was thinking of. After years of Python I can
still get trapped on those. But maybe that's just me :)


-- 
Tarek Ziad? | http://ziade.org


From tjreedy at udel.edu  Fri Nov 19 04:02:55 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 18 Nov 2010 22:02:55 -0500
Subject: [Python-ideas] In-process interpreters
In-Reply-To: <AANLkTi=eyWz=EwwtAuer+SQC+_jOZxWymN-eeFtr03PS@mail.gmail.com>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>	<AANLkTik-jcBUZqqQ4zSTAj4OnMHYC_C9_iVtP_hVif+E@mail.gmail.com>	<AANLkTin_PPAr9ko4ZrAxF-j7zsbmXaRvoKjdoOJofFj-@mail.gmail.com>	<2E034B571A5CE44E949B9FCC3B6D24EE57872A3D@exchcn.ccp.ad.local>
	<AANLkTi=eyWz=EwwtAuer+SQC+_jOZxWymN-eeFtr03PS@mail.gmail.com>
Message-ID: <ic4pct$jk1$1@dough.gmane.org>

On 11/18/2010 4:08 AM, Tim Lesher wrote:
> 2010/11/18 Kristj?n Valur J?nsson<kristjan-hPzAiOG7rkJWk0Htik3J/w at public.gmane.org>:
>> This is one of the things I?d have liked to see as part of py3k:  ?fixing?
>> python for embedding in applications and defices.
>
> My current work project has similar issues (also an embedded Python on
> embedded hardware). We've examined Py_Initialize()/Py_Finalize()
> pairing, subinterpreters, and out-of-proc interpreters, and are in the
> same predicament.
>
> I believe one of the main difficulties will be doing it in a way that
> doesn't penalize the 99% use case that most of the Python world
> experiences (one interpreter/one process, named python).  That's been,
> for example, the fatal issue with most of the efforts to eliminate the
> global interpreter lock.
>
> It doesn't matter how nice it makes life for us if it causes
> performance or other headaches for the vast majority of Python users.

I keep wondering whether it would be possible to rearrange the current 
code in any way that a) would not hinder performance significantly but 
b) would make it easier to maintain patches for compiler variations for 
the minority use cases.

-- 
Terry Jan Reedy




From stephen at xemacs.org  Fri Nov 19 05:32:24 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 19 Nov 2010 13:32:24 +0900
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <AANLkTinZ-RnWd67G=ze6JunwO0NMV1ahp5eEU0SV35Cz@mail.gmail.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<20101118162236.680ddca7@pitrou.net> <ic3hif$u6i$1@dough.gmane.org>
	<4CE5B1AF.6080502@canterbury.ac.nz> <4CE5B338.1010607@egenix.com>
	<AANLkTikbQ5uVedUfr5_EWqZGUpSu79jRgoPoyK5cUTrJ@mail.gmail.com>
	<AANLkTinZ-RnWd67G=ze6JunwO0NMV1ahp5eEU0SV35Cz@mail.gmail.com>
Message-ID: <87vd3uaq7b.fsf@uwakimon.sk.tsukuba.ac.jp>

Tarek Ziad? writes:

 > Yes that's the case I was thinking of. After years of Python I can
 > still get trapped on those. But maybe that's just me :)

So you're suggesting (pseudo-code, not Python!)

    message = usual_message
    if object is instance of ModuleType:
        message += "\nreferenced object is a module; are your imports OK?"
    raise AttributeError(message)

or something like that?


From ncoghlan at gmail.com  Fri Nov 19 14:15:31 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 19 Nov 2010 23:15:31 +1000
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
Message-ID: <AANLkTimgo9T5hx1mKgRdKFq6Dk_b+fFvzpBNmag=yTUh@mail.gmail.com>

On Fri, Nov 19, 2010 at 1:14 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> This morning I tried to fix an issue for a while before I realized I
> had a circular import. This issue is not obvious because you get a
> cryptic error, like an AttributeError and it can tak a while before
> finding out.
>
> I don't know of this was mentioned before, or how hard it would be.
> But it would be nice if Python had a specific "CircularImportError"
> raised in that case, or something..
> That would be a fabulous hint for developers.

Feel free to take a shot at it - the relevant bug report [1] is only a
little over 6 years old! :)

If we could detect the situation well enough to give a specific error
message for it, we could probably do something smarter about it than
that.

I find my naive early messages about the import subsystem from back
then are kind of amusing now, though... little did I know what evils
lurked beneath the surface of the import statement...

Cheers,
Nick.

[1] http://bugs.python.org/issue992389

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Fri Nov 19 14:31:09 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 19 Nov 2010 23:31:09 +1000
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <AANLkTimgo9T5hx1mKgRdKFq6Dk_b+fFvzpBNmag=yTUh@mail.gmail.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<AANLkTimgo9T5hx1mKgRdKFq6Dk_b+fFvzpBNmag=yTUh@mail.gmail.com>
Message-ID: <AANLkTikW_+4Gby=G8CzT-jQE6B08BN-z66NhOQY0H5kU@mail.gmail.com>

On Fri, Nov 19, 2010 at 11:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Feel free to take a shot at it - the relevant bug report [1] is only a
> little over 6 years old! :)
> [1] http://bugs.python.org/issue992389

Less facetiously, there are 3 different things suggested in that bug
report that people could try:

1. Implement the set-and-rollback feature for submodule attributes in
parent packages during import
2. Implement the "try looking for the fully qualified name in
sys.modules" idea(/hack)
3. Track in-progress imports and throw specific errors if the import
code trips over one

The last option does have the advantage of requiring less coordination
with the developers of other implementations (given the subtleties of
this issue, I'd want feedback from at least PyPy, IronPython and
Jython as to what they can handle before we started messing with the
semantics of what the language allows as far as circular imports go)

The issue suffers from the fact that at least some of the people with
the knowledge to do something about it are not inclined to do so due
to strong stylistic objections (*ahem*).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From niki.spahiev at gmail.com  Fri Nov 19 10:38:19 2010
From: niki.spahiev at gmail.com (Niki Spahiev)
Date: Fri, 19 Nov 2010 11:38:19 +0200
Subject: [Python-ideas] In-process interpreters
In-Reply-To: <ic4pct$jk1$1@dough.gmane.org>
References: <AANLkTimARNO3we=ui4mfy4ZBrKtd4wPJW0XGnRZRiMYP@mail.gmail.com>	<AANLkTik-jcBUZqqQ4zSTAj4OnMHYC_C9_iVtP_hVif+E@mail.gmail.com>	<AANLkTin_PPAr9ko4ZrAxF-j7zsbmXaRvoKjdoOJofFj-@mail.gmail.com>	<2E034B571A5CE44E949B9FCC3B6D24EE57872A3D@exchcn.ccp.ad.local>	<AANLkTi=eyWz=EwwtAuer+SQC+_jOZxWymN-eeFtr03PS@mail.gmail.com>
	<ic4pct$jk1$1@dough.gmane.org>
Message-ID: <ic5gib$5rh$1@dough.gmane.org>

On 19.11.2010 05:02, Terry Reedy wrote:
>
> I keep wondering whether it would be possible to rearrange the current
> code in any way that a) would not hinder performance significantly but
> b) would make it easier to maintain patches for compiler variations for
> the minority use cases.
>

Restating problem this way leads to this idea:

struct PythonStaticState {
    ...
};

#ifdef EMBED
#define GLOBAL_STATE_MACRO PythoStaticState globalPythonState,
#else
#define GLOBAL_STATE_MACRO
PythonStaticState globalPythonState;
#endif

PyObject * PyAnyApi(GLOBAL_STATE_MACRO, a, b, c, d)
{
	globalPythonState.var1 = 0;
}

and using good compiler and linker for nonembeded case ;^)

Niki



From mal at egenix.com  Fri Nov 19 16:37:40 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 19 Nov 2010 16:37:40 +0100
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <AANLkTikW_+4Gby=G8CzT-jQE6B08BN-z66NhOQY0H5kU@mail.gmail.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>	<AANLkTimgo9T5hx1mKgRdKFq6Dk_b+fFvzpBNmag=yTUh@mail.gmail.com>
	<AANLkTikW_+4Gby=G8CzT-jQE6B08BN-z66NhOQY0H5kU@mail.gmail.com>
Message-ID: <4CE699C4.9020005@egenix.com>

Nick Coghlan wrote:
> On Fri, Nov 19, 2010 at 11:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Feel free to take a shot at it - the relevant bug report [1] is only a
>> little over 6 years old! :)
>> [1] http://bugs.python.org/issue992389
> 
> Less facetiously, there are 3 different things suggested in that bug
> report that people could try:
> 
> 1. Implement the set-and-rollback feature for submodule attributes in
> parent packages during import
> 2. Implement the "try looking for the fully qualified name in
> sys.modules" idea(/hack)
> 3. Track in-progress imports and throw specific errors if the import
> code trips over one
> 
> The last option does have the advantage of requiring less coordination
> with the developers of other implementations (given the subtleties of
> this issue, I'd want feedback from at least PyPy, IronPython and
> Jython as to what they can handle before we started messing with the
> semantics of what the language allows as far as circular imports go)
> 
> The issue suffers from the fact that at least some of the people with
> the knowledge to do something about it are not inclined to do so due
> to strong stylistic objections (*ahem*).

How would the implementation differentiate between a real attribute
error (missing symbol while setting up a module) and one caused by
a circular import (symbol available, but not yet defined) ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 19 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Fri Nov 19 16:55:15 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 20 Nov 2010 01:55:15 +1000
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <4CE699C4.9020005@egenix.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<AANLkTimgo9T5hx1mKgRdKFq6Dk_b+fFvzpBNmag=yTUh@mail.gmail.com>
	<AANLkTikW_+4Gby=G8CzT-jQE6B08BN-z66NhOQY0H5kU@mail.gmail.com>
	<4CE699C4.9020005@egenix.com>
Message-ID: <AANLkTim_pjVr=2NT=m+QbadtseCw-dme75NBu3qLVBF3@mail.gmail.com>

2010/11/20 M.-A. Lemburg <mal at egenix.com>:
> Nick Coghlan wrote:
>> The issue suffers from the fact that at least some of the people with
>> the knowledge to do something about it are not inclined to do so due
>> to strong stylistic objections (*ahem*).
>
> How would the implementation differentiate between a real attribute
> error (missing symbol while setting up a module) and one caused by
> a circular import (symbol available, but not yet defined) ?

I'm the wrong person to ask - there's a reason my efforts when it
comes to resolving this consist almost entirely of trying to nudge the
people that actually care about fixing it in potentially promising
directions :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ianb at colorstudy.com  Fri Nov 19 18:18:00 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 19 Nov 2010 11:18:00 -0600
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <AANLkTinZ-RnWd67G=ze6JunwO0NMV1ahp5eEU0SV35Cz@mail.gmail.com>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<20101118162236.680ddca7@pitrou.net> <ic3hif$u6i$1@dough.gmane.org>
	<4CE5B1AF.6080502@canterbury.ac.nz> <4CE5B338.1010607@egenix.com>
	<AANLkTikbQ5uVedUfr5_EWqZGUpSu79jRgoPoyK5cUTrJ@mail.gmail.com>
	<AANLkTinZ-RnWd67G=ze6JunwO0NMV1ahp5eEU0SV35Cz@mail.gmail.com>
Message-ID: <AANLkTinmRpLjKHPyUx-+iRCSNHfjHajQxMnmsjERxtXK@mail.gmail.com>

On Thu, Nov 18, 2010 at 5:29 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> > If you use the "from module import something" form it will be an
> > ImportError. If you do "import module" and then "module.something" you
> will
> > see an AttributeError. These are the ones that can be confusing.
>
> Yes that's the case I was thinking of. After years of Python I can
> still get trapped on those. But maybe that's just me :)
>

I certainly have problems with circular imports (especially involving
__init__.py), and (as far as I've noticed) the error messages are seldom
helpful unless you happen to be experienced in the problem and remember to
look for circular imports.  Simply giving a better error message for this
case would solve the problem in my opinion, without the complexity of
actually making anything work that doesn't work now.

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101119/29fae3f5/attachment.html>

From dreamingforward at gmail.com  Fri Nov 19 21:47:12 2010
From: dreamingforward at gmail.com (average)
Date: Fri, 19 Nov 2010 13:47:12 -0700
Subject: [Python-ideas]  syntax for set
In-Reply-To: <AANLkTik=w27i7xn4WLVzzDSqaGzbmZ4SOyG51T0k4=P3@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>
	<ibtrn5$e8m$1@dough.gmane.org>
	<4CE2CE43.3040907@mrabarnett.plus.com>
	<AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>
	<87pqu4csbi.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimbW5fMxRcHpod6NpUPZ7dMPpNwfbi2Da3ud55O@mail.gmail.com>
	<4CE5ADCD.5050904@pearwood.info>
	<AANLkTik=w27i7xn4WLVzzDSqaGzbmZ4SOyG51T0k4=P3@mail.gmail.com>
Message-ID: <AANLkTinjE4oefdd2Yi_NCnpEcGQH6xyA8=QoLfAigu=B@mail.gmail.com>

On Thu, Nov 18, 2010 at 3:50 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> average wrote:
>
>  Truly.  But the point is that the universe will inevitably evolve towards
>> elegance.
>>
>
> "Inevitably"? I think the only think we can say that the universe will
> inevitably evolve towards is a state of maximum entropy. But frankly, I
> don't see the relevance to the future heat death of the universe in hundreds
> of billions of years to Python's syntax.
>

Come on now :)  I was speaking of the programmer's universe.  But to the
extent the universe is logical, it will also evolve (or has evolved?)
towards elegance (that is, after all what's pushing the physicist's ideal of
a Theory of Everything!).  Which brings us to the mathematicians dilemma:
do we create the universe or do we discover it?


> To only "go on with the information it has and the users it has"

> ensures that it remains merely a branch off the trunk of the ideal
> (consequently drawing few new users), that eventually will need either to
> be
> *deliberately* pruned, or will wither and break off unexpectedly at some
> future point.  Remember, the bulk of new, loyal, users are drawn by an
> ideal
> (you might remember yourself being among them), not even necessarily by how
> big the library is.
>

I beg to differ... it seems to me that users are drawn to languages for many
> reasons, and while elegance can be one of them, it is fairly low down the
> list. Why else would C++, VB and PHP be so popular, while Haskell and Scheme
> remain tiny and unpopular (although very influential)?
>

Well, you certainly have the issue of momentum for one part of the
explanation.  Many are not willing do ditch their old toolset even when a
better one is presented.  The other is that the most elegant languages are
sometimes those understood by the fewest....


> Besides, how do you define "elegant"? I've heard it said, in full
> seriousness, that BrainF*** is the most elegant language because it is a
> fully Turing-complete language in only six commands (plus two more for IO).
>
>
I can define it quite succinctly:  Komolgorov semantic density.  Each
element of syntax of a language can be seen as a condensing down to its
Kolmolgorov ideal the tightest expression of a common programmatic need.
Completeness is a necessary but insufficient condition (so BrainFuck is out
of the question).

Personally, I see Python as an extremely elegant language, but that elegance
> is tempered by a very strong dose of practicality and realism. Gratuitously
> breaking backwards compatibility just so that there is a literal for the
> empty set is not what I call either practical or realistic.
>

The breaking of backwards compatibility is in no way gratuitous.  It may be
demolishing of a long-standing local-maxima, but hardly gratuitous.

Just a few ideas, nothing I'm trying to make a religious war on...

marcos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101119/17d89178/attachment.html>

From __peter__ at web.de  Sat Nov 20 10:59:55 2010
From: __peter__ at web.de (Peter Otten)
Date: Sat, 20 Nov 2010 10:59:55 +0100
Subject: [Python-ideas] Detecting circular imports ?
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<AANLkTimgo9T5hx1mKgRdKFq6Dk_b+fFvzpBNmag=yTUh@mail.gmail.com>
	<AANLkTikW_+4Gby=G8CzT-jQE6B08BN-z66NhOQY0H5kU@mail.gmail.com>
	<4CE699C4.9020005@egenix.com>
Message-ID: <ic866n$s7g$1@dough.gmane.org>

M.-A. Lemburg wrote:

> How would the implementation differentiate between a real attribute
> error (missing symbol while setting up a module) and one caused by
> a circular import (symbol available, but not yet defined) ?

Informing the user that the module code hasn't completed yet when the 
AttributeError occurs may be good enough. 




From fdrake at acm.org  Sat Nov 20 12:53:14 2010
From: fdrake at acm.org (Fred Drake)
Date: Sat, 20 Nov 2010 06:53:14 -0500
Subject: [Python-ideas] Detecting circular imports ?
In-Reply-To: <ic866n$s7g$1@dough.gmane.org>
References: <AANLkTimRnLLgtbTTnw0X5tGAR_gb0z9FH6uRJwROQJ-x@mail.gmail.com>
	<AANLkTimgo9T5hx1mKgRdKFq6Dk_b+fFvzpBNmag=yTUh@mail.gmail.com>
	<AANLkTikW_+4Gby=G8CzT-jQE6B08BN-z66NhOQY0H5kU@mail.gmail.com>
	<4CE699C4.9020005@egenix.com> <ic866n$s7g$1@dough.gmane.org>
Message-ID: <AANLkTikALDW88YdROMTy+C9GOzyxBa+K00Ecay3925s-@mail.gmail.com>

On Sat, Nov 20, 2010 at 4:59 AM, Peter Otten <__peter__ at web.de> wrote:
> Informing the user that the module code hasn't completed yet when the
> AttributeError occurs may be good enough.

Perhaps it makes sense that this is an attribute on the AttributeError
that gets set, and AttributeError.__str__ adds to the message when
set.

Yes, that *could* be a subclass, but I'd hate to get an
IncompleteModuleMaybeACircularImportAttributeError when it's just a
typo.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein


From andy at insectnation.org  Sun Nov 21 20:39:25 2010
From: andy at insectnation.org (Andy Buckley)
Date: Sun, 21 Nov 2010 19:39:25 +0000
Subject: [Python-ideas] Adding an optional function argument to all() and
	any() builtins
Message-ID: <4CE9756D.4020508@insectnation.org>

I may be missing a standard idiom, but it seems to me that the any(...)
and all(...) builtins are unnecessarily limited by requiring that the
iterable they take as an argument is already in a form suitable for the
intended kind of boolean comparison.

Most of the time, when I want to check that any or all of a collection
matches some test criterion, my iterable is not already in a valid form
to pass to any() or all(), and so I either have to explicitly re-code a
slightly modified version of the builtin, or wastefully use map() to
apply my test to *all* the items in the list. This is only a few lines
extra, but IMHO it would make these functions more useful and improve
the readability of this common idiom if the test function could be
supplied as an optional argument (which, if None, would default to the
standard boolean comparison):

For example, this unclear code:

    testval = False
    for i in mylist:
        if mytestfunction(i):
            testval = True
            break
    if testval:
        foo()

or this wasteful (and still unclear, IMO) version:

    mylist_bool = map(mytestfunction, mylist)
    if any(mylist_bool):
        foo()

would be replaced by the very natural form:

    if any(mylist, test=mytestfunction):
        foo()

The "test" keyword arg is perhaps not the best name, I admit, but no
other jumps prominently to mind ("match", "fn"?).

I think this would be a nice feature, and would improve the expressivity
of the language, much as the "key" optional arg to sort() does. Of
course, maybe I just don't know the nice way to do these sorts of
tests... please let me know if I missed something and am reinventing the
wheel ;)

Cheers,
Andy


From python at mrabarnett.plus.com  Sun Nov 21 21:11:38 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 21 Nov 2010 20:11:38 +0000
Subject: [Python-ideas] Adding an optional function argument to all()
 and	any() builtins
In-Reply-To: <4CE9756D.4020508@insectnation.org>
References: <4CE9756D.4020508@insectnation.org>
Message-ID: <4CE97CFA.1010702@mrabarnett.plus.com>

On 21/11/2010 19:39, Andy Buckley wrote:
> I may be missing a standard idiom, but it seems to me that the any(...)
> and all(...) builtins are unnecessarily limited by requiring that the
> iterable they take as an argument is already in a form suitable for the
> intended kind of boolean comparison.
>
> Most of the time, when I want to check that any or all of a collection
> matches some test criterion, my iterable is not already in a valid form
> to pass to any() or all(), and so I either have to explicitly re-code a
> slightly modified version of the builtin, or wastefully use map() to
> apply my test to *all* the items in the list. This is only a few lines
> extra, but IMHO it would make these functions more useful and improve
> the readability of this common idiom if the test function could be
> supplied as an optional argument (which, if None, would default to the
> standard boolean comparison):
>
> For example, this unclear code:
>
>      testval = False
>      for i in mylist:
>          if mytestfunction(i):
>              testval = True
>              break
>      if testval:
>          foo()
>
> or this wasteful (and still unclear, IMO) version:
>
>      mylist_bool = map(mytestfunction, mylist)
>      if any(mylist_bool):
>          foo()
>
> would be replaced by the very natural form:
>
>      if any(mylist, test=mytestfunction):
>          foo()
>
> The "test" keyword arg is perhaps not the best name, I admit, but no
> other jumps prominently to mind ("match", "fn"?).
>
> I think this would be a nice feature, and would improve the expressivity
> of the language, much as the "key" optional arg to sort() does. Of
> course, maybe I just don't know the nice way to do these sorts of
> tests... please let me know if I missed something and am reinventing the
> wheel ;)
>
if any(mytestfunction(i) for i in mylist):
     foo()


From masklinn at masklinn.net  Sun Nov 21 21:20:45 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 21 Nov 2010 21:20:45 +0100
Subject: [Python-ideas] Adding an optional function argument to all()
	and	any() builtins
In-Reply-To: <4CE97CFA.1010702@mrabarnett.plus.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE97CFA.1010702@mrabarnett.plus.com>
Message-ID: <9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>

On 2010-11-21, at 21:11 , MRAB wrote:
> 
> if any(mytestfunction(i) for i in mylist):
>    foo()

is not an improvement over `if any(map(mytestfunction, mylist))` or `if any(imap(mytestfunction, mylist))`

From alexander.belopolsky at gmail.com  Sun Nov 21 21:43:52 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sun, 21 Nov 2010 15:43:52 -0500
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>
References: <4CE9756D.4020508@insectnation.org>
	<4CE97CFA.1010702@mrabarnett.plus.com>
	<9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>
Message-ID: <AANLkTimhzKr+HhK8EKWFg7xK-i4KORUAHX6gkq77rVY2@mail.gmail.com>

On Sun, Nov 21, 2010 at 3:20 PM, Masklinn <masklinn at masklinn.net> wrote:
> On 2010-11-21, at 21:11 , MRAB wrote:
>>
>> if any(mytestfunction(i) for i in mylist):
>> ? ?foo()
>
> is not an improvement over `if any(map(mytestfunction, mylist))` or `if any(imap(mytestfunction, mylist))`

In 2.7, both MRAB's generator expression solution and your imap() are
improvements over map() because they don't create a temporary list.
In 3.x, map = imap, but generator expression is more readable IMO.


From bruce at leapyear.org  Sun Nov 21 22:21:15 2010
From: bruce at leapyear.org (Bruce Leban)
Date: Sun, 21 Nov 2010 13:21:15 -0800
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>
References: <4CE9756D.4020508@insectnation.org>
	<4CE97CFA.1010702@mrabarnett.plus.com>
	<9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>
Message-ID: <AANLkTi=Mu3oRwMDZ-S+CbVkuKFOhMpWg5=q+na=wDQD9@mail.gmail.com>

It is an improvement if there's no function. I think

    any(i > 3 for i in foo)

is more readable than

    any(map(lambda i: i > 3, foo))

As to the original suggestion, the length of any(foo, test=bar) vs
any(map(bar, foo)) is about the same. So I think this doesn't make anything
simpler and unnecessarily adds complexity to any/all which then will
inevitably lead to suggestions to make other things more complicated to
match them.

--- Bruce
Learn about security: http://j.mp/gruyere-security



On Sun, Nov 21, 2010 at 12:20 PM, Masklinn <masklinn at masklinn.net> wrote:

> On 2010-11-21, at 21:11 , MRAB wrote:
> >
> > if any(mytestfunction(i) for i in mylist):
> >    foo()
>
> is not an improvement over `if any(map(mytestfunction, mylist))` or `if
> any(imap(mytestfunction, mylist))`
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101121/1d3c52a3/attachment.html>

From masklinn at masklinn.net  Sun Nov 21 22:35:41 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 21 Nov 2010 22:35:41 +0100
Subject: [Python-ideas] Adding an optional function argument to all()
	and any() builtins
In-Reply-To: <AANLkTimhzKr+HhK8EKWFg7xK-i4KORUAHX6gkq77rVY2@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE97CFA.1010702@mrabarnett.plus.com>
	<9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>
	<AANLkTimhzKr+HhK8EKWFg7xK-i4KORUAHX6gkq77rVY2@mail.gmail.com>
Message-ID: <8F54992A-400E-4921-8780-E10D43BF5E1F@masklinn.net>

On 2010-11-21, at 21:43 , Alexander Belopolsky wrote:
> On Sun, Nov 21, 2010 at 3:20 PM, Masklinn <masklinn at masklinn.net> wrote:
>> On 2010-11-21, at 21:11 , MRAB wrote:
>>> if any(mytestfunction(i) for i in mylist):
>>>    foo()
>> is not an improvement over `if any(map(mytestfunction, mylist))` or `if any(imap(mytestfunction, mylist))`
> 
> In 2.7, both MRAB's generator expression solution and your imap() are
> improvements over map() because they don't create a temporary list.

Which doesn't actually mean it's an improvement, gencomps are not always faster than map when the transformer is a function in all cases.

From masklinn at masklinn.net  Sun Nov 21 22:36:43 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 21 Nov 2010 22:36:43 +0100
Subject: [Python-ideas] Adding an optional function argument to all()
	and any() builtins
In-Reply-To: <AANLkTi=Mu3oRwMDZ-S+CbVkuKFOhMpWg5=q+na=wDQD9@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE97CFA.1010702@mrabarnett.plus.com>
	<9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>
	<AANLkTi=Mu3oRwMDZ-S+CbVkuKFOhMpWg5=q+na=wDQD9@mail.gmail.com>
Message-ID: <33CC6A11-7D84-410E-89B7-B633A5C0F88B@masklinn.net>

On 2010-11-21, at 22:21 , Bruce Leban wrote:
> It is an improvement if there's no function. I think
> 
>    any(i > 3 for i in foo)
> 
> is more readable than
> 
>    any(map(lambda i: i > 3, foo))
Yes, but that's a different situation than the one Andy posted.


From pyideas at rebertia.com  Sun Nov 21 23:03:07 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Sun, 21 Nov 2010 14:03:07 -0800
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <8F54992A-400E-4921-8780-E10D43BF5E1F@masklinn.net>
References: <4CE9756D.4020508@insectnation.org>
	<4CE97CFA.1010702@mrabarnett.plus.com>
	<9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>
	<AANLkTimhzKr+HhK8EKWFg7xK-i4KORUAHX6gkq77rVY2@mail.gmail.com>
	<8F54992A-400E-4921-8780-E10D43BF5E1F@masklinn.net>
Message-ID: <AANLkTi=m4mPm-22KtUD+PzLDftbn-taoD1Cg93MG=Wc0@mail.gmail.com>

On Sun, Nov 21, 2010 at 1:35 PM, Masklinn <masklinn at masklinn.net> wrote:
> On 2010-11-21, at 21:43 , Alexander Belopolsky wrote:
>> On Sun, Nov 21, 2010 at 3:20 PM, Masklinn <masklinn at masklinn.net> wrote:
>>> On 2010-11-21, at 21:11 , MRAB wrote:
>>>> if any(mytestfunction(i) for i in mylist):
>>>> ? ?foo()
>>> is not an improvement over `if any(map(mytestfunction, mylist))` or `if any(imap(mytestfunction, mylist))`
>>
>> In 2.7, both MRAB's generator expression solution and your imap() are
>> improvements over map() because they don't create a temporary list.
>
> Which doesn't actually mean it's an improvement, gencomps are not always faster than map when the transformer is a function in all cases.

In many cases it will be though, because the gencomp is not eagerly
evaluated, unlike listcomps and old map()s. gencomps are also
definitely more memory efficient.

Cheers,
Chris


From ben+python at benfinney.id.au  Sun Nov 21 23:31:14 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 22 Nov 2010 09:31:14 +1100
Subject: [Python-ideas] Adding an optional function argument to all()
	and any() builtins
References: <4CE9756D.4020508@insectnation.org>
Message-ID: <87oc9i8g25.fsf@benfinney.id.au>

Masklinn <masklinn at masklinn.net> writes:

> On 2010-11-21, at 21:11 , MRAB wrote:
> > if any(mytestfunction(i) for i in mylist):
> >    foo()
>
> is not an improvement over `if any(map(mytestfunction, mylist))` or
> `if any(imap(mytestfunction, mylist))`

It is definitely an improvement by the criterion the OP mentioned, which
was:

Andy Buckley <andy at insectnation.org> writes:

> I either have to explicitly re-code a slightly modified version of the
> builtin, or wastefully use map() to apply my test to *all* the items
> in the list.

-- 
 \           ?If [a technology company] has confidence in their future |
  `\      ability to innovate, the importance they place on protecting |
_o__)     their past innovations really should decline.? ?Gary Barnett |
Ben Finney



From rob.cliffe at btinternet.com  Sun Nov 21 23:43:25 2010
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Sun, 21 Nov 2010 22:43:25 +0000
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <AANLkTi=m4mPm-22KtUD+PzLDftbn-taoD1Cg93MG=Wc0@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>	<4CE97CFA.1010702@mrabarnett.plus.com>	<9999A09C-35CB-4111-AA84-CB55FBC729F3@masklinn.net>	<AANLkTimhzKr+HhK8EKWFg7xK-i4KORUAHX6gkq77rVY2@mail.gmail.com>	<8F54992A-400E-4921-8780-E10D43BF5E1F@masklinn.net>
	<AANLkTi=m4mPm-22KtUD+PzLDftbn-taoD1Cg93MG=Wc0@mail.gmail.com>
Message-ID: <4CE9A08D.5050701@btinternet.com>



On 21/11/2010 22:03, Chris Rebert wrote:
> On Sun, Nov 21, 2010 at 1:35 PM, Masklinn<masklinn at masklinn.net>  wrote:
>> On 2010-11-21, at 21:43 , Alexander Belopolsky wrote:
>>> On Sun, Nov 21, 2010 at 3:20 PM, Masklinn<masklinn at masklinn.net>  wrote:
>>>> On 2010-11-21, at 21:11 , MRAB wrote:
>>>>> if any(mytestfunction(i) for i in mylist):
>>>>>     foo()
>>>> is not an improvement over `if any(map(mytestfunction, mylist))` or `if any(imap(mytestfunction, mylist))`
>>> In 2.7, both MRAB's generator expression solution and your imap() are
>>> improvements over map() because they don't create a temporary list.
>> Which doesn't actually mean it's an improvement, gencomps are not always faster than map when the transformer is a function in all cases.
> In many cases it will be though, because the gencomp is not eagerly
> evaluated, unlike listcomps and old map()s. gencomps are also
> definitely more memory efficient.
>
> Cheers,
> Chris
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
For me,

     if any(mytestfunction(i) for i in mylist):
         foo()

wins hands down.  Not only is it, I would guess, quite efficient, but it 
is immediately readable, in particular to newbies.*  Whereas the 
readability of
     if any(map(mytestfunction, mylist))
depends on how familiar you are with map().  If you've never heard of 
it, it's quite impenetrable, as the name "map" is not IMHO very 
meaningful.  (Not that I can think of anything better to call it, or 
that there is any chance of it being renamed.  Still ... perhaps 
"apply_all" or some suc?)

[* I hope I can be forgiven for speaking on behalf of newbies, when I 
don't - quite - class myself as one any more.]

Best wishes
Rob Cliffe


From andy at insectnation.org  Mon Nov 22 00:00:01 2010
From: andy at insectnation.org (Andy Buckley)
Date: Sun, 21 Nov 2010 23:00:01 +0000
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <87oc9i8g25.fsf@benfinney.id.au>
References: <4CE9756D.4020508@insectnation.org>
	<87oc9i8g25.fsf@benfinney.id.au>
Message-ID: <4CE9A471.6050302@insectnation.org>

On 21/11/10 22:31, Ben Finney wrote:
> Masklinn <masklinn at masklinn.net> writes:
> 
>> On 2010-11-21, at 21:11 , MRAB wrote:
>>> if any(mytestfunction(i) for i in mylist):
>>>    foo()
>>
>> is not an improvement over `if any(map(mytestfunction, mylist))` or
>> `if any(imap(mytestfunction, mylist))`
> 
> It is definitely an improvement by the criterion the OP mentioned, which
> was:
> 
> Andy Buckley <andy at insectnation.org> writes:
> 
>> I either have to explicitly re-code a slightly modified version of the
>> builtin, or wastefully use map() to apply my test to *all* the items
>> in the list.

Yes, at least syntactically: I had an odd blind spot here and never
considered using a comprehension/generator expression! Neat.

Still, Masklinn has suggested that this isn't necessarily performant
even in Py 2.7 or 3.x, so there still appears to be a use-case for an
extra argument to any/all. Or did I misunderstand the gencomp issue?

It might be nice to add a suggestion of gencomp use to the
documentation, as I'm sure there are others like me with (temporary)
comprehension blind spots who think those functions have more limited
applicability than is the case.

Thanks,
Andy



From steve at pearwood.info  Mon Nov 22 00:12:30 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 22 Nov 2010 10:12:30 +1100
Subject: [Python-ideas] Adding an optional function argument to all()
 and	any() builtins
In-Reply-To: <4CE9756D.4020508@insectnation.org>
References: <4CE9756D.4020508@insectnation.org>
Message-ID: <4CE9A75E.9050100@pearwood.info>

Andy Buckley wrote:
> I may be missing a standard idiom, but it seems to me that the any(...)
> and all(...) builtins are unnecessarily limited by requiring that the
> iterable they take as an argument is already in a form suitable for the
> intended kind of boolean comparison.

Yes, just like every function requires its input to already be in a form 
suitable for whatever the function expects to do. Just as any() and 
all() expect their argument to be truth-values (not necessarily True or 
False, but values to be inspected for their truthness), so sum() expects 
input consisting of numbers. Would you expect to be able to write this?

mylist = ["a12", "b13", "c14", "d15"]
sum(mylist, convert=lambda s: int(s[1:]))

I hope not. Converting its input into a suitable format is not the 
responsibility of sum. sum's responsibility begins and ends with adding 
the input, any conversions should be done either externally:

mylist = ["a12", "b13", "c14", "d15"]
newlist = [int(s[1:]) for s in mylist]
sum(newlist)

or lazily using a generator expression as the argument to sum:

mylist = ["a12", "b13", "c14", "d15"]
sum(int(s[1:]) for s in mylist)

There are variations on this using built-in map, itertools.imap, and 
similar.

The problems with shifting the responsibility to any and all (or sum) 
instead of keeping it with the caller include:

* It adds complexity to the API for all and any, and that makes the API 
harder to learn and harder to remember.

* It duplicates functionality: now two more functions are expected to 
know how to accept a second argument and call it against the input 
before testing.

* It can lead to "Jack of All Trades, Master of None" functions:

oven.roast(
     chicken, add_stuffing="sage and onion",
     serve=["pre-dinner drinks", "appetizer", "main course", "desert"],
     wash_pots_afterward=False
     )

Why is the oven being asked to prepare the stuffing and serve the food? 
Now this is an extreme case, but any time you consider adding extra 
responsibility to a function, you're taking a step in that direction.

There should be a clear benefit to the caller before accepting these 
extra costs. For example, list.sort(), sorted(), min() and max() now all 
take a key argument which is superficially similar to your suggested 
"test" argument. You would need to demonstrate similar benefit.


> Most of the time, when I want to check that any or all of a collection
> matches some test criterion, my iterable is not already in a valid form
> to pass to any() or all(), and so I either have to explicitly re-code a
> slightly modified version of the builtin, or wastefully use map() to
> apply my test to *all* the items in the list. 

Re-coding all and any is usually a big No. That's almost always the 
wrong solution.

In Python 3, map becomes a lazy iterator, which means it no longer 
applies the test to every item up front. In Python 2, the simple 
alternative is itertools.imap, which saves you from re-coding map:

def lazymap(func, iterable):
     # Don't use this, use itertools.imap
     for obj in iterable:
         yield func(obj)


But probably the best solution is to use a generator expression.

# Like a list comp, except with round brackets.
genexp = (mytestfunction(x) for x in mylist)
any(genexp)


This becomes even easier when you embed the generator expression 
directly in the function call:

any(mytestfunction(x) for x in mylist)

If mytestfunction is simple enough, you can do it in-place without 
needing to write a function:

any(x >= 42 for x in mylist)



[...]
> The "test" keyword arg is perhaps not the best name, I admit, but no
> other jumps prominently to mind ("match", "fn"?).

When you have difficulty thinking of a name which clearly and concisely 
describes what the argument does, that's often a good hint that you are 
dumping too much responsibility onto the function.

Having said that, I'd suggest that an appropriate name might be the same 
name used by sort and friends: key.

-0 on the idea -- I don't find it compelling or necessary, but nor do I 
hate it.



-- 
Steven


From cmjohnson.mailinglist at gmail.com  Mon Nov 22 00:40:46 2010
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Sun, 21 Nov 2010 13:40:46 -1000
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <4CE9A75E.9050100@pearwood.info>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
Message-ID: <AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>

On Sun, Nov 21, 2010 at 1:12 PM, Steven D'Aprano wrote:

When you have difficulty thinking of a name which clearly and concisely
> describes what the argument does, that's often a good hint that you are
> dumping too much responsibility onto the function.
>
> Having said that, I'd suggest that an appropriate name might be the same
> name used by sort and friends: key.
>

I agree with almost everything you wrote, except the suggestion of "key". In
sorted et al., the point of key is that you want a certain form used when
comparing, but *not* the sorted list that is the final product. That's why
you can't just use sorted(name.lower() for name in names) as a substitute
for sorted(names, key=str.lower). Compare the following:

>>> sorted( [u"MacDonald", u"Macan"])
[u'MacDonald', u'Macan']
>>> sorted(s.lower() for s in [u"MacDonald", u"Macan"])
[u'macan', u'macdonald']
>>> sorted([u"MacDonald", u"Macan"], key=unicode.lower)
[u'Macan', u'MacDonald']

None of this applies to any/all, since they just return a True or a False,
no matter what kind of iterable input they get. So, key is not a good name,
and your earlier analysis of why there shouldn't be another keyword on
any/all stands.

-- Carl Johnson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101121/b8a54284/attachment.html>

From rrr at ronadam.com  Mon Nov 22 01:29:25 2010
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 21 Nov 2010 18:29:25 -0600
Subject: [Python-ideas] Adding an optional function argument to all()
 and	any() builtins
In-Reply-To: <4CE9756D.4020508@insectnation.org>
References: <4CE9756D.4020508@insectnation.org>
Message-ID: <4CE9B965.1080200@ronadam.com>



On 11/21/2010 01:39 PM, Andy Buckley wrote:
> I may be missing a standard idiom, but it seems to me that the any(...)
> and all(...) builtins are unnecessarily limited by requiring that the
> iterable they take as an argument is already in a form suitable for the
> intended kind of boolean comparison.
>
> Most of the time, when I want to check that any or all of a collection
> matches some test criterion, my iterable is not already in a valid form
> to pass to any() or all(), and so I either have to explicitly re-code a
> slightly modified version of the builtin, or wastefully use map() to
> apply my test to *all* the items in the list. This is only a few lines
> extra, but IMHO it would make these functions more useful and improve
> the readability of this common idiom if the test function could be
> supplied as an optional argument (which, if None, would default to the
> standard boolean comparison):
>
> For example, this unclear code:
>
>      testval = False
>      for i in mylist:
>          if mytestfunction(i):
>              testval = True
>              break
>      if testval:
>          foo()

This has the advantage that it doesn't need to iterate the whole list. 
Sometimes you can't beat a nice old fashion loop. ;-)

I would write it like this...

     def test_any(iterable, test=bool):
         for i in iterable:
             if test(i):
                 return True
         return False


Then it would read fine in your program.

     if test_any(my_list, my_test_function):
         foo()


Cheers,
   Ron



From pyideas at rebertia.com  Mon Nov 22 01:35:45 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Sun, 21 Nov 2010 16:35:45 -0800
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
Message-ID: <AANLkTi=jWXsmfPOPWEhVT0HSf=uppMwFEs+05VLfwm4D@mail.gmail.com>

On Sun, Nov 21, 2010 at 3:40 PM, Carl M. Johnson
<cmjohnson.mailinglist at gmail.com> wrote:
> On Sun, Nov 21, 2010 at 1:12 PM, Steven D'Aprano?wrote:
>> When you have difficulty thinking of a name which clearly and concisely
>> describes what the argument does, that's often a good hint that you are
>> dumping too much responsibility onto the function.
>>
>> Having said that, I'd suggest that an appropriate name might be the same
>> name used by sort and friends: key.
>
> I agree with almost everything you wrote, except the suggestion of "key". In
> sorted et al., the point of key is that you want a certain form used when
> comparing, but *not* the sorted list that is the final product. That's why
> you can't just use sorted(name.lower() for name in names) as a substitute
> for sorted(names, key=str.lower). Compare the following:
>>>> sorted( [u"MacDonald", u"Macan"])
> [u'MacDonald', u'Macan']
>>>> sorted(s.lower() for s in [u"MacDonald", u"Macan"])
> [u'macan', u'macdonald']
>>>> sorted([u"MacDonald", u"Macan"], key=unicode.lower)
> [u'Macan', u'MacDonald']
> None of this applies to any/all, since they just return a True or a False,
> no matter what kind of iterable input they get. So, key is not a good name,
> and your earlier analysis of why there shouldn't be another keyword on
> any/all stands.

-1 on adding such a keyword, for the reasons already brought up; but
just to play Devil's Advocate, "predicate" (or perhaps "pred" or even
"p" depending on how terse you want to be) would be a good name for
the hypothetical parameter. Because one is testing whether any() or
all() of the items in a collection satisfy a given predicate (in the
trivial case, the "identity" bool() predicate).

Cheers,
Chris


From andy at insectnation.org  Mon Nov 22 02:34:57 2010
From: andy at insectnation.org (Andy Buckley)
Date: Mon, 22 Nov 2010 01:34:57 +0000
Subject: [Python-ideas] Adding an optional function argument to all()
 and	any() builtins
In-Reply-To: <4CE9A75E.9050100@pearwood.info>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
Message-ID: <4CE9C8C1.7030700@insectnation.org>

On 21/11/10 23:12, Steven D'Aprano wrote:
> Andy Buckley wrote:
>> I may be missing a standard idiom, but it seems to me that the any(...)
>> and all(...) builtins are unnecessarily limited by requiring that the
>> iterable they take as an argument is already in a form suitable for the
>> intended kind of boolean comparison.
> 
> Yes, just like every function requires its input to already be in a form
> suitable for whatever the function expects to do. Just as any() and
> all() expect their argument to be truth-values (not necessarily True or
> False, but values to be inspected for their truthness), so sum() expects
> input consisting of numbers. Would you expect to be able to write this?
> 
> mylist = ["a12", "b13", "c14", "d15"]
> sum(mylist, convert=lambda s: int(s[1:]))
> 
> I hope not. Converting its input into a suitable format is not the
> responsibility of sum. sum's responsibility begins and ends with adding
> the input, any conversions should be done either externally:
> 
> mylist = ["a12", "b13", "c14", "d15"]
> newlist = [int(s[1:]) for s in mylist]
> sum(newlist)
> 
> or lazily using a generator expression as the argument to sum:
> 
> mylist = ["a12", "b13", "c14", "d15"]
> sum(int(s[1:]) for s in mylist)
> 
> There are variations on this using built-in map, itertools.imap, and
> similar.

Ok, I like this argument: thanks for making it ;)  If there weren't
generator expressions, I would still argue for a keyword in this case,
as any/all have usefulness on objects more general than one would pass
to sum... but there are gen exps, so I'm more than happy.

To reiterate, though, for those who like me have a blank day and fail to
realise that generator expressions are a neat and powerful idiom in this
case, I think it would be nice to mention this in their documentation.

Thanks all,
Andy



From cmjohnson.mailinglist at gmail.com  Mon Nov 22 02:56:11 2010
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Sun, 21 Nov 2010 15:56:11 -1000
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <4CE9C8C1.7030700@insectnation.org>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<4CE9C8C1.7030700@insectnation.org>
Message-ID: <AANLkTik54uK-S81a-rLXjbEZoSCzbCfg9OHsb35S9th-@mail.gmail.com>

On Sun, Nov 21, 2010 at 3:34 PM, Andy Buckley wrote:

>
> To reiterate, though, for those who like me have a blank day and fail to
> realise that generator expressions are a neat and powerful idiom in this
> case, I think it would be nice to mention this in their documentation.


If you read the original blog post where GvR mentions any/all, he does so
specifically in the context of making generator expressions more useful. :-)
That said, probably very few Python users will ever read that post, so it
should be better documented.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101121/dd0c197e/attachment.html>

From dangyogi at gmail.com  Mon Nov 22 02:57:57 2010
From: dangyogi at gmail.com (Bruce Frederiksen)
Date: Sun, 21 Nov 2010 20:57:57 -0500
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
Message-ID: <AANLkTik+4mghGEgYea6do2FttKfRozS5w7x+4b8o3n+Q@mail.gmail.com>

On Sun, Nov 21, 2010 at 6:40 PM, Carl M. Johnson <
cmjohnson.mailinglist at gmail.com> wrote:

>
> On Sun, Nov 21, 2010 at 1:12 PM, Steven D'Aprano wrote:
>
> Having said that, I'd suggest that an appropriate name might be the same
>> name used by sort and friends: key.
>>
>
> I agree with almost everything you wrote, except the suggestion of "key".
> [...]
>
> None of this applies to any/all, since they just return a True or a False,
> no matter what kind of iterable input they get.
>

However, if any/all acted like or/and and returned the last value examined
(rather than simply True/False), then "key" would make sense.  Then they
turn into more useful selection tools.

There have been several times when I wished that any/all acted like or/and
because I needed the offending value for further use...

-Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101121/e0d413e4/attachment.html>

From raymond.hettinger at gmail.com  Mon Nov 22 03:52:35 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sun, 21 Nov 2010 18:52:35 -0800
Subject: [Python-ideas] Adding an optional function argument to all()
	and any() builtins
In-Reply-To: <AANLkTik+4mghGEgYea6do2FttKfRozS5w7x+4b8o3n+Q@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
	<AANLkTik+4mghGEgYea6do2FttKfRozS5w7x+4b8o3n+Q@mail.gmail.com>
Message-ID: <51B6E02F-82AB-404F-BBB0-4970513B0081@gmail.com>


On Nov 21, 2010, at 5:57 PM, Bruce Frederiksen wrote:

> There have been several times when I wished that any/all acted like or/and because I needed the offending value for further use...

This is such a trivial function.
Why not just roll your own and dump it in your personal utils?


Raymond

From steve at pearwood.info  Mon Nov 22 05:19:28 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 22 Nov 2010 15:19:28 +1100
Subject: [Python-ideas] Adding an optional function argument to all()
 and	any() builtins
In-Reply-To: <4CE9B965.1080200@ronadam.com>
References: <4CE9756D.4020508@insectnation.org> <4CE9B965.1080200@ronadam.com>
Message-ID: <4CE9EF50.705@pearwood.info>

Ron Adam wrote:
> On 11/21/2010 01:39 PM, Andy Buckley wrote:
[...]
>> For example, this unclear code:
>>
>>      testval = False
>>      for i in mylist:
>>          if mytestfunction(i):
>>              testval = True
>>              break
>>      if testval:
>>          foo()
> 
> This has the advantage that it doesn't need to iterate the whole list. 
> Sometimes you can't beat a nice old fashion loop. ;-)

Both any() and all() have lazy semantics and will stop as early as 
possible. They will only traverse the entire list if they have to. 
There's no need to write your own function to get that behaviour.


> I would write it like this...
> 
>     def test_any(iterable, test=bool):
>         for i in iterable:
>             if test(i):
>                 return True
>         return False

That would be more easily written as:

def test_any(iterable, test=bool):
     return any(test(obj) for obj in iterable)

But of course that raises the question of what advantage test_any() has 
over an inline call to any(test(obj) for obj in iterable).


Aside: for what little it's worth, defaulting to bool as the test 
function causes a significant slow-down:

[steve at sylar ~]$ python3 -m timeit -s "L=[0]*1000000" "any(L)"
10 loops, best of 3: 28 msec per loop
[steve at sylar ~]$ python3 -m timeit -s "L=[0]*1000000" "any(bool(x) for x 
in L)"
10 loops, best of 3: 615 msec per loop


In fairness, some of that slow-down is due to the overhead of the 
generator expression itself:

[steve at sylar ~]$ python3 -m timeit -s "L=[0]*1000000" "any(x for x in L)"
10 loops, best of 3: 241 msec per loop

but it's best to avoid calling bool() unless you absolutely need an 
actual bool. test_any should probably be written as:

def test_any(iterable, test=None):
     if test: return any(test(obj) for obj in iterable)
     else: return any(iterable)


although my care-factor is insufficient to actually bother timing it :)



-- 
Steven



From steve at pearwood.info  Mon Nov 22 05:29:12 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 22 Nov 2010 15:29:12 +1100
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <51B6E02F-82AB-404F-BBB0-4970513B0081@gmail.com>
References: <4CE9756D.4020508@insectnation.org>	<4CE9A75E.9050100@pearwood.info>	<AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>	<AANLkTik+4mghGEgYea6do2FttKfRozS5w7x+4b8o3n+Q@mail.gmail.com>
	<51B6E02F-82AB-404F-BBB0-4970513B0081@gmail.com>
Message-ID: <4CE9F198.3060106@pearwood.info>

Raymond Hettinger wrote:
> On Nov 21, 2010, at 5:57 PM, Bruce Frederiksen wrote:
> 
>> There have been several times when I wished that any/all acted like or/and because I needed the offending value for further use...
> 
> This is such a trivial function.
> Why not just roll your own and dump it in your personal utils?

In fairness, the same could be said about any() and all() in the first 
place. The advantages are that the built-ins are significantly faster 
than the pure-Python version (a factor of 3 times faster on my machine 
using Python 3.1), and they're available everywhere you go, without 
re-writing the functions or importing them.

I'm (mildly) with Bruce on this one, but I think we missed the boat. 
Perhaps in Python4000 any() and all() should return the relevant object 
rather than True/False.


-- 
Steven



From rrr at ronadam.com  Mon Nov 22 05:47:58 2010
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 21 Nov 2010 22:47:58 -0600
Subject: [Python-ideas] Adding an optional function argument to all()
 and	any() builtins
In-Reply-To: <4CE9EF50.705@pearwood.info>
References: <4CE9756D.4020508@insectnation.org> <4CE9B965.1080200@ronadam.com>
	<4CE9EF50.705@pearwood.info>
Message-ID: <4CE9F5FE.90003@ronadam.com>



On 11/21/2010 10:19 PM, Steven D'Aprano wrote:
> Ron Adam wrote:
>> On 11/21/2010 01:39 PM, Andy Buckley wrote:
> [...]
>>> For example, this unclear code:
>>>
>>> testval = False
>>> for i in mylist:
>>> if mytestfunction(i):
>>> testval = True
>>> break
>>> if testval:
>>> foo()
>>
>> This has the advantage that it doesn't need to iterate the whole list.
>> Sometimes you can't beat a nice old fashion loop. ;-)
>
> Both any() and all() have lazy semantics and will stop as early as
> possible. They will only traverse the entire list if they have to. There's
> no need to write your own function to get that behaviour.
>
>
>> I would write it like this...
>>
>> def test_any(iterable, test=bool):
>> for i in iterable:
>> if test(i):
>> return True
>> return False
>
> That would be more easily written as:
>
> def test_any(iterable, test=bool):
> return any(test(obj) for obj in iterable)

Yep, you're right.

 >>> def test(value):
...   print(value)
...   return value > 0
...
 >>> my_list = list(range(-5, 5))
 >>> any(test(obj) for obj in my_list)
-5
-4
-3
-2
-1
0
1
True


I was thinking of this..

 >>> any([test(obj) for obj in my_list])
-5
-4
-3
-2
-1
0
1
2
3
4
True

This is a nice example of the differences between a generator expression 
and a list comprehension.


> But of course that raises the question of what advantage test_any() has
> over an inline call to any(test(obj) for obj in iterable).
>
>
> Aside: for what little it's worth, defaulting to bool as the test function
> causes a significant slow-down:
>
> [steve at sylar ~]$ python3 -m timeit -s "L=[0]*1000000" "any(L)"
> 10 loops, best of 3: 28 msec per loop
> [steve at sylar ~]$ python3 -m timeit -s "L=[0]*1000000" "any(bool(x) for x in
> L)"
> 10 loops, best of 3: 615 msec per loop
>
>
> In fairness, some of that slow-down is due to the overhead of the generator
> expression itself:
>
> [steve at sylar ~]$ python3 -m timeit -s "L=[0]*1000000" "any(x for x in L)"
> 10 loops, best of 3: 241 msec per loop
>
> but it's best to avoid calling bool() unless you absolutely need an actual
> bool. test_any should probably be written as:
>
> def test_any(iterable, test=None):
> if test: return any(test(obj) for obj in iterable)
> else: return any(iterable)
>
>
> although my care-factor is insufficient to actually bother timing it :)

I agree. ;-)

Ron



From denis.spir at gmail.com  Mon Nov 22 10:44:23 2010
From: denis.spir at gmail.com (spir)
Date: Mon, 22 Nov 2010 10:44:23 +0100
Subject: [Python-ideas] Adding an optional function argument to all()
 and	any() builtins
In-Reply-To: <4CE9C8C1.7030700@insectnation.org>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<4CE9C8C1.7030700@insectnation.org>
Message-ID: <20101122104423.7b0d6b64@o>

On Mon, 22 Nov 2010 01:34:57 +0000
Andy Buckley <andy at insectnation.org> wrote:

> To reiterate, though, for those who like me have a blank day and fail to
> realise that generator expressions are a neat and powerful idiom in this
> case, I think it would be nice to mention this in their documentation.

Yes; generally speaking, generator expressions should be mentioned in "official" doc along with every feature they are a useful parameter of, with a sensible example use case. Genexprs are extremely useful, but under-used in real code because they are somewhat new in the programming culture. Doc should help in making them more famous.

Denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From zuo at chopin.edu.pl  Mon Nov 22 11:16:41 2010
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Mon, 22 Nov 2010 11:16:41 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <ibrv3a$pk6$1@dough.gmane.org>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
Message-ID: <20101122101641.GA2326@chopin.edu.pl>

Terry Reedy dixit (2010-11-15, 13:45):

> The *only* glitch is the lack of an empty set notation. The idea of
> dict() == {:}; set()=={};
> which would be the obvious choice if starting fresh,
> was considered but rejected by Guido as too disruptive
> and introducing another barrier between Python 2 and 3.

I'd be happy with:

* {:} for empty dict() (as a collection of key-value *pairs*)
* {.} for empty set() (as a similar collection of *single* elements)

And {} for empty dict() as well -- to keep compatibility (maybe to be
deprecated later).

Regards,
*j



From zuo at chopin.edu.pl  Mon Nov 22 11:23:41 2010
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Mon, 22 Nov 2010 11:23:41 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <20101122101641.GA2326@chopin.edu.pl>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
Message-ID: <20101122102341.GA2977@chopin.edu.pl>

Jan Kaliszewski dixit (2010-11-22, 11:16):

> I'd be happy with:
> 
> * {:} for empty dict() (as a collection of key-value *pairs*)
> * {.} for empty set() (as a similar collection of *single* elements)
> 
> And {} for empty dict() as well -- to keep compatibility (maybe to be
> deprecated later).

PS. Another argument: {} is criticized for its similarity to () (when using some fonts).


From tlesher at gmail.com  Mon Nov 22 12:32:41 2010
From: tlesher at gmail.com (Tim Lesher)
Date: Mon, 22 Nov 2010 06:32:41 -0500
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <4CE9F198.3060106@pearwood.info>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
	<AANLkTik+4mghGEgYea6do2FttKfRozS5w7x+4b8o3n+Q@mail.gmail.com>
	<51B6E02F-82AB-404F-BBB0-4970513B0081@gmail.com>
	<4CE9F198.3060106@pearwood.info>
Message-ID: <AANLkTikq7DyW34KT+6xxMiLjtOJnDUiO-Hsw2ypaPCow@mail.gmail.com>

On Sun, Nov 21, 2010 at 23:29, Steven D'Aprano <steve at pearwood.info> wrote:
> I'm (mildly) with Bruce on this one, but I think we missed the boat. Perhaps
> in Python4000 any() and all() should return the relevant object rather than
> True/False.

That could be the source of some frightening, hard-to-detect regressions:

if any([-1,0,1], test=is_even):
    # never taken, because 0 is returned instead of True


-- 
Tim Lesher <tlesher at gmail.com>


From dangyogi at gmail.com  Mon Nov 22 16:58:57 2010
From: dangyogi at gmail.com (Bruce Frederiksen)
Date: Mon, 22 Nov 2010 10:58:57 -0500
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <AANLkTikq7DyW34KT+6xxMiLjtOJnDUiO-Hsw2ypaPCow@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
	<AANLkTik+4mghGEgYea6do2FttKfRozS5w7x+4b8o3n+Q@mail.gmail.com>
	<51B6E02F-82AB-404F-BBB0-4970513B0081@gmail.com>
	<4CE9F198.3060106@pearwood.info>
	<AANLkTikq7DyW34KT+6xxMiLjtOJnDUiO-Hsw2ypaPCow@mail.gmail.com>
Message-ID: <AANLkTimGz+dsjA1uQVvcNa7U9O7XtZ-T32016rBLEnhL@mail.gmail.com>

On Mon, Nov 22, 2010 at 6:32 AM, Tim Lesher <tlesher at gmail.com> wrote:

> That could be the source of some frightening, hard-to-detect regressions:
>
> if any([-1,0,1], test=is_even):
>    # never taken, because 0 is returned instead of True
>

Good point!  The only way out of this for what I was wanting that I can
think of becomes:

first_even = next(filter(is_even, [-1, 0, 1]), None)

In which case, I already have all of the tools that I need!

I withdraw my suggestion.

-Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101122/d98744ef/attachment.html>

From gerald.britton at gmail.com  Mon Nov 22 17:02:19 2010
From: gerald.britton at gmail.com (Gerald Britton)
Date: Mon, 22 Nov 2010 11:02:19 -0500
Subject: [Python-ideas] Set Syntax
Message-ID: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>

Hi -- thought I'd chime in on this one:

> I'd be happy with:
>
> * {:} for empty dict() (as a collection of key-value *pairs*)
> * {.} for empty set() (as a similar collection of *single* elements)
>
> And {} for empty dict() as well -- to keep compatibility (maybe to be
> deprecated later).

Note that dictionaries and sets have clear() methods:

>>> d = {1:2}
>>> d
{1: 2}
>>> d.clear()
>>> d
{}
>>> s = set([1,2])
>>> s
set([1, 2])
>>> s.clear()
>>> s
set([])

Deques have this method as well:

>>> from collections import deque
>>> q = deque([1,2])
>>> q
deque([1, 2])
>>> q.clear()
>>> q
deque([])

If lists had the clear() method, then the basic collection objects
would be in alignment.

-- 
Gerald Britton


From ncoghlan at gmail.com  Mon Nov 22 17:19:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 23 Nov 2010 02:19:55 +1000
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <AANLkTimGz+dsjA1uQVvcNa7U9O7XtZ-T32016rBLEnhL@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<AANLkTin1_PYZmpcSKP2e=NT5EzrUQSKO+KyFAuijzVHH@mail.gmail.com>
	<AANLkTik+4mghGEgYea6do2FttKfRozS5w7x+4b8o3n+Q@mail.gmail.com>
	<51B6E02F-82AB-404F-BBB0-4970513B0081@gmail.com>
	<4CE9F198.3060106@pearwood.info>
	<AANLkTikq7DyW34KT+6xxMiLjtOJnDUiO-Hsw2ypaPCow@mail.gmail.com>
	<AANLkTimGz+dsjA1uQVvcNa7U9O7XtZ-T32016rBLEnhL@mail.gmail.com>
Message-ID: <AANLkTikZP+UTmHgSm3K7yTEnp5EYXied03m+GYOMeHSn@mail.gmail.com>

On Tue, Nov 23, 2010 at 1:58 AM, Bruce Frederiksen <dangyogi at gmail.com> wrote:
> On Mon, Nov 22, 2010 at 6:32 AM, Tim Lesher <tlesher at gmail.com> wrote:
>>
>> That could be the source of some frightening, hard-to-detect regressions:
>>
>> if any([-1,0,1], test=is_even):
>> ? ?# never taken, because 0 is returned instead of True
>
> Good point!? The only way out of this for what I was wanting that I can
> think of becomes:
>
> first_even = next(filter(is_even, [-1, 0, 1]), None)
>
> In which case, I already have all of the tools that I need!

Again, the generator expression version will be more readable for most folks:

  first_even = next((x for x in seq if is_even(x)), None)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From rrr at ronadam.com  Mon Nov 22 17:49:15 2010
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 22 Nov 2010 10:49:15 -0600
Subject: [Python-ideas] syntax for set
In-Reply-To: <20101122101641.GA2326@chopin.edu.pl>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
Message-ID: <4CEA9F0B.7070701@ronadam.com>



On 11/22/2010 04:16 AM, Jan Kaliszewski wrote:
> Terry Reedy dixit (2010-11-15, 13:45):
>
>> The *only* glitch is the lack of an empty set notation. The idea of
>> dict() == {:}; set()=={};
>> which would be the obvious choice if starting fresh,
>> was considered but rejected by Guido as too disruptive
>> and introducing another barrier between Python 2 and 3.
>
> I'd be happy with:
>
> * {:} for empty dict() (as a collection of key-value *pairs*)
> * {.} for empty set() (as a similar collection of *single* elements)
>
> And {} for empty dict() as well -- to keep compatibility (maybe to be
> deprecated later).

Maybe it's not intentional, but where ever a colon is used, it tends to 
mean "make an association to".  (except in slicing)

       Key : value                   # This is also name : object.

       class name : class body

       function name : function body


And where ever a '.' is, it tends to mean "get a sub-part of:

       class name . method name


So the {.} doesn't really fit with this idea.  Probably the correct set 
notation would have been {} if it was put into python before dictionaries.

If you follow this line of reasoning, then a dictionary is a set of 
associations.  ie... {} for set, and : for associated to, and put those 
together and we have {:}.

An interesting "wild" idea might be to delay defining weather it's a 
dictionary or a set depending on how it's first used.  Then {} could be 
both an empty set and an empty dictionary, but I think there are probably 
are a lot of reasons not to do that.

Ron























From bborcic at gmail.com  Mon Nov 22 19:20:33 2010
From: bborcic at gmail.com (Boris Borcic)
Date: Mon, 22 Nov 2010 19:20:33 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CEA9F0B.7070701@ronadam.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<20101122101641.GA2326@chopin.edu.pl>
	<4CEA9F0B.7070701@ronadam.com>
Message-ID: <icec9h$rsa$1@dough.gmane.org>

Ron Adam wrote:
>
> An interesting "wild" idea might be to delay defining weather it's a
> dictionary or a set depending on how it's first used. Then {} could be
> both an empty set and an empty dictionary, but I think there are
> probably are a lot of reasons not to do that.

I'd say the challenge is wider, there are various corners where a "generic zero 
value/factory" feels like a desirable feature, just waiting to be correctly 
designed...

Cheers, BB



From raymond.hettinger at gmail.com  Mon Nov 22 19:40:21 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 22 Nov 2010 10:40:21 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <20101122101641.GA2326@chopin.edu.pl>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
Message-ID: <856C9747-91B9-4E8D-8937-AA10A658F8E1@gmail.com>


> I'd be happy with:
> 
> * {:} for empty dict() (as a collection of key-value *pairs*)
> * {.} for empty set() (as a similar collection of *single* elements)
> 
> And {} for empty dict() as well -- to keep compatibility (maybe to be
> deprecated later).

I'm curious why you guys think you *need* an empty set literal.
The current spelling using set() and frozenset() is unambiguous.
So what's the point of trying to shoehorn-in a new literal?

AFAICT, this discussion has been solely motivated by
a shallow itch, "dicts have one, so sets have to have one too".

If there were a clean, beautiful, obvious correct answer,
it would have already been done.   Since there isn't,
we have to ask, who cares?


Raymond


From bborcic at gmail.com  Mon Nov 22 19:47:42 2010
From: bborcic at gmail.com (Boris Borcic)
Date: Mon, 22 Nov 2010 19:47:42 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinjE4oefdd2Yi_NCnpEcGQH6xyA8=QoLfAigu=B@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<AANLkTi=19=yhTfxq0wLcBrsNfbZBK1BeL9i07og2C10B@mail.gmail.com>	<ibtrn5$e8m$1@dough.gmane.org>	<4CE2CE43.3040907@mrabarnett.plus.com>	<AANLkTi=fpQ8yFQ3LXkjGGrnwxn2wPfrnv0ZfXvc02KDt@mail.gmail.com>	<87pqu4csbi.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTimbW5fMxRcHpod6NpUPZ7dMPpNwfbi2Da3ud55O@mail.gmail.com>	<4CE5ADCD.5050904@pearwood.info>	<AANLkTik=w27i7xn4WLVzzDSqaGzbmZ4SOyG51T0k4=P3@mail.gmail.com>
	<AANLkTinjE4oefdd2Yi_NCnpEcGQH6xyA8=QoLfAigu=B@mail.gmail.com>
Message-ID: <icedse$4q1$1@dough.gmane.org>

average wrote:
[...]

>
>
> Well, you certainly have the issue of momentum for one part of the
> explanation.  Many are not willing do ditch their old toolset even when
> a better one is presented.

So the real issue is the order in which toolsets get presented to newcomers to 
the field.

> The other is that the most elegant languages
> are sometimes those understood by the fewest....
>
>     Besides, how do you define "elegant"? I've heard it said, in full
>     seriousness, that BrainF*** is the most elegant language because it
>     is a fully Turing-complete language in only six commands (plus two
>     more for IO).
>
>
> I can define it quite succinctly:  Komolgorov semantic density.  Each
> element of syntax of a language can be seen as a condensing down to its
> Kolmolgorov ideal the tightest expression of a common programmatic
> need.

An experience that early computers provided to programmers was that of having to 
save on code size because memory was too tight; and a lesson from that 
experience is that gains in information density of code have less to do with 
local optimization of expressions than with global optimizations in the sense of 
finding ways to fold apparently different code execution paths into variants of 
the same.

IOW, algorithmic information density is concept overkill, when arguing about the 
standard representation of empty collections factory calls.

And as far as elegance is concerned, I am in fact tempted to argue that the call 
syntax in list(), set() and dict() has over [] and {} the virtuous side-effect 
of reminding the reader that a distinct empty collection is made at each call, 
despite the invariability of the expression.

Cheers, BB

  Completeness is a necessary but insufficient condition (so
> BrainFuck is out of the question).
>
>     Personally, I see Python as an extremely elegant language, but that
>     elegance is tempered by a very strong dose of practicality and
>     realism. Gratuitously breaking backwards compatibility just so that
>     there is a literal for the empty set is not what I call either
>     practical or realistic.
>
>
> The breaking of backwards compatibility is in no way gratuitous.  It may
> be demolishing of a long-standing local-maxima, but hardly gratuitous.
>
> Just a few ideas, nothing I'm trying to make a religious war on...
>
> marcos
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas




From brett at python.org  Mon Nov 22 20:42:21 2010
From: brett at python.org (Brett Cannon)
Date: Mon, 22 Nov 2010 11:42:21 -0800
Subject: [Python-ideas] Adding an optional function argument to all()
 and any() builtins
In-Reply-To: <AANLkTik54uK-S81a-rLXjbEZoSCzbCfg9OHsb35S9th-@mail.gmail.com>
References: <4CE9756D.4020508@insectnation.org>
	<4CE9A75E.9050100@pearwood.info>
	<4CE9C8C1.7030700@insectnation.org>
	<AANLkTik54uK-S81a-rLXjbEZoSCzbCfg9OHsb35S9th-@mail.gmail.com>
Message-ID: <AANLkTikea1vxdB2kqMGd8B9meLQODEc-oaJ_bKcLqzqW@mail.gmail.com>

On Sun, Nov 21, 2010 at 17:56, Carl M. Johnson
<cmjohnson.mailinglist at gmail.com> wrote:
> On Sun, Nov 21, 2010 at 3:34 PM, Andy Buckley?wrote:
>>
>> To reiterate, though, for those who like me have a blank day and fail to
>> realise that generator expressions are a neat and powerful idiom in this
>> case, I think it would be nice to mention this in their documentation.
>
> If you read the original blog post where GvR mentions any/all, he does so
> specifically in the context of making generator expressions more useful. :-)
> That said, probably very few Python users will ever read that post, so it
> should be better documented.

But the blind spot of not thinking in terms of genexps is for "old"
Python developers, not new ones who started off with them and probably
don't use listcomps that often (I know I don't anymore). Documenting
this specific suggestion leads down a rabbit hole of adding coding
suggestions all over Python for how we feel something *should* be used
when it isn't really our place to do so.

But if someone wants to submit a patch to fix the docs another core
developer might be willing to make the change other than me (I'm -0 on
the change).


From bborcic at gmail.com  Tue Nov 23 15:41:10 2010
From: bborcic at gmail.com (Boris Borcic)
Date: Tue, 23 Nov 2010 15:41:10 +0100
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
Message-ID: <icgjq6$8hi$1@dough.gmane.org>

Gerald Britton wrote:
>
> If lists had the clear() method, then the basic collection objects
> would be in alignment.
>

Wouldn't this be too pure to be practical ? ;)




From pyideas at rebertia.com  Tue Nov 23 20:08:45 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Tue, 23 Nov 2010 11:08:45 -0800
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
Message-ID: <AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>

On Mon, Nov 22, 2010 at 8:02 AM, Gerald Britton
<gerald.britton at gmail.com> wrote:
<snip>
> Note that dictionaries and sets have clear() methods:
>
>>>> d = {1:2}
>>>> d
> {1: 2}
>>>> d.clear()
>>>> d
> {}
>>>> s = set([1,2])
>>>> s
> set([1, 2])
>>>> s.clear()
>>>> s
> set([])
>
> Deques have this method as well:
>
>>>> from collections import deque
>>>> q = deque([1,2])
>>>> q
> deque([1, 2])
>>>> q.clear()
>>>> q
> deque([])
>
> If lists had the clear() method, then the basic collection objects
> would be in alignment.

Possible objection: Violates TOOWTDI.

There are already 2 ways to clear a list:
del lst[:]
lst[:] = []

Do we necessarily need a third?

Unlike lists, dicts, sets, and deques don't support slice subscripts,
which is arguably the only reason they need .clear() methods.

Also, is it that common to treat such significantly differently
collection types generically?

Cheers,
Chris
--
http://blog.rebertia.com


From mwm-keyword-python.b4bdba at mired.org  Tue Nov 23 20:33:11 2010
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Tue, 23 Nov 2010 14:33:11 -0500
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
	<AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>
Message-ID: <20101123143311.40cd3b56@bhuda.mired.org>

On Tue, 23 Nov 2010 11:08:45 -0800
Chris Rebert <pyideas at rebertia.com> wrote:
> Also, is it that common to treat such significantly differently
> collection types generically?

What's common - or at least not unknown - is for the changing nature
of a problem to dictate that you change which collection type you're
using. The more methods they share, the easier making that change
is.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From guido at python.org  Tue Nov 23 20:59:22 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 23 Nov 2010 11:59:22 -0800
Subject: [Python-ideas] Set Syntax
In-Reply-To: <20101123143311.40cd3b56@bhuda.mired.org>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
	<AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>
	<20101123143311.40cd3b56@bhuda.mired.org>
Message-ID: <AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>

FWIW I'm fine with adding list.clear() to 3.3.

On Tue, Nov 23, 2010 at 11:33 AM, Mike Meyer
<mwm-keyword-python.b4bdba at mired.org> wrote:
> On Tue, 23 Nov 2010 11:08:45 -0800
> Chris Rebert <pyideas at rebertia.com> wrote:
>> Also, is it that common to treat such significantly differently
>> collection types generically?
>
> What's common - or at least not unknown - is for the changing nature
> of a problem to dictate that you change which collection type you're
> using. The more methods they share, the easier making that change
> is.
>
> ? ?<mike
> --
> Mike Meyer <mwm at mired.org> ? ? ? ? ? ? ?http://www.mired.org/consulting.html
> Independent Network/Unix/Perforce consultant, email for more information.
>
> O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (python.org/~guido)


From merwok at netwok.org  Tue Nov 23 21:25:49 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 23 Nov 2010 21:25:49 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
Message-ID: <4CEC234D.6020301@netwok.org>

Hi,

> Recently I had the need to filter objects based on whether they're picklable
> or not:
> 
> http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items
> 
> I'm not sure what's a good way to check for a specific object whether it's
> picklable.
> 
> <http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items>This
> led me to think: Maybe we should have an `Unpicklable` abstract base class
> in the `collections` module? Then various unpicklable classes, like locks,
> files or widgets, could inherit from this class to signify that they cannot
> be pickled.
> 
> What do you think?

This sounds useful.  I?d rather spell the ABC pickle.Picklable, though.

Regards



From cool-rr at cool-rr.com  Tue Nov 23 21:46:51 2010
From: cool-rr at cool-rr.com (cool-RR)
Date: Tue, 23 Nov 2010 22:46:51 +0200
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <4CEC234D.6020301@netwok.org>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
Message-ID: <AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>

On Tue, Nov 23, 2010 at 10:25 PM, ?ric Araujo <merwok at netwok.org> wrote:

> Hi,
>
> > Recently I had the need to filter objects based on whether they're
> picklable
> > or not:
> >
> >
> http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items
> >
> > I'm not sure what's a good way to check for a specific object whether
> it's
> > picklable.
> >
> > <
> http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items
> >This
> > led me to think: Maybe we should have an `Unpicklable` abstract base
> class
> > in the `collections` module? Then various unpicklable classes, like
> locks,
> > files or widgets, could inherit from this class to signify that they
> cannot
> > be pickled.
> >
> > What do you think?
>
> This sounds useful.  I?d rather spell the ABC pickle.Picklable, though.
>
> Regards
>
>
(Spelling note: People told me that "pickleable" (with an "e" in the middle)
makes more sense, so I'm using that now.)

The best solution might be to have both a `Pickleable` class and
an `Unpickleable` class. The reason to have the former is that
`isinstance(thing, Pickleable)` is more natural, and the reason to have the
latter is because we can't require people to inherit from `Pickleable` for
every single class that they define. (Since pickleability is the rule and
unpickleability is the exception.)

So `Pickleable` could have a `__subclasshook__` that would do the real work,
similarly to `Iterable`.


Ram.

-- 
Sincerely,
Ram Rachum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101123/8eae2968/attachment.html>

From solipsis at pitrou.net  Tue Nov 23 21:54:58 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 23 Nov 2010 21:54:58 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
Message-ID: <20101123215458.53eed0b0@pitrou.net>

On Tue, 23 Nov 2010 22:46:51 +0200
cool-RR <cool-rr at cool-rr.com> wrote:
> (Spelling note: People told me that "pickleable" (with an "e" in the middle)
> makes more sense, so I'm using that now.)
> 
> The best solution might be to have both a `Pickleable` class and
> an `Unpickleable` class. The reason to have the former is that
> `isinstance(thing, Pickleable)` is more natural, and the reason to have the
> latter is because we can't require people to inherit from `Pickleable` for
> every single class that they define. (Since pickleability is the rule and
> unpickleability is the exception.)
> 
> So `Pickleable` could have a `__subclasshook__` that would do the real work,
> similarly to `Iterable`.

The problem is that "doing the real work" can be CPU-intensive, and I
don't think we would like isinstance() calls to become arbitrarily
expensive (even if technically it's possible).

An alternative is simply to catch the exception when trying to pickle a
value. That implies that the type takes care to raise an exception if
pickling would work but unpickling wouldn't, which is what file objects
now do.

By the way, "Unpickleable" doesn't work since it's ambiguous: you don't
know whether it means you can unpickle the thing, or you can't pickle
it.

Regards

Antoine.




From fdrake at acm.org  Tue Nov 23 22:04:29 2010
From: fdrake at acm.org (Fred Drake)
Date: Tue, 23 Nov 2010 16:04:29 -0500
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
Message-ID: <AANLkTik9JwE-HPNE=iHQmMvJC_L2hqhUDZvpf6fWyW2=@mail.gmail.com>

On Tue, Nov 23, 2010 at 3:46 PM, cool-RR <cool-rr at cool-rr.com> wrote:
> (Spelling note: People told me that "pickleable" (with an "e" in the middle)
> makes more sense, so I'm using that now.)

Strange; doesn't fit my expectations.

> The best solution might be to have both a `Pickleable` class and
> an?`Unpickleable` class. The reason to have the former is that
> `isinstance(thing, Pickleable)` is more natural, and the reason to have the
> latter is because we can't require people to inherit from `Pickleable` for
> every single class that they define. (Since pickleability is the rule and
> unpickleability is the exception.)

Well, that certainly matches current (broken) expectations.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein


From cool-rr at cool-rr.com  Tue Nov 23 22:22:13 2010
From: cool-rr at cool-rr.com (Ram Rachum)
Date: Tue, 23 Nov 2010 21:22:13 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Adding_=60Unpicklable=60_to_the_=60colle?=
	=?utf-8?q?ctions=60_module?=
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
Message-ID: <loom.20101123T221933-185@post.gmane.org>

Antoine Pitrou <solipsis at ...> writes:

> 
> On Tue, 23 Nov 2010 22:46:51 +0200
> cool-RR <cool-rr at ...> wrote:
> > (Spelling note: People told me that "pickleable" (with an "e" in the middle)
> > makes more sense, so I'm using that now.)
> > 
> > The best solution might be to have both a `Pickleable` class and
> > an `Unpickleable` class. The reason to have the former is that
> > `isinstance(thing, Pickleable)` is more natural, and the reason to have the
> > latter is because we can't require people to inherit from `Pickleable` for
> > every single class that they define. (Since pickleability is the rule and
> > unpickleability is the exception.)
> > 
> > So `Pickleable` could have a `__subclasshook__` that would do the real work,
> > similarly to `Iterable`.
> 
> The problem is that "doing the real work" can be CPU-intensive, and I
> don't think we would like isinstance() calls to become arbitrarily
> expensive (even if technically it's possible).
> 
> An alternative is simply to catch the exception when trying to pickle a
> value. That implies that the type takes care to raise an exception if
> pickling would work but unpickling wouldn't, which is what file objects
> now do.
> 
> By the way, "Unpickleable" doesn't work since it's ambiguous: you don't
> know whether it means you can unpickle the thing, or you can't pickle
> it.
> 
> Regards
> 
> Antoine.


Sorry, I didn't explain well enough: I meant that only inherently unpickleable 
objects, like files and locks, will be instances of `Unpickleable`. So if we 
have a list containing a file or a lock, it will still be an instance of 
`Pickleable`, as well as an object which refers to a file/lock as an attribute.

Maybe this is a good case for having only the negative `Unpickleable` and not 
the positive `Pickleable`.


Ram.



From solipsis at pitrou.net  Tue Nov 23 22:48:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 23 Nov 2010 22:48:41 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<loom.20101123T221933-185@post.gmane.org>
Message-ID: <20101123224841.3c762279@pitrou.net>

On Tue, 23 Nov 2010 21:22:13 +0000 (UTC)
Ram Rachum <cool-rr at cool-rr.com> wrote:
> > 
> > By the way, "Unpickleable" doesn't work since it's ambiguous: you don't
> > know whether it means you can unpickle the thing, or you can't pickle
> > it.
> > 
> > Regards
> > 
> > Antoine.
> 
> 
> Sorry, I didn't explain well enough: I meant that only inherently unpickleable 
> objects, like files and locks, will be instances of `Unpickleable`. So if we 
> have a list containing a file or a lock, it will still be an instance of 
> `Pickleable`, as well as an object which refers to a file/lock as an attribute.
> 
> Maybe this is a good case for having only the negative `Unpickleable` and not 
> the positive `Pickleable`.

Read what I said above about the ambiguity of the name, though.




From fdrake at acm.org  Tue Nov 23 22:58:24 2010
From: fdrake at acm.org (Fred Drake)
Date: Tue, 23 Nov 2010 16:58:24 -0500
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <20101123224841.3c762279@pitrou.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<loom.20101123T221933-185@post.gmane.org>
	<20101123224841.3c762279@pitrou.net>
Message-ID: <AANLkTikim7GU-3m2wXCcHXsyA6ONRQZ-RcZoJA7LkMSy@mail.gmail.com>

On Tue, Nov 23, 2010 at 4:48 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Read what I said above about the ambiguity of the name, though.

Indeed.  But there's always pickle.Me.Not.  :-)


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein


From dreamingforward at gmail.com  Tue Nov 23 23:16:04 2010
From: dreamingforward at gmail.com (average)
Date: Tue, 23 Nov 2010 15:16:04 -0700
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
	<AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>
	<20101123143311.40cd3b56@bhuda.mired.org>
	<AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>
Message-ID: <AANLkTimOSWGA_ay3r+1CoSUzeNkG5oiW9o3e3R28bdmQ@mail.gmail.com>

On Tue, Nov 23, 2010 at 12:59 PM, Guido van Rossum <guido at python.org> wrote:

> FWIW I'm fine with adding list.clear() to 3.3.
>
> How about calling it container.clear()?  ;^)

marcos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101123/3b4ce002/attachment.html>

From pyideas at rebertia.com  Tue Nov 23 23:18:23 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Tue, 23 Nov 2010 14:18:23 -0800
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <4CEC234D.6020301@netwok.org>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
Message-ID: <AANLkTiktqFjFuzs+i8xdNzeKq5m4qXK+44RU_HADPNbt@mail.gmail.com>

On Tue, Nov 23, 2010 at 12:25 PM, ?ric Araujo <merwok at netwok.org> wrote:
> Hi,
>
>> Recently I had the need to filter objects based on whether they're picklable
>> or not:
>>
>> http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items
>>
>> I'm not sure what's a good way to check for a specific object whether it's
>> picklable.
>>
>> <http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items>This
>> led me to think: Maybe we should have an `Unpicklable` abstract base class
>> in the `collections` module? Then various unpicklable classes, like locks,
>> files or widgets, could inherit from this class to signify that they cannot
>> be pickled.
>>
>> What do you think?
>
> This sounds useful. ?I?d rather spell the ABC pickle.Picklable, though.

Indeed; let's not turn the `collections` module into a repository of
miscellaneous ABCs.
I'm looking at you, collections.Callable!

Cheers,
Chris
--
Burn the witch!
http://blog.rebertia.com


From mikegraham at gmail.com  Tue Nov 23 23:18:57 2010
From: mikegraham at gmail.com (Mike Graham)
Date: Tue, 23 Nov 2010 17:18:57 -0500
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTimOSWGA_ay3r+1CoSUzeNkG5oiW9o3e3R28bdmQ@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
	<AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>
	<20101123143311.40cd3b56@bhuda.mired.org>
	<AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>
	<AANLkTimOSWGA_ay3r+1CoSUzeNkG5oiW9o3e3R28bdmQ@mail.gmail.com>
Message-ID: <AANLkTikANPaQtQ71BgdbECvKsQv0P-9K8Nndv8hHFSeH@mail.gmail.com>

On Tue, Nov 23, 2010 at 5:16 PM, average <dreamingforward at gmail.com> wrote:
> On Tue, Nov 23, 2010 at 12:59 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> FWIW I'm fine with adding list.clear() to 3.3.
>>
> How about calling it container.clear()?? ;^)
>
> marcos

1) Because that's not how Python's type system works, and
2) Because many containers are immutable?

Mike


From solipsis at pitrou.net  Wed Nov 24 00:01:04 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 24 Nov 2010 00:01:04 +0100
Subject: [Python-ideas] Bring back callable()
Message-ID: <20101124000104.573a85a9@pitrou.net>


Hello,

Python 3 has removed callable() under the justification that's it's not
very useful and duck typing (EAFP) should be used instead. However,
it has since been felt by many people that it was an annoying loss;
there are situations where you truly want to know whether something is a
callable without actually calling it (for example when writing
sophisticated decorators, or simply when you want to inform the user
of an API misuse).

The substitute of writing `isinstance(x, collections.Callable)` is
not good, 1) because it's wordier 2) because collections is really not
an intuitive place where to look for a Callable ABC.

So, I would advocate bringing back the callable() builtin, which was
easy to use, helpful and semantically sane.

Regards

Antoine.




From fuzzyman at voidspace.org.uk  Wed Nov 24 00:14:29 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 23 Nov 2010 23:14:29 +0000
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <20101124000104.573a85a9@pitrou.net>
References: <20101124000104.573a85a9@pitrou.net>
Message-ID: <AANLkTikmTBwiAoOvCLKOEZTdO1s2YkLyyB0j8cCDP4Rn@mail.gmail.com>

On 23 November 2010 23:01, Antoine Pitrou <solipsis at pitrou.net> wrote:

>
> Hello,
>
> Python 3 has removed callable() under the justification that's it's not
> very useful and duck typing (EAFP) should be used instead. However,
> it has since been felt by many people that it was an annoying loss;
> there are situations where you truly want to know whether something is a
> callable without actually calling it (for example when writing
> sophisticated decorators, or simply when you want to inform the user
> of an API misuse).
>
> The substitute of writing `isinstance(x, collections.Callable)` is
> not good, 1) because it's wordier 2) because collections is really not
> an intuitive place where to look for a Callable ABC.
>
> So, I would advocate bringing back the callable() builtin, which was
> easy to use, helpful and semantically sane.
>

+1 I find it useful in Python 2. You have to know its limitations, but it is
still useful.

Michael


>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101123/4772ef96/attachment.html>

From alexandre.conrad at gmail.com  Wed Nov 24 00:15:46 2010
From: alexandre.conrad at gmail.com (Alexandre Conrad)
Date: Tue, 23 Nov 2010 15:15:46 -0800
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <20101124000104.573a85a9@pitrou.net>
References: <20101124000104.573a85a9@pitrou.net>
Message-ID: <AANLkTi=Tr1Q8TugDPxh0BUv+OkuweEDoM2Tzb_hPVU=n@mail.gmail.com>

2010/11/23 Antoine Pitrou <solipsis at pitrou.net>:
> So, I would advocate bringing back the callable() builtin, which was
> easy to use, helpful and semantically sane.

What about:

hasattr(obj, '__call__')

?

-- 
Alex | twitter.com/alexconrad


From robertc at robertcollins.net  Wed Nov 24 00:17:34 2010
From: robertc at robertcollins.net (Robert Collins)
Date: Wed, 24 Nov 2010 12:17:34 +1300
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <20101124000104.573a85a9@pitrou.net>
References: <20101124000104.573a85a9@pitrou.net>
Message-ID: <AANLkTimJaZjxYHE9M-Pg7o0htoRMYnbev6SREWym2d-3@mail.gmail.com>

On Wed, Nov 24, 2010 at 12:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> So, I would advocate bringing back the callable() builtin, which was
> easy to use, helpful and semantically sane.

+1


From guido at python.org  Wed Nov 24 00:19:13 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 23 Nov 2010 15:19:13 -0800
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <AANLkTikmTBwiAoOvCLKOEZTdO1s2YkLyyB0j8cCDP4Rn@mail.gmail.com>
References: <20101124000104.573a85a9@pitrou.net>
	<AANLkTikmTBwiAoOvCLKOEZTdO1s2YkLyyB0j8cCDP4Rn@mail.gmail.com>
Message-ID: <AANLkTin7zyvSJA4BAeYkFB1tGC6PvAbsjScqcTs=DFj2@mail.gmail.com>

I admit defeat on this one.

On Tue, Nov 23, 2010 at 3:14 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>
>
> On 23 November 2010 23:01, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> Hello,
>>
>> Python 3 has removed callable() under the justification that's it's not
>> very useful and duck typing (EAFP) should be used instead. However,
>> it has since been felt by many people that it was an annoying loss;
>> there are situations where you truly want to know whether something is a
>> callable without actually calling it (for example when writing
>> sophisticated decorators, or simply when you want to inform the user
>> of an API misuse).
>>
>> The substitute of writing `isinstance(x, collections.Callable)` is
>> not good, 1) because it's wordier 2) because collections is really not
>> an intuitive place where to look for a Callable ABC.
>>
>> So, I would advocate bringing back the callable() builtin, which was
>> easy to use, helpful and semantically sane.
>
> +1 I find it useful in Python 2. You have to know its limitations, but it is
> still useful.
>
> Michael
>
>>
>> Regards
>>
>> Antoine.
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
>
>
> --
> http://www.voidspace.org.uk
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
--Guido van Rossum (python.org/~guido)


From solipsis at pitrou.net  Wed Nov 24 00:20:24 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 24 Nov 2010 00:20:24 +0100
Subject: [Python-ideas] Bring back callable()
References: <20101124000104.573a85a9@pitrou.net>
	<AANLkTi=Tr1Q8TugDPxh0BUv+OkuweEDoM2Tzb_hPVU=n@mail.gmail.com>
Message-ID: <20101124002024.3abb12c0@pitrou.net>

On Tue, 23 Nov 2010 15:15:46 -0800
Alexandre Conrad
<alexandre.conrad at gmail.com> wrote:
> 2010/11/23 Antoine Pitrou <solipsis at pitrou.net>:
> > So, I would advocate bringing back the callable() builtin, which was
> > easy to use, helpful and semantically sane.
> 
> What about:
> 
> hasattr(obj, '__call__')

It's not the right code.
You can take a look at the implementation of Callable.__subclasshook__
here:
http://code.python.org/hg/branches/py3k/file/tip/Lib/_abcoll.py#l139

Regards

Antoine.




From pyideas at rebertia.com  Wed Nov 24 00:29:29 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Tue, 23 Nov 2010 15:29:29 -0800
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <20101124000104.573a85a9@pitrou.net>
References: <20101124000104.573a85a9@pitrou.net>
Message-ID: <AANLkTinP0SdgcCXhALrQXgSb0C42BnnjSaSAOjjEePja@mail.gmail.com>

On Tue, Nov 23, 2010 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Hello,
>
> Python 3 has removed callable() under the justification that's it's not
> very useful and duck typing (EAFP) should be used instead. However,
> it has since been felt by many people that it was an annoying loss;
> there are situations where you truly want to know whether something is a
> callable without actually calling it (for example when writing
> sophisticated decorators, or simply when you want to inform the user
> of an API misuse).
>
> The substitute of writing `isinstance(x, collections.Callable)` is
> not good, 1) because it's wordier 2) because collections is really not
> an intuitive place where to look for a Callable ABC.
>
> So, I would advocate bringing back the callable() builtin, which was
> easy to use, helpful and semantically sane.

Perhaps it should also be renamed iscallable() for consistency with
isinstance() and issubclass() and to free up the nice name "callable"
for user use.

Cheers,
Chris
--
http://blog.rebertia.com


From greg.ewing at canterbury.ac.nz  Wed Nov 24 00:32:47 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 24 Nov 2010 12:32:47 +1300
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <20101123215458.53eed0b0@pitrou.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
Message-ID: <4CEC4F1F.2090007@canterbury.ac.nz>

Antoine Pitrou wrote:

> By the way, "Unpickleable" doesn't work since it's ambiguous: you don't
> know whether it means you can unpickle the thing, or you can't pickle
> it.

Nonpickleable?

-- 
Greg


From tjreedy at udel.edu  Wed Nov 24 00:38:15 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 23 Nov 2010 18:38:15 -0500
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>	<AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>	<20101123143311.40cd3b56@bhuda.mired.org>
	<AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>
Message-ID: <ichj97$87e$1@dough.gmane.org>

On 11/23/2010 2:59 PM, Guido van Rossum wrote:
> FWIW I'm fine with adding list.clear() to 3.3.

http://bugs.python.org/issue10516

-- 
Terry Jan Reedy



From python at mrabarnett.plus.com  Wed Nov 24 00:50:10 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 23 Nov 2010 23:50:10 +0000
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <AANLkTinP0SdgcCXhALrQXgSb0C42BnnjSaSAOjjEePja@mail.gmail.com>
References: <20101124000104.573a85a9@pitrou.net>
	<AANLkTinP0SdgcCXhALrQXgSb0C42BnnjSaSAOjjEePja@mail.gmail.com>
Message-ID: <4CEC5332.3010801@mrabarnett.plus.com>

On 23/11/2010 23:29, Chris Rebert wrote:
> On Tue, Nov 23, 2010 at 3:01 PM, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>> Hello,
>>
>> Python 3 has removed callable() under the justification that's it's not
>> very useful and duck typing (EAFP) should be used instead. However,
>> it has since been felt by many people that it was an annoying loss;
>> there are situations where you truly want to know whether something is a
>> callable without actually calling it (for example when writing
>> sophisticated decorators, or simply when you want to inform the user
>> of an API misuse).
>>
>> The substitute of writing `isinstance(x, collections.Callable)` is
>> not good, 1) because it's wordier 2) because collections is really not
>> an intuitive place where to look for a Callable ABC.
>>
>> So, I would advocate bringing back the callable() builtin, which was
>> easy to use, helpful and semantically sane.
>
> Perhaps it should also be renamed iscallable() for consistency with
> isinstance() and issubclass() and to free up the nice name "callable"
> for user use.
>
Seems reasonable to me.


From tjreedy at udel.edu  Wed Nov 24 00:50:13 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 23 Nov 2010 18:50:13 -0500
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <20101124000104.573a85a9@pitrou.net>
References: <20101124000104.573a85a9@pitrou.net>
Message-ID: <ichjvm$b0s$1@dough.gmane.org>

On 11/23/2010 6:01 PM, Antoine Pitrou wrote:

> The substitute of writing `isinstance(x, collections.Callable)` is
> not good, 1) because it's wordier 2) because collections is really not
> an intuitive place where to look for a Callable ABC.

I thnk it should be in the abc module, along with WeakSet

The funny thing is that your rationale for not adding Picklable (just 
try and catch exception) is essentially the rationale for removing 
callable() ;-).

-- 
Terry Jan Reedy



From dreamingforward at gmail.com  Wed Nov 24 00:53:45 2010
From: dreamingforward at gmail.com (average)
Date: Tue, 23 Nov 2010 16:53:45 -0700
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTikANPaQtQ71BgdbECvKsQv0P-9K8Nndv8hHFSeH@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
	<AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>
	<20101123143311.40cd3b56@bhuda.mired.org>
	<AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>
	<AANLkTimOSWGA_ay3r+1CoSUzeNkG5oiW9o3e3R28bdmQ@mail.gmail.com>
	<AANLkTikANPaQtQ71BgdbECvKsQv0P-9K8Nndv8hHFSeH@mail.gmail.com>
Message-ID: <AANLkTim+H0ZKvwnB2jBdtAQpN4N3vCkmUWewhvLw5KH-@mail.gmail.com>

On Tue, Nov 23, 2010 at 3:18 PM, Mike Graham <mikegraham at gmail.com> wrote:

> On Tue, Nov 23, 2010 at 5:16 PM, average <dreamingforward at gmail.com>
> wrote:
> > On Tue, Nov 23, 2010 at 12:59 PM, Guido van Rossum <guido at python.org>
> wrote:
> >>
> >> FWIW I'm fine with adding list.clear() to 3.3.
> >>
> > How about calling it container.clear()?  ;^)
> > marcos
>
> 1) Because that's not how Python's type system works, and
> 2) Because many containers are immutable?
>
> That's a decent point.  But in the case of a non-uniform container scheme
and in absence of a obvious syntax to distinguish between mutable and
immutable types, then it doesn't really make much sense to add the clear
method to lists.... yet

mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101123/6e15f66e/attachment.html>

From dreamingforward at gmail.com  Wed Nov 24 01:21:52 2010
From: dreamingforward at gmail.com (average)
Date: Tue, 23 Nov 2010 17:21:52 -0700
Subject: [Python-ideas] syntax for set
In-Reply-To: <20101122102341.GA2977@chopin.edu.pl>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
Message-ID: <AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>

On Mon, Nov 22, 2010 at 3:23 AM, Jan Kaliszewski <zuo at chopin.edu.pl> wrote:
>
> > I'd be happy with:
> >
> > * {:} for empty dict() (as a collection of key-value *pairs*)
> > * {.} for empty set() (as a similar collection of *single* elements)
> >
> > And {} for empty dict() as well -- to keep compatibility (maybe to be
> > deprecated later).
>
> PS. Another argument: {} is criticized for its similarity to () (when using
> some fonts).
>
> That is sometimes true, but one should bend towards fixing the font problem
rather than adapting to brokenness....

m
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101123/a8dbc63f/attachment.html>

From solipsis at pitrou.net  Wed Nov 24 01:25:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 24 Nov 2010 01:25:47 +0100
Subject: [Python-ideas] Bring back callable()
References: <20101124000104.573a85a9@pitrou.net> <ichjvm$b0s$1@dough.gmane.org>
Message-ID: <20101124012547.35795f9c@pitrou.net>

On Tue, 23 Nov 2010 18:50:13 -0500
Terry Reedy <tjreedy at udel.edu> wrote:
> On 11/23/2010 6:01 PM, Antoine Pitrou wrote:
> 
> > The substitute of writing `isinstance(x, collections.Callable)` is
> > not good, 1) because it's wordier 2) because collections is really not
> > an intuitive place where to look for a Callable ABC.
> 
> I thnk it should be in the abc module, along with WeakSet
> 
> The funny thing is that your rationale for not adding Picklable (just 
> try and catch exception) is essentially the rationale for removing 
> callable() ;-).

Well, I have never needed to know in advance whether an object was
picklable or not, while knowing whether it's callable has come often.

Regards

Antoine.




From cmjohnson.mailinglist at gmail.com  Wed Nov 24 01:53:49 2010
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Tue, 23 Nov 2010 14:53:49 -1000
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <20101123215458.53eed0b0@pitrou.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
Message-ID: <AANLkTikWbLdR-+GuHE7ikrxxh6g26PQzo=wz5mj13vQ2@mail.gmail.com>

On Tue, Nov 23, 2010 at 10:54 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:


>  > So `Pickleable` could have a `__subclasshook__` that would do the real
> work,
> > similarly to `Iterable`.
>
> The problem is that "doing the real work" can be CPU-intensive, and I
> don't think we would like isinstance() calls to become arbitrarily
> expensive (even if technically it's possible).
>
> An alternative is simply to catch the exception when trying to pickle a
> value. That implies that the type takes care to raise an exception if
> pickling would work but unpickling wouldn't, which is what file objects
> now do.
>

Yes, but isn't the advantage of having an ABC that you only have to "do the
real work" one time and after that the class will be registered as
Pickleable/NonPickleable? So, the first time you check a random class it
would see if it can call the Pickling methods and if so register that class
so future isinstance checks are virtually free and don't have to be dealt
with using try/except.

Maybe I'm misunderstanding something.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101123/eee831eb/attachment.html>

From benjamin at python.org  Wed Nov 24 04:02:11 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 24 Nov 2010 03:02:11 +0000 (UTC)
Subject: [Python-ideas] Bring back callable()
References: <20101124000104.573a85a9@pitrou.net>
	<AANLkTinP0SdgcCXhALrQXgSb0C42BnnjSaSAOjjEePja@mail.gmail.com>
Message-ID: <loom.20101124T040048-126@post.gmane.org>

Chris Rebert <pyideas at ...> writes:
> Perhaps it should also be renamed iscallable() for consistency with
> isinstance() and issubclass() and to free up the nice name "callable"
> for user use.

Let's not. We shouldn't make it anymore difficult to cope with having a gap of
no callable() by renaming it. Anyway, they're different kinds of words.
"instance()" or "subclass()" wouldn't make any sense.






From dreamingforward at gmail.com  Wed Nov 24 05:15:25 2010
From: dreamingforward at gmail.com (average)
Date: Tue, 23 Nov 2010 21:15:25 -0700
Subject: [Python-ideas] syntax for set
In-Reply-To: <856C9747-91B9-4E8D-8937-AA10A658F8E1@gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<856C9747-91B9-4E8D-8937-AA10A658F8E1@gmail.com>
Message-ID: <AANLkTin-7BTSn=GwA+R95J1eeRda0kL9bfiFqVfd2Bgy@mail.gmail.com>

On Mon, Nov 22, 2010 at 11:40 AM, Raymond Hettinger <
raymond.hettinger at gmail.com> wrote:

> > I'd be happy with:
> >
> > * {:} for empty dict() (as a collection of key-value *pairs*)
> > * {.} for empty set() (as a similar collection of *single* elements)
> >
> > And {} for empty dict() as well -- to keep compatibility (maybe to be
> > deprecated later).
>
> I'm curious why you guys think you *need* an empty set literal.
> The current spelling using set() and frozenset() is unambiguous.
> So what's the point of trying to shoehorn-in a new literal?
>
> AFAICT, this discussion has been solely motivated by
> a shallow itch, "dicts have one, so sets have to have one too".
>

Because those forms (to the starting programmer especially) make it seem
like a second-class citizen.  That may seem a trivial objection, but those
"sensitive initial conditions" set up trajectories which have (eventually)
far-reaching consequences.  Separately, but along those lines,  if
mutability and immutability are frequent enough concerns, then perhaps some
thought should be given to how one might address that with a simple
syntactical or general, object-wise methodology.  Perhaps this thread is not
appropriate to pose the question, but should there be a general mutable bit
attached to any given object?

If there were a clean, beautiful, obvious correct answer,
> it would have already been done.   Since there isn't,
> we have to ask, who cares?
>
> No! there is a clean, beautiful, obvious correct answer!  The problem is
*only* backwards-compatibility.  That is why we care.

Cordially, in any case... :)

mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101123/01d9d306/attachment.html>

From solipsis at pitrou.net  Wed Nov 24 09:45:12 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 24 Nov 2010 09:45:12 +0100
Subject: [Python-ideas] syntax for set
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<856C9747-91B9-4E8D-8937-AA10A658F8E1@gmail.com>
Message-ID: <20101124094512.0da1566f@pitrou.net>

On Mon, 22 Nov 2010 10:40:21 -0800
Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
> 
> If there were a clean, beautiful, obvious correct answer,
> it would have already been done.

Are you predicting everything we will add from now on to the language
will be filthy, ugly and confusingly (in)correct? :)

Regards

Antoine.




From masklinn at masklinn.net  Wed Nov 24 09:58:34 2010
From: masklinn at masklinn.net (Masklinn)
Date: Wed, 24 Nov 2010 09:58:34 +0100
Subject: [Python-ideas] Set Syntax
In-Reply-To: <AANLkTim+H0ZKvwnB2jBdtAQpN4N3vCkmUWewhvLw5KH-@mail.gmail.com>
References: <AANLkTin7GdCShVbWg2ySWDp_gV3tOBci2944m=AzG08s@mail.gmail.com>
	<AANLkTi=qb9MF+2kKEkW6E9VgOzzE3pJJv=sykFqC5Jqa@mail.gmail.com>
	<20101123143311.40cd3b56@bhuda.mired.org>
	<AANLkTimutNrJp0x14zgJwvcfjGWqChMjyc0UdYsz9T3t@mail.gmail.com>
	<AANLkTimOSWGA_ay3r+1CoSUzeNkG5oiW9o3e3R28bdmQ@mail.gmail.com>
	<AANLkTikANPaQtQ71BgdbECvKsQv0P-9K8Nndv8hHFSeH@mail.gmail.com>
	<AANLkTim+H0ZKvwnB2jBdtAQpN4N3vCkmUWewhvLw5KH-@mail.gmail.com>
Message-ID: <6DF15753-652F-4246-AD12-7C62BA9B580D@masklinn.net>

On 2010-11-24, at 00:53 , average wrote:
> On Tue, Nov 23, 2010 at 3:18 PM, Mike Graham <mikegraham at gmail.com> wrote:
>> On Tue, Nov 23, 2010 at 5:16 PM, average <dreamingforward at gmail.com>
>> wrote:
>>> On Tue, Nov 23, 2010 at 12:59 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>>> 
>>>> FWIW I'm fine with adding list.clear() to 3.3.
>>>> 
>>> How about calling it container.clear()?  ;^)
>>> marcos
>> 
>> 1) Because that's not how Python's type system works, and
>> 2) Because many containers are immutable?
>> 
>> That's a decent point.  But in the case of a non-uniform container scheme
> and in absence of a obvious syntax to distinguish between mutable and
> immutable types, then it doesn't really make much sense to add the clear
> method to lists.... yet
That's a pretty terrible point: sets have a .clear() method, frozensets don't. Dicts have a .clear() method, if functional maps were added to the language they probably wouldn't. Mutable collections have a .clear() method, immutable collections don't. You can only mix them (mutable and immutable methods) when reading them (in which case you don't care for the .clear() method anyway) so it's not like it matters.

From g.brandl at gmx.net  Wed Nov 24 10:02:54 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 24 Nov 2010 10:02:54 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<20101122101641.GA2326@chopin.edu.pl>	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
Message-ID: <icikc0$41n$1@dough.gmane.org>

Am 24.11.2010 01:21, schrieb average:
> 
> 
> On Mon, Nov 22, 2010 at 3:23 AM, Jan Kaliszewski <zuo at chopin.edu.pl
> <mailto:zuo at chopin.edu.pl>> wrote:
> 
>     > I'd be happy with:
>     >
>     > * {:} for empty dict() (as a collection of key-value *pairs*)
>     > * {.} for empty set() (as a similar collection of *single* elements)
>     >
>     > And {} for empty dict() as well -- to keep compatibility (maybe to be
>     > deprecated later).
> 
>     PS. Another argument: {} is criticized for its similarity to () (when using
>     some fonts).
> 
> That is sometimes true, but one should bend towards fixing the font problem
> rather than adapting to brokenness....

Yep. Otherwise you would also need to forbid "1" in identifiers because it
can easily be confused with "l" in some fonts.  (And of course, let's not
speak of identifiers in Python 3 in this context...)

Georg



From masklinn at masklinn.net  Wed Nov 24 10:40:58 2010
From: masklinn at masklinn.net (Masklinn)
Date: Wed, 24 Nov 2010 10:40:58 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <icikc0$41n$1@dough.gmane.org>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<20101122101641.GA2326@chopin.edu.pl>	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
Message-ID: <03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>

On 2010-11-24, at 10:02 , Georg Brandl wrote:
Am 24.11.2010 01:21, schrieb average:
>> On Mon, Nov 22, 2010 at 3:23 AM, Jan Kaliszewski <zuo at chopin.edu.pl
>> <mailto:zuo at chopin.edu.pl>> wrote:
>>> I'd be happy with:
>>> 
>>> * {:} for empty dict() (as a collection of key-value *pairs*)
>>> * {.} for empty set() (as a similar collection of *single* elements)
>>> 
>>> And {} for empty dict() as well -- to keep compatibility (maybe to be
>>> deprecated later).
>>    PS. Another argument: {} is criticized for its similarity to () (when using
>>    some fonts).
>> 
>> That is sometimes true, but one should bend towards fixing the font problem
>> rather than adapting to brokenness....
> 
> Yep. Otherwise you would also need to forbid "1" in identifiers because it
> can easily be confused with "l" in some fonts.  (And of course, let's not
> speak of identifiers in Python 3 in this context...)
> 
> Georg
> 
Since 1s will be in the code source in any case (as free-standing integers or as part of floats) wouldn't it be more sensible to ban `l` from identifiers? As well as uppercase O just to be on the safe side?

As an alternative, I would modestly propose banning integer literals altogether in favor of english-spelled constants: `ONE` can hardly be confused with the letter `l`, and having to write `FIVE HUNDRED FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE MILLION NINE HUNDRED EIGHTY ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than 514451981921 would surely reduce the usage of redundant magic numbers in the code.

From mwm-keyword-python.b4bdba at mired.org  Wed Nov 24 11:05:17 2010
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Wed, 24 Nov 2010 05:05:17 -0500
Subject: [Python-ideas] syntax for set
In-Reply-To: <03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
Message-ID: <140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>

"Masklinn" <masklinn at masklinn.net> wrote:

>As an alternative, I would modestly propose banning integer literals
>altogether in favor of english-spelled constants: `ONE` can hardly be
>confused with the letter `l`, and having to write `FIVE HUNDRED
>FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE MILLION NINE HUNDRED EIGHTY
>ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than 514451981921 would
>surely reduce the usage of redundant magic numbers in the code.

Given that roman numeral literals have already been rejected, I'd like to propose them as an alternative alternative.
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.


From cool-rr at cool-rr.com  Wed Nov 24 11:54:54 2010
From: cool-rr at cool-rr.com (Ram Rachum)
Date: Wed, 24 Nov 2010 10:54:54 +0000 (UTC)
Subject: [Python-ideas] Bring back callable()
References: <20101124000104.573a85a9@pitrou.net>
Message-ID: <loom.20101124T115206-756@post.gmane.org>

Antoine Pitrou <solipsis at ...> writes:
> Python 3 has removed callable() under the justification that's it's not
> very useful and duck typing (EAFP) should be used instead. However,
> it has since been felt by many people that it was an annoying loss;
> there are situations where you truly want to know whether something is a
> callable without actually calling it (for example when writing
> sophisticated decorators, or simply when you want to inform the user
> of an API misuse).
> 
> The substitute of writing `isinstance(x, collections.Callable)` is
> not good, 1) because it's wordier 2) because collections is really not
> an intuitive place where to look for a Callable ABC.
> 
> So, I would advocate bringing back the callable() builtin, which was
> easy to use, helpful and semantically sane.
> 
> Antoine.

I actually really prefer `isinstance(x, collections.Callable)`...

If this was accepted, I would at least want it to be called `iscallable` instead 
of `callable`, since I sometimes want to use the name `callable` in my code.


Ram.



From mal at egenix.com  Wed Nov 24 11:59:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 24 Nov 2010 11:59:34 +0100
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <loom.20101124T115206-756@post.gmane.org>
References: <20101124000104.573a85a9@pitrou.net>
	<loom.20101124T115206-756@post.gmane.org>
Message-ID: <4CECF016.6020106@egenix.com>

Ram Rachum wrote:
> Antoine Pitrou <solipsis at ...> writes:
>> Python 3 has removed callable() under the justification that's it's not
>> very useful and duck typing (EAFP) should be used instead. However,
>> it has since been felt by many people that it was an annoying loss;
>> there are situations where you truly want to know whether something is a
>> callable without actually calling it (for example when writing
>> sophisticated decorators, or simply when you want to inform the user
>> of an API misuse).
>>
>> The substitute of writing `isinstance(x, collections.Callable)` is
>> not good, 1) because it's wordier 2) because collections is really not
>> an intuitive place where to look for a Callable ABC.
>>
>> So, I would advocate bringing back the callable() builtin, which was
>> easy to use, helpful and semantically sane.
>>
>> Antoine.
> 
> I actually really prefer `isinstance(x, collections.Callable)`...
>
> If this was accepted, I would at least want it to be called `iscallable` instead 
> of `callable`, since I sometimes want to use the name `callable` in my code.

That's still possible. callable() is a built-in function,
not a keyword.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 24 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From g.brandl at gmx.net  Wed Nov 24 11:59:57 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 24 Nov 2010 11:59:57 +0100
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <AANLkTin7zyvSJA4BAeYkFB1tGC6PvAbsjScqcTs=DFj2@mail.gmail.com>
References: <20101124000104.573a85a9@pitrou.net>	<AANLkTikmTBwiAoOvCLKOEZTdO1s2YkLyyB0j8cCDP4Rn@mail.gmail.com>
	<AANLkTin7zyvSJA4BAeYkFB1tGC6PvAbsjScqcTs=DFj2@mail.gmail.com>
Message-ID: <icir7f$429$1@dough.gmane.org>

Falls under "no new builtins" though.

Georg

Am 24.11.2010 00:19, schrieb Guido van Rossum:
> I admit defeat on this one.
> 
> On Tue, Nov 23, 2010 at 3:14 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>>
>>
>> On 23 November 2010 23:01, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>
>>> Hello,
>>>
>>> Python 3 has removed callable() under the justification that's it's not
>>> very useful and duck typing (EAFP) should be used instead. However,
>>> it has since been felt by many people that it was an annoying loss;
>>> there are situations where you truly want to know whether something is a
>>> callable without actually calling it (for example when writing
>>> sophisticated decorators, or simply when you want to inform the user
>>> of an API misuse).
>>>
>>> The substitute of writing `isinstance(x, collections.Callable)` is
>>> not good, 1) because it's wordier 2) because collections is really not
>>> an intuitive place where to look for a Callable ABC.
>>>
>>> So, I would advocate bringing back the callable() builtin, which was
>>> easy to use, helpful and semantically sane.
>>
>> +1 I find it useful in Python 2. You have to know its limitations, but it is
>> still useful.
>>
>> Michael
>>
>>>
>>> Regards
>>>
>>> Antoine.
>>>
>>>
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>>
>>
>> --
>> http://www.voidspace.org.uk
>>
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>>
> 
> 
> 




From cmjohnson.mailinglist at gmail.com  Wed Nov 24 11:57:57 2010
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Wed, 24 Nov 2010 00:57:57 -1000
Subject: [Python-ideas] syntax for set
In-Reply-To: <140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>
Message-ID: <AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>

On Wed, Nov 24, 2010 at 12:05 AM, Mike Meyer <
mwm-keyword-python.b4bdba at mired.org> wrote:

> "Masklinn" <masklinn at masklinn.net> wrote:
>
> >As an alternative, I would modestly propose banning integer literals
> >altogether in favor of english-spelled constants: `ONE` can hardly be
> >confused with the letter `l`, and having to write `FIVE HUNDRED
> >FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE MILLION NINE HUNDRED EIGHTY
> >ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than 514451981921 would
> >surely reduce the usage of redundant magic numbers in the code.
>

+1 If it's good enough for my banker when writing checks, it should be good
enough for Python. But it might be safer if we required the Arabic numerals
in parentheses following the English. If the two don't match, we could raise
a CheckingError. Seems to be a basically self-documenting error type to me.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101124/31b22833/attachment.html>

From cool-rr at cool-rr.com  Wed Nov 24 12:12:20 2010
From: cool-rr at cool-rr.com (Ram Rachum)
Date: Wed, 24 Nov 2010 11:12:20 +0000 (UTC)
Subject: [Python-ideas] Bring back callable()
References: <20101124000104.573a85a9@pitrou.net>
	<loom.20101124T115206-756@post.gmane.org>
	<4CECF016.6020106@egenix.com>
Message-ID: <loom.20101124T121138-880@post.gmane.org>

M.-A. Lemburg <mal at ...> writes:

> 
> Ram Rachum wrote:
> > If this was accepted, I would at least want it to be called `iscallable` 
instead 
> > of `callable`, since I sometimes want to use the name `callable` in my code.
> 
> That's still possible. callable() is a built-in function,
> not a keyword.
> 

I know, but I really don't like overshadowing built-ins.


Ram.



From masklinn at masklinn.net  Wed Nov 24 13:37:37 2010
From: masklinn at masklinn.net (Masklinn)
Date: Wed, 24 Nov 2010 13:37:37 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>
	<AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
Message-ID: <8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>

On 2010-11-24, at 11:57 , Carl M. Johnson wrote:
> On Wed, Nov 24, 2010 at 12:05 AM, Mike Meyer <
> mwm-keyword-python.b4bdba at mired.org> wrote:
>> "Masklinn" <masklinn at masklinn.net> wrote:
>>> As an alternative, I would modestly propose banning integer literals
>>> altogether in favor of english-spelled constants: `ONE` can hardly be
>>> confused with the letter `l`, and having to write `FIVE HUNDRED
>>> FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE MILLION NINE HUNDRED EIGHTY
>>> ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than 514451981921 would
>>> surely reduce the usage of redundant magic numbers in the code.
> +1 If it's good enough for my banker when writing checks, it should be good
> enough for Python. But it might be safer if we required the Arabic numerals
> in parentheses following the English. If the two don't match, we could raise
> a CheckingError. Seems to be a basically self-documenting error type to me.

But then we're back with the possible confusion between arabic numerals and ascii characters.

Still, I think this idea is a good one, we need some kind of checksum. Maybe it could be merged with Mike's proposal: the "basic english" number is followed by the same number written using roman numerals and the two versions are checked against one another to ensure consistency.



From steve at pearwood.info  Wed Nov 24 14:28:41 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 25 Nov 2010 00:28:41 +1100
Subject: [Python-ideas] syntax for set
In-Reply-To: <8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<20101122101641.GA2326@chopin.edu.pl>	<20101122102341.GA2977@chopin.edu.pl>	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>	<icikc0$41n$1@dough.gmane.org>	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>	<AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
	<8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>
Message-ID: <4CED1309.3070305@pearwood.info>

Masklinn wrote:
> On 2010-11-24, at 11:57 , Carl M. Johnson wrote:
>> On Wed, Nov 24, 2010 at 12:05 AM, Mike Meyer <
>> mwm-keyword-python.b4bdba at mired.org> wrote:
>>> "Masklinn" <masklinn at masklinn.net> wrote:
>>>> As an alternative, I would modestly propose banning integer literals
>>>> altogether in favor of english-spelled constants: `ONE` can hardly be
>>>> confused with the letter `l`, and having to write `FIVE HUNDRED
>>>> FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE MILLION NINE HUNDRED EIGHTY
>>>> ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than 514451981921 would
>>>> surely reduce the usage of redundant magic numbers in the code.
>> +1 If it's good enough for my banker when writing checks, it should be good
>> enough for Python. But it might be safer if we required the Arabic numerals
>> in parentheses following the English. If the two don't match, we could raise
>> a CheckingError. Seems to be a basically self-documenting error type to me.
> 
> But then we're back with the possible confusion between arabic numerals and ascii characters.
> 
> Still, I think this idea is a good one, we need some kind of checksum. Maybe it could be merged with Mike's proposal: the "basic english" number is followed by the same number written using roman numerals and the two versions are checked against one another to ensure consistency.

Is this thread happening four months early or eight months late?

But in the spirit of this discussion, can I propose that Python require 
all numeric literals be written out in full English words, followed by 
the binary expansion in brackets, so as to ensure that the value 
generated by Python matches the programmers expectation?

n = ONE THOUSAND FOUR HUNDRED AND FIFTY SEVEN (10110110001)

This will especially be useful for float literals, since the programmer 
will no longer be able to feign ignorance of the complexities of binary 
floating point, but will have to deal with them explicitly:

x = FORTY-TWO POINT ONE \
     (100000001000101000011001100110011001100110011001100110011001101)

Never again will we have to answer questions about why Python's floating 
point maths is inaccurate!

Should we enter the binary expansion in big-endian or little-endian format?



-- 
Steven



From python at mrabarnett.plus.com  Wed Nov 24 17:25:04 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 24 Nov 2010 16:25:04 +0000
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <icir7f$429$1@dough.gmane.org>
References: <20101124000104.573a85a9@pitrou.net>	<AANLkTikmTBwiAoOvCLKOEZTdO1s2YkLyyB0j8cCDP4Rn@mail.gmail.com>	<AANLkTin7zyvSJA4BAeYkFB1tGC6PvAbsjScqcTs=DFj2@mail.gmail.com>
	<icir7f$429$1@dough.gmane.org>
Message-ID: <4CED3C60.1060608@mrabarnett.plus.com>

On 24/11/2010 10:59, Georg Brandl wrote:
> Falls under "no new builtins" though.
>
It's not really new because it's in Python 2. :-)

> Georg
>
> Am 24.11.2010 00:19, schrieb Guido van Rossum:
>> I admit defeat on this one.
>>
>> On Tue, Nov 23, 2010 at 3:14 PM, Michael Foord
>> <fuzzyman at voidspace.org.uk>  wrote:
>>>
>>>
>>> On 23 November 2010 23:01, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>>>>
>>>> Hello,
>>>>
>>>> Python 3 has removed callable() under the justification that's it's not
>>>> very useful and duck typing (EAFP) should be used instead. However,
>>>> it has since been felt by many people that it was an annoying loss;
>>>> there are situations where you truly want to know whether something is a
>>>> callable without actually calling it (for example when writing
>>>> sophisticated decorators, or simply when you want to inform the user
>>>> of an API misuse).
>>>>
>>>> The substitute of writing `isinstance(x, collections.Callable)` is
>>>> not good, 1) because it's wordier 2) because collections is really not
>>>> an intuitive place where to look for a Callable ABC.
>>>>
>>>> So, I would advocate bringing back the callable() builtin, which was
>>>> easy to use, helpful and semantically sane.
>>>
>>> +1 I find it useful in Python 2. You have to know its limitations, but it is
>>> still useful.
>>>


From mwm-keyword-python.b4bdba at mired.org  Wed Nov 24 20:21:34 2010
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Wed, 24 Nov 2010 14:21:34 -0500
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CED1309.3070305@pearwood.info>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>
	<AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
	<8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>
	<4CED1309.3070305@pearwood.info>
Message-ID: <20101124142134.7d544c4d@bhuda.mired.org>

On Thu, 25 Nov 2010 00:28:41 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> But in the spirit of this discussion, can I propose that Python require 
> all numeric literals be written out in full English words, followed by 
> the binary expansion in brackets, so as to ensure that the value 
> generated by Python matches the programmers expectation?
> 
> n = ONE THOUSAND FOUR HUNDRED AND FIFTY SEVEN (10110110001)
> 
> This will especially be useful for float literals, since the programmer 
> will no longer be able to feign ignorance of the complexities of binary 
> floating point, but will have to deal with them explicitly:
> 
> x = FORTY-TWO POINT ONE \
>      (100000001000101000011001100110011001100110011001100110011001101)
> 
> Never again will we have to answer questions about why Python's floating 
> point maths is inaccurate!
> 
> Should we enter the binary expansion in big-endian or little-endian format?

Programmers choice. Use the struct module notation:

(=....) Native order.
(<....) Little-endian.
(>....) Big-endian

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From dreamingforward at gmail.com  Thu Nov 25 00:52:57 2010
From: dreamingforward at gmail.com (average)
Date: Wed, 24 Nov 2010 16:52:57 -0700
Subject: [Python-ideas] syntax for set
In-Reply-To: <8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>
	<AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
	<8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>
Message-ID: <AANLkTimYXbUepB0w=yqXKxWMyC+H_mCTNNARHNhL356J@mail.gmail.com>

On Wed, Nov 24, 2010 at 5:37 AM, Masklinn <masklinn at masklinn.net> wrote:

> On 2010-11-24, at 11:57 , Carl M. Johnson wrote:
> > On Wed, Nov 24, 2010 at 12:05 AM, Mike Meyer <
> > mwm-keyword-python.b4bdba at mired.org> wrote:
> >> "Masklinn" <masklinn at masklinn.net> wrote:
> >>> As an alternative, I would modestly propose banning integer literals
> >>> altogether in favor of english-spelled constants: `ONE` can hardly be
> >>> confused with the letter `l`, and having to write `FIVE HUNDRED
> >>> FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE MILLION NINE HUNDRED EIGHTY
> >>> ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than 514451981921 would
> >>> surely reduce the usage of redundant magic numbers in the code.
> > +1 If it's good enough for my banker when writing checks, it should be
> good
> > enough for Python. But it might be safer if we required the Arabic
> numerals
> > in parentheses following the English. If the two don't match, we could
> raise
> > a CheckingError. Seems to be a basically self-documenting error type to
> me.
>
> But then we're back with the possible confusion between arabic numerals and
> ascii characters.
>
> Still, I think this idea is a good one, we need some kind of checksum.
> Maybe it could be merged with Mike's proposal: the "basic english" number is
> followed by the same number written using roman numerals and the two
> versions are checked against one another to ensure consistency.
>
>
Wow.  We are in the Twilight Zone.  Queue music...

mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101124/7454ba4d/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Nov 25 03:42:55 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 25 Nov 2010 15:42:55 +1300
Subject: [Python-ideas] syntax for set
In-Reply-To: <03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
Message-ID: <4CEDCD2F.6010504@canterbury.ac.nz>

On 24/11/10 22:40, Masklinn wrote:

> Since 1s will be in the code source in any case (as free-standing integers
> or as part of floats) wouldn't it be more sensible to ban `l` from
> identifiers? As well as uppercase O just to be on the safe side?

Don't forget Z, A and S to eliminate confusion with 2, 4 and 5.

> having to write `FIVE HUNDRED FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE
> MILLION NINE HUNDRED EIGHTY ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than
> 514451981921 would surely reduce the usage of redundant magic numbers in the code.

Please, no. We'd have endless arguments over whether it should be
NINE HUNDRED TWENTY ONE or NINE HUNDRED AND TWENTY ONE. :-(

-- 
Greg


From ben+python at benfinney.id.au  Thu Nov 25 03:56:18 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 25 Nov 2010 13:56:18 +1100
Subject: [Python-ideas] syntax for set
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<4CEDCD2F.6010504@canterbury.ac.nz>
Message-ID: <87vd3m6rhp.fsf@benfinney.id.au>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

> Please, no. We'd have endless arguments over whether it should be
> NINE HUNDRED TWENTY ONE or NINE HUNDRED AND TWENTY ONE. :-(

Clearly there needs to be a compile-time option to enable the user to
select which one is correct.

-- 
 \        ?Odious ideas are not entitled to hide from criticism behind |
  `\          the human shield of their believers' feelings.? ?Richard |
_o__)                                                         Stallman |
Ben Finney



From python at mrabarnett.plus.com  Thu Nov 25 04:01:20 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 25 Nov 2010 03:01:20 +0000
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CEDCD2F.6010504@canterbury.ac.nz>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>	<20101122101641.GA2326@chopin.edu.pl>	<20101122102341.GA2977@chopin.edu.pl>	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>	<icikc0$41n$1@dough.gmane.org>	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<4CEDCD2F.6010504@canterbury.ac.nz>
Message-ID: <4CEDD180.1030906@mrabarnett.plus.com>

On 25/11/2010 02:42, Greg Ewing wrote:
> On 24/11/10 22:40, Masklinn wrote:
>
>> Since 1s will be in the code source in any case (as free-standing
>> integers
>> or as part of floats) wouldn't it be more sensible to ban `l` from
>> identifiers? As well as uppercase O just to be on the safe side?
>
> Don't forget Z, A and S to eliminate confusion with 2, 4 and 5.
>
>> having to write `FIVE HUNDRED FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE
>> MILLION NINE HUNDRED EIGHTY ONE THOUSAND NINE HUNDRED TWENTY ONE`
>> rather than
>> 514451981921 would surely reduce the usage of redundant magic numbers
>> in the code.
>
> Please, no. We'd have endless arguments over whether it should be
> NINE HUNDRED TWENTY ONE or NINE HUNDRED AND TWENTY ONE. :-(
>
Shouldn't it be in Dutch?


From greg.ewing at canterbury.ac.nz  Thu Nov 25 04:33:43 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 25 Nov 2010 16:33:43 +1300
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>
	<AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
Message-ID: <4CEDD917.7040009@canterbury.ac.nz>

On 24/11/10 23:57, Carl M. Johnson wrote:
> But it might be safer if we required the Arabic
> numerals in parentheses following the English. If the two don't match,
> we could raise a CheckingError.

Also require the rest of the line to be filled with "*" characters
(up to the PEP-8 standard line length of 80 chars) to verify that
nothing has been added or removed since the program was written.
You can't be too careful when numeric-literal-eating nanoviruses
are around...

-- 
Greg


From denis.spir at gmail.com  Thu Nov 25 11:20:53 2010
From: denis.spir at gmail.com (spir)
Date: Thu, 25 Nov 2010 11:20:53 +0100
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CEDD180.1030906@mrabarnett.plus.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<4CEDCD2F.6010504@canterbury.ac.nz>
	<4CEDD180.1030906@mrabarnett.plus.com>
Message-ID: <20101125112053.0b75b431@o>

On Thu, 25 Nov 2010 03:01:20 +0000
MRAB <python at mrabarnett.plus.com> wrote:

> On 25/11/2010 02:42, Greg Ewing wrote:
> > On 24/11/10 22:40, Masklinn wrote:
> >
> >> Since 1s will be in the code source in any case (as free-standing
> >> integers
> >> or as part of floats) wouldn't it be more sensible to ban `l` from
> >> identifiers? As well as uppercase O just to be on the safe side?
> >
> > Don't forget Z, A and S to eliminate confusion with 2, 4 and 5.
> >
> >> having to write `FIVE HUNDRED FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE
> >> MILLION NINE HUNDRED EIGHTY ONE THOUSAND NINE HUNDRED TWENTY ONE`
> >> rather than
> >> 514451981921 would surely reduce the usage of redundant magic numbers
> >> in the code.
> >
> > Please, no. We'd have endless arguments over whether it should be
> > NINE HUNDRED TWENTY ONE or NINE HUNDRED AND TWENTY ONE. :-(
> >
> Shouldn't it be in Dutch?

+1

PS: google translate: VIJFHONDERD eenenveertig miljard vierhonderd Fifty One miljoen negenhonderd eenentachtig duizend Negen honderd twintig ONE

(I wonder about its casing algorithm ;-) ...and don't want to have to fix it.)


denis
-- -- -- -- -- -- --
vit esse estrany ?

spir.wikidot.com



From rob.cliffe at btinternet.com  Thu Nov 25 13:21:06 2010
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 25 Nov 2010 12:21:06 +0000
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTimYXbUepB0w=yqXKxWMyC+H_mCTNNARHNhL356J@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>	<ibrv3a$pk6$1@dough.gmane.org>	<20101122101641.GA2326@chopin.edu.pl>	<20101122102341.GA2977@chopin.edu.pl>	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>	<icikc0$41n$1@dough.gmane.org>	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>	<AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>	<8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>
	<AANLkTimYXbUepB0w=yqXKxWMyC+H_mCTNNARHNhL356J@mail.gmail.com>
Message-ID: <4CEE54B2.9060905@btinternet.com>

The idea is flawed.  No, really!  People would start writing integer 
literals as floats.  And the scheme doesn't cover floats (there is no 
standard way of writing decimal places in Roman numerals AFAIK).
Rob Cliffe

On 24/11/2010 23:52, average wrote:
> On Wed, Nov 24, 2010 at 5:37 AM, Masklinn <masklinn at masklinn.net 
> <mailto:masklinn at masklinn.net>> wrote:
>
>     On 2010-11-24, at 11:57 , Carl M. Johnson wrote:
>     > On Wed, Nov 24, 2010 at 12:05 AM, Mike Meyer <
>     > mwm-keyword-python.b4bdba at mired.org
>     <mailto:mwm-keyword-python.b4bdba at mired.org>> wrote:
>     >> "Masklinn" <masklinn at masklinn.net
>     <mailto:masklinn at masklinn.net>> wrote:
>     >>> As an alternative, I would modestly propose banning integer
>     literals
>     >>> altogether in favor of english-spelled constants: `ONE` can
>     hardly be
>     >>> confused with the letter `l`, and having to write `FIVE HUNDRED
>     >>> FORTY-ONE BILLION FOUR HUNDRED FIFTY ONE MILLION NINE HUNDRED
>     EIGHTY
>     >>> ONE THOUSAND NINE HUNDRED TWENTY ONE` rather than 514451981921
>     would
>     >>> surely reduce the usage of redundant magic numbers in the code.
>     > +1 If it's good enough for my banker when writing checks, it
>     should be good
>     > enough for Python. But it might be safer if we required the
>     Arabic numerals
>     > in parentheses following the English. If the two don't match, we
>     could raise
>     > a CheckingError. Seems to be a basically self-documenting error
>     type to me.
>
>     But then we're back with the possible confusion between arabic
>     numerals and ascii characters.
>
>     Still, I think this idea is a good one, we need some kind of
>     checksum. Maybe it could be merged with Mike's proposal: the
>     "basic english" number is followed by the same number written
>     using roman numerals and the two versions are checked against one
>     another to ensure consistency.
>
>
> Wow.  We are in the Twilight Zone.  Queue music...
>
> mark
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101125/289aab70/attachment.html>

From bruce at leapyear.org  Thu Nov 25 18:37:35 2010
From: bruce at leapyear.org (Bruce Leban)
Date: Thu, 25 Nov 2010 09:37:35 -0800
Subject: [Python-ideas] syntax for set
In-Reply-To: <AANLkTinpF8dBq8CnPAq5+WZFL56U16-7KDnfPFFijddc@mail.gmail.com>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<140d2cec-5f60-4e06-9bb3-298a9603ddcb@email.android.com>
	<AANLkTinEw-N+3=3LRNYnafveM0YTLOwbHK0qQiMTxA-Z@mail.gmail.com>
	<8612CBF2-6A71-4651-8215-F871F78CD79C@masklinn.net>
	<AANLkTimYXbUepB0w=yqXKxWMyC+H_mCTNNARHNhL356J@mail.gmail.com>
	<4CEE54B2.9060905@btinternet.com>
	<AANLkTinpF8dBq8CnPAq5+WZFL56U16-7KDnfPFFijddc@mail.gmail.com>
Message-ID: <AANLkTim=A6QynbW7rvrwke8D6E0W_YfoTOoyCx0Smcpw@mail.gmail.com>

> The idea is flawed. No, really! People would start
> writing integer literals as floats. And the scheme
> doesn't cover floats (there is no standard way of
> writing decimal places in Roman numerals AFAIK).
> Rob Cliffe

3.14 == CCCXIVE-II

(unscientific notation)

I've always thought COBOL did a better job of EIBTI with spelled out
keywords like ADD instead of + etc. This would clearly one up COBOL. (ADD
ONE TO COBOL GIVING PYTHON)

--- Bruce
(via android)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101125/2f1bbe68/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Nov 25 21:07:02 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 26 Nov 2010 09:07:02 +1300
Subject: [Python-ideas] syntax for set
In-Reply-To: <20101125112053.0b75b431@o>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>
	<4CE0A7BD.1070801@mrabarnett.plus.com> <ibrv3a$pk6$1@dough.gmane.org>
	<20101122101641.GA2326@chopin.edu.pl>
	<20101122102341.GA2977@chopin.edu.pl>
	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>
	<icikc0$41n$1@dough.gmane.org>
	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>
	<4CEDCD2F.6010504@canterbury.ac.nz>
	<4CEDD180.1030906@mrabarnett.plus.com> <20101125112053.0b75b431@o>
Message-ID: <4CEEC1E6.9050701@canterbury.ac.nz>

spir wrote:

> PS: google translate: VIJFHONDERD eenenveertig miljard vierhonderd Fifty One
> miljoen negenhonderd eenentachtig duizend Negen honderd twintig ONE
> 
> (I wonder about its casing algorithm ;-) ...and don't want to have to fix it.)

I also wonder about its knowledge of basic vocabulary, since Fifty
and One don't seem to have been translated...

-- 
Greg



From rrr at ronadam.com  Fri Nov 26 04:20:40 2010
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 25 Nov 2010 21:20:40 -0600
Subject: [Python-ideas] syntax for set
In-Reply-To: <4CEEC1E6.9050701@canterbury.ac.nz>
References: <AANLkTinDCBB5JVf+DH=pfNfAh2d7_1RBwoPFGzoUFsFj@mail.gmail.com>	<4CE0A7BD.1070801@mrabarnett.plus.com>
	<ibrv3a$pk6$1@dough.gmane.org>	<20101122101641.GA2326@chopin.edu.pl>	<20101122102341.GA2977@chopin.edu.pl>	<AANLkTikH8-yYv8TDpevVBf=wf=VeNzfZp2GWhLigDnOF@mail.gmail.com>	<icikc0$41n$1@dough.gmane.org>	<03FA3BD2-6119-4E07-AF95-2EAF26698733@masklinn.net>	<4CEDCD2F.6010504@canterbury.ac.nz>	<4CEDD180.1030906@mrabarnett.plus.com>
	<20101125112053.0b75b431@o> <4CEEC1E6.9050701@canterbury.ac.nz>
Message-ID: <4CEF2788.8050408@ronadam.com>



On 11/25/2010 02:07 PM, Greg Ewing wrote:
> spir wrote:
>
>> PS: google translate: VIJFHONDERD eenenveertig miljard vierhonderd Fifty One
>> miljoen negenhonderd eenentachtig duizend Negen honderd twintig ONE
>>
>> (I wonder about its casing algorithm ;-) ...and don't want to have to fix
>> it.)
>
> I also wonder about its knowledge of basic vocabulary, since Fifty
> and One don't seem to have been translated...

Why bother, it's just matter of time before we will be switching to a 
quantum based decimal module with infinite precision for intermediate 
operations.

As for set syntax, I don't think it's needed. -1

Cheers,
    Ron


From kristjan at ccpgames.com  Fri Nov 26 11:18:48 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Fri, 26 Nov 2010 18:18:48 +0800
Subject: [Python-ideas] compile() getting an optmize flag
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE578730AD@exchcn.ccp.ad.local>

When using python to compile to bytecode, it is really inconvenient that the compiler is fixed into using whatever opmiziation level the interpreter was started with.  This makes it impossible to create e.g. .zip libraries of different optimization levels from a single python session.  What is more, a program that creates optimized .zip files cannot itself have active asserts and so on.
Adding the optimize state to the compiler?s state outht not to be too difficult.  We could then:
compile(source, filename, mode[, flags[, dont_inherit[, optimize]]])
with optmize defaulting to the interpreter's current level.

Same with py_compile, and so on.

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101126/a8ba752f/attachment.html>

From mal at egenix.com  Fri Nov 26 13:27:08 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 26 Nov 2010 13:27:08 +0100
Subject: [Python-ideas] compile() getting an optmize flag
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE578730AD@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE578730AD@exchcn.ccp.ad.local>
Message-ID: <4CEFA79C.8050407@egenix.com>

Kristj?n Valur J?nsson wrote:
> When using python to compile to bytecode, it is really inconvenient that the compiler is fixed into using whatever opmiziation level the interpreter was started with.  This makes it impossible to create e.g. .zip libraries of different optimization levels from a single python session.  What is more, a program that creates optimized .zip files cannot itself have active asserts and so on.
> Adding the optimize state to the compiler?s state outht not to be too difficult.  We could then:
> compile(source, filename, mode[, flags[, dont_inherit[, optimize]]])
> with optmize defaulting to the interpreter's current level.
> 
> Same with py_compile, and so on.

+1

We could then remove the tricks in distutils to compile a package
using an external spawned Python process to get around the
limitation.

See e.g. mxTools for an alternative, but not thread-safe approach:

http://www.egenix.com/products/python/mxBase/mxTools/doc/#_Toc199521822

mx.Tools.optimization([level])

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 26 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From flub at devork.be  Fri Nov 26 23:54:09 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Fri, 26 Nov 2010 22:54:09 +0000
Subject: [Python-ideas] compile() getting an optmize flag
In-Reply-To: <4CEFA79C.8050407@egenix.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE578730AD@exchcn.ccp.ad.local>
	<4CEFA79C.8050407@egenix.com>
Message-ID: <AANLkTik-x++eW6Ho9iEybdA1JYOm_ZzvEPjYXYDPSC6d@mail.gmail.com>

2010/11/26 M.-A. Lemburg <mal at egenix.com>:
> Kristj?n Valur J?nsson wrote:
>> When using python to compile to bytecode, it is really inconvenient that the compiler is fixed into using whatever opmiziation level the interpreter was started with. ?This makes it impossible to create e.g. .zip libraries of different optimization levels from a single python session. ?What is more, a program that creates optimized .zip files cannot itself have active asserts and so on.
>> Adding the optimize state to the compiler?s state outht not to be too difficult. ?We could then:
>> compile(source, filename, mode[, flags[, dont_inherit[, optimize]]])
>> with optmize defaulting to the interpreter's current level.
>>
>> Same with py_compile, and so on.
>
> +1
>
> We could then remove the tricks in distutils to compile a package
> using an external spawned Python process to get around the
> limitation.

FWIW I'm +1 as well

It would be nice getting rid of that spawning hack in our company's
build process too.

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From zac256 at gmail.com  Sat Nov 27 03:58:57 2010
From: zac256 at gmail.com (Zac Burns)
Date: Fri, 26 Nov 2010 21:58:57 -0500
Subject: [Python-ideas] Bring back callable()
In-Reply-To: <loom.20101124T115206-756@post.gmane.org>
References: <20101124000104.573a85a9@pitrou.net>
	<loom.20101124T115206-756@post.gmane.org>
Message-ID: <AANLkTinhbJdYs3QAUY16qT04P_8TpuBbnqvFydR_qb5=@mail.gmail.com>

> I actually really prefer `isinstance(x, collections.Callable)`...
>
> If this was accepted, I would at least want it to be called `iscallable`
> instead
> of `callable`, since I sometimes want to use the name `callable` in my
> code.
>
>
> Ram.
>
>
+1 to making a Callable abstract baseclass

-1 to putting it in collections... it's not iterable nor does it hold items.

--
Zachary Burns
(407)590-4814
Aim - Zac256FL
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101126/66f40bc9/attachment.html>

From g.brandl at gmx.net  Sat Nov 27 21:39:05 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 27 Nov 2010 21:39:05 +0100
Subject: [Python-ideas] compile() getting an optmize flag
In-Reply-To: <AANLkTik-x++eW6Ho9iEybdA1JYOm_ZzvEPjYXYDPSC6d@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE578730AD@exchcn.ccp.ad.local>	<4CEFA79C.8050407@egenix.com>
	<AANLkTik-x++eW6Ho9iEybdA1JYOm_ZzvEPjYXYDPSC6d@mail.gmail.com>
Message-ID: <icrq9i$gup$1@dough.gmane.org>

Am 26.11.2010 23:54, schrieb Floris Bruynooghe:
> 2010/11/26 M.-A. Lemburg <mal at egenix.com>:
>> Kristj?n Valur J?nsson wrote:
>>> When using python to compile to bytecode, it is really inconvenient that the compiler is fixed into using whatever opmiziation level the interpreter was started with.  This makes it impossible to create e.g. .zip libraries of different optimization levels from a single python session.  What is more, a program that creates optimized .zip files cannot itself have active asserts and so on.
>>> Adding the optimize state to the compiler?s state outht not to be too difficult.  We could then:
>>> compile(source, filename, mode[, flags[, dont_inherit[, optimize]]])
>>> with optmize defaulting to the interpreter's current level.
>>>
>>> Same with py_compile, and so on.
>>
>> +1
>>
>> We could then remove the tricks in distutils to compile a package
>> using an external spawned Python process to get around the
>> limitation.
> 
> FWIW I'm +1 as well
> 
> It would be nice getting rid of that spawning hack in our company's
> build process too.

I've posted a patch at http://bugs.python.org/issue10553.  Reviews are welcome.

Georg



From scott+python-ideas at scottdial.com  Sun Nov 28 03:31:02 2010
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Sat, 27 Nov 2010 21:31:02 -0500
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <4CEC4F1F.2090007@canterbury.ac.nz>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
Message-ID: <4CF1BEE6.7080605@scottdial.com>

On 11/23/2010 6:32 PM, Greg Ewing wrote:
> Antoine Pitrou wrote:
> 
>> By the way, "Unpickleable" doesn't work since it's ambiguous: you don't
>> know whether it means you can unpickle the thing, or you can't pickle
>> it.
> 
> Nonpickleable?
> 

Ephemeral?

As an added bonus, twisted already uses this terminology, see:
twisted.persisted.styles.Ephemeral.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From solipsis at pitrou.net  Sun Nov 28 10:11:39 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Nov 2010 10:11:39 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
Message-ID: <20101128101139.77ccf7e7@pitrou.net>

On Sat, 27 Nov 2010 21:31:02 -0500
Scott Dial <scott+python-ideas at scottdial.com>
wrote:
> On 11/23/2010 6:32 PM, Greg Ewing wrote:
> > Antoine Pitrou wrote:
> > 
> >> By the way, "Unpickleable" doesn't work since it's ambiguous: you don't
> >> know whether it means you can unpickle the thing, or you can't pickle
> >> it.
> > 
> > Nonpickleable?
> > 
> 
> Ephemeral?
> 
> As an added bonus, twisted already uses this terminology, see:
> twisted.persisted.styles.Ephemeral.

Twisted has a taste for "smart" obscure names (can you guess what
Avatar and Portal are for?), which is not in core Python's tradition.
NonPickleable would be fine IMO.

Regards

Antoine.




From scott+python-ideas at scottdial.com  Sun Nov 28 11:24:02 2010
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Sun, 28 Nov 2010 05:24:02 -0500
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <20101128101139.77ccf7e7@pitrou.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
Message-ID: <4CF22DC2.5030105@scottdial.com>

On 11/28/2010 4:11 AM, Antoine Pitrou wrote:
> On Sat, 27 Nov 2010 21:31:02 -0500
> Scott Dial <scott+python-ideas at scottdial.com>
> wrote:
>> On 11/23/2010 6:32 PM, Greg Ewing wrote:
>>> Nonpickleable?
>>>
>>
>> Ephemeral?
>>
>> As an added bonus, twisted already uses this terminology, see:
>> twisted.persisted.styles.Ephemeral.
> 
> Twisted has a taste for "smart" obscure names (can you guess what
> Avatar and Portal are for?),

"pickle" is an ironic context to complain about "smart" obscure names.
Anyways, yes, I do know what those classes are, but I have used them
before. Analogously, I suspect that is also the only reason why
"Nonpickleable" seems like an "obvious" choice to you.

But, next you are gonna want a "NonMarshalable" and "NonJSONable" and
"NonBananaable" and "NonJellyable" and every other version of
persistence. Or, you pick a name that describes the property that you
really want to describe (e.g., ephemeral).

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From solipsis at pitrou.net  Sun Nov 28 11:31:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Nov 2010 11:31:27 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <4CF22DC2.5030105@scottdial.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net>
	<4CF22DC2.5030105@scottdial.com>
Message-ID: <1290940287.3534.3.camel@localhost.localdomain>

Le dimanche 28 novembre 2010 ? 05:24 -0500, Scott Dial a ?crit :
> On 11/28/2010 4:11 AM, Antoine Pitrou wrote:
> > On Sat, 27 Nov 2010 21:31:02 -0500
> > Scott Dial <scott+python-ideas at scottdial.com>
> > wrote:
> >> On 11/23/2010 6:32 PM, Greg Ewing wrote:
> >>> Nonpickleable?
> >>>
> >>
> >> Ephemeral?
> >>
> >> As an added bonus, twisted already uses this terminology, see:
> >> twisted.persisted.styles.Ephemeral.
> > 
> > Twisted has a taste for "smart" obscure names (can you guess what
> > Avatar and Portal are for?),
> 
> "pickle" is an ironic context to complain about "smart" obscure names.
> Anyways, yes, I do know what those classes are, but I have used them
> before.
> Analogously, I suspect that is also the only reason why
> "Nonpickleable" seems like an "obvious" choice to you.

Nonpickleable (spelling and casing notwithstanding) quite obviously
means "which can't be pickled". I'm not sure what you're arguing about,
or if you're just arguing for the sake of having an argument :)

Regards

Antoine.




From scott+python-ideas at scottdial.com  Sun Nov 28 12:06:05 2010
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Sun, 28 Nov 2010 06:06:05 -0500
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <1290940287.3534.3.camel@localhost.localdomain>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>	<4CF22DC2.5030105@scottdial.com>
	<1290940287.3534.3.camel@localhost.localdomain>
Message-ID: <4CF2379D.5050605@scottdial.com>

On 11/28/2010 5:31 AM, Antoine Pitrou wrote:
> Le dimanche 28 novembre 2010 ? 05:24 -0500, Scott Dial a ?crit :
>> On 11/28/2010 4:11 AM, Antoine Pitrou wrote:
>>> On Sat, 27 Nov 2010 21:31:02 -0500
>>> Scott Dial <scott+python-ideas at scottdial.com>
>>> wrote:
>>>> Ephemeral?
>>>>
>>>> As an added bonus, twisted already uses this terminology, see:
>>>> twisted.persisted.styles.Ephemeral.
>>>
>>> Twisted has a taste for "smart" obscure names (can you guess what
>>> Avatar and Portal are for?),
>>
>> "pickle" is an ironic context to complain about "smart" obscure names.
>> Anyways, yes, I do know what those classes are, but I have used them
>> before.
>> Analogously, I suspect that is also the only reason why
>> "Nonpickleable" seems like an "obvious" choice to you.
> 
> Nonpickleable (spelling and casing notwithstanding) quite obviously
> means "which can't be pickled". I'm not sure what you're arguing about,
> or if you're just arguing for the sake of having an argument :)

I don't know why you snipped and ignored the part where I explained why
"Ephemeral" was a better choice:

On 11/28/2010 5:24 AM, Scott Dial wrote:
> But, next you are gonna want a "NonMarshalable" and "NonJSONable" and
> "NonBananaable" and "NonJellyable" and every other version of
> persistence. Or, you pick a name that describes the property that you
> really want to describe (e.g., ephemeral).

The OPs problem happens to be using "pickle", but this is not a problem
exclusive to pickling; there are a bunch of serialization methods in the
stdlib and elsewhere, and his question generalizes to all of them.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From solipsis at pitrou.net  Sun Nov 28 12:27:39 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Nov 2010 12:27:39 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
	<4CF22DC2.5030105@scottdial.com>
	<1290940287.3534.3.camel@localhost.localdomain>
	<4CF2379D.5050605@scottdial.com>
Message-ID: <20101128122739.3942b728@pitrou.net>

On Sun, 28 Nov 2010 06:06:05 -0500
Scott Dial <scott+python-ideas at scottdial.com>
wrote:
> 
> I don't know why you snipped and ignored the part where I explained why
> "Ephemeral" was a better choice:

Because it is simply mistaken. "Ephemeral" doesn't equate "can't be
pickled".
Ephemeral basically means very short-lived, but this says nothing about
the ability to transport said data over the network or a local pipe or
socket. Moreover, a datum can be short-lived on disk, or long-lived
in RAM. You are confusing serialization with persistence.

> The OPs problem happens to be using "pickle", but this is not a problem
> exclusive to pickling; there are a bunch of serialization methods in the
> stdlib and elsewhere, and his question generalizes to all of them.

It really doesn't. There are lots of things which are pickleable but
can't be serialized with e.g. JSON or XMLRPC.

Regards

Antoine.




From masklinn at masklinn.net  Sun Nov 28 13:40:45 2010
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 28 Nov 2010 13:40:45 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <20101128101139.77ccf7e7@pitrou.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
Message-ID: <9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>

On 2010-11-28, at 10:11 , Antoine Pitrou wrote:
> On Sat, 27 Nov 2010 21:31:02 -0500
> Scott Dial <scott+python-ideas at scottdial.com>
> wrote:
>> On 11/23/2010 6:32 PM, Greg Ewing wrote:
>>> Antoine Pitrou wrote:
>>> 
>>>> By the way, "Unpickleable" doesn't work since it's ambiguous: you don't
>>>> know whether it means you can unpickle the thing, or you can't pickle
>>>> it.
>>> 
>>> Nonpickleable?
>>> 
>> 
>> Ephemeral?
>> 
>> As an added bonus, twisted already uses this terminology, see:
>> twisted.persisted.styles.Ephemeral.
> 
> Twisted has a taste for "smart" obscure names (can you guess what
> Avatar and Portal are for?), which is not in core Python's tradition.
> NonPickleable would be fine IMO.
> 

Why not `pickle.Incompatible`? It's not like the ABC has anything to do with collections, so it would make more sense for it to live in `pickle` (which you've already imported if you're trying to pickle an object) rather than collections.

From g.brandl at gmx.net  Sun Nov 28 15:12:38 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Nov 2010 15:12:38 +0100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>	<20101128101139.77ccf7e7@pitrou.net>
	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>
Message-ID: <icto10$tul$1@dough.gmane.org>

Am 28.11.2010 13:40, schrieb Masklinn:

>> Twisted has a taste for "smart" obscure names (can you guess what Avatar
>> and Portal are for?), which is not in core Python's tradition. 
>> NonPickleable would be fine IMO.
>> 
> 
> Why not `pickle.Incompatible`? It's not like the ABC has anything to do with
> collections, so it would make more sense for it to live in `pickle` (which
> you've already imported if you're trying to pickle an object) rather than
> collections.

pickle.Indigestable?

Georg



From fuzzyman at voidspace.org.uk  Sun Nov 28 17:44:23 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 28 Nov 2010 16:44:23 +0000
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <icto10$tul$1@dough.gmane.org>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>
	<icto10$tul$1@dough.gmane.org>
Message-ID: <AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>

On 28 November 2010 14:12, Georg Brandl <g.brandl at gmx.net> wrote:

> Am 28.11.2010 13:40, schrieb Masklinn:
>
> >> Twisted has a taste for "smart" obscure names (can you guess what Avatar
> >> and Portal are for?), which is not in core Python's tradition.
> >> NonPickleable would be fine IMO.
> >>
> >
> > Why not `pickle.Incompatible`? It's not like the ABC has anything to do
> with
> > collections, so it would make more sense for it to live in `pickle`
> (which
> > you've already imported if you're trying to pickle an object) rather than
> > collections.
>
> pickle.Indigestable?
>

Undigestable surely...

Michael


>
> Georg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101128/8beac85e/attachment.html>

From tjreedy at udel.edu  Sun Nov 28 19:48:39 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 28 Nov 2010 13:48:39 -0500
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>	<20101128101139.77ccf7e7@pitrou.net>	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>	<icto10$tul$1@dough.gmane.org>
	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
Message-ID: <icu863$vdt$1@dough.gmane.org>


>      > Why not `pickle.Incompatible`? It's not like the ABC has anything

>     pickle.Indigestable?
>
>
> Undigestable surely...

I agree with putting whatever in the pickle module.

If there were ever demand for json.Incompatible or whatever, there we 
have it.

-- 
Terry Jan Reedy



From cool-rr at cool-rr.com  Sun Nov 28 23:37:12 2010
From: cool-rr at cool-rr.com (cool-RR)
Date: Mon, 29 Nov 2010 00:37:12 +0200
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
	non-type inputs
Message-ID: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>

`issubclass(1, list)` raises an Exception, complaining that `1` is not a
class. This is wrong in my opinion. It should just return False.

Use case: I have an object which can be either a list, or a string, or a
callable, or a type. And I want to check whether it's a sub-class of some
base class.

So I don't think I should be taking extra precautions before using
`issubclass`: If my object is not a subclass of the given base class, I
should just get `False`.


Ram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101129/bc5e4900/attachment.html>

From fuzzyman at voidspace.org.uk  Sun Nov 28 23:43:51 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 28 Nov 2010 22:43:51 +0000
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
Message-ID: <AANLkTi=arbVqON6NFh+Z-pSzN+WCyFYB6fg=nmv+Gf6k@mail.gmail.com>

On 28 November 2010 22:37, cool-RR <cool-rr at cool-rr.com> wrote:

> `issubclass(1, list)` raises an Exception, complaining that `1` is not a
> class. This is wrong in my opinion. It should just return False.
>
> Use case: I have an object which can be either a list, or a string, or a
> callable, or a type. And I want to check whether it's a sub-class of some
> base class.
>
> So I don't think I should be taking extra precautions before using
> `issubclass`: If my object is not a subclass of the given base class, I
> should just get `False`.
>
>
FWIW (which isn't much I guess) it annoys me that I have to protect calls to
issubclass with if isinstance(obj, type).

It isn't a subclass, so a False would be fine... The advantage of type
checking is earlier failures when you're doing something wrong. The
disadvantage is, well, all the disadvantages of type checking...

All the best,

Michael Foord



>
> Ram.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101128/2dad8541/attachment.html>

From cool-rr at cool-rr.com  Sun Nov 28 23:49:57 2010
From: cool-rr at cool-rr.com (Ram Rachum)
Date: Sun, 28 Nov 2010 22:49:57 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Adding_=60Unpicklable=60_to_the_=60colle?=
	=?utf-8?q?ctions=60_module?=
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>	<20101128101139.77ccf7e7@pitrou.net>	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>	<icto10$tul$1@dough.gmane.org>
	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
	<icu863$vdt$1@dough.gmane.org>
Message-ID: <loom.20101128T234519-229@post.gmane.org>

Terry Reedy <tjreedy at ...> writes:

> 
> 
> >      > Why not `pickle.Incompatible`? It's not like the ABC has anything
> 
> >     pickle.Indigestable?
> >
> >
> > Undigestable surely...
> 
> I agree with putting whatever in the pickle module.
> 
> If there were ever demand for json.Incompatible or whatever, there we 
> have it.
> 

I think that naming it `pickle.Incompatible` would be the best indeed. The 
reason I wanted to avoid having a positive `Pickleable` class is that people 
might think that if something is an instance of it then it's pickleable, which 
is false, since a list is "inherently pickleable" but a list containing a lock 
object is not pickleable.

So I think it will be best to have both a `pickle.Incompatible` and a 
`pickle.Compatible`. The reason to have the negative is to let people inherit 
from it, the reason to have the positive is to make `isinstance` calls more 
natural. (i.e. avoid a double negative.)

What do you think?


Ram.



From fuzzyman at voidspace.org.uk  Mon Nov 29 00:21:25 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 28 Nov 2010 23:21:25 +0000
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
Message-ID: <AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>

On 28 November 2010 22:37, cool-RR <cool-rr at cool-rr.com> wrote:

> `issubclass(1, list)` raises an Exception, complaining that `1` is not a
> class. This is wrong in my opinion. It should just return False.
>
> Use case: I have an object which can be either a list, or a string, or a
> callable, or a type. And I want to check whether it's a sub-class of some
> base class.
>
> So I don't think I should be taking extra precautions before using
> `issubclass`: If my object is not a subclass of the given base class, I
> should just get `False`.
>
>
Unfortunately it would be a backwards incompatible change. Currently
catching the TypeError from issubclass is a way of detecting that an object
*isn't* a type.

Maybe one to chalk up for Python 4...

Michael



>
> Ram.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101128/2a5c81ad/attachment.html>

From cool-rr at cool-rr.com  Mon Nov 29 00:23:47 2010
From: cool-rr at cool-rr.com (cool-RR)
Date: Mon, 29 Nov 2010 01:23:47 +0200
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
Message-ID: <AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>

On Mon, Nov 29, 2010 at 1:21 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>
>
> On 28 November 2010 22:37, cool-RR <cool-rr at cool-rr.com> wrote:
>
>> `issubclass(1, list)` raises an Exception, complaining that `1` is not a
>> class. This is wrong in my opinion. It should just return False.
>>
>> Use case: I have an object which can be either a list, or a string, or a
>> callable, or a type. And I want to check whether it's a sub-class of some
>> base class.
>>
>> So I don't think I should be taking extra precautions before using
>> `issubclass`: If my object is not a subclass of the given base class, I
>> should just get `False`.
>>
>>
> Unfortunately it would be a backwards incompatible change. Currently
> catching the TypeError from issubclass is a way of detecting that an object
> *isn't* a type.
>

Who is doing that?! What's wrong with something like `isinstance(thing,
(type, types.ClassType))`?


Ram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101129/8212a443/attachment.html>

From fuzzyman at voidspace.org.uk  Mon Nov 29 00:26:08 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 28 Nov 2010 23:26:08 +0000
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
	<AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
Message-ID: <AANLkTi=wLb7p_z7HBogEeCJywyo4By4er69p=NAmjWqY@mail.gmail.com>

On 28 November 2010 23:23, cool-RR <cool-rr at cool-rr.com> wrote:

> On Mon, Nov 29, 2010 at 1:21 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:
>
>>
>>
>> On 28 November 2010 22:37, cool-RR <cool-rr at cool-rr.com> wrote:
>>
>>> `issubclass(1, list)` raises an Exception, complaining that `1` is not a
>>> class. This is wrong in my opinion. It should just return False.
>>>
>>> Use case: I have an object which can be either a list, or a string, or a
>>> callable, or a type. And I want to check whether it's a sub-class of some
>>> base class.
>>>
>>> So I don't think I should be taking extra precautions before using
>>> `issubclass`: If my object is not a subclass of the given base class, I
>>> should just get `False`.
>>>
>>>
>> Unfortunately it would be a backwards incompatible change. Currently
>> catching the TypeError from issubclass is a way of detecting that an object
>> *isn't* a type.
>>
>
> Who is doing that?! What's wrong with something like `isinstance(thing,
> (type, types.ClassType))`?
>
>
It doesn't matter who is doing it (or why). If they are writing valid python
code we shouldn't break it for them. That's only my opinion - personally I
always write the guard and would *like* to see the change. I don't think
that's sufficient to break backwards compatibility in this way though.

Michael


>
> Ram.
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101128/4781da3e/attachment.html>

From solipsis at pitrou.net  Mon Nov 29 00:44:18 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 29 Nov 2010 00:44:18 +0100
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
Message-ID: <20101129004418.134d4c16@pitrou.net>

On Mon, 29 Nov 2010 00:37:12 +0200
cool-RR <cool-rr at cool-rr.com> wrote:
> `issubclass(1, list)` raises an Exception, complaining that `1` is not a
> class. This is wrong in my opinion. It should just return False.

It raises an exception for the same reason that
    1 in 'a'
raises an exception. Because there's no way that an int can belong in a
string, or an int can be a subclass of anything; and so passing an int
there is very likely to be a programming error.

Python is dynamically typed, it doesn't mean it is untyped (contrast
with PHP). If you want arbitrary polymorphism, you have to write the
supporting code yourself.

Regards

Antoine.




From cool-rr at cool-rr.com  Mon Nov 29 00:45:33 2010
From: cool-rr at cool-rr.com (cool-RR)
Date: Mon, 29 Nov 2010 01:45:33 +0200
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <AANLkTi=wLb7p_z7HBogEeCJywyo4By4er69p=NAmjWqY@mail.gmail.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
	<AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
	<AANLkTi=wLb7p_z7HBogEeCJywyo4By4er69p=NAmjWqY@mail.gmail.com>
Message-ID: <AANLkTikSNBE+ERQiEWTZGO8Nd9JBX+-jmBj-+CLreadX@mail.gmail.com>

On Mon, Nov 29, 2010 at 1:26 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>
> On 28 November 2010 23:23, cool-RR <cool-rr at cool-rr.com> wrote:
>
>> On Mon, Nov 29, 2010 at 1:21 AM, Michael Foord <fuzzyman at voidspace.org.uk
>> > wrote:
>>
>>>
>>>
>>> On 28 November 2010 22:37, cool-RR <cool-rr at cool-rr.com> wrote:
>>>
>>>> `issubclass(1, list)` raises an Exception, complaining that `1` is not a
>>>> class. This is wrong in my opinion. It should just return False.
>>>>
>>>> Use case: I have an object which can be either a list, or a string, or a
>>>> callable, or a type. And I want to check whether it's a sub-class of some
>>>> base class.
>>>>
>>>> So I don't think I should be taking extra precautions before using
>>>> `issubclass`: If my object is not a subclass of the given base class, I
>>>> should just get `False`.
>>>>
>>>>
>>> Unfortunately it would be a backwards incompatible change. Currently
>>> catching the TypeError from issubclass is a way of detecting that an object
>>> *isn't* a type.
>>>
>>
>> Who is doing that?! What's wrong with something like `isinstance(thing,
>> (type, types.ClassType))`?
>>
>>
> It doesn't matter who is doing it (or why). If they are writing valid
> python code we shouldn't break it for them. That's only my opinion -
> personally I always write the guard and would *like* to see the change. I
> don't think that's sufficient to break backwards compatibility in this way
> though.
>
> Michael
>

I guess it depends on how many people do that. Doing `issubclass` and
excepting `TypeError` is a very convoluted way of checking if something is a
type. I think that the kind of person who would write that code is not the
kind of person who would care to port their program in Python 3.x.

Also, isn't it possible to break backwards compatibility on such things by
deprecating it in one release and changing it in the next?


Ram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101129/ce3c7d54/attachment.html>

From steve at pearwood.info  Mon Nov 29 01:13:00 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 29 Nov 2010 11:13:00 +1100
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
	<AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
Message-ID: <4CF2F00C.4070801@pearwood.info>

cool-RR wrote:

>> Unfortunately it would be a backwards incompatible change. Currently
>> catching the TypeError from issubclass is a way of detecting that an object
>> *isn't* a type.
>>
> 
> Who is doing that?! What's wrong with something like `isinstance(thing,
> (type, types.ClassType))`?

Is that a serious question? Trying something, and catching the exception 
if it fails, is a general Python idiom. In many case, it's often 
considered more "pythonic" to write:

try:
     handle normal case (which might fail)
except SomeException:
     handle exceptional case

than:

if operation will fail:
    handle exceptional case
else:
    handle normal case (which hopefully won't fail)


since at least Python 1.5. It's been long considered "best practice" 
under many circumstances to catch exceptions rather than to try to 
preemptively guess what might go wrong.


-- 
Steven



From guido at python.org  Mon Nov 29 04:01:09 2010
From: guido at python.org (Guido van Rossum)
Date: Sun, 28 Nov 2010 19:01:09 -0800
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <4CF2F00C.4070801@pearwood.info>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
	<AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
	<4CF2F00C.4070801@pearwood.info>
Message-ID: <AANLkTimCYzt02unODZ-JYgm9j4WOGFJ2ZOVr7pWv=+We@mail.gmail.com>

On Sun, Nov 28, 2010 at 4:13 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> cool-RR wrote:
>
>>> Unfortunately it would be a backwards incompatible change. Currently
>>> catching the TypeError from issubclass is a way of detecting that an
>>> object
>>> *isn't* a type.
>>>
>>
>> Who is doing that?! What's wrong with something like `isinstance(thing,
>> (type, types.ClassType))`?
>
> Is that a serious question? Trying something, and catching the exception if
> it fails, is a general Python idiom. In many case, it's often considered
> more "pythonic" to write:
>
> try:
> ? ?handle normal case (which might fail)
> except SomeException:
> ? ?handle exceptional case
>
> than:
>
> if operation will fail:
> ? handle exceptional case
> else:
> ? handle normal case (which hopefully won't fail)
>
>
> since at least Python 1.5. It's been long considered "best practice" under
> many circumstances to catch exceptions rather than to try to preemptively
> guess what might go wrong.

Hm. The try/except version should IMO be used as a last resort only
(similar to using eval() / exec()). It usually reads more clumsily
than the if-based version, and often it is not clear to the reader
what error condition is being tested. One of the most common things I
see in Python code reviews at work is try/except clauses that catch an
overly general exception and/or out a try/except around an overly long
piece of code, masking potential errors and obscuring the meaning of
the code. (Another common one is legitimate try/except clauses that
are nevertheless lacking a comment explaining what error condition is
being caught and why.)

That is not to say that try/except should not be used, but I think the
question of what's wrong with an explicit test should not be cast off
as rhetorical.

As to the original issue, Antoine got it: issubclass(X, Y) does not
make sense if you're not even sure that X is a class.

-- 
--Guido van Rossum (python.org/~guido)


From stephen at xemacs.org  Mon Nov 29 04:47:40 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 29 Nov 2010 12:47:40 +0900
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>
	<icto10$tul$1@dough.gmane.org>
	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
Message-ID: <87oc986bab.fsf@uwakimon.sk.tsukuba.ac.jp>

Michael Foord writes:
 > On 28 November 2010 14:12, Georg Brandl <g.brandl at gmx.net> wrote:
 > 
 > > Am 28.11.2010 13:40, schrieb Masklinn:
 > >
 > > >> Twisted has a taste for "smart" obscure names (can you guess what Avatar
 > > >> and Portal are for?), which is not in core Python's tradition.
 > > >> NonPickleable would be fine IMO.
 > > >>
 > > >
 > > > Why not `pickle.Incompatible`? It's not like the ABC has anything to do
 > > with
 > > > collections, so it would make more sense for it to live in `pickle`
 > > (which
 > > > you've already imported if you're trying to pickle an object) rather than
 > > > collections.
 > >
 > > pickle.Indigestable?
 > >
 > 
 > Undigestable surely...

Only if you're a Consitituational historian.



From guido at python.org  Mon Nov 29 06:10:35 2010
From: guido at python.org (Guido van Rossum)
Date: Sun, 28 Nov 2010 21:10:35 -0800
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <4CF22DC2.5030105@scottdial.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net>
	<4CF22DC2.5030105@scottdial.com>
Message-ID: <AANLkTimJvjUVZBrqQ0O98HexyxS5gSQfnptcydKQVg7H@mail.gmail.com>

On Sun, Nov 28, 2010 at 2:24 AM, Scott Dial
<scott+python-ideas at scottdial.com> wrote:
> On 11/28/2010 4:11 AM, Antoine Pitrou wrote:
>> On Sat, 27 Nov 2010 21:31:02 -0500
>> Scott Dial <scott+python-ideas at scottdial.com>
>> wrote:
>>> On 11/23/2010 6:32 PM, Greg Ewing wrote:
>>>> Nonpickleable?
>>>>
>>>
>>> Ephemeral?
>>>
>>> As an added bonus, twisted already uses this terminology, see:
>>> twisted.persisted.styles.Ephemeral.
>>
>> Twisted has a taste for "smart" obscure names (can you guess what
>> Avatar and Portal are for?),
>
> "pickle" is an ironic context to complain about "smart" obscure names.
> Anyways, yes, I do know what those classes are, but I have used them
> before. Analogously, I suspect that is also the only reason why
> "Nonpickleable" seems like an "obvious" choice to you.
>
> But, next you are gonna want a "NonMarshalable" and "NonJSONable" and
> "NonBananaable" and "NonJellyable" and every other version of
> persistence. Or, you pick a name that describes the property that you
> really want to describe (e.g., ephemeral).

I see your point but I still veto ephemeral -- as a non-native English
speaker with a limited vocabulary it is too obscure for me (isn't it
something to do with astrology? :-).

Antoine is channeling me well today. :-)

-- 
--Guido van Rossum (python.org/~guido)


From kristjan at ccpgames.com  Mon Nov 29 06:12:42 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 29 Nov 2010 13:12:42 +0800
Subject: [Python-ideas] gc callbacks
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5787318A@exchcn.ccp.ad.local>

Hi there.
Some months back I mentioned that I had other stuff in store for GC.
Here is an idea for you.  This particular idea is just a generalization of a system we've used in EVE for years now: garbage collection callbacks.

The need, originally, was to be able to quantify the time spent in garbage collection runs.  Since they occur out of direct control of the application, we needed to have python tell us about it somehow.

We added gc.register_callback(), a function that added a callable to an internal list of functions that get called on two occations:

1)      When a garbage collection run Is about to start

2)      When a garbage collection run has finished.
The cases are distinguished using an integer argument.  The callbacks are invoked from gc with gc inhibited from reentry, so that the callbacks cannot themselves cause another gc run to commence.

What we traditionally use this for is to start and stop a performance timer and other stats.
More recently though, we have found another very important use for this.  When gc finds uncollectable objects, they are put in the gc.garbage list.  This then needs to be handled by the application.  However, there is no particularly good way for the application to do this, except to periodically check this list.

With the above callback, modules that create uncollectable objects, such as classes with __del__ methods, can register their callback.  At the end of a gc run, they can then walk gc.garbage and take appropriate action for those objects it recognizes.

So, what do you think?  This is  a very simple addition to gc, orthogonal to everything and easily implemented.  I also think it is very useful.

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101129/c3aff220/attachment.html>

From guido at python.org  Mon Nov 29 06:43:03 2010
From: guido at python.org (Guido van Rossum)
Date: Sun, 28 Nov 2010 21:43:03 -0800
Subject: [Python-ideas] gc callbacks
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5787318A@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5787318A@exchcn.ccp.ad.local>
Message-ID: <AANLkTi=QJH4X614de8MA7iJ28vc-abbBLm07t3yAoztO@mail.gmail.com>

Sounds great. One could quibble over the details of the callback
signature, but I'll leave the bikeshedding to others. Do you have code
that could be a basis for a patch? Is it encumbered by copyrights?

--Guido

2010/11/28 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> Hi there.
>
> Some months back I mentioned that I had other stuff in store for GC.
>
> Here is an idea for you.? This particular idea is just a generalization of a
> system we?ve used in EVE for years now: garbage collection callbacks.
>
>
>
> The need, originally, was to be able to quantify the time spent in garbage
> collection runs.? Since they occur out of direct control of the application,
> we needed to have python tell us about it somehow.
>
>
>
> We added gc.register_callback(), a function that added a callable to an
> internal list of functions that get called on two occations:
>
> 1)????? When a garbage collection run Is about to start
>
> 2)????? When a garbage collection run has finished.
>
> The cases are distinguished using an integer argument.? The callbacks are
> invoked from gc with gc inhibited from reentry, so that the callbacks cannot
> themselves cause another gc run to commence.
>
>
>
> What we traditionally use this for is to start and stop a performance timer
> and other stats.
>
> More recently though, we have found another very important use for this.
> When gc finds uncollectable objects, they are put in the gc.garbage list.
> This then needs to be handled by the application.? However, there is no
> particularly good way for the application to do this, except to periodically
> check this list.
>
>
>
> With the above callback, modules that create uncollectable objects, such as
> classes with __del__ methods, can register their callback.? At the end of a
> gc run, they can then walk gc.garbage and take appropriate action for those
> objects it recognizes.
>
>
>
> So, what do you think?? This is? a very simple addition to gc, orthogonal to
> everything and easily implemented.? I also think it is very useful.
>
>
>
> K
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
--Guido van Rossum (python.org/~guido)


From kristjan at ccpgames.com  Mon Nov 29 07:01:39 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 29 Nov 2010 14:01:39 +0800
Subject: [Python-ideas] gc callbacks
In-Reply-To: <AANLkTi=QJH4X614de8MA7iJ28vc-abbBLm07t3yAoztO@mail.gmail.com>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5787318A@exchcn.ccp.ad.local>
	<AANLkTi=QJH4X614de8MA7iJ28vc-abbBLm07t3yAoztO@mail.gmail.com>
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE578731A6@exchcn.ccp.ad.local>

I'll submit our current code as a patch to the tracker.  Like all our contributions it comes with no strings attached :)
K

-----Original Message-----
From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum
Sent: Monday, November 29, 2010 13:43
To: Kristj?n Valur J?nsson
Cc: python-ideas at python.org
Subject: Re: [Python-ideas] gc callbacks

Sounds great. One could quibble over the details of the callback signature, but I'll leave the bikeshedding to others. Do you have code that could be a basis for a patch? Is it encumbered by copyrights?

--Guido

2010/11/28 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> Hi there.
>
> Some months back I mentioned that I had other stuff in store for GC.
>
> Here is an idea for you.? This particular idea is just a 
> generalization of a system we've used in EVE for years now: garbage collection callbacks.
>
>
>
> The need, originally, was to be able to quantify the time spent in 
> garbage collection runs.? Since they occur out of direct control of 
> the application, we needed to have python tell us about it somehow.
>
>
>
> We added gc.register_callback(), a function that added a callable to 
> an internal list of functions that get called on two occations:
>
> 1)????? When a garbage collection run Is about to start
>
> 2)????? When a garbage collection run has finished.
>
> The cases are distinguished using an integer argument.? The callbacks 
> are invoked from gc with gc inhibited from reentry, so that the 
> callbacks cannot themselves cause another gc run to commence.
>
>
>
> What we traditionally use this for is to start and stop a performance 
> timer and other stats.
>
> More recently though, we have found another very important use for this.
> When gc finds uncollectable objects, they are put in the gc.garbage list.
> This then needs to be handled by the application.? However, there is 
> no particularly good way for the application to do this, except to 
> periodically check this list.
>
>
>
> With the above callback, modules that create uncollectable objects, 
> such as classes with __del__ methods, can register their callback.? At 
> the end of a gc run, they can then walk gc.garbage and take 
> appropriate action for those objects it recognizes.
>
>
>
> So, what do you think?? This is? a very simple addition to gc, 
> orthogonal to everything and easily implemented.? I also think it is very useful.
>
>
>
> K
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



--
--Guido van Rossum (python.org/~guido)



From kristjan at ccpgames.com  Mon Nov 29 13:33:39 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 29 Nov 2010 20:33:39 +0800
Subject: [Python-ideas] gc callbacks
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE578731A6@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5787318A@exchcn.ccp.ad.local>
	<AANLkTi=QJH4X614de8MA7iJ28vc-abbBLm07t3yAoztO@mail.gmail.com>
	<2E034B571A5CE44E949B9FCC3B6D24EE578731A6@exchcn.ccp.ad.local>
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5795C7C0@exchcn.ccp.ad.local>

FYI:
http://bugs.python.org/issue10576

K

-----Original Message-----
From: python-ideas-bounces+kristjan=ccpgames.com at python.org [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Kristj?n Valur J?nsson
Sent: Monday, November 29, 2010 14:02
To: Guido van Rossum
Cc: python-ideas at python.org
Subject: Re: [Python-ideas] gc callbacks

I'll submit our current code as a patch to the tracker.  Like all our contributions it comes with no strings attached :) K

-----Original Message-----
From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum
Sent: Monday, November 29, 2010 13:43
To: Kristj?n Valur J?nsson
Cc: python-ideas at python.org
Subject: Re: [Python-ideas] gc callbacks

Sounds great. One could quibble over the details of the callback signature, but I'll leave the bikeshedding to others. Do you have code that could be a basis for a patch? Is it encumbered by copyrights?

--Guido

2010/11/28 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> Hi there.
>
> Some months back I mentioned that I had other stuff in store for GC.
>
> Here is an idea for you.? This particular idea is just a 
> generalization of a system we've used in EVE for years now: garbage collection callbacks.
>
>
>
> The need, originally, was to be able to quantify the time spent in 
> garbage collection runs.? Since they occur out of direct control of 
> the application, we needed to have python tell us about it somehow.
>
>
>
> We added gc.register_callback(), a function that added a callable to 
> an internal list of functions that get called on two occations:
>
> 1)????? When a garbage collection run Is about to start
>
> 2)????? When a garbage collection run has finished.
>
> The cases are distinguished using an integer argument.? The callbacks 
> are invoked from gc with gc inhibited from reentry, so that the 
> callbacks cannot themselves cause another gc run to commence.
>
>
>
> What we traditionally use this for is to start and stop a performance 
> timer and other stats.
>
> More recently though, we have found another very important use for this.
> When gc finds uncollectable objects, they are put in the gc.garbage list.
> This then needs to be handled by the application.? However, there is 
> no particularly good way for the application to do this, except to 
> periodically check this list.
>
>
>
> With the above callback, modules that create uncollectable objects, 
> such as classes with __del__ methods, can register their callback.? At 
> the end of a gc run, they can then walk gc.garbage and take 
> appropriate action for those objects it recognizes.
>
>
>
> So, what do you think?? This is? a very simple addition to gc, 
> orthogonal to everything and easily implemented.? I also think it is very useful.
>
>
>
> K
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



--
--Guido van Rossum (python.org/~guido)

_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
http://mail.python.org/mailman/listinfo/python-ideas



From janssen at parc.com  Mon Nov 29 18:35:55 2010
From: janssen at parc.com (Bill Janssen)
Date: Mon, 29 Nov 2010 09:35:55 PST
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
	non-type inputs
In-Reply-To: <4CF2F00C.4070801@pearwood.info>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
	<AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
	<4CF2F00C.4070801@pearwood.info>
Message-ID: <12037.1291052155@parc.com>

Steven D'Aprano <steve at pearwood.info> wrote:

> since at least Python 1.5. It's been long considered "best practice"
> under many circumstances to catch exceptions rather than to try to
> preemptively guess what might go wrong.

You're kidding, right?  *Programming* is about preemptively guessing
what might go wrong :-).

Bill


From tjreedy at udel.edu  Mon Nov 29 20:42:16 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 29 Nov 2010 14:42:16 -0500
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <AANLkTimJvjUVZBrqQ0O98HexyxS5gSQfnptcydKQVg7H@mail.gmail.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>	<4CF22DC2.5030105@scottdial.com>
	<AANLkTimJvjUVZBrqQ0O98HexyxS5gSQfnptcydKQVg7H@mail.gmail.com>
Message-ID: <id0vml$ka3$1@dough.gmane.org>

On 11/29/2010 12:10 AM, Guido van Rossum wrote:

> I see your point but I still veto ephemeral -- as a non-native English
> speaker with a limited vocabulary it is too obscure for me (isn't it
> something to do with astrology? :-).

You are thinking of 'ephemeris': "a tabular statement of the assigned 
places of a celestial body for regular intervals ". Such tables are 
produced and use by astronomers and used by navigators to determine ship 
position at night. And yes, astrologers piggyback their crap on such 
tables also ;-).

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Mon Nov 29 20:49:41 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 29 Nov 2010 14:49:41 -0500
Subject: [Python-ideas] gc callbacks
In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5787318A@exchcn.ccp.ad.local>
References: <2E034B571A5CE44E949B9FCC3B6D24EE5787318A@exchcn.ccp.ad.local>
Message-ID: <id104h$mhp$1@dough.gmane.org>

On 11/29/2010 12:12 AM, Kristj?n Valur J?nsson wrote:
> Hi there.
>
> Some months back I mentioned that I had other stuff in store for GC.
>
> Here is an idea for you.  This particular idea is just a generalization
> of a system we?ve used in EVE for years now: garbage collection callbacks.
>
> The need, originally, was to be able to quantify the time spent in
> garbage collection runs.  Since they occur out of direct control of the
> application, we needed to have python tell us about it somehow.
>
> We added gc.register_callback(), a function that added a callable to an
> internal list of functions that get called on two occations:
>
> 1)When a garbage collection run Is about to start
>
> 2)When a garbage collection run has finished.
>
> The cases are distinguished using an integer argument.  The callbacks
> are invoked from gc with gc inhibited from reentry, so that the
> callbacks cannot themselves cause another gc run to commence.
>
> What we traditionally use this for is to start and stop a performance
> timer and other stats.
>
> More recently though, we have found another very important use for
> this.  When gc finds uncollectable objects, they are put in the
> gc.garbage list.  This then needs to be handled by the application.
> However, there is no particularly good way for the application to do
> this, except to periodically check this list.
>
> With the above callback, modules that create uncollectable objects, such
> as classes with __del__ methods, can register their callback.  At the
> end of a gc run, they can then walk gc.garbage and take appropriate
> action for those objects it recognizes.
>
> So, what do you think?  This is  a very simple addition to gc,
> orthogonal to everything and easily implemented.  I also think it is
> very useful.

I presume the slowdown in the normal case consists of a very quick 'if 
callback list is not empty' check. If others agree and you open an issue 
with a ready-to-go patch (code, doc, test, what's-new), getting in 3.2b1 
might be possible.

-- 
Terry Jan Reedy




From ben+python at benfinney.id.au  Mon Nov 29 22:27:13 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 30 Nov 2010 08:27:13 +1100
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz> <4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net> <4CF22DC2.5030105@scottdial.com>
	<AANLkTimJvjUVZBrqQ0O98HexyxS5gSQfnptcydKQVg7H@mail.gmail.com>
	<id0vml$ka3$1@dough.gmane.org>
Message-ID: <87zksr3jny.fsf@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> On 11/29/2010 12:10 AM, Guido van Rossum wrote:
>
> > I see your point but I still veto ephemeral -- as a non-native
> > English speaker with a limited vocabulary it is too obscure for me
> > (isn't it something to do with astrology? :-).
>
> You are thinking of 'ephemeris':

I interpreted the Dutch smiley as meaning that Guido was aware of that
difference :-)

> "a tabular statement of the assigned places of a celestial body for
> regular intervals ". Such tables are produced and use by astronomers
> and used by navigators to determine ship position at night. And yes,
> astrologers piggyback their crap on such tables also ;-).

To be fair, ephemerides have been produced for many thousands of years,
by people who were simultaneously practicing what we would today
distinguish as both astrology and astronomy. The astrology crap was not
piggybacked, but integral to the process and purpose of the ephemeris
from its beginning, until we finally teased them apart.

-- 
 \       ?One of the most important things you learn from the internet |
  `\   is that there is no ?them? out there. It's just an awful lot of |
_o__)                                            ?us?.? ?Douglas Adams |
Ben Finney



From ncoghlan at gmail.com  Tue Nov 30 02:55:58 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 30 Nov 2010 11:55:58 +1000
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
 non-type inputs
In-Reply-To: <12037.1291052155@parc.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
	<AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
	<4CF2F00C.4070801@pearwood.info> <12037.1291052155@parc.com>
Message-ID: <AANLkTi=mf8HO2ssZLA0wM-HCTfFmj=J7zYFZ8=SRmQf4@mail.gmail.com>

On Tue, Nov 30, 2010 at 3:35 AM, Bill Janssen <janssen at parc.com> wrote:
> Steven D'Aprano <steve at pearwood.info> wrote:
>
>> since at least Python 1.5. It's been long considered "best practice"
>> under many circumstances to catch exceptions rather than to try to
>> preemptively guess what might go wrong.
>
> You're kidding, right? ?*Programming* is about preemptively guessing
> what might go wrong :-).

LBYL (Look Before You Leap) and EAFPI (Easier to Ask Forgiveness than
Permission) is a style argument that is never going to go away though,
since the answer is "it depends".

My personal answer is usually "make the common case fast". That means
defaulting to using a try-except block (with an else clause to prevent
overreaching) since the overhead of a try-except that doesn't need to
catch an exception is close to zero and the exception normally
reflects the uncommon case. However, since raising and catching an
exception is quite expensive, if it is a situation where average
and/or worst-case performance matters more than common case
performance, testing a condition first may start to appear more
attractive if the preemptive check is faster than the exception
handling would be.

One thing to keep in mind, though, is that some pre-emptive checks are
using the moral equivalent of a try/except block under the covers
(with hasattr() being the classic example of this), so the performance
gain in the exceptional case actually isn't all that great. You really
need to profile it to figure out which is faster.

You do need to take care that the try/except isn't covering too much
though, and, depending on how obscure the exception is, potentially
add a comment as to the error condition you're checking for.

As far as the original post's request goes, no. issubclass expects to
be given two types. If you have an insanely polymorphic variable that
can contain a wide variety of objects, only some of which are type
instances, then either use isinstance to check first as you are
already doing, or a try-except block to convert the TypeError to a
False result (and bundle the combined test into a helper function if
you're doing it a lot).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From janssen at parc.com  Tue Nov 30 03:22:04 2010
From: janssen at parc.com (Bill Janssen)
Date: Mon, 29 Nov 2010 18:22:04 PST
Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for
	non-type inputs
In-Reply-To: <AANLkTi=mf8HO2ssZLA0wM-HCTfFmj=J7zYFZ8=SRmQf4@mail.gmail.com>
References: <AANLkTikHQvPzfw1qu8u2raqCLbBmScd0fY8uFWECFxki@mail.gmail.com>
	<AANLkTi=GQ7tYsFQ_8giq6Uf4+4J3ZvmXj-09ptjv6S13@mail.gmail.com>
	<AANLkTin_FvCbSgJWJph_2U+_V_PGGMpGw6bwT5oeac+K@mail.gmail.com>
	<4CF2F00C.4070801@pearwood.info> <12037.1291052155@parc.com>
	<AANLkTi=mf8HO2ssZLA0wM-HCTfFmj=J7zYFZ8=SRmQf4@mail.gmail.com>
Message-ID: <28819.1291083724@parc.com>

Nick Coghlan <ncoghlan at gmail.com> wrote:

> As far as the original post's request goes, no. issubclass expects to
> be given two types. If you have an insanely polymorphic variable that
> can contain a wide variety of objects, only some of which are type
> instances, then either use isinstance to check first as you are
> already doing, or a try-except block to convert the TypeError to a
> False result (and bundle the combined test into a helper function if
> you're doing it a lot).

Right.  Use isinstance instead; that's what it's there for.

But handling exceptions properly really requires thinking about which
might occur, and what to do if they do occur.  If you don't do that
preemptive thinking about exceptionality, you might as well not bother
catching them at all.

Bill


From cmjohnson.mailinglist at gmail.com  Tue Nov 30 05:38:39 2010
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Mon, 29 Nov 2010 18:38:39 -1000
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <87zksr3jny.fsf@benfinney.id.au>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net>
	<4CF22DC2.5030105@scottdial.com>
	<AANLkTimJvjUVZBrqQ0O98HexyxS5gSQfnptcydKQVg7H@mail.gmail.com>
	<id0vml$ka3$1@dough.gmane.org> <87zksr3jny.fsf@benfinney.id.au>
Message-ID: <AANLkTimB-NYqxmi6f0r9fgkok=b14ZqeD8zgb3Qj8pT0@mail.gmail.com>

On Mon, Nov 29, 2010 at 11:27 AM, Ben Finney
<ben+python at benfinney.id.au<ben%2Bpython at benfinney.id.au>
> wrote:


>  > "a tabular statement of the assigned places of a celestial body for
> > regular intervals ". Such tables are produced and use by astronomers
> > and used by navigators to determine ship position at night. And yes,
> > astrologers piggyback their crap on such tables also ;-).
>
>
> by people who were simultaneously practicing what we would today
> distinguish as both astrology and astronomy. The astrology crap was not
> piggybacked, but integral to the process and purpose of the ephemeris
> from its beginning, until we finally teased them apart.
>

"Astrology" and "astronomy" are clearly misnamed.

X-ology = the study of X. What do astronomers do? They study stars.

X+nomy = the laws of X. What do astrologers do? They "show" how the laws of
the stars rule our lives.

Clearly, we need to fire up the time machine and fix this one, or astronomy
will never get the increased userbase it deserves.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101129/53efd197/attachment.html>

From python at mrabarnett.plus.com  Tue Nov 30 05:44:55 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 30 Nov 2010 04:44:55 +0000
Subject: [Python-ideas] Adding `Unpicklable` to the `collections` module
In-Reply-To: <AANLkTimB-NYqxmi6f0r9fgkok=b14ZqeD8zgb3Qj8pT0@mail.gmail.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>	<4CF22DC2.5030105@scottdial.com>	<AANLkTimJvjUVZBrqQ0O98HexyxS5gSQfnptcydKQVg7H@mail.gmail.com>	<id0vml$ka3$1@dough.gmane.org>
	<87zksr3jny.fsf@benfinney.id.au>
	<AANLkTimB-NYqxmi6f0r9fgkok=b14ZqeD8zgb3Qj8pT0@mail.gmail.com>
Message-ID: <4CF48147.90401@mrabarnett.plus.com>

On 30/11/2010 04:38, Carl M. Johnson wrote:
>
>
> On Mon, Nov 29, 2010 at 11:27 AM, Ben Finney <ben+python at benfinney.id.au
> <mailto:ben%2Bpython at benfinney.id.au>> wrote:
>
>      > "a tabular statement of the assigned places of a celestial body for
>      > regular intervals ". Such tables are produced and use by astronomers
>      > and used by navigators to determine ship position at night. And yes,
>      > astrologers piggyback their crap on such tables also ;-).
>
>
>     by people who were simultaneously practicing what we would today
>     distinguish as both astrology and astronomy. The astrology crap was not
>     piggybacked, but integral to the process and purpose of the ephemeris
>     from its beginning, until we finally teased them apart.
>
>
> "Astrology" and "astronomy" are clearly misnamed.
>
> X-ology = the study of X. What do astronomers do? They study stars.
>
> X+nomy = the laws of X. What do astrologers do? They "show" how the laws
> of the stars rule our lives.
>
> Clearly, we need to fire up the time machine and fix this one, or
> astronomy will never get the increased userbase it deserves.
>
There's a theory that it's not possible to travel back in a time
machine to before it was created. Since Guido wasn't even born then,
it's not possible to fix that.


From dimaqq at gmail.com  Tue Nov 30 08:14:14 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Tue, 30 Nov 2010 00:14:14 -0700
Subject: [Python-ideas] In-process interpreters
Message-ID: <AANLkTimRLsXqDb_mLMjSY7E24JGx8Xf49UBARrRxs6NH@mail.gmail.com>

Perhaps you can clarify what exactly you want to do.
I can see at least 2 distict cases

1. Multithreaded web server (or even browser)
* interpreters need separate imports
  at least pure python modules should be loaded and unloaded,
  different versions of same python modules could be used by different
interpreters
  if different versions of C extensions are needed, different dynamic
loading might be needed
  btw, is it still the case that C extensions cannot be unloaded?
* interpreters need separate memory regions
  so that individual interpreters can be killed quickly

2. Long-running process that executes many small independent user
scripts (e.g. phone)
* memory could be shared as long as cross-references are forbidden
  garbage collector hopefully kill circular references after
interpreter is terminated
* long-running process might want to reload a C extension, ouch
* a possible workaround would be execute only one interpreter at a time,
  somehow pickling user script state?


From cool-rr at cool-rr.com  Tue Nov 30 14:14:35 2010
From: cool-rr at cool-rr.com (Ram Rachum)
Date: Tue, 30 Nov 2010 13:14:35 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Adding=60Unpicklable=60_to_the_=60collec?=
	=?utf-8?q?tions=60_module?=
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>	<20101128101139.77ccf7e7@pitrou.net>	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>	<icto10$tul$1@dough.gmane.org>
	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
	<icu863$vdt$1@dough.gmane.org>
	<loom.20101128T234519-229@post.gmane.org>
Message-ID: <loom.20101130T141116-682@post.gmane.org>

Ram Rachum <cool-rr at ...| writes:
| | I agree with putting whatever in the pickle module.
| | 
| | If there were ever demand for json.Incompatible or whatever, there we 
| | have it.
| | 
| 
| I think that naming it `pickle.Incompatible` would be the best indeed. The 
| reason I wanted to avoid having a positive `Pickleable` class is that people 
| might think that if something is an instance of it then it's pickleable, which 
| is false, since a list is "inherently pickleable" but a list containing a lock 
| object is not pickleable.
| 
| So I think it will be best to have both a `pickle.Incompatible` and a 
| `pickle.Compatible`. The reason to have the negative is to let people inherit 
| from it, the reason to have the positive is to make `isinstance` calls more 
| natural. (i.e. avoid a double negative.)
| 
| What do you think?
| 
| Ram.
| 

So... are people in favor of this idea?


Ram.



From dimaqq at gmail.com  Tue Nov 30 17:52:10 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Tue, 30 Nov 2010 09:52:10 -0700
Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module
In-Reply-To: <loom.20101130T141116-682@post.gmane.org>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>
	<icto10$tul$1@dough.gmane.org>
	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
	<icu863$vdt$1@dough.gmane.org>
	<loom.20101128T234519-229@post.gmane.org>
	<loom.20101130T141116-682@post.gmane.org>
Message-ID: <AANLkTi=8QGhuz8Z5xJU5Gtgq2yj4=w=W=NQZRVigPHZH@mail.gmail.com>

Ahem, what happened to duck typing?

To me, the whole isinstance(datum, type) appears no more elegant than
the Java way (class implements this interface, implements that
interface, ad infinitum), anyhow it's not pythonic.
I have similar reservations about metaclass hacks.

On the other hand I am often confronted with similar problem:
  say I have a function foo(s): .... s.lower() .... which was written
to expect a string
  however all this function really cares about is that it's argument
can do .lower()
  what is the pythonic way to test for this?
  how to test that object can do .lower(), .startswith(), .decode(),
but not all methods of str/unicode?

  so far the best I can manage is let it be, don't test anything, just
let it throw an exception.
  does anyone know better?

d.

On 30 November 2010 06:14, Ram Rachum <cool-rr at cool-rr.com> wrote:
> Ram Rachum <cool-rr at ...| writes:
> | | I agree with putting whatever in the pickle module.
> | |
> | | If there were ever demand for json.Incompatible or whatever, there we
> | | have it.
> | |
> |
> | I think that naming it `pickle.Incompatible` would be the best indeed. The
> | reason I wanted to avoid having a positive `Pickleable` class is that people
> | might think that if something is an instance of it then it's pickleable, which
> | is false, since a list is "inherently pickleable" but a list containing a lock
> | object is not pickleable.
> |
> | So I think it will be best to have both a `pickle.Incompatible` and a
> | `pickle.Compatible`. The reason to have the negative is to let people inherit
> | from it, the reason to have the positive is to make `isinstance` calls more
> | natural. (i.e. avoid a double negative.)
> |
> | What do you think?
> |
> | Ram.
> |
>
> So... are people in favor of this idea?
>
>
> Ram.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From dimaqq at gmail.com  Tue Nov 30 18:46:12 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Tue, 30 Nov 2010 10:46:12 -0700
Subject: [Python-ideas] Truly international Python
Message-ID: <AANLkTinyum5q+8WkN0NLKiLT2oaZs7_-4qs71CMyyXtP@mail.gmail.com>

Let's recall Guido's old Computer Programming for Everybody (CP4E) proposal.

Nowadays that Python is established, it's high time to push Python
into education, especially first programming language education. I
think, in the modern world it means pre-school.

Now the larger part of the world's children doesn't learn English
before school, therefore we need to have truly localized Python.

Some might recall a Python derivative demo with unicode variable names
(link anyone?).

I think we ought to go further. For example, consider imaginary
language pig latin:


"""This does that"""  --> """Thiso acto thato"""   # docstrings
__version__ = (1,2,3) --> __versio__ = (1,2,3)     # variable names
import time           --> importo chrono           # standard module names
def foo(): pass       --> defo foo(): passo        # Python keywords
"foo".upper()         --> "foo".uppero()           # standard library
raise Xx("undefined") --> raisio Xx("indifinito")  # errors
#!/usr/bin/python     --> #!/usr/bin/pythono       # executable name
#!/usr/bin/python     --> #!/usero/binaro/pythono  # name and path


Of course there are concerns for many languages:
Each language needs to establish stable translations for keywords,
basic types, standard modules, methods in standard modules, etc.
Some languages don't support word spaces natively
Some languages have different punctuation rules, e.g. comma for decimal point
Some languages use different quotes
RTL languages spell words RTL yet (some/all?) spell numbers LTR
Hopefully none has to recreate 10,000-separator system ;-)

Anyhow, it's not the issue of core Python to support particular
languages, what is needed is:
the concept that this is needed, and
the base where from a particular localization can evolve from

Here, a fun example, how Python might look like in
google-translate-simplified-chinese. Blame google, not me as I know
very little about this language.

"""This does that"""  --> """??"""           # docstrings
__version__ = (1,2,3) --> __??__ = (1,2,3)   # variable names
import time           --> ?? ??             # standard module names
def foo(): pass       --> ? ????? ??     # Python keywords
"foo".upper()         --> ??? ????         # standard library
raise Xx("undefined") --> ???????????? # errors
#!/usr/bin/python     --> #!/usr/bin/?        # executable name
#!/usr/bin/python     --> #!/??/???/?      # name and path

I track this here and will update with the received feedback:

http://pythonic-wisdom.blogspot.com/2010/11/truly-international-python.html


From masklinn at masklinn.net  Tue Nov 30 18:58:11 2010
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 30 Nov 2010 18:58:11 +0100
Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module
In-Reply-To: <AANLkTi=8QGhuz8Z5xJU5Gtgq2yj4=w=W=NQZRVigPHZH@mail.gmail.com>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>
	<icto10$tul$1@dough.gmane.org>
	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
	<icu863$vdt$1@dough.gmane.org>
	<loom.20101128T234519-229@post.gmane.org>
	<loom.20101130T141116-682@post.gmane.org>
	<AANLkTi=8QGhuz8Z5xJU5Gtgq2yj4=w=W=NQZRVigPHZH@mail.gmail.com>
Message-ID: <A371A350-5B0C-40D0-986B-1F5512A82896@masklinn.net>

On 2010-11-30, at 17:52 , Dima Tisnek wrote:
> Ahem, what happened to duck typing?
Nothing.

> To me, the whole isinstance(datum, type) appears no more elegant than
> the Java way (class implements this interface, implements that
> interface, ad infinitum)
You're wrong. In fact, it could hardly be more different than what Java does. Python's ABCs are simply used to name ducks (and maybe one day tools can use them to implement a restricted structural static type system for python, who knows).

For instance, as of Python 2.7 these blocks of code are roughly equivalent (the first two are in fact identical in effect):

    if isinstance(obj, collections.Sized):
        doSomethingWith(len(obj))

    if hasattr(obj, '__len__'):
        doSomethingWith(len(obj))

    try:
        doSomethingWith(len(obj))
    except TypeError:
        pass

They all test for the same thing: that you can call len() on the object. They don't test anything about your type hierarchy or which interfaces you inherit, which would be the only thing doable in Java (barring abuses of reflection). `collections.Sized` is simply a name for "I can call len() on this object".

> anyhow it's not pythonic.
The inclusion of abcs in Python strongly seems to disagree with that statement.

>  how to test that object can do .lower(), .startswith(), .decode(),
> but not all methods of str/unicode?
Depends if you can give a name to that structure. That's what ABCs do when they're used as "instance testers" (they can also be used as mixins which is very nice in that you don't have to reimplement every single method of `set` yourself to create a new and separate set type): they check that the instance provided has the right structure. Not that it inherits from the right  class or implement the right methods. If you check what `isinstance(obj, collections.Iterator)` does for instance, it aliases to this:

    hasattr(obj, "next") and hasattr(obj, "__iter__")

the former looks much more readable than the latter, doesn't it? And they mean the exact same thing. It's not like ABCs remove your ability to EAFP

From dimaqq at gmail.com  Tue Nov 30 19:50:22 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Tue, 30 Nov 2010 11:50:22 -0700
Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module
In-Reply-To: <A371A350-5B0C-40D0-986B-1F5512A82896@masklinn.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>
	<4CEC234D.6020301@netwok.org>
	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>
	<20101123215458.53eed0b0@pitrou.net>
	<4CEC4F1F.2090007@canterbury.ac.nz>
	<4CF1BEE6.7080605@scottdial.com>
	<20101128101139.77ccf7e7@pitrou.net>
	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>
	<icto10$tul$1@dough.gmane.org>
	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>
	<icu863$vdt$1@dough.gmane.org>
	<loom.20101128T234519-229@post.gmane.org>
	<loom.20101130T141116-682@post.gmane.org>
	<AANLkTi=8QGhuz8Z5xJU5Gtgq2yj4=w=W=NQZRVigPHZH@mail.gmail.com>
	<A371A350-5B0C-40D0-986B-1F5512A82896@masklinn.net>
Message-ID: <AANLkTinT9NG5M_yMzHLYQUobfmJTx8XDaJGZm57Z+QEN@mail.gmail.com>

On 30 November 2010 10:58, Masklinn <masklinn at masklinn.net> wrote:
[snip]
> For instance, as of Python 2.7 these blocks of code are roughly equivalent (the first two are in fact identical in effect):
>
> ? ?if isinstance(obj, collections.Sized):
> ? ? ? ?doSomethingWith(len(obj))
>
> ? ?if hasattr(obj, '__len__'):
> ? ? ? ?doSomethingWith(len(obj))
>
> ? ?try:
> ? ? ? ?doSomethingWith(len(obj))
> ? ?except TypeError:
> ? ? ? ?pass
>
> They all test for the same thing: that you can call len() on the object. They don't test anything about your type hierarchy or which interfaces you inherit, which would be the only thing doable in Java (barring abuses of reflection). `collections.Sized` is simply a name for "I can call len() on this object".

I do get this, it just seems a bit overboard to make programmers learn
that Sized == len, Iterator == next+__iter__, and by induction
Stringed == upper,lower,translate,foo, Integral == +,-,*,/ etc.

What's the point to introduce all these new terms for old concepts?

As far as Java connotaions go, it's the -able suffix that makes it
very Java-like, and the concept that a semantic "interface" needs to
be defined and oddly named to get something done.
Surely Python is much more relaxed, yet the naming currently used is
not the best.

What I'm concerned about is that insinstance(anobj, pickle.Compatible)
this doesn't look very Python-like, if not Pythonic.


In the end, I guess, what 'user' is interested in, is not whether a
particular object has attribute "len", but the semantics of this
attribute, that it is callable, it returns something int-like and it
represents some notion like number of elements in something
container-like.

Of course we all need something like that, moreover I'm sure one day
we'll get to the point where objects gain and loose abc's during their
lifetime.


From merwok at netwok.org  Tue Nov 30 20:14:34 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 30 Nov 2010 20:14:34 +0100
Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module
In-Reply-To: <A371A350-5B0C-40D0-986B-1F5512A82896@masklinn.net>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>	<20101128101139.77ccf7e7@pitrou.net>	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>	<icto10$tul$1@dough.gmane.org>	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>	<icu863$vdt$1@dough.gmane.org>	<loom.20101128T234519-229@post.gmane.org>	<loom.20101130T141116-682@post.gmane.org>	<AANLkTi=8QGhuz8Z5xJU5Gtgq2yj4=w=W=NQZRVigPHZH@mail.gmail.com>
	<A371A350-5B0C-40D0-986B-1F5512A82896@masklinn.net>
Message-ID: <4CF54D1A.6060106@netwok.org>

> For instance, as of Python 2.7 these blocks of code are roughly
> equivalent (the first two are in fact identical in effect):
> 
>     if isinstance(obj, collections.Sized):
>         doSomethingWith(len(obj))
> 
>     if hasattr(obj, '__len__'):
>         doSomethingWith(len(obj))
> 
>     try:
>         doSomethingWith(len(obj))
>     except TypeError:
>         pass
> 
> They all test for the same thing: that you can call len() on the object.

Not exactly: magic methods are looked up on the class, not on the
instance.  That?s why isintance+ABCs is the right test to use here.  See
http://docs.python.org/dev/reference/datamodel#special-method-names

(The third block also suffers from an over-catching except clause.  Put
as little code as possible in the try block and use an else clause.)


ABCs are not contradictory to duck typing at all:

>>> import collections
>>> class Demo:
...     def __len__(self):
...         return 42
...
>>> isinstance(Demo(), collections.Sized)
True

Demo is not required to subclass Sized or register.  ABCs are an
optional mechanism that implements duck typing, so to speak.
http://docs.python.org/dev/glossary
http://docs.python.org/whatsnew/2.6#pep-3119-abstract-base-classes
http://www.python.org/dev/peps/pep-3119/#abcs-vs-duck-typing

Regards



From masklinn at masklinn.net  Tue Nov 30 23:08:54 2010
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 30 Nov 2010 23:08:54 +0100
Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module
In-Reply-To: <4CF54D1A.6060106@netwok.org>
References: <AANLkTi=ToD+S=eWybcFaeprrUqEfh_h2R9KN4fsdCtwk@mail.gmail.com>	<4CEC234D.6020301@netwok.org>	<AANLkTimavbeRX+VYqEBNp9qAsvz72GXMfdrNYftazqPe@mail.gmail.com>	<20101123215458.53eed0b0@pitrou.net>	<4CEC4F1F.2090007@canterbury.ac.nz>	<4CF1BEE6.7080605@scottdial.com>	<20101128101139.77ccf7e7@pitrou.net>	<9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net>	<icto10$tul$1@dough.gmane.org>	<AANLkTinUJ+7XLh+gkbVhWVxfmBQpnyT27vni6aja9sd+@mail.gmail.com>	<icu863$vdt$1@dough.gmane.org>	<loom.20101128T234519-229@post.gmane.org>	<loom.20101130T141116-682@post.gmane.org>	<AANLkTi=8QGhuz8Z5xJU5Gtgq2yj4=w=W=NQZRVigPHZH@mail.gmail.com>
	<A371A350-5B0C-40D0-986B-1F5512A82896@masklinn.net>
	<4CF54D1A.6060106@netwok.org>
Message-ID: <5214C323-9A3C-40ED-B018-D4C20F0C44BB@masklinn.net>

On 2010-11-30, at 20:14 , ?ric Araujo wrote:
>> For instance, as of Python 2.7 these blocks of code are roughly
>> equivalent (the first two are in fact identical in effect):
>> 
>>    if isinstance(obj, collections.Sized):
>>        doSomethingWith(len(obj))
>> 
>>    if hasattr(obj, '__len__'):
>>        doSomethingWith(len(obj))
>> 
>>    try:
>>        doSomethingWith(len(obj))
>>    except TypeError:
>>        pass
>> 
>> They all test for the same thing: that you can call len() on the object.
> 
> Not exactly: magic methods are looked up on the class, not on the
> instance.
So?

>  That?s why isintance+ABCs is the right test to use here.  See
> http://docs.python.org/dev/reference/datamodel#special-method-names
Due to Python's lookup rule, looking up most (if not all) of the special method on the instance is going to give the same result.

> (The third block also suffers from an over-catching except clause.
technically.

> ABCs are not contradictory to duck typing at all:
> 
>>>> import collections
>>>> class Demo:
> ...     def __len__(self):
> ...         return 42
> ...
>>>> isinstance(Demo(), collections.Sized)
> True
> 
> Demo is not required to subclass Sized or register.  ABCs are an
> optional mechanism that implements duck typing, so to speak.
> http://docs.python.org/dev/glossary
> http://docs.python.org/whatsnew/2.6#pep-3119-abstract-base-classes
> http://www.python.org/dev/peps/pep-3119/#abcs-vs-duck-typing

May I know why you're replying that to *my* mail?