From mwh at python.net  Wed Jun  1 10:46:36 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 01 Jun 2005 09:46:36 +0100
Subject: [Python-Dev] Closing old bugs
In-Reply-To: <000001c565d7$65bebc20$5807a044@oemcomputer> (Raymond
	Hettinger's message of "Tue, 31 May 2005 07:53:44 -0400")
References: <000001c565d7$65bebc20$5807a044@oemcomputer>
Message-ID: <2md5r6wjkj.fsf@starship.python.net>

"Raymond Hettinger" <raymond.hettinger at verizon.net> writes:

> There should be some greater care exercised in closing old bugs.

Possibly.  OTOH, we have something like 900 open bugs to work on, and
it's not like bug reporters can't re-open a bug report if they think
it's been closed in error (this has happened a few times already, and
it's a good thing, IMHO).

> Marking them "deprecated" and then erasing them is only a good strategy
> if we have no means of reproducing the error or ascertaining what the OP
> was talking about. 
>
> For instance, in www.python.org/sf/640553 , it was possible for a
> reviewer to directly verify whether usr/local local was still being used
> in setup.py.

Well, that one in particular was always in the "is it really a bug?"
category.

> Likewise, www.python.org/sf/728515 should not have been
> closed (Martin's post could have been taken as a clue that the bug was
> valid and simply waiting for some volunteer to submit a patch).
>
> Old age and a missing OP is not sufficient reason to close a bug. 

But if closing a bug is an effective way of kicking things into life
again...

Cheers,
mwh

-- 
  ARTHUR:  Don't ask me how it works or I'll start to whimper.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 11

From gvanrossum at gmail.com  Wed Jun  1 17:16:44 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Wed, 1 Jun 2005 08:16:44 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
Message-ID: <ca471dc2050601081626b6ef19@mail.gmail.com>

I hope that I've got the rewrite of PEP 343 to include generator
extensions right now. I've chosen the 'with' keyword. Please review
here; I think this is ready for review by the unwashed masses. :-)

  http://www.python.org/peps/pep-0343.html

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Wed Jun  1 17:23:49 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Wed, 1 Jun 2005 08:23:49 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050601081626b6ef19@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
Message-ID: <ca471dc2050601082310118eae@mail.gmail.com>

>   http://www.python.org/peps/pep-0343.html

I should add that IMO this obsoletes PEP 288 and PEP 325; I plan to
reject those when PEP 343 is accepted. I've already withdrawn PEP 340.
PEP 342 is separate (but I'll probably present it together with PEP
343).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Jun  1 18:16:56 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 01 Jun 2005 12:16:56 -0400
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050601081626b6ef19@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>

At 08:16 AM 6/1/2005 -0700, Guido van Rossum wrote:
>I hope that I've got the rewrite of PEP 343 to include generator
>extensions right now. I've chosen the 'with' keyword. Please review
>here; I think this is ready for review by the unwashed masses. :-)
>
>   http://www.python.org/peps/pep-0343.html


Looks great.  A few questions/comments:

* What's the rationale for raising TypeError from close()?  Wasn't 
RuntimeError discussed previously for that role?  (and it's also used by 
the with_template example)  OTOH, maybe that means we want a 
ControlFlowError or some such that can be used for both.

* The "opening" example under "Generator Decorator" seems to be missing a 
try/finally block.

* The transaction handler could also be written as:

     @with_template
     def transactional(db):
         db.begin()
         try:
             yield db
         except:
             db.rollback()
         else:
             db.commit()

at least, if I understand it correctly.
  


From gvanrossum at gmail.com  Wed Jun  1 18:50:39 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Wed, 1 Jun 2005 09:50:39 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>
Message-ID: <ca471dc2050601095012b6b93@mail.gmail.com>

[Guido van Rossum]
> >   http://www.python.org/peps/pep-0343.html

[Phillip J. Eby]
> Looks great.  A few questions/comments:
> 
> * What's the rationale for raising TypeError from close()?  Wasn't
> RuntimeError discussed previously for that role?  (and it's also used by
> the with_template example)  OTOH, maybe that means we want a
> ControlFlowError or some such that can be used for both.

I really don't want a new exception for this, since it's just a bug in
the generator's code. I could go with RuntimeError here too, but I
figured TypeError's meaning in a wider sense applies: the generator
doesn't respond appropriately to the throw() call, which is similar to
not handling a particular argument (list) correctly.

> * The "opening" example under "Generator Decorator" seems to be missing a
> try/finally block.

Good catch. I've fixed this in the PEP.

> * The transaction handler could also be written as:
> 
>      @with_template
>      def transactional(db):
>          db.begin()
>          try:
>              yield db
>          except:
>              db.rollback()
>          else:
>              db.commit()
> 
> at least, if I understand it correctly.

Ah, of course. I've updated the PEP.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric.nieuwland at xs4all.nl  Wed Jun  1 20:46:20 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Wed, 1 Jun 2005 20:46:20 +0200
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050601082310118eae@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
Message-ID: <1cb1006397f1deed5a6670201ef95147@xs4all.nl>

Nice going! But ...

Could we extend the 'try' syntax for this instead of introducing 
'with'? If I look at the translation it an augmented 'try'.
	with EXPR as VAR:
		BLOCK1
	except EXCEPTION:
		BLOCK2
could then be translated to
	abc = EXPR
	exc = (None, None, None)
	VAR = abc.__enter__()
	try:
		try:
			BLOCK1
		except EXCEPTION:
			BLOCK2
		except:
			exc = sys.exc_info()
			raise
	finally:
		abc.__exit__(*exc)


Can the 'throw()' method be renamed 'raise()'? IMHO that makes much 
clearer what happens.

Same thing with 'GeneratorExit', 'StopGeneration' more closely matches 
'StopIteration'.

--eric


From eric.nieuwland at xs4all.nl  Wed Jun  1 20:57:47 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Wed, 1 Jun 2005 20:57:47 +0200
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <1cb1006397f1deed5a6670201ef95147@xs4all.nl>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<1cb1006397f1deed5a6670201ef95147@xs4all.nl>
Message-ID: <ec26b5d30b9eac0b7352f3ab35c702dc@xs4all.nl>

Eric Nieuwland wrote:

> If I look at the translation it an augmented 'try'.
> 	with EXPR as VAR:
> 		BLOCK1
> 	except EXCEPTION:
> 		BLOCK2

Oops, that should read:

	try EXPR as VAR:
		BLOCK1
	except EXCEPTION:
		BLOCK2

--eric


From pje at telecommunity.com  Wed Jun  1 21:18:28 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 01 Jun 2005 15:18:28 -0400
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <1cb1006397f1deed5a6670201ef95147@xs4all.nl>
References: <ca471dc2050601082310118eae@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>

At 08:46 PM 6/1/2005 +0200, Eric Nieuwland wrote:
>Nice going! But ...
>
>Could we extend the 'try' syntax for this instead of introducing
>'with'? If I look at the translation it an augmented 'try'.
>         with EXPR as VAR:
>                 BLOCK1
>         except EXCEPTION:
>                 BLOCK2
>could then be translated to

-1, too confusing.


>Can the 'throw()' method be renamed 'raise()'? IMHO that makes much
>clearer what happens.

No, 'raise' is a reserved word.  It would have to be 'raise_()'.  -0.


>Same thing with 'GeneratorExit', 'StopGeneration' more closely matches
>'StopIteration'.

StopIteration is raised the *other* way, so closely matching isn't really a 
benefit.  -1.


From MFC at uk.ibm.com  Wed Jun  1 21:44:56 2005
From: MFC at uk.ibm.com (Mike Cowlishaw)
Date: Wed, 1 Jun 2005 20:44:56 +0100
Subject: [Python-Dev] Adventures with Decimal
In-Reply-To: <429140D5.7010908@canterbury.ac.nz>
Message-ID: <OF2DD54E98.96138825-ON80257013.006BE858-80257013.006C7BDB@uk.ibm.com>

> Raymond Hettinger wrote:
> > IMO, user input (or
> > the full numeric strings in a text data file) is sacred and presumably
> > done for a reason -- the explicitly requested digits should not be
> > throw-away without good reason.
> 
> I still don't understand what's so special about the
> input phase that it should be treated sacredly, while
> happily desecrating the result of any *other* operation.

The 'difference' here is, with unlimited precision decimal 
representations, there is no "input phase".  The decimal number can 
represent the value, sign, and exponent in the character string the user 
provided _exactly_, and indeed it could be implemented using strings as 
the internal representation -- in which case the 'construction' of a new 
number is simply a string copy operation.

There is no operation taking place as there is no narrowing necessary. 
This is quite unlike (for example) converting an ASCII string "1.01" to a 
binary floating-point double which has a fixed precision and no base-5 
component.

mfc

From eric.nieuwland at xs4all.nl  Wed Jun  1 22:00:20 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Wed, 1 Jun 2005 22:00:20 +0200
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
References: <ca471dc2050601082310118eae@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
Message-ID: <f40a60ed315dc986486f178899065709@xs4all.nl>

Phillip J. Eby wrote:
> At 08:46 PM 6/1/2005 +0200, Eric Nieuwland wrote:
>> If I look at the translation it an augmented 'try'.
>>         with EXPR as VAR:
>>                 BLOCK1
>>         except EXCEPTION:
>>                 BLOCK2
>> could then be translated to
>
> -1, too confusing.

A matter of taste, I guess. IMHO 'with' secretly handling exceptions is 
confusing.

>> Can the 'throw()' method be renamed 'raise()'? IMHO that makes much
>> clearer what happens.
>
> No, 'raise' is a reserved word.  It would have to be 'raise_()'.  -0.

My bad. Should have thought about that.

>> Same thing with 'GeneratorExit', 'StopGeneration' more closely matches
>> 'StopIteration'.
>
> StopIteration is raised the *other* way, so closely matching isn't 
> really a benefit.  -1.

Yep! Misread that one.

--eric


From pje at telecommunity.com  Wed Jun  1 22:26:44 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 01 Jun 2005 16:26:44 -0400
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <f40a60ed315dc986486f178899065709@xs4all.nl>
References: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>

At 10:00 PM 6/1/2005 +0200, Eric Nieuwland wrote:
>Phillip J. Eby wrote:
> > At 08:46 PM 6/1/2005 +0200, Eric Nieuwland wrote:
> >> If I look at the translation it an augmented 'try'.
> >>         with EXPR as VAR:
> >>                 BLOCK1
> >>         except EXCEPTION:
> >>                 BLOCK2
> >> could then be translated to
> >
> > -1, too confusing.
>
>A matter of taste, I guess. IMHO 'with' secretly handling exceptions is
>confusing.

It doesn't secretly handle them; it simply gets access to them, which is an 
entirely different thing.

By confusing, I mean that it is not clear from your construct what 
exceptions are caught by the 'except' clause, due to its structural 
layout.  It's also not clear whether the __enter__/__exit__ of EXPR wrap 
BLOCK1 only, or both BLOCK1 and BLOCK2.  These aspects are "confusing" 
because whatever decision you make about the semantics, someone will have 
to *remember* them, as opposed to being unambiguously represented by the 
block structure.

By contrast, if you remove the except: clause from your construct, it is 
clear that BLOCK1 is what is wrapped, and there is no possible confusion 
about who sees what exceptions.  Exceptions inside the block are 
communicated to __exit__, exceptions outside (including those in the 'with' 
statement itself) are not.


From kbk at shore.net  Thu Jun  2 06:40:11 2005
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu, 2 Jun 2005 00:40:11 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200506020440.j524eB3m025074@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  344 open ( +2) /  2845 closed ( +6) /  3189 total ( +8)
Bugs    :  916 open (-20) /  5014 closed (+40) /  5930 total (+20)
RFE     :  191 open ( +2) /   163 closed ( +4) /   354 total ( +6)

New / Reopened Patches
______________________

Optimization for textwrap  (2005-05-26)
       http://python.org/sf/1209527  opened by  Connelly

Build Python2.4.1 on AIX5 using xlc v6  (2005-05-27)
       http://python.org/sf/1209781  opened by  Gangadhar NPK

Split email headers near a space  (2005-05-29)
       http://python.org/sf/1210680  opened by  Noam Raphael

Add st_flags support to (l)stat function  (2005-05-31)
       http://python.org/sf/1212117  opened by  Diego Petten?

mode argument for fileinput class  (2005-05-31)
       http://python.org/sf/1212287  opened by  Reinhold Birkenfeld

Improved profiler  (2005-06-01)
       http://python.org/sf/1212837  opened by  Brett Rosen

option to allow reload to affect existing instances  (2005-06-01)
       http://python.org/sf/1212921  opened by  Brett Rosen

new patch for fixing skipitem() in getargs.c  (2005-06-01)
       http://python.org/sf/1212928  opened by  Reinhold Birkenfeld

note that os.chown can have -1 as an argument  (2005-06-01)
       http://python.org/sf/1213031  opened by  Reinhold Birkenfeld

Patches Closed
______________

updates for the compiler package  (2005-05-21)
       http://python.org/sf/1206077  closed by  sxanth

make float packing copy bytes when they can  (2005-04-12)
       http://python.org/sf/1181301  closed by  mwh

webbrowser.Netscape.open bug fix  (2005-02-20)
       http://python.org/sf/1144816  closed by  birkenfeld

cgitb: make more usable for 'binary-only' software  (2003-06-10)
       http://python.org/sf/751943  closed by  birkenfeld

bug skipping optional keyword arguments of type "w#"  (2004-07-06)
       http://python.org/sf/985713  closed by  birkenfeld

Optional keyword unicode args not handled correctly  (2003-12-04)
       http://python.org/sf/853890  closed by  birkenfeld

New / Reopened Bugs
___________________

urllib2's urlopen() method causes a memory leak  (2005-05-25)
       http://python.org/sf/1208304  opened by  Petr Toman

no CoreGraphics library under Python 2.4  (2005-05-25)
CLOSED http://python.org/sf/1208468  opened by  Jurjen N.E. Bos

longs should be pickled in hexadecimal  (2005-05-26)
CLOSED http://python.org/sf/1209324  opened by  Fredrik Johansson

divmod documentation shd reference // not /  (2005-05-26)
CLOSED http://python.org/sf/1209411  opened by  Alan

os.path.join() fails if 2nd arg is a UNC path  (2005-05-26)
       http://python.org/sf/1209447  opened by  John Ehresman

spurious blank page in dist.pdf  (2005-05-27)
       http://python.org/sf/1209560  opened by  paul rubin

dict.popitem documentation should mention empty dict case  (2005-05-27)
CLOSED http://python.org/sf/1209671  opened by  William Chang

doc bug in Lock.acquire  (2005-05-27)
       http://python.org/sf/1209880  opened by  Chris Perkins

Typo in "Differences from mimelib"  (2005-05-27)
       http://python.org/sf/1210001  opened by  Zumi

comma separated cookie values  (2005-05-28)
       http://python.org/sf/1210326  opened by  tvogt

Cursors not correctly closed after exception.  (2005-05-28)
       http://python.org/sf/1210377  opened by  Ragnar Ouchterlony

An error in Python Tutorial  (2005-05-29)
CLOSED http://python.org/sf/1210832  opened by  Gene

mmap's resize method resizes the file in win32 but not unix  (2003-04-27)
CLOSED http://python.org/sf/728515  reopened by  facundobatista

parser tells invalid syntax with correct code  (2005-05-31)
CLOSED http://python.org/sf/1211639  opened by  ntrunk

itertools.groupby ungraceful, un-Pythonic  (2005-05-31)
CLOSED http://python.org/sf/1212077  opened by  Mike Coleman

str.lower() to have an IMPORTANT NOTE or it's for magicians  (2005-05-31)
       http://python.org/sf/1212195  opened by  Nikos Kouremenos

anydbm and 'n' flag  (2005-05-31)
CLOSED http://python.org/sf/1212223  opened by  Jack Moffitt

Incorrect result for regular expression - "|(hello)|(world)"  (2005-06-01)
       http://python.org/sf/1212411  opened by  Vijay Kumar

Python 2.5 CVS broken for HP-UX platform?  (2005-06-01)
       http://python.org/sf/1212703  opened by  Vincent Jamart

Python segfaults on OpenBSD (tested 3.4 and 3.5)  (2005-06-01)
       http://python.org/sf/1212900  opened by  Fabien Devaux

Bugs Closed
___________

bug in unichr() documentation  (2005-02-11)
       http://python.org/sf/1120777  closed by  fdrake

Line ending documentation is misleading  (2005-03-21)
       http://python.org/sf/1167922  closed by  fdrake

no CoreGraphics library under Python 2.4  (2005-05-25)
       http://python.org/sf/1208468  closed by  mwh

Issue in grammar  (2005-05-24)
       http://python.org/sf/1207501  closed by  mwh

Problem with abs function  (2005-05-16)
       http://python.org/sf/1202946  closed by  rhettinger

longs should be pickled in hexadecimal  (2005-05-26)
       http://python.org/sf/1209324  closed by  rhettinger

divmod documentation shd reference // not /  (2005-05-26)
       http://python.org/sf/1209411  closed by  rhettinger

dict.popitem documentation should mention empty dict case  (2005-05-27)
       http://python.org/sf/1209671  closed by  rhettinger

An error in Python Tutorial  (2005-05-29)
       http://python.org/sf/1210832  closed by  birkenfeld

Windows os.path.isdir bad if drive only  (2002-02-11)
       http://python.org/sf/516232  closed by  facundobatista

test_signal hangs -- signal broken on OpenBSD?  (2002-04-26)
       http://python.org/sf/549081  closed by  facundobatista

popen4 doesn't close filedescriptors when in Threads  (2003-07-09)
       http://python.org/sf/768649  closed by  facundobatista

Sudden death with SIGSEGV in RtlEnterCriticalSection  (2003-06-30)
       http://python.org/sf/763190  closed by  facundobatista

can't CNTRL-C when running os.system in a thread  (2003-06-18)
       http://python.org/sf/756940  closed by  facundobatista

socketmodule.c: inet_pton() expects 4-byte packed_addr  (2003-04-30)
       http://python.org/sf/730222  closed by  facundobatista

mmap's resize method resizes the file in win32 but not unix  (2003-04-27)
       http://python.org/sf/728515  closed by  facundobatista

Core Dumps : Python2.2.2   (2003-04-24)
       http://python.org/sf/727241  closed by  facundobatista

&quot;build_ext&quot; &quot;libraries&quot; subcommand not s  (2003-04-07)
       http://python.org/sf/716634  closed by  facundobatista

Thread running (os.system or popen#)  (2003-03-11)
       http://python.org/sf/701836  closed by  facundobatista

Canvas origin is off-canvas in create_&lt;item&gt;().  Worka  (2003-03-10)
       http://python.org/sf/700650  closed by  facundobatista

Canvas Widget origin is off-screen  (2003-03-08)
       http://python.org/sf/699816  closed by  facundobatista

Profilier hooked into SystemExit  (2003-02-15)
       http://python.org/sf/687297  closed by  facundobatista

String formatting operation Unicode problem.  (2003-01-28)
       http://python.org/sf/676346  closed by  facundobatista

os.popen+() can take string list and bypass shell.  (2003-01-12)
       http://python.org/sf/666700  closed by  facundobatista

doctest and exception messages  (2002-12-16)
       http://python.org/sf/654783  closed by  facundobatista

Misuse of /usr/local/in setup.py  (2002-11-19)
       http://python.org/sf/640553  closed by  facundobatista

parser tells invalid syntax with correct code  (2005-05-31)
       http://python.org/sf/1211639  closed by  doerwalter

urllib has spurious print statement  (2005-05-20)
       http://python.org/sf/1205544  closed by  birkenfeld

Description of string.lstrip() needs improvement  (2005-05-15)
       http://python.org/sf/1202395  closed by  rhettinger

FutureWarning: %u/%o/%x/%X of negative int will return a sig  (2003-06-19)
       http://python.org/sf/757365  closed by  birkenfeld

digraphs on komment lines / xlib  (2003-07-16)
       http://python.org/sf/772176  closed by  birkenfeld

itertools.groupby ungraceful, un-Pythonic  (2005-05-31)
       http://python.org/sf/1212077  closed by  rhettinger

anydbm and 'n' flag  (2005-05-31)
       http://python.org/sf/1212223  closed by  jafo

memory leak in pickle/cPickle [2.2.3]  (2003-07-14)
       http://python.org/sf/770997  closed by  birkenfeld

file.read returns reaches EOF when it shouldn't  (2004-01-11)
       http://python.org/sf/875157  closed by  birkenfeld

Minor bug in urllib docs  (2005-05-03)
       http://python.org/sf/1194249  closed by  akuchling

asyncore.loop() documentation  (2005-04-12)
       http://python.org/sf/1181939  closed by  akuchling

Typo in Curses-Function doc  (2005-02-15)
       http://python.org/sf/1123268  closed by  akuchling

SMTPHandler argument misdescribed  (2005-02-13)
       http://python.org/sf/1121875  closed by  akuchling

incorrect constant names in curses window objects page  (2005-01-19)
       http://python.org/sf/1105706  closed by  akuchling

curses.textpad raises error  (2005-02-27)
       http://python.org/sf/1152762  closed by  akuchling

New / Reopened RFE
__________________

expat binding for XML_ParserReset  (2005-05-25)
       http://python.org/sf/1208730  opened by  John Eikenberry

add single html files  (2005-05-27)
       http://python.org/sf/1209562  opened by  paul rubin

calling it revert is fine with me  (2005-05-27)
CLOSED http://python.org/sf/1209664  opened by  paul rubin

"break" and "continue"-ing out of nested 'for' loops  (2005-05-29)
CLOSED http://python.org/sf/1210975  opened by  Geraint Luff

sets needs an 'arbitrary element' method  (2005-05-31)
       http://python.org/sf/1212091  opened by  Mike Coleman

Prepending Text  (2005-05-31)
       http://python.org/sf/1212169  opened by  Aaron Elbaz

RFE Closed
__________

Clipboard Cleared on Close  (2005-05-24)
       http://python.org/sf/1207592  closed by  kbk

Bottom Scroll Bar  (2005-05-24)
       http://python.org/sf/1207613  closed by  kbk

calling it revert is fine with me  (2005-05-27)
       http://python.org/sf/1209664  closed by  rhettinger

"break" and "continue"-ing out of nested 'for' loops  (2005-05-29)
       http://python.org/sf/1210975  closed by  rhettinger


From facundobatista at gmail.com  Thu Jun  2 06:41:51 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 2 Jun 2005 01:41:51 -0300
Subject: [Python-Dev] Closing old bugs
In-Reply-To: <2md5r6wjkj.fsf@starship.python.net>
References: <000001c565d7$65bebc20$5807a044@oemcomputer>
	<2md5r6wjkj.fsf@starship.python.net>
Message-ID: <e04bdf31050601214137f4e82@mail.gmail.com>

On 6/1/05, Michael Hudson <mwh at python.net> wrote:

> > Old age and a missing OP is not sufficient reason to close a bug.
> 
> But if closing a bug is an effective way of kicking things into life
> again...

I'm seeing this effect in a lot of bugs I closed as old ones. I think
that using the mail-OP-and-commenters property of the SF bug tracking
is a good thing here, and that that mail is enough alert to the
interested people for them to reopen the bug if not closed correctly.

Take note that for closing it, first there's a warning, and if in a
*month* (which really happens to delay into several months, my fault)
the interested people don't take care again of that bug...

However, I'm not opposing myself to a change in our behaviour about
this old bugs. Let's just define a new procedure (if we want to) and
then I'll follow it.

Thanks.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From raymond.hettinger at verizon.net  Thu Jun  2 08:05:13 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 02 Jun 2005 02:05:13 -0400
Subject: [Python-Dev] Closing old bugs
Message-ID: <000001c56739$0afee180$28b69d8d@oemcomputer>

> > Old age and a missing OP is not sufficient reason to close a bug.
> > 
> > But if closing a bug is an effective way of kicking things into life
> > again...
> 
> I'm seeing this effect in a lot of bugs I closed as old ones.

That means they shouldn't have been closed and that we almost lost a valid report.

Also, for the most part, "kicking to life" means getting a qualified reviewer to take time to decide an appropriate course of action.  Typically, the OP is not that person.  Usually, the only "kick to life" we need from an OP is clarification if their post was not sufficiently specific; otherwise, they usually shouldn't have to do anything.



> Take note that for closing it, first there's a warning, and if in a
> *month* (which really happens to delay into several months, my fault)
> the interested people don't take care again of that bug...

A better use of time is to BE one of the interested people and take care of the bug.  Just closing it doesn't make the problem go away.

Also, inactivity does not imply that a bug is not a recurring irritant.  We encourage posters to scan existing bug reports before filing a new one.  Likewise, we immediately close duplicates.  If the original report then disappears without having been cleared, then we've broken our promises to the othesr who did or would have posted a more current report.  The existence of an old report means the problem has been registered and is awaiting a thoughtful response.  Because of the way SF is setup, the other interested people are not likely to receive your "one month warnings".

Closing reports without analyzing their contents is not progress.  AFAICT, that has never been our policy.  Is there anyone else on python-dev who thinks it's a bad idea to automatically close reports without checking whether the issue is real?



Raymond


From gmccaughan at synaptics-uk.com  Thu Jun  2 13:51:54 2005
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu, 2 Jun 2005 12:51:54 +0100
Subject: [Python-Dev] AST manipulation and source code generation
In-Reply-To: <d74hus$3i6$1@sea.gmane.org>
References: <Pine.LNX.4.58.0505240117240.4932@server1.LFW.org>
	<e8bf7a530505260607f00d30f@mail.gmail.com>
	<d74hus$3i6$1@sea.gmane.org>
Message-ID: <200506021251.54661.gmccaughan@synaptics-uk.com>

> ?!ng: Well, i should refine that a bit to say that the Lisp macro system
> is a little more specific.  Whereas AST transformations in Python
> are open-ended (you could generate any result you want), the key
> interesting property of Lisp macros is that they are constrained
> to be "safe", in the sense that the bindings of variable names are
> always preserved.

I'm not quite sure what ?!ng means by "the bindings of
variable names are always preserved", but I conjecture
that he is thinking of the "hygienic macros" in Scheme
rather than the macros in Common Lisp which permit
arbitrary code transformations.

-- 
g


From mcherm at mcherm.com  Thu Jun  2 14:28:36 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu,  2 Jun 2005 05:28:36 -0700
Subject: [Python-Dev] Closing old bugs
Message-ID: <20050602052836.f2cx4lj85ps04wkw@login.werra.lunarpages.com>

Raymond writes:
> Is there anyone else on python-dev who thinks it's a bad idea to automatically
> close reports without checking whether the issue is real?

Raymond:

I'm speaking up with some trepidation here, since I am NOT one of those
who frequently closes bugs. But I think that at least sometimes there
IS an advantage to closing old bugs.

I can envision a world in which there would be only 5-10 open bug reports
at any given time, with enough developer volunteers available to respond
to new reports as they come in. And I realize that we are nowhere near
such a world and we're not going to be getting there anytime soon.

But it is still the case that MOST people are intimidated by the enormous
stack of open bugs. Perhaps "intimidated" is the wrong word... what I want
to convey is that most people have no idea what's important or where to
start -- it's just too big a pile. Perhaps all of the people who would
actually work to close Python bugs are perfectly happy with the existing
system. Or perhaps we scare off some potential volunteers because they
have no idea where to start.

I've seen some systems that solve this problem by allowing users to "vote"
for favorite bugs... then you can tell the "important" bugs because they
are more likely to have lots of votes. As I see it, Facundo is using a
variant of that system. He is asking whether there is *ONE PERSON* out
there who cares enough about a bug to subscribe to it and then to respond
to his inquiry. If there's not even one such person, then he's closing
the bug (but if one such person comes along later, they can re-report it).

Sure, we may be throwing away some useful information by closing a bug
report without proper investigation. But we're not in a situation where
we are short of information... we've got plenty of information (bug
reports), what we lack is developer time. If throwing away information
helps to better focus the developer time we have, then it may be a
useful process!

And someday when nirvana arrives and there are only 5-10 open bugs, we
can send intrepid volunteers digging through the archives to examine
bugs that got closed without proper investigation. I'm not holding my
breath.

-- Michael Chermside


From amk at amk.ca  Thu Jun  2 14:52:05 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 2 Jun 2005 08:52:05 -0400
Subject: [Python-Dev] Vestigial code in threadmodule?
Message-ID: <20050602125205.GA3501@rogue.amk.ca>

Looking at bug #1209880, the following function from threadmodule.c is
referenced.  I think the args==NULL case, which can return None
instead of a Boolean value, can never be reached because
PyArg_ParseTuple() will fail if args==NULL.

Before ripping the args==NULL code out, I wanted to be sure my
analysis is correct; is there some subtlety here I'm missing that
makes args==NULL possible?

--amk

static PyObject *
lock_PyThread_acquire_lock(lockobject *self, PyObject *args)
{
   int i = 1;

   if (!PyArg_ParseTuple(args, "|i:acquire", &i))
      return NULL;

   Py_BEGIN_ALLOW_THREADS
   i = PyThread_acquire_lock(self->lock_lock, i);
   Py_END_ALLOW_THREADS

   if (args == NULL) {
      Py_INCREF(Py_None);
      return Py_None;
   }
   else
      return PyBool_FromLong((long)i);
}

From python at rcn.com  Thu Jun  2 15:01:09 2005
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 2 Jun 2005 09:01:09 -0400
Subject: [Python-Dev] Closing old bugs
In-Reply-To: <20050602052836.f2cx4lj85ps04wkw@login.werra.lunarpages.com>
Message-ID: <000101c56773$2663c5a0$3bb02c81@oemcomputer>

> I've seen some systems that solve this problem by allowing users to
"vote"
> for favorite bugs... then you can tell the "important" bugs because
they
> are more likely to have lots of votes. As I see it, Facundo is using a
> variant of that system. He is asking whether there is *ONE PERSON* out
> there who cares enough about a bug to subscribe to it and then to
respond
> to his inquiry.  If there's not even one such person, then he's
closing
> the bug (but if one such person comes along later, they can re-report
it).

-1  This is both silly and harmful.  It in no way resembles a
professional approach to bug resolution.  It throws away valuable
information based on some vague theory of developer marketing (i.e.
threatening to close a bug will cause a qualified, interested developer
to suddenly have both the time and inclination to address it properly).

If the real goal is to "kick some life" into bug resolution, then do
something that directly fulfills that goal.  Host a bug day.  Make a
newsgroup posting requesting thoughts on your favorite ten bugs.  Write
email to people who you think are capable of addressing the particular
issue in question.  Go recruit some qualified developers.  Or just find
a bug that interests you and fix it.

Closing bugs without reading them is an anti-contribution.  If it were a
best practice, then there would already be a cron job in place to do it
automatically.



Raymond

From reinhold-birkenfeld-nospam at wolke7.net  Thu Jun  2 15:14:36 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Thu, 02 Jun 2005 15:14:36 +0200
Subject: [Python-Dev] Closing old bugs
In-Reply-To: <000101c56773$2663c5a0$3bb02c81@oemcomputer>
References: <20050602052836.f2cx4lj85ps04wkw@login.werra.lunarpages.com>
	<000101c56773$2663c5a0$3bb02c81@oemcomputer>
Message-ID: <d7n0i2$n92$1@sea.gmane.org>

Raymond Hettinger wrote:
>> I've seen some systems that solve this problem by allowing users to
> "vote"
>> for favorite bugs... then you can tell the "important" bugs because
> they
>> are more likely to have lots of votes. As I see it, Facundo is using a
>> variant of that system. He is asking whether there is *ONE PERSON* out
>> there who cares enough about a bug to subscribe to it and then to
> respond
>> to his inquiry.  If there's not even one such person, then he's
> closing
>> the bug (but if one such person comes along later, they can re-report
> it).
> 
> -1  This is both silly and harmful.  It in no way resembles a
> professional approach to bug resolution.  It throws away valuable
> information based on some vague theory of developer marketing (i.e.
> threatening to close a bug will cause a qualified, interested developer
> to suddenly have both the time and inclination to address it properly).

ACK so far.

> If the real goal is to "kick some life" into bug resolution, then do
> something that directly fulfills that goal.  Host a bug day.  Make a
> newsgroup posting requesting thoughts on your favorite ten bugs.  Write
> email to people who you think are capable of addressing the particular
> issue in question.  Go recruit some qualified developers.  Or just find
> a bug that interests you and fix it.

Well, to "fix it" is not so easy for most people. You have to post a patch
or attach it to the bug and then wait for someone to check it in. It's not
sure that this is done anytime soon (no complaint; time is short, I know).
Even fixes that are agreed upon by several people are not always checked in
for a long time.

Reinhold

-- 
Mail address is perfectly valid!


From nnorwitz at gmail.com  Thu Jun  2 15:19:36 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 2 Jun 2005 09:19:36 -0400
Subject: [Python-Dev] Vestigial code in threadmodule?
In-Reply-To: <20050602125205.GA3501@rogue.amk.ca>
References: <20050602125205.GA3501@rogue.amk.ca>
Message-ID: <ee2a432c050602061958b19ba0@mail.gmail.com>

On 6/2/05, A.M. Kuchling <amk at amk.ca> wrote:
> Looking at bug #1209880, the following function from threadmodule.c is
> referenced.  I think the args==NULL case, which can return None
> instead of a Boolean value, can never be reached because
> PyArg_ParseTuple() will fail if args==NULL.
> 
> Before ripping the args==NULL code out, I wanted to be sure my
> analysis is correct; is there some subtlety here I'm missing that
> makes args==NULL possible?

I think the args == NULL code should be ripped out, but there seems to
be a different problem.

If args is NULL to PyArg_ParseTuple() an assertion will be triggered
(or it'll just crash).  See vgetargs1() in Python/getargs.c::138.

args can be NULL if load_args() in Python/ceval.c fails (line 3724). 
The trace starts at line 3551, PyCFunction_Call() will be called with
the NULL args on line 3553.  PyCFunction_Call() will call a PyMethod
that will likely call PyArg_ParseTuple() or something like it.

I think the following patch should fix this (it just adds an if condition).

Does this make sense or am I missing something?

n
--
Index: Python/ceval.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v
retrieving revision 2.422
diff -u -r2.422 ceval.c
--- Python/ceval.c      4 Apr 2005 15:49:02 -0000       2.422
+++ Python/ceval.c      2 Jun 2005 13:16:14 -0000
@@ -3549,9 +3549,13 @@
                else {
                        PyObject *callargs;
                        callargs = load_args(pp_stack, na);
-                       READ_TIMESTAMP(*pintr0);
-                       C_TRACE(x=PyCFunction_Call(func,callargs,NULL));
-                       READ_TIMESTAMP(*pintr1);
+                       if (callargs) {
+                               READ_TIMESTAMP(*pintr0);
+                              
C_TRACE(x=PyCFunction_Call(func,callargs,NULL));+                     
         READ_TIMESTAMP(*pintr1);
+                       }
+                       else
+                               x = NULL;
                        Py_XDECREF(callargs);
                }
        } else {

From adv at langdale.com.au  Thu Jun  2 15:47:57 2005
From: adv at langdale.com.au (Arnold deVos)
Date: Thu, 02 Jun 2005 23:47:57 +1000
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050601095012b6b93@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>	<5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>
	<ca471dc2050601095012b6b93@mail.gmail.com>
Message-ID: <429F0E0D.2020709@langdale.com.au>

Guido van Rossum wrote:
> [Phillip J. Eby]
>>* The transaction handler could also be written as:
>>
>>     @with_template
>>     def transactional(db):
>>         db.begin()
>>         try:
>>             yield db
>>         except:
>>             db.rollback()
>>         else:
>>             db.commit()
>>
>>at least, if I understand it correctly.
> 
> 
> Ah, of course. I've updated the PEP.
> 

This template eats eats the exception, which will cause a RuntimeError
in the proposed Wrapper, I think.  A raise after rollback is needed.

- Arnold

From adv at langdale.com.au  Thu Jun  2 15:47:57 2005
From: adv at langdale.com.au (Arnold deVos)
Date: Thu, 02 Jun 2005 23:47:57 +1000
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050601095012b6b93@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>	<5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>
	<ca471dc2050601095012b6b93@mail.gmail.com>
Message-ID: <429F0E0D.2020709@langdale.com.au>

Guido van Rossum wrote:
> [Phillip J. Eby]
>>* The transaction handler could also be written as:
>>
>>     @with_template
>>     def transactional(db):
>>         db.begin()
>>         try:
>>             yield db
>>         except:
>>             db.rollback()
>>         else:
>>             db.commit()
>>
>>at least, if I understand it correctly.
> 
> 
> Ah, of course. I've updated the PEP.
> 

This template eats eats the exception, which will cause a RuntimeError
in the proposed Wrapper, I think.  A raise after rollback is needed.

- Arnold


From anothermax at gmail.com  Thu Jun  2 15:57:14 2005
From: anothermax at gmail.com (Jeremy Maxfield)
Date: Thu, 2 Jun 2005 15:57:14 +0200
Subject: [Python-Dev] Vestigial code in threadmodule?
In-Reply-To: <20050602125205.GA3501@rogue.amk.ca>
References: <20050602125205.GA3501@rogue.amk.ca>
Message-ID: <93dc9c320506020657580ca956@mail.gmail.com>

If you're digging  into the threadmodule.c could you take a look at
bug #1163563 
(http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&group_id=5470&atid=105470)
I've posted a patch
(http://sourceforge.net/tracker/index.php?
func=detail&aid=1203393&group_id=5470&atid=305470)

Regards,
Max


On 6/2/05, A.M. Kuchling <amk at amk.ca> wrote:
> Looking at bug #1209880, the following function from threadmodule.c is
> referenced.  I think the args==NULL case, which can return None
> instead of a Boolean value, can never be reached because
> PyArg_ParseTuple() will fail if args==NULL.
> 
> Before ripping the args==NULL code out, I wanted to be sure my
> analysis is correct; is there some subtlety here I'm missing that
> makes args==NULL possible?
> 
> --amk
> 
> static PyObject *
> lock_PyThread_acquire_lock(lockobject *self, PyObject *args)
> {
>   int i = 1;
> 
>   if (!PyArg_ParseTuple(args, "|i:acquire", &i))
>      return NULL;
> 
>   Py_BEGIN_ALLOW_THREADS
>   i = PyThread_acquire_lock(self->lock_lock, i);
>   Py_END_ALLOW_THREADS
> 
>   if (args == NULL) {
>      Py_INCREF(Py_None);
>      return Py_None;
>   }
>   else
>      return PyBool_FromLong((long)i);
> }
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com
> 


-- 
flickr: http://www.flickr.com/photos/anothermax/sets

From tim.peters at gmail.com  Thu Jun  2 16:12:16 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 2 Jun 2005 10:12:16 -0400
Subject: [Python-Dev] Vestigial code in threadmodule?
In-Reply-To: <20050602125205.GA3501@rogue.amk.ca>
References: <20050602125205.GA3501@rogue.amk.ca>
Message-ID: <1f7befae0506020712132e4067@mail.gmail.com>

[A.M. Kuchling]
> Looking at bug #1209880, the following function from threadmodule.c is
> referenced.  I think the args==NULL case, which can return None
> instead of a Boolean value, can never be reached because
> PyArg_ParseTuple() will fail if args==NULL.

It would assert-fail in a debug build.  In a release build the most
likely outcome would be a segfault (NULL-pointer dereference in the
expansion of vgetargs1's "PyTuple_Check(args)").

> Before ripping the args==NULL code out, I wanted to be sure my
> analysis is correct; is there some subtlety here I'm missing that
> makes args==NULL possible?

Rip it out; blame me <wink>.

> --amk
>
> static PyObject *
> lock_PyThread_acquire_lock(lockobject *self, PyObject *args)

Noe that this is a file-local function.  The only references are here:

static PyMethodDef lock_methods[] = {
	{"acquire_lock", (PyCFunction)lock_PyThread_acquire_lock, 
	 METH_VARARGS, acquire_doc},
	{"acquire",      (PyCFunction)lock_PyThread_acquire_lock, 
	 METH_VARARGS, acquire_doc},

METH_VARARGS always passes a tuple (possibly empty).  These are very
old functions, so I bet they used to use METH_OLDARGS (implied by
absence at the time) ... yup, METH_VARARGS was introduced here in rev
2.48, and unintentionally changed the return contract of this
function.  So that was a backward incompatibility introduced in Python
2.3a1.  Since nobody complained then or since, I vote to keep "the
new" return contract and fiddle the docs to match it.

From adv at langdale.com.au  Thu Jun  2 16:14:49 2005
From: adv at langdale.com.au (Arnold deVos)
Date: Fri, 03 Jun 2005 00:14:49 +1000
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <429F0E0D.2020709@langdale.com.au>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>	<5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>	<ca471dc2050601095012b6b93@mail.gmail.com>
	<429F0E0D.2020709@langdale.com.au>
Message-ID: <d7n42v$3it$1@sea.gmane.org>

Arnold deVos wrote:
> 
> This template eats eats the exception, which will cause a RuntimeError
> in the proposed Wrapper, I think.  A raise after rollback is needed.

Actually, the Wrapper as written in the PEP does not raise RuntimeError
if the generator catches a block's exception.

Shouldn't the relevant clause in the Wrapper go like this:

try:
    self.gen.throw(type, value, traceback)
except type:
    return
except StopIteration:
    raise RuntimeError("generator caught exception")
else:
    raise RuntimeError("generator didn't stop")

And the transaction template would go like this (re-raising the exception):

@with_template
def transactional(db):
    db.begin()
        try:
            yield None
        except:
            db.rollback()
            raise
        else:
            db.commit()

At least this is what I gleaned from the earlier threads.  It means that
the template does not appear to supress an exception that it cannot
actually supress.

- Arnold


From gvanrossum at gmail.com  Thu Jun  2 16:38:48 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 2 Jun 2005 07:38:48 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <d7n42v$3it$1@sea.gmane.org>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>
	<ca471dc2050601095012b6b93@mail.gmail.com>
	<429F0E0D.2020709@langdale.com.au> <d7n42v$3it$1@sea.gmane.org>
Message-ID: <ca471dc20506020738332e7a7e@mail.gmail.com>

[Arnold deVos, responding to himself]
> > This template eats eats the exception, which will cause a RuntimeError
> > in the proposed Wrapper, I think.  A raise after rollback is needed.

No, the generator returns after rolling back, which causes throw() to
raise StopIteration, which is good enough for the wrapper as written.

> Actually, the Wrapper as written in the PEP does not raise RuntimeError
> if the generator catches a block's exception.
> 
> Shouldn't the relevant clause in the Wrapper go like this:
> 
> try:
>     self.gen.throw(type, value, traceback)
> except type:
>     return
> except StopIteration:
>     raise RuntimeError("generator caught exception")
> else:
>     raise RuntimeError("generator didn't stop")

I considered that, but decided that it should be okay for the
generator to respond to a throw() by returning (thus replacing the
exception thrown by StopIteration) since the call to __exit__() is
contained inside a finally-clause, so that when __exit__() returns
normally, the finally-clause will re-raise the original exception
anyway.

Note that there are currently no other use cases for throw() except
the with_template decorator and the close() method. Both allow the
generator to respond either by letting the exception pass through it
unchanged (after executing finally-clauses if present) or by simply
returning (which will raise StopIteration in the caller of throw()).
Erroneous behaviors are, in both cases, raising some other exception
or *yielding* another value. There may be *other* use cases for
yielding a value, for example, a future (looping) block-statement a la
PEP 343. Raising another exception is always an indication of a bug in
the generator.

> And the transaction template would go like this (re-raising the exception):
> 
> @with_template
> def transactional(db):
>     db.begin()
>         try:
>             yield None
>         except:
>             db.rollback()
>             raise
>         else:
>             db.commit()
> 
> At least this is what I gleaned from the earlier threads.  It means that
> the template does not appear to supress an exception that it cannot
> actually supress.

I disagree (at the -0 to -0.5 level); given that the with_template
decorator allows StopIteration, I think that "appearing to suppress an
exception it doesn't actually suppress" is a minor sin -- the key
point is that the cleanup gets executed and doesn't raise a new
exception or reaches a yield-statement.

I'll summarize this discussion in the PEP.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Thu Jun  2 16:58:46 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 2 Jun 2005 07:58:46 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc20506020738332e7a7e@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>
	<ca471dc2050601095012b6b93@mail.gmail.com>
	<429F0E0D.2020709@langdale.com.au> <d7n42v$3it$1@sea.gmane.org>
	<ca471dc20506020738332e7a7e@mail.gmail.com>
Message-ID: <ca471dc2050602075879c06699@mail.gmail.com>

[me]
> I'll summarize this discussion in the PEP.

I've added this section to the PEP. Is anyone dead set against the
tentative resolutions here?

Open Issues

    Discussion on python-dev has revealed some open issues.  I list
    them here, with my preferred resolution and its motivation.  The
    PEP as currently written reflects this preferred resolution.

    1. What exception should be raised by close() when the generator
       yields another value as a response to the GeneratorExit
       exception?

       I originally chose TypeError because it represents gross
       misbehavior of the generator function, which should be fixed by
       changing the code.  But the with_template decorator class uses
       RuntimeError for similar offenses.  Arguably they should all
       use the same exception.  I'd rather not introduce a new
       exception class just for this purpose, since it's not an
       exception that I want people to catch: I want it to turn into a
       traceback which is seen by the programmer who then fixes the
       code.  So now I believe they should both raise RuntimeError.
       There are some precedents for that: it's raised by the core
       Python code in situations where endless recursion is detected,
       and for uninitialized objects (and for a variety of
       miscellaneous conditions).

    2. Both the generator close() method and the __exit__() method of
       the with_template decorator class catch StopIteration and
       consider it equivalent to re-raising the exception passed to
       throw().  Is allowing StopIteration right here?

       This is so that a generator doing cleanup depending on the
       exception thrown (like the transactional() example below) can
       *catch* the exception thrown if it wants to and doesn't have to
       worry about re-raising it.  I find this more convenient for the
       generator writer.  Against this was brought in that the
       generator *appears* to suppress an exception that it cannot
       suppress: the transactional() example would be more clear
       according to this view if it re-raised the original exception
       after the call to db.rollback().  I personally would find the
       requirement to re-raise the exception an annoyance in a
       generator used as a with-template, since all the code after
       yield is used for is cleanup, and it is invoked from a
       finally-clause (the one implicit in the with-statement) which
       re-raises the original exception anyway.


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Jun  2 17:08:27 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 03 Jun 2005 01:08:27 +1000
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050601081626b6ef19@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
Message-ID: <429F20EB.8070800@gmail.com>

Guido van Rossum wrote:
> I hope that I've got the rewrite of PEP 343 to include generator
> extensions right now. I've chosen the 'with' keyword. Please review
> here; I think this is ready for review by the unwashed masses. :-)
> 
>   http://www.python.org/peps/pep-0343.html
> 

Looks pretty good to me (looking at version 1.19).

One comment is that, in the 'optional extensions' section, where you 
say 'such mistakes are easily diagnosed', you could point to your 
generator wrapper as an example where attempting to reuse the block 
handler raises a RuntimeError on the second attempt.

Also, I'm wondering if it would be useful to have a 'closing' template 
that looked like:

   @with_template
   def closing(obj):
       try:
           yield obj
       finally:
           obj.close()

That can be used to deterministically close anything with a close 
method, be it file, generator, or something else:

   with closing(open("argument.txt")) as contradiction:
      for line in contradiction:
          print line

   with closing(some_gen()) as data:
      for datum in data:
          process(datum)


Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From pje at telecommunity.com  Thu Jun  2 17:51:26 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 02 Jun 2005 11:51:26 -0400
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <429F20EB.8070800@gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com>

At 01:08 AM 6/3/2005 +1000, Nick Coghlan wrote:
>Also, I'm wondering if it would be useful to have a 'closing' template
>that looked like:
>
>    @with_template
>    def closing(obj):
>        try:
>            yield obj
>        finally:
>            obj.close()

+1 if you make it 'if hasattr(obj,"close"): obj.close()' in the finally 
clause, so that it will still work if the type of the object changes.


From gvanrossum at gmail.com  Thu Jun  2 18:20:14 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 2 Jun 2005 09:20:14 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<429F20EB.8070800@gmail.com>
	<5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com>
Message-ID: <ca471dc205060209206ebcc593@mail.gmail.com>

On 6/2/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 01:08 AM 6/3/2005 +1000, Nick Coghlan wrote:
> >Also, I'm wondering if it would be useful to have a 'closing' template
> >that looked like:
> >
> >    @with_template
> >    def closing(obj):
> >        try:
> >            yield obj
> >        finally:
> >            obj.close()
> 
> +1 if you make it 'if hasattr(obj,"close"): obj.close()' in the finally
> clause, so that it will still work if the type of the object changes.

But then you'd get back the bug where it would silently do nothing
when you pass it the wrong thing; wasn't that argument used against an
earlier proposal to skip calling __exit__() when it doesn't exist?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Jun  2 18:58:43 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 02 Jun 2005 12:58:43 -0400
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc205060209206ebcc593@mail.gmail.com>
References: <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<429F20EB.8070800@gmail.com>
	<5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050602122814.02202cc0@mail.telecommunity.com>

At 09:20 AM 6/2/2005 -0700, Guido van Rossum wrote:
>On 6/2/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 01:08 AM 6/3/2005 +1000, Nick Coghlan wrote:
> > >Also, I'm wondering if it would be useful to have a 'closing' template
> > >that looked like:
> > >
> > >    @with_template
> > >    def closing(obj):
> > >        try:
> > >            yield obj
> > >        finally:
> > >            obj.close()
> >
> > +1 if you make it 'if hasattr(obj,"close"): obj.close()' in the finally
> > clause, so that it will still work if the type of the object changes.
>
>But then you'd get back the bug where it would silently do nothing
>when you pass it the wrong thing;

That's not a bug, it's a feature.  If the object doesn't have a 'close()' 
method, clearly it doesn't need to be closed.  If it's the "wrong" object 
for what you're using it for in the body of the 'with' block, it'll show up 
there, so this doesn't hide any errors.

The idea is that "with closing(foo):" is just an assertion that you only 
intend to use 'foo' in the body of that block, and not afterwards, so if 
'foo' is something that needs closing, go ahead and close it.

The specific use case I have in mind is situations where a piece of code 
expects an iterable, but there are callers (or callees) that need to do 
cleanup.  On the other hand some callers (or callees) would like to just 
pass in or return a list, or are passing or returning an object from 
somewhere else that may or may not have a close() method.

Thus, a whole bunch of code would suddenly be strewed with assumptions 
about whether things can be closed or not, which seems like pure 
administrivia.  The important functionality is what the object does 
*before* you close it; that's what the programmer should be able to remain 
focused on.


>wasn't that argument used against an
>earlier proposal to skip calling __exit__() when it doesn't exist?

I thought the resolution of that discussion was that you should use an 
explicit wrapper if you want this behavior -- which is what 'closing()' 
is.  Anyway, if you don't like it, don't put it in.  I just thought it 
would be a good example to include for a wrapper whose behavior is more 
optional in nature.


From python at rcn.com  Thu Jun  2 21:34:17 2005
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 2 Jun 2005 15:34:17 -0400
Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib
	sre_compile.py, 1.57, 1.58
In-Reply-To: <E1DdprX-0000f2-Dp@sc8-pr-cvs1.sourceforge.net>
Message-ID: <002501c567aa$11428300$3bb02c81@oemcomputer>

> Index: sre_compile.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/sre_compile.py,v
> retrieving revision 1.57
> retrieving revision 1.58
> diff -u -d -r1.57 -r1.58
> --- sre_compile.py	28 Feb 2005 19:27:52 -0000	1.57
> +++ sre_compile.py	2 Jun 2005 13:35:52 -0000	1.58
> @@ -167,7 +167,7 @@
>              emit(av-1)
>          elif op is GROUPREF_EXISTS:
>              emit(OPCODES[op])
> -            emit((av[0]-1)*2)
> +            emit(av[0]-1)
>              skipyes = _len(code); emit(0)
>              _compile(code, av[1], flags)
>              if av[2]:


The times two operation also occurs twice in nearby code.  Are those
also incorrect?


Raymond Hettinger

From amk at amk.ca  Thu Jun  2 21:51:40 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 2 Jun 2005 15:51:40 -0400
Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib
	sre_compile.py, 1.57, 1.58
In-Reply-To: <002501c567aa$11428300$3bb02c81@oemcomputer>
References: <E1DdprX-0000f2-Dp@sc8-pr-cvs1.sourceforge.net>
	<002501c567aa$11428300$3bb02c81@oemcomputer>
Message-ID: <20050602195140.GA5899@rogue.amk.ca>

On Thu, Jun 02, 2005 at 03:34:17PM -0400, Raymond Hettinger wrote:
> The times two operation also occurs twice in nearby code.  Are those
> also incorrect?

I believe they're correct.  

EXPN: The regex engine refers to both 'groups', where group #N means
the corresponding group in the pattern, and 'marks', which is a table
of indexes into the string; group #0 -> mark 0 and 1, group #1 -> mark
2 and 3, etc.  The compiler was storing the mark value, but then the
code for the GROUPREF_EXISTS opcode was doubling it, converting group
to mark:

        case SRE_OP_GROUPREF_EXISTS:
            TRACE(("|%p|%p|GROUPREF_EXISTS %d\n", ctx->pattern,
                   ctx->ptr, ctx->pattern[0]));
            /* <GROUPREF_EXISTS> <group> <skip> codeyes <JUMP> codeno ... */
            i = ctx->pattern[0];
            {
                int groupref = i+i;
                if (groupref >= state->lastmark) {
                    ctx->pattern += ctx->pattern[1];
                    break;
                } else {
                ...
             }

The two other uses of *2 are for the MARK opcode, which stores the
current position in the string in the corresponding entry in the
table; it really does want the mark number, so the doubling is
correct.

The 'groupref = i+i' in the above code is a bit odd -- why not write 2*i?  I
thought it might be a typo for i+1, but that wouldn't make sense
either; eventually I decided this was just a stylistic thing.

--am"Only the effbot knows for sure"k




From eric.nieuwland at xs4all.nl  Thu Jun  2 22:04:20 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Thu, 2 Jun 2005 22:04:20 +0200
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
References: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
Message-ID: <9330e57c131b13f7d0327412fbf4926f@xs4all.nl>

Phillip J. Eby wrote:
> At 10:00 PM 6/1/2005 +0200, Eric Nieuwland wrote:
>> Phillip J. Eby wrote:
>> > -1, too confusing.
>>
>> A matter of taste, I guess. IMHO 'with' secretly handling exceptions 
>> is
>> confusing.
>
> It doesn't secretly handle them; it simply gets access to them, which 
> is an entirely different thing.
>
> By confusing, I mean that it is not clear from your construct what 
> exceptions are caught by the 'except' clause, due to its structural 
> layout.  It's also not clear whether the __enter__/__exit__ of EXPR 
> wrap BLOCK1 only, or both BLOCK1 and BLOCK2.  These aspects are 
> "confusing" because whatever decision you make about the semantics, 
> someone will have to *remember* them, as opposed to being 
> unambiguously represented by the block structure.
>
> By contrast, if you remove the except: clause from your construct, it 
> is clear that BLOCK1 is what is wrapped, and there is no possible 
> confusion about who sees what exceptions.  Exceptions inside the block 
> are communicated to __exit__, exceptions outside (including those in 
> the 'with' statement itself) are not.

OK. This forwarding (is that the proper expression here?) of an 
exception to __exit__ is what I meant by 'secretly handling'.

If everybody agrees I'll write
	with EXPR as VAR:
		try:
			BLOCK1
		except EXCEPTION:
			BLOCK2
instead. Seems a waiste to me, though.
I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles 
uncaught exceptions by forwarding it to EXPR's __exit__ method. No 
confusion with me.

--eric


From pje at telecommunity.com  Thu Jun  2 22:12:03 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 02 Jun 2005 16:12:03 -0400
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <9330e57c131b13f7d0327412fbf4926f@xs4all.nl>
References: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com>

At 10:04 PM 6/2/2005 +0200, Eric Nieuwland wrote:
>I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles uncaught 
>exceptions by forwarding it to EXPR's __exit__ method. No confusion with me.

No doubt.  However, it's not obvious what happens to an exception in EXPR; 
surely it can't be passed to EXPR's __exit__ method.  So, is it handled by 
the try, or does it pass out of the block?  Whichever answer you give, 
there is somebody who will think the opposite.  And this is precisely the 
ambiguity I've been talking about.

In contrast, a 'with' unmixed with 'try' is absolutely unambiguous as to 
which except: clauses handle what exceptions where.


From eric.nieuwland at xs4all.nl  Thu Jun  2 22:16:42 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Thu, 2 Jun 2005 22:16:42 +0200
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com>
References: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
	<5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com>
Message-ID: <989928a3906aa6b0258875486e16f4cf@xs4all.nl>


On 2 jun 2005, at 22:12, Phillip J. Eby wrote:

> At 10:04 PM 6/2/2005 +0200, Eric Nieuwland wrote:
>> I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles 
>> uncaught exceptions by forwarding it to EXPR's __exit__ method. No 
>> confusion with me.
>
> No doubt.  However, it's not obvious what happens to an exception in 
> EXPR; surely it can't be passed to EXPR's __exit__ method.  So, is it 
> handled by the try, or does it pass out of the block?  Whichever 
> answer you give, there is somebody who will think the opposite.  And 
> this is precisely the ambiguity I've been talking about.
>
> In contrast, a 'with' unmixed with 'try' is absolutely unambiguous as 
> to which except: clauses handle what exceptions where.

<slap forehead> I never thought of that! Now I see what you mean.
I could only save my idea by stating "the scope of 'try' only starts 
after the ':'", but that seems too artificial.

--eric


From pje at telecommunity.com  Thu Jun  2 22:24:46 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 02 Jun 2005 16:24:46 -0400
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <989928a3906aa6b0258875486e16f4cf@xs4all.nl>
References: <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com>
	<5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<ca471dc2050601081626b6ef19@mail.gmail.com>
	<ca471dc2050601082310118eae@mail.gmail.com>
	<5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com>
	<5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com>
	<5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050602162313.01fd0558@mail.telecommunity.com>

At 10:16 PM 6/2/2005 +0200, Eric Nieuwland wrote:

>On 2 jun 2005, at 22:12, Phillip J. Eby wrote:
>
> > At 10:04 PM 6/2/2005 +0200, Eric Nieuwland wrote:
> >> I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles
> >> uncaught exceptions by forwarding it to EXPR's __exit__ method. No
> >> confusion with me.
> >
> > No doubt.  However, it's not obvious what happens to an exception in
> > EXPR; surely it can't be passed to EXPR's __exit__ method.  So, is it
> > handled by the try, or does it pass out of the block?  Whichever
> > answer you give, there is somebody who will think the opposite.  And
> > this is precisely the ambiguity I've been talking about.
> >
> > In contrast, a 'with' unmixed with 'try' is absolutely unambiguous as
> > to which except: clauses handle what exceptions where.
>
><slap forehead> I never thought of that! Now I see what you mean.
>I could only save my idea by stating "the scope of 'try' only starts
>after the ':'", but that seems too artificial.

That's what I mean: if you have to solve it by decree, it becomes a rule 
that people have to *remember* (or at least read carefully and think about 
it), which goes against the "executable pseudocode" nature.


From tdelaney at avaya.com  Thu Jun  2 22:45:19 2005
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Fri, 3 Jun 2005 06:45:19 +1000
Subject: [Python-Dev] PEP 343 rewrite complete
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE0252057E@au3010avexu1.global.avaya.com>

Phillip J. Eby wrote:

> That's not a bug, it's a feature.  If the object doesn't have a
> 'close()' 
> method, clearly it doesn't need to be closed.  If it's the "wrong"
> object 
> for what you're using it for in the body of the 'with' block, it'll
> show up 
> there, so this doesn't hide any errors.

For those semantics, I think one of the following would be better:

    with local(obj):

    with scoped(obj):

but those semantics apply better to __enter__ and __exit__ anyway.

I think a "closing adaptor" should only work with objects that have a
close() method.

Perhaps:

    with scoped_closable(obj):

Tim Delaney

From gvanrossum at gmail.com  Thu Jun  2 22:58:05 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 2 Jun 2005 13:58:05 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <429F20EB.8070800@gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<429F20EB.8070800@gmail.com>
Message-ID: <ca471dc20506021358196d1598@mail.gmail.com>

[Nick Coghlan]
> Also, I'm wondering if it would be useful to have a 'closing' template
> that looked like:
> 
>    @with_template
>    def closing(obj):
>        try:
>            yield obj
>        finally:
>            obj.close()
> 
> That can be used to deterministically close anything with a close
> method, be it file, generator, or something else:
> 
>    with closing(open("argument.txt")) as contradiction:
>       for line in contradiction:
>           print line
> 
>    with closing(some_gen()) as data:
>       for datum in data:
>           process(datum)

I just realized this has a race condition. The bytecode for the
expression closing(open("...")) must necessarily contain a bytecode
that calls open() followed by another bytecode that calls closing().
If a ^C happens between these two byte codes, the stack contains an
open file object that won't be closed explicitly.

With the original opening() template, this race can be avoided (and I
intend to do so) by implementing opening() in C (as a class with
__enter__ and __exit__ methods), and by making sure that the
interpreter does *not* check for interrupts between the call to
__enter__ and the start of the try-finally-statement in the
translation of the with-statement.

The only way to avoid the race that I can see with the closing()
template would be to disable signals for the duration of the
evaluation of the expression in the with-statement, but I really don't
like that solution at all -- system calls like that can be
excruciatingly expensive compared to bytecode execution. Most
applications don't catch ^C so they don't need this extra protection
-- but the compiler doesn't know that so everybody pays for it. The
solution for avoiding interrupts between the __enter__() call and the
try start doesn't require disabling signals; there can be a single
opcode that calls __enter__ and sets up the try-finally context.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From reinhold-birkenfeld-nospam at wolke7.net  Thu Jun  2 23:14:44 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Thu, 02 Jun 2005 23:14:44 +0200
Subject: [Python-Dev] sys.path in interactive session
Message-ID: <d7nsm7$sem$1@sea.gmane.org>

While looking at bug #779191, I saw that sys.path's first element
is '' in interactive sessions, but the current dir otherwise. Is this
intentional?

Reinhold

-- 
Mail address is perfectly valid!


From gvanrossum at gmail.com  Fri Jun  3 01:50:07 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 2 Jun 2005 16:50:07 -0700
Subject: [Python-Dev] sys.path in interactive session
In-Reply-To: <d7nsm7$sem$1@sea.gmane.org>
References: <d7nsm7$sem$1@sea.gmane.org>
Message-ID: <ca471dc20506021650739cf3c5@mail.gmail.com>

I've always liked it this way; using "" instead of "." means that if
you os.path.join() it with a script name you don't get a spurious "./"
prepended.

I think that the "absolutizing" of sys.path entries is relatively new
(seems to have started in 2.3).

Also note that it's not really the current directory but the directory
containing the script; that is definitely intentional.

On 6/2/05, Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:
> While looking at bug #779191, I saw that sys.path's first element
> is '' in interactive sessions, but the current dir otherwise. Is this
> intentional?
> 
> Reinhold
> 
> --
> Mail address is perfectly valid!
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From bob at redivi.com  Fri Jun  3 02:08:54 2005
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 2 Jun 2005 17:08:54 -0700
Subject: [Python-Dev] sys.path in interactive session
In-Reply-To: <ca471dc20506021650739cf3c5@mail.gmail.com>
References: <d7nsm7$sem$1@sea.gmane.org>
	<ca471dc20506021650739cf3c5@mail.gmail.com>
Message-ID: <5A9AFDC6-6CFD-46A6-9B2D-2C91A59FD00D@redivi.com>

On Jun 2, 2005, at 4:50 PM, Guido van Rossum wrote:

> On 6/2/05, Reinhold Birkenfeld <reinhold-birkenfeld- 
> nospam at wolke7.net> wrote:
>
>> While looking at bug #779191, I saw that sys.path's first element
>> is '' in interactive sessions, but the current dir otherwise. Is this
>> intentional?
>>
>
> I've always liked it this way; using "" instead of "." means that if
> you os.path.join() it with a script name you don't get a spurious "./"
> prepended.
>
> I think that the "absolutizing" of sys.path entries is relatively new
> (seems to have started in 2.3).
>
> Also note that it's not really the current directory but the directory
> containing the script; that is definitely intentional.

The absolutizing of sys.path in site.py is misbehavior anyway.. it  
spits in the face of path hooks, for example <http://mail.python.org/ 
pipermail/python-dev/2005-April/052885.html>.

I still haven't committed that patch, I haven't had a whole lot of  
Python-time lately with co-founding a new company, preparing to move,  
WWDC, etc... but nobody bothered to complain, so if someone wants to  
commit <http://python.org/sf/1174614> before I get a chance, feel free!

-bob


From pje at telecommunity.com  Fri Jun  3 02:48:27 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 02 Jun 2005 20:48:27 -0400
Subject: [Python-Dev] sys.path in interactive session
In-Reply-To: <5A9AFDC6-6CFD-46A6-9B2D-2C91A59FD00D@redivi.com>
References: <ca471dc20506021650739cf3c5@mail.gmail.com>
	<d7nsm7$sem$1@sea.gmane.org>
	<ca471dc20506021650739cf3c5@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050602203346.021fe9e0@mail.telecommunity.com>

At 05:08 PM 6/2/2005 -0700, Bob Ippolito wrote:
>On Jun 2, 2005, at 4:50 PM, Guido van Rossum wrote:
> >
> > I think that the "absolutizing" of sys.path entries is relatively new
> > (seems to have started in 2.3).
> >
> > Also note that it's not really the current directory but the directory
> > containing the script; that is definitely intentional.
>
>The absolutizing of sys.path in site.py is misbehavior anyway..

Your patch doesn't fix it doing this to __file__; probably this should not 
touch __file__ if a module has a __loader__ set, or if the path it's 
absolutizing doesn't exist.

Actually, your patch doesn't fix it doing this to all the sys.path entries, 
either.  It appears that site.py may be quite broken with respect to PEP 
302.  Fixing it is going to be interesting, since it's quite possible that 
you can have stuff on PYTHONPATH for which the sys.path_hooks entries don't 
exist yet.  Thus, you can't guarantee when you absolutize a path entry that 
it is in fact a filename, unless you check for its existence - which in 
principle could be a coincidence.

It may be that we need to define more clearly what is allowed on sys.path, 
such that site.py can tell the difference between filesystem paths, and 
non-filesystem PEP 302 specifiers.  Right now, the only major PEP 302 
implementation that I know of is zipimport, but zipimport uses strings that 
are effectively filesystem paths, so site.py doesn't break them.



From adv at langdale.com.au  Fri Jun  3 02:49:49 2005
From: adv at langdale.com.au (Arnold deVos)
Date: Fri, 03 Jun 2005 10:49:49 +1000
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050602075879c06699@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>	<5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com>	<ca471dc2050601095012b6b93@mail.gmail.com>	<429F0E0D.2020709@langdale.com.au>
	<d7n42v$3it$1@sea.gmane.org>	<ca471dc20506020738332e7a7e@mail.gmail.com>
	<ca471dc2050602075879c06699@mail.gmail.com>
Message-ID: <d7o99g$u1$1@sea.gmane.org>

Guido van Rossum wrote:
> 
>        [...] a generator doing cleanup depending on the
>        exception thrown (like the transactional() example below) can
>        *catch* the exception thrown if it wants to and doesn't have to
>        worry about re-raising it.  I find this more convenient for the
>        generator writer.  Against this was brought in that the
>        generator *appears* to suppress an exception that it cannot
>        suppress: the transactional() example would be more clear
>        according to this view if it re-raised the original exception
>        after the call to db.rollback().  [...]

Of course, the explicit re-raise is only needed in a minority of use 
cases where the exception is caught. Two additional points in favour of 
this:

- refactoring a naked try as a with + template is more direct and 
uniform across all use cases.

- it is upwards compatible if the prohibition on templates suppressing 
exceptions is ever reconsidered. (Flow control macro discussion not 
withstanding.)

- Arnold


From gvanrossum at gmail.com  Fri Jun  3 05:09:02 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 2 Jun 2005 20:09:02 -0700
Subject: [Python-Dev] For review: PEP 343: Anonymous Block Redux and
	Generator Enhancements
Message-ID: <ca471dc205060220096b1dfd41@mail.gmail.com>

After many rounds of discussion on python-dev, I'm inviting public
comments for PEP 343. Rather than posting the entire PEP text here,
I'm inviting everyone to read it on line
(http://www.python.org/peps/pep-0343.html) and then post comments on a
Wiki page I've created for this purpose
(http://wiki.python.org/moin/WithStatement).

I think this is a good one; I hope people agree. Its acceptance will
obsolete about 4 other PEPs! (A sign that it fulfills a need and that
the proposed solution is powerful.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fcano at ono.com  Sat Jun  4 19:49:47 2005
From: fcano at ono.com (Florencio Cano Gabarda)
Date: Sat, 04 Jun 2005 19:49:47 +0200
Subject: [Python-Dev] Summer of Code: Developing complete SSL support for
	Python
Message-ID: <20050604193203.03D9.FCANO@ono.com>

Hello,
This is my first post to the list but I wish to be an active developer
of Python in the future. This project of Summer of Code from Google has
opened my eyes to the open source community. The money has been a great
motivation but  thinking about coding something that will be remembered
in the future and specially USEFUL to the community motivates me very
much too.

After this introduction I want to talk about SSL support in Python. I
have read in the Summer of Code section of Python web that a possible
project is coding a new complete module that implements SSL to Python.
And possibly add this module to the python standard library. Appart from
this the project would include adding functionality to the modules (like
urlib) to support SSL as trasparent as possible.
?What do you think about it?

**I'm actually looking for information in the archives of this list and
web** but I would like to ask the list something:
- Is necessary a complete new SSL module for Python?
- What do you need about SSL ?
- What do you think about incorporing SSL to act trasparently in the
modules that use it in the actuality ?
- Any kind of information, references, would be appreciate.

I would like to do the new SSL module as good as possible. A piece of
art and efficiency if possible and obviusly having in mind all
programming standards.

Thanks very much for any help. 
-- 
Florencio Cano Gabarda <fcano at ono.com>



From aahz at pythoncraft.com  Sat Jun  4 20:10:16 2005
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 4 Jun 2005 11:10:16 -0700
Subject: [Python-Dev] Summer of Code: Developing complete SSL support
	for Python
In-Reply-To: <20050604193203.03D9.FCANO@ono.com>
References: <20050604193203.03D9.FCANO@ono.com>
Message-ID: <20050604181016.GA13484@panix.com>

On Sat, Jun 04, 2005, Florencio Cano Gabarda wrote:
>
> This is my first post to the list but I wish to be an active developer
> of Python in the future. This project of Summer of Code from Google has
> opened my eyes to the open source community. The money has been a great
> motivation but  thinking about coding something that will be remembered
> in the future and specially USEFUL to the community motivates me very
> much too.
> 
> After this introduction I want to talk about SSL support in Python. 

Great!  Given the nature of your project, python-dev probably is the
best place for you to get information, but I also encourage you to join
the Summer of Code mailing list:
http://mail.python.org/mailman/listinfo/summerofcode
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"The only problem with Microsoft is they just have no taste." --Steve Jobs

From shane at hathawaymix.org  Sat Jun  4 20:26:47 2005
From: shane at hathawaymix.org (Shane Hathaway)
Date: Sat, 04 Jun 2005 12:26:47 -0600
Subject: [Python-Dev] Summer of Code: Developing complete SSL support
 for	Python
In-Reply-To: <20050604193203.03D9.FCANO@ono.com>
References: <20050604193203.03D9.FCANO@ono.com>
Message-ID: <42A1F267.6000504@hathawaymix.org>

Florencio Cano Gabarda wrote:
> I would like to do the new SSL module as good as possible. A piece of
> art and efficiency if possible and obviusly having in mind all
> programming standards.

Guido and much of the community would certainly be appreciative of a new
SSL module, especially if you can overcome the problems that plague
M2Crypto.

http://www.artima.com/weblogs/viewpost.jsp?thread=95863

I would say that the criteria for success would be:

1) A module, expected to be included in the standard library, that makes
it easy to create both client and server SSL sockets.

2) No leaks or segfaults.

3) An API that any programmer can use without knowing much about
cryptography.

I want to be able to write code that's as simple as this:

    import socket
    import ssl

    def open_ssl_socket(address):
        base = socket.socket()
        base.connect(address)
        sock = ssl.client(base)
        return sock

    def run_server(port, handler, pki_files):
        keys = ssl.load_keys(pki_files)
        s = socket.socket()
        s.bind(('', port))
        s.listen(5)
        while True:
            base, address = s.accept()
            sock = ssl.server(base, keys)
            handler(sock)
            sock.close()

"pki_filenames" in the example is a list of key files, certificate
files, certificiate signing requests, and perhaps other PKI files.  I
want the ssl module to figure out for itself what each file means, so
that I as a mere human can forget about those details. :-)  However, if
there's any ambiguity in the set of files provided, the SSL module
should throw an exception rather than try to guess the intent.

If you're ambitious, you could also figure out how to make this work
with non-blocking sockets.  I believe Twisted has made progress there.

Shane

From gjc at inescporto.pt  Sun Jun  5 14:05:25 2005
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Sun, 05 Jun 2005 13:05:25 +0100
Subject: [Python-Dev] Summer of Code: Developing complete SSL
	support	for	Python
In-Reply-To: <42A1F267.6000504@hathawaymix.org>
References: <20050604193203.03D9.FCANO@ono.com>
	<42A1F267.6000504@hathawaymix.org>
Message-ID: <1117973125.8137.6.camel@emperor>

On Sat, 2005-06-04 at 12:26 -0600, Shane Hathaway wrote:
> Florencio Cano Gabarda wrote:
> > I would like to do the new SSL module as good as possible. A piece of
> > art and efficiency if possible and obviusly having in mind all
> > programming standards.
> 
> Guido and much of the community would certainly be appreciative of a new
> SSL module, especially if you can overcome the problems that plague
> M2Crypto.
> 
> http://www.artima.com/weblogs/viewpost.jsp?thread=95863
> 
> I would say that the criteria for success would be:
> 
> 1) A module, expected to be included in the standard library, that makes
> it easy to create both client and server SSL sockets.
> 
> 2) No leaks or segfaults.
> 
> 3) An API that any programmer can use without knowing much about
> cryptography.
> 
> I want to be able to write code that's as simple as this:
> 
>     import socket
>     import ssl
> 
>     def open_ssl_socket(address):
>         base = socket.socket()
>         base.connect(address)
>         sock = ssl.client(base)
>         return sock
> 
>     def run_server(port, handler, pki_files):
>         keys = ssl.load_keys(pki_files)
>         s = socket.socket()
>         s.bind(('', port))
>         s.listen(5)
>         while True:
>             base, address = s.accept()
>             sock = ssl.server(base, keys)
>             handler(sock)
>             sock.close()
> 
> "pki_filenames" in the example is a list of key files, certificate
> files, certificiate signing requests, and perhaps other PKI files.  I
> want the ssl module to figure out for itself what each file means, so
> that I as a mere human can forget about those details. :-)  However, if
> there's any ambiguity in the set of files provided, the SSL module
> should throw an exception rather than try to guess the intent.
> 
> If you're ambitious, you could also figure out how to make this work
> with non-blocking sockets.  I believe Twisted has made progress there.

  4. In the socket module documentation:


ssl(
sock[, keyfile, certfile])
        Initiate a SSL connection over the socket sock. keyfile is the
        name of a PEM formatted file that contains your private key.
        certfile is a PEM formatted certificate chain file. On success,
        a new SSLObject is returned.
        
Warning: This does not do any certificate verification!

   I would make it a top priority to enable certificate verification in
ssl sockets.  I don't see the point in doing SSL without certificate
verification.  It's just false security.  Maybe adding a callback asking
the application what to do if certificate validation fails, so that
application writers can show a GUI dialogue or something like that...

  Best regards.

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic


From p.f.moore at gmail.com  Sun Jun  5 18:10:17 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 5 Jun 2005 17:10:17 +0100
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc2050601081626b6ef19@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
Message-ID: <79990c6b050605091051094e4@mail.gmail.com>

On 6/1/05, Guido van Rossum <gvanrossum at gmail.com> wrote:
> I hope that I've got the rewrite of PEP 343 to include generator
> extensions right now. I've chosen the 'with' keyword. Please review
> here; I think this is ready for review by the unwashed masses. :-)
> 
>   http://www.python.org/peps/pep-0343.html

A fairly minor point - with_template read wrongly to me for quite a
while: I read it as something along the lines of "with a template"
(which then doesn't make much sense...) rather than "template for the
with statement".

Unfortunately, I don't have a better naming suggestion (other than
simply "template", which is probably too general to work), so I just
raise this as something you should expect to need to clarify a few
times...

Paul.

From martin at v.loewis.de  Sun Jun  5 18:24:43 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 05 Jun 2005 18:24:43 +0200
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <ca471dc20506021358196d1598@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>	<429F20EB.8070800@gmail.com>
	<ca471dc20506021358196d1598@mail.gmail.com>
Message-ID: <42A3274B.6090407@v.loewis.de>

Guido van Rossum wrote:
>>   @with_template
>>   def closing(obj):
>>       try:
>>           yield obj
>>       finally:
>>           obj.close()
>>
> I just realized this has a race condition. The bytecode for the
> expression closing(open("...")) must necessarily contain a bytecode
> that calls open() followed by another bytecode that calls closing().

This is not a convincing point. That race condition always existed,
e.g. in the traditional

  f = open(filename)
  try:
    process(f)
  finally:
    f.close()

as you could always get an async exception between open returns and
f is assigned. This isn't much of an issue, since CPython would always
release the file immediately as the stack frame is cleared due to
the exception.

I think would should explicitly weaken our guarantees for
"asynchronous" exceptions (of which we currently only have
KeyboardInterrupt). The PEP should point out that an async
exception between the beginning of the with statement and
the assignment to the variable may or may not cause __exit__
to be called (depending on how far you are into __enter__)

Regards,
Martin

From gvanrossum at gmail.com  Sun Jun  5 18:33:28 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun, 5 Jun 2005 09:33:28 -0700
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <42A3274B.6090407@v.loewis.de>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<429F20EB.8070800@gmail.com>
	<ca471dc20506021358196d1598@mail.gmail.com>
	<42A3274B.6090407@v.loewis.de>
Message-ID: <ca471dc2050605093325f15a59@mail.gmail.com>

On 6/5/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Guido van Rossum wrote:
> >>   @with_template
> >>   def closing(obj):
> >>       try:
> >>           yield obj
> >>       finally:
> >>           obj.close()
> >>
> > I just realized this has a race condition. The bytecode for the
> > expression closing(open("...")) must necessarily contain a bytecode
> > that calls open() followed by another bytecode that calls closing().
> 
> This is not a convincing point. That race condition always existed,
> e.g. in the traditional
> 
>   f = open(filename)
>   try:
>     process(f)
>   finally:
>     f.close()
> 
> as you could always get an async exception between open returns and
> f is assigned. This isn't much of an issue, since CPython would always
> release the file immediately as the stack frame is cleared due to
> the exception.
> 
> I think would should explicitly weaken our guarantees for
> "asynchronous" exceptions (of which we currently only have
> KeyboardInterrupt). The PEP should point out that an async
> exception between the beginning of the with statement and
> the assignment to the variable may or may not cause __exit__
> to be called (depending on how far you are into __enter__)

That is pretty clear from the translation given in the PEP.

What is not so clear is that the cace can be completely avoided *if*
the call to __enter__ and the try-finally setup are combined in one
opcode, *and* __enter__ is implemented in C, *and* the reversible
actions are all done by __enter__.

This is also why I don't like giving file objects __enter__ and
__exit__ methods.

I know all this doesn't matter in most situations, but sometimes it
does, and it would be good if there was a solution -- today, there
really isn't one except to rely on CPython's reference counting.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From noamraph at gmail.com  Mon Jun  6 00:30:06 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Mon, 6 Jun 2005 00:30:06 +0200
Subject: [Python-Dev] Split MIME headers into multiple lines near a space
In-Reply-To: <429A3BFD.2040805@gmail.com>
References: <b348a0850505281723415f1a6f@mail.gmail.com>
	<429A3BFD.2040805@gmail.com>
Message-ID: <b348a085050605153025c534f0@mail.gmail.com>

On 5/30/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Noam's suggestion seems reasonable to me, but I'm not
> sure what the performance implications are.

I think that they are not critical. The number of lines can grow by at
most twice, because shorter words would not have a line of their own.
The added rfind call seems not very significant to me, since it is
preceded by about log2n string encodings, to test if an encoded prefix
fits the required line length.

Have a good day,
Noam

From ncoghlan at gmail.com  Mon Jun  6 11:31:56 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 06 Jun 2005 19:31:56 +1000
Subject: [Python-Dev] PEP 343 rewrite complete
In-Reply-To: <79990c6b050605091051094e4@mail.gmail.com>
References: <ca471dc2050601081626b6ef19@mail.gmail.com>
	<79990c6b050605091051094e4@mail.gmail.com>
Message-ID: <42A4180C.1030009@gmail.com>

Paul Moore wrote:
> Unfortunately, I don't have a better naming suggestion (other than
> simply "template", which is probably too general to work), so I just
> raise this as something you should expect to need to clarify a few
> times...

PEP 346 used "statement_template" for the generator decorator. That 
could be shortened to "stmt_template" without losing much in 
readability ("stmt" being a fairly common abbreviation of "statement").

Avoiding the problem with the English meaning of the phrases "do 
template" and "with template" was part of the reason for PEP 346's 
choice of name (the other part being that the name tied in nicely with 
that PEP's "user defined statement" terminology).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From reinhold-birkenfeld-nospam at wolke7.net  Mon Jun  6 20:20:26 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Mon, 06 Jun 2005 20:20:26 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
Message-ID: <d823u9$gs3$1@sea.gmane.org>

Hello,

I am currently having some thoughts about the standard library, with regard
to Python 2.5 and 3.0. Since I don't want to withhold them from you, here
are they ;)

- Flat namespace: Should we tend to a more hierarchic library (e.g.
  inet.url, inet.http, inet.nntp)? This would increase clarity when
  searching for a module.

- 3rd party modules: There are many superb modules out there, some of which
  really do have a "standard" character. Examples include PIL, numarray,
  ElementTree, [wxPython - I know this is a hairy issue],

- User interface: Tkinter may be "the" standard, but other Toolkits deserve
  notice, notably wxPython and PyGtk, the latter of which might be even
  easier to package.

Reinhold, please don't hit me!

-- 
Mail address is perfectly valid!


From python at rcn.com  Mon Jun  6 20:36:06 2005
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 6 Jun 2005 14:36:06 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d823u9$gs3$1@sea.gmane.org>
Message-ID: <004901c56ac6$be0e3c40$7105a044@oemcomputer>

> - 3rd party modules: There are many superb modules out there, some of
> which
>   really do have a "standard" character. Examples include PIL,
numarray,
>   ElementTree, [wxPython - I know this is a hairy issue],

If Fred were up for it, I think ElementTree would be a wonderful,
must-have addition.


Raymond Hettinger

From skip at pobox.com  Mon Jun  6 20:38:36 2005
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 6 Jun 2005 13:38:36 -0500
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d823u9$gs3$1@sea.gmane.org>
References: <d823u9$gs3$1@sea.gmane.org>
Message-ID: <17060.38956.60212.120855@montanaro.dyndns.org>



    Reinhold> - Flat namespace: Should we tend to a more hierarchic library
    Reinhold>   (e.g.  inet.url, inet.http, inet.nntp)? This would increase
    Reinhold>   clarity when searching for a module.

We've talked about this before.  The main technical challenge seems to be
backward compatibility.  You need to support both flat ("import urllib") and
packaged namespaces ("from www import urllib"), possibly within the same
application.  That is, postulating a www package, if I execute

    import urllib
    from www.urllib import urlopen

the module-level code should only be executed once, and

    urlopen == urllib.urlopen

should evaluate to True.

    Reinhold> - 3rd party modules: There are many superb modules out there,
    Reinhold>   some of which really do have a "standard"
    Reinhold>   character. Examples include PIL, numarray, ElementTree,
    Reinhold>   [wxPython - I know this is a hairy issue],

I think a good first step would be deciding on a package hierarchy.  Third
party packages could plug in wherever they fit best.

Skip

From barry at python.org  Mon Jun  6 21:43:22 2005
From: barry at python.org (Barry Warsaw)
Date: Mon, 06 Jun 2005 15:43:22 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <17060.38956.60212.120855@montanaro.dyndns.org>
References: <d823u9$gs3$1@sea.gmane.org>
	<17060.38956.60212.120855@montanaro.dyndns.org>
Message-ID: <1118087002.3503.18.camel@geddy.wooz.org>

On Mon, 2005-06-06 at 14:38, Skip Montanaro wrote:

>     import urllib
>     from www.urllib import urlopen
> 
> the module-level code should only be executed once, and
> 
>     urlopen == urllib.urlopen
> 
> should evaluate to True.

Not to mention "urlopen is urllib.urlopen"

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050606/8b324452/attachment.pgp

From skip at pobox.com  Mon Jun  6 22:06:42 2005
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 6 Jun 2005 15:06:42 -0500
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <1118087002.3503.18.camel@geddy.wooz.org>
References: <d823u9$gs3$1@sea.gmane.org>
	<17060.38956.60212.120855@montanaro.dyndns.org>
	<1118087002.3503.18.camel@geddy.wooz.org>
Message-ID: <17060.44242.802034.654426@montanaro.dyndns.org>

>>>>> "Barry" == Barry Warsaw <barry at python.org> writes:

    Barry> On Mon, 2005-06-06 at 14:38, Skip Montanaro wrote:
    >> import urllib
    >> from www.urllib import urlopen
    >> 
    >> the module-level code should only be executed once, and
    >> 
    >> urlopen == urllib.urlopen
    >> 
    >> should evaluate to True.

    Barry> Not to mention "urlopen is urllib.urlopen"

Whoops, yeah.  I was thinking in terms of module-level functions and
classes, where I think == is sufficient:

    >>> import foo2
    >>> foo2.f
    <function f at 0x38e6b0>
    >>> g = foo2.f
    >>> reload(foo2)
    <module 'foo2' from 'foo2.pyc'>
    >>> foo2.f == g
    False

Obviously, for data objects "is" is what you want.

Skip

From eric.nieuwland at xs4all.nl  Mon Jun  6 22:35:07 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Mon, 6 Jun 2005 22:35:07 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <17060.38956.60212.120855@montanaro.dyndns.org>
References: <d823u9$gs3$1@sea.gmane.org>
	<17060.38956.60212.120855@montanaro.dyndns.org>
Message-ID: <eef6695e212d1317fd62712d4989bc85@xs4all.nl>

Skip Montanaro wrote:
> The main technical challenge seems to be
> backward compatibility.  You need to support both flat ("import 
> urllib") and
> packaged namespaces ("from www import urllib"), possibly within the 
> same
> application.  That is, postulating a www package, if I execute
>
>     import urllib
>     from www.urllib import urlopen
>
> the module-level code should only be executed once, and
>
>     urlopen == urllib.urlopen
>
> should evaluate to True.

Unless
	from __future__ import new_std_library

and we don't allow mixed use within the same module.

--eric


From martin at v.loewis.de  Mon Jun  6 23:55:43 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 06 Jun 2005 23:55:43 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <004901c56ac6$be0e3c40$7105a044@oemcomputer>
References: <004901c56ac6$be0e3c40$7105a044@oemcomputer>
Message-ID: <42A4C65F.6070301@v.loewis.de>

Raymond Hettinger wrote:
>>  ElementTree, [wxPython - I know this is a hairy issue],
> 
> 
> If Fred were up for it, I think ElementTree would be a wonderful,
> must-have addition.

I might be missing fine details of the English language here
(what does "to be up for something" mean?), however, I believe
ElementTree is an unlikely addition to the standard library.

I also might be confusing people: AFAIK, ElementTree is
Fredrik Lundh's work, not Fred Drake's, so it would be
Fredrik who would need to contribute. My understanding is
that Fredrik has indicated a dislike to contribute certain
pieces of work, primarily out of fear to lose control. My
experience with integrating his work without his explicit
cooperation (specifically: sgmlop in PyXML) show that
he won't object to somebody else doing such integration,
but also won't try to cooperate afterwards.

Regards,
Martin

From python at rcn.com  Tue Jun  7 00:10:26 2005
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 6 Jun 2005 18:10:26 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <42A4C65F.6070301@v.loewis.de>
Message-ID: <000b01c56ae4$8afdbe20$7105a044@oemcomputer>

> > If Fred were up for it, I think ElementTree would be a wonderful,
> > must-have addition.


> I might be missing fine details of the English language here
> (what does "to be up for something" mean?), however, I believe
> ElementTree is an unlikely addition to the standard library.

Rewritten:

If Fredrik Lundh were supportive of the idea, I think the Standard
Library would benefit greatly from incorporating ElementTree.


Raymond

From bob at redivi.com  Tue Jun  7 00:17:54 2005
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 6 Jun 2005 15:17:54 -0700
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <000b01c56ae4$8afdbe20$7105a044@oemcomputer>
References: <000b01c56ae4$8afdbe20$7105a044@oemcomputer>
Message-ID: <0A1F09D1-415A-4E02-9622-FBFBE63E35E0@redivi.com>


On Jun 6, 2005, at 3:10 PM, Raymond Hettinger wrote:

>>> If Fred were up for it, I think ElementTree would be a wonderful,
>>> must-have addition.
>>>
>
>
>
>> I might be missing fine details of the English language here
>> (what does "to be up for something" mean?), however, I believe
>> ElementTree is an unlikely addition to the standard library.
>>
>
> Rewritten:
>
> If Fredrik Lundh were supportive of the idea, I think the Standard
> Library would benefit greatly from incorporating ElementTree.
>

Personally I'd like to see the standard library get smaller rather  
than larger.  There's a whole lot of bit rot in there, and since  
sys.path prefers the standard library over anything else it's a  
really huge pain to integrate patches on a faster release schedule  
than Python's while remaining sane at the same time.

-bob


From tlesher at gmail.com  Tue Jun  7 00:36:48 2005
From: tlesher at gmail.com (Tim Lesher)
Date: Mon, 6 Jun 2005 18:36:48 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d823u9$gs3$1@sea.gmane.org>
References: <d823u9$gs3$1@sea.gmane.org>
Message-ID: <9613db600506061536520abece@mail.gmail.com>

On 6/6/05, Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:
> - Flat namespace: Should we tend to a more hierarchic library (e.g.
>   inet.url, inet.http, inet.nntp)? This would increase clarity when
>   searching for a module.

-1. I feel the opposite way: when trying to figure out where something
"lives", I prefer Python's flat namespace to (for example) Java's
com.company.baz.bar.foo hierarchy.

> - 3rd party modules: There are many superb modules out there, some of which
>   really do have a "standard" character. Examples include PIL, numarray,
>   ElementTree, [wxPython - I know this is a hairy issue],

I think the most important question for each of these is "is the
module's release schedule at least as stable as Python's?".  For many
of these, I suspect the answer is "no".

--
Tim Lesher <tlesher at gmail.com>

From pje at telecommunity.com  Tue Jun  7 00:46:59 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 06 Jun 2005 18:46:59 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <0A1F09D1-415A-4E02-9622-FBFBE63E35E0@redivi.com>
References: <000b01c56ae4$8afdbe20$7105a044@oemcomputer>
	<000b01c56ae4$8afdbe20$7105a044@oemcomputer>
Message-ID: <5.1.1.6.0.20050606182742.01f7d118@mail.telecommunity.com>

At 03:17 PM 6/6/2005 -0700, Bob Ippolito wrote:
>Personally I'd like to see the standard library get smaller rather
>than larger.  There's a whole lot of bit rot in there, and since
>sys.path prefers the standard library over anything else it's a
>really huge pain to integrate patches on a faster release schedule
>than Python's while remaining sane at the same time.

You know, before you said that, it hadn't occurred to me that the Python 
standard library is subject to the same economic forces that cause 
mega-packages like Twisted, SciPy, Zope, etc. to develop.  Specifically, 
the cost incurred by relying on an externally-distributed dependency causes 
anyone with non-trivial needs to create "batteries included" libraries.

One of my goals for Python Eggs and EasyInstall was to lower this 
dependency-cost barrier by reducing the "dependency cost" to zero at the 
point of installation, by making it as easy to install ten packages as 
one.  (Another was to reduce the dependency cost for the developer, who 
need only add package metadata for the dependency to be fulfilled at 
installation time.)

Now that you've pointed out the parallel between the stdlib and the other 
modules, I wonder if Python 3.0 might be able to take a more minimalist 
approach to the standard library, if it included the equivalents of 
easy_install and pkg_resources.


From skip at pobox.com  Tue Jun  7 01:21:11 2005
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 6 Jun 2005 18:21:11 -0500
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <9613db600506061536520abece@mail.gmail.com>
References: <d823u9$gs3$1@sea.gmane.org>
	<9613db600506061536520abece@mail.gmail.com>
Message-ID: <17060.55911.922987.863770@montanaro.dyndns.org>


    Tim> On 6/6/05, Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:

    >> - Flat namespace: Should we tend to a more hierarchic library (e.g.
    >> inet.url, inet.http, inet.nntp)? This would increase clarity when
    >> searching for a module.

    Tim> -1. I feel the opposite way: when trying to figure out where
    Tim> something "lives", I prefer Python's flat namespace to (for
    Tim> example) Java's com.company.baz.bar.foo hierarchy.

I think Reinhold's suggestion (e.g., inet.*) was for a flatter namespace
than Java's scheme, but for a slightly more structured namespace than the
current standard library provides.  Some amount of structure helps to
collect modules together whose names don't obviously suggest who their
neighbors are in the functionality space.  For example, to the naive user it
might not be obvious that these groups of modules and packages are related:

    * getopt and optparse
    * bsddb, gdbm and anydbm
    * email, mhlib, quopri
    * heapq, collections
    * user, site, sitecustomize
    * profile, hotshot, pstats
    * pipes, subprocess, os

I wouldn't mind a stdlib that defined a set of top-level packages (some of
which might be wholly unpopulated by modules in the standard distribution)
It might, for example, define a gui package and gui.Tkinter and gui._tkinter
modules, leaving the remainder of gui namespace available for 3rd party
modules.  Such a scheme would probably work best if there was some fairly
lightweight registration/approval process in the community to avoid needless
collisions.  For example, it might be a bit confusing if two organizations
began installing their packages in a subpackage named gui.widgets.  An
unsuspecting person downloading an application that used one version of
gui.widgets into environment with the conflicting gui.widgets would run into
trouble.  Is the CPAN namespace wide open?  If I wanted to create a Perl
module to fit into the HTML namespace is there some procedure involved or is
it an example of squatters' rights?

    >> - 3rd party modules: There are many superb modules out there, some of
    >>   which really do have a "standard" character. Examples include PIL,
    >>   numarray, ElementTree, [wxPython - I know this is a hairy issue],

    Tim> I think the most important question for each of these is "is the
    Tim> module's release schedule at least as stable as Python's?".  For
    Tim> many of these, I suspect the answer is "no".

If you provide the necessary namespace structure for them to nestle into, I
suspect most of them could be maintained outside the stdlib just fine.

Skip

From fperez.net at gmail.com  Tue Jun  7 02:12:26 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 06 Jun 2005 18:12:26 -0600
Subject: [Python-Dev] Thoughts on stdlib evolvement
References: <d823u9$gs3$1@sea.gmane.org>
	<9613db600506061536520abece@mail.gmail.com>
	<17060.55911.922987.863770@montanaro.dyndns.org>
Message-ID: <d82oi7$g22$1@sea.gmane.org>

Skip Montanaro wrote:

> I wouldn't mind a stdlib that defined a set of top-level packages (some of
> which might be wholly unpopulated by modules in the standard distribution)
> It might, for example, define a gui package and gui.Tkinter and gui._tkinter
> modules, leaving the remainder of gui namespace available for 3rd party
> modules.  Such a scheme would probably work best if there was some fairly
> lightweight registration/approval process in the community to avoid needless
> collisions.  For example, it might be a bit confusing if two organizations
> began installing their packages in a subpackage named gui.widgets.  An
> unsuspecting person downloading an application that used one version of
> gui.widgets into environment with the conflicting gui.widgets would run into
> trouble.

I've wondered if it wouldn't be better if the std lib were all stuffed into its
own namespace:

from std import urllib

If a more structured approach is desired, it could be 

from std.www import urllib

for example.  But having std. as the top-level namespace for the stdlib, could
simplify (I think) life a lot in the long run.  If a decision for a more
structured namespace is made, then it might be nice to have the same top-level
structure in site-packages, albeit empty by default:

from std.www import foo  -> standard library www packages

from www import bar  -> third-party www packages

Third-party packages can still be put into base site-packages, of course, but
developers could be encouraged to transition into putting things into these
categories.

This would also ease the process of 'staging' a module as external for a while
before deciding whether it meets the requirement for being put into the
stdlib.

Just an idea (sorry if it's been discussed and shot down before).

best,

f


From jcarlson at uci.edu  Tue Jun  7 06:24:52 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 06 Jun 2005 21:24:52 -0700
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d82oi7$g22$1@sea.gmane.org>
References: <d823u9$gs3$1@sea.gmane.org>	<9613db600506061536520abece@mail.gmail.com>	<17060.55911.922987.863770@montanaro.dyndns.org>
	<d82oi7$g22$1@sea.gmane.org>
Message-ID: <42A52194.9020008@uci.edu>

Fernando Perez wrote:
> Skip Montanaro wrote:
> 
> 
>>I wouldn't mind a stdlib that defined a set of top-level packages (some of
>>which might be wholly unpopulated by modules in the standard distribution)
>>It might, for example, define a gui package and gui.Tkinter and gui._tkinter
>>modules, leaving the remainder of gui namespace available for 3rd party
>>modules.  Such a scheme would probably work best if there was some fairly
>>lightweight registration/approval process in the community to avoid needless
>>collisions.  For example, it might be a bit confusing if two organizations
>>began installing their packages in a subpackage named gui.widgets.  An
>>unsuspecting person downloading an application that used one version of
>>gui.widgets into environment with the conflicting gui.widgets would run into
>>trouble.
> 
> 
> I've wondered if it wouldn't be better if the std lib were all stuffed into its
> own namespace:
> 
> from std import urllib
> 
> If a more structured approach is desired, it could be 
> 
> from std.www import urllib

This generally brings up the intersection of stdlib and nonstdlib naming 
hierarchy.  More specifically, what does "import email" mean? 
Presumably it means to import the email module or package, but from the 
current module directory, or from the standard library?

I posted about this months ago, and got to writing an import hook that 
used a method suggested by Guido by naming things (sys.modules names) 
based on whether they were imported relative to the module with a 
__name__ of '__main__', and using the prefix dot semantics for 
package-relative imports to mean any relative imports.

I eventually got hung up on some nested import bits, and got busy with 
work.  I may have time to hack on it next week to finish the 'proof of 
concept' (currently in the Bay area for work), but if someone wants or 
needs more explanation, I would be happy to offer it.

  - Josiah

From stephen at xemacs.org  Tue Jun  7 06:26:41 2005
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 07 Jun 2005 13:26:41 +0900
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <17060.55911.922987.863770@montanaro.dyndns.org> (Skip
	Montanaro's message of "Mon, 6 Jun 2005 18:21:11 -0500")
References: <d823u9$gs3$1@sea.gmane.org>
	<9613db600506061536520abece@mail.gmail.com>
	<17060.55911.922987.863770@montanaro.dyndns.org>
Message-ID: <87r7fe7pxa.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Skip" == Skip Montanaro <skip at pobox.com> writes:

    Skip> If you provide the necessary namespace structure for them to
    Skip> nestle into, I suspect most of them could be maintained
    Skip> outside the stdlib just fine.

FWIW, this has worked well for XEmacs; it's one of our most popular
features vis-a-vis GNU Emacs.  Users like it because we do provide a
centralized directory of modules (with many mirrors) that they can
download updates from, as well as a "batteries included" library
distribution (which we call "SUMO tarballs" for historical reasons).
Package developers like it because they can maintain closer control,
yet have up-to-date versions distributed from our repository.  Finally
the core maintainers like it because there's a clear delegation of
responsibility.

We provide a centralized repository, which is mirrored by a couple
score sites; I'm not sure how much harder it would be to make it
distributed.  The essential feature is the centralized directory.

We do have problems.  One is the megapackage issue that Phillip Eby
referred in his reply to Bob Ippolito
<5.1.1.6.0.20050606182742.01f7d118 at mail.telecommunity.com>.  Our
packaging system doesn't work well for them.  Some of the problems are
our fault, but I think it's actually a hard problem; we could fix up
some details, but not really address the core issues of megapackages.

The other one is that our initial division into "core" and "package"
was pretty radical, and several core developers still want to pull
certain packages back into core.  But this creates backward
compatibility problems.  I think this would be less of a problem for
Python.  Creating the package repository could be more conservative:
Python's standard library has plenty of modules that would not present
such problems.  In any case, Python has better facilities for dealing
with backward compatibility issues (such as deprecation warnings and
the like) than we do, and the language itself is simply more suited to
preserving modularity than Emacs Lisp is (with its single dynamically
scoped namespace).

Our package namespace structure is trivial (it's basically flat); what
is more important than the namespace structure, I think, is provision
of package organization and directory utilities as Phillip Eby
mentions.  If they're reasonably flexible and Python invests some
effort in helping package maintainers with different current
distribution methods to adapt, I think this would work well for Python
too.  One thing we have found is that it takes only a small amount of
incremental work to maintain the package directory and repository as
such, even as it has grown dramatically.  By contrast, I find the Zope
wiki-based organization of Products quite annoying to work with.

Of course the packaged code itself can and does bitrot if the external
maintainer goes AWOL, or simply doesn't like hanging out with XEmacs
people.  In such cases we've found that users will often step up to
help with the nitty-gritty management of "their" package, such as
tracking bug reports, liaison with an "unsociable" upstream,
regression testing, and integrating changes (anybody can run make and
patch), even if they have to ask for help with code-related work.  Ie,
the fact that there is a well-defined package requiring specific
management tasks encourages them to take on the work they can handle.

For XEmacs the effort of separating out the packages was moderately
large; it took about a year, with three or four developers spending a
substantial fraction of their time on it, either organizing the
repository or creating infrastructure.  It was a very worthwhile
investment.  Python's investment probably would be proportionately
smaller, as much of the necessary infrastructure seems to be already
available.

For XEmacs, I think the main returns come from creating well-defined
chunks that people can "own" and take responsibility for.  This helps
to mitigate the "dormant bug" problem, and generally keeps the
packaged code fresher than it was in the old days, even though we have
a much larger volume of code and probably less manpower now.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From fperez.net at gmail.com  Tue Jun  7 06:27:41 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 06 Jun 2005 22:27:41 -0600
Subject: [Python-Dev] Thoughts on stdlib evolvement
References: <d823u9$gs3$1@sea.gmane.org>	<9613db600506061536520abece@mail.gmail.com>	<17060.55911.922987.863770@montanaro.dyndns.org>
	<d82oi7$g22$1@sea.gmane.org> <42A52194.9020008@uci.edu>
Message-ID: <d837ib$f7b$1@sea.gmane.org>

Josiah Carlson wrote:

> Fernando Perez wrote:

>> I've wondered if it wouldn't be better if the std lib were all stuffed into
>> its own namespace:
>> 
>> from std import urllib
>> 
>> If a more structured approach is desired, it could be
>> 
>> from std.www import urllib
> 
> This generally brings up the intersection of stdlib and nonstdlib naming
> hierarchy.  More specifically, what does "import email" mean?
> Presumably it means to import the email module or package, but from the
> current module directory, or from the standard library?

Well, I've thought of this (ligthly) mostly as a py3k thing, since it would
require that 'import email' in the naked fails, as it would become 'import
std.email', or 'import std.www.email' or whatever.  A plain 'import email'
would then refer to some third-party 'email' module, not part of the standard
library.

Since this would mean a massive break of exisiting code, it would necessarily be
a py3k issue.  But nonetheless the idea of confinign the stdlib to the 'std'
namespace does have some appeal, at least to me.

Best,

f


From jcarlson at uci.edu  Tue Jun  7 07:24:14 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 06 Jun 2005 22:24:14 -0700
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d837ib$f7b$1@sea.gmane.org>
References: <d823u9$gs3$1@sea.gmane.org>	<9613db600506061536520abece@mail.gmail.com>	<17060.55911.922987.863770@montanaro.dyndns.org>	<d82oi7$g22$1@sea.gmane.org>
	<42A52194.9020008@uci.edu> <d837ib$f7b$1@sea.gmane.org>
Message-ID: <42A52F7E.3050206@uci.edu>

Fernando Perez wrote:
> Josiah Carlson wrote:
>>Fernando Perez wrote:
>>>I've wondered if it wouldn't be better if the std lib were all stuffed into
>>>its own namespace:
>>>
>>>from std import urllib
>>>
>>>If a more structured approach is desired, it could be
>>>
>>>from std.www import urllib
>>
>>This generally brings up the intersection of stdlib and nonstdlib naming
>>hierarchy.  More specifically, what does "import email" mean?
>>Presumably it means to import the email module or package, but from the
>>current module directory, or from the standard library?
> 
> 
> Well, I've thought of this (ligthly) mostly as a py3k thing, since it would
> require that 'import email' in the naked fails, as it would become 'import
> std.email', or 'import std.www.email' or whatever.  A plain 'import email'
> would then refer to some third-party 'email' module, not part of the standard
> library.
> 
> Since this would mean a massive break of exisiting code, it would necessarily be
> a py3k issue.  But nonetheless the idea of confinign the stdlib to the 'std'
> namespace does have some appeal, at least to me.

Actually no.

Re: std.*
If there were a single path (or set of paths) know at install time where 
standard library modules are, then that can be placed in a PYSDTLIB 
environment variable (or some other specified place).  Inside Python, 
whenever an import is done relative to std, one merely examines the 
paths offered via the PYSTDLIB environment variable for the looked-for 
module.  For modules/packages in site-packages, one could use another 
name, perhaps "packages" or somesuch, to refer to it.  sys.path is not 
usable as-is because it can contain the path of the module named 
__main__, and is (not uncommonly) modified by user code.

This can be used with a proper import hook, which is turned on 
per-module via a __future__ import, hence no backwards compatibility issues.


What I had been working on is the use of special import semantics for 
non-stdlib/site-packages modules.  Specifically if you were to write 
'mymodule.py' and import your other module 'myothermodule.py' in the 
same path, you would use "from . import myothermodule".  It doesn't 
solve any problems with standard library naming, but it does allow one 
to do cousin imports... "from ..uncle import cousin", as well as all 
sorts of other things, which allows (in writing software) for modules 
and packages to be shared by other packages.

  - Josiah

From dstanek at dstanek.com  Tue Jun  7 13:09:26 2005
From: dstanek at dstanek.com (David Stanek)
Date: Tue, 7 Jun 2005 07:09:26 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <17060.38956.60212.120855@montanaro.dyndns.org>
References: <d823u9$gs3$1@sea.gmane.org>
	<17060.38956.60212.120855@montanaro.dyndns.org>
Message-ID: <20050607110926.GA7863@goliath.hsd1.oh.comcast.net>

On Mon, Jun 06, 2005 at 01:38:36PM -0500, Skip Montanaro wrote:
> 
> 
>     Reinhold> - Flat namespace: Should we tend to a more hierarchic library
>     Reinhold>   (e.g.  inet.url, inet.http, inet.nntp)? This would increase
>     Reinhold>   clarity when searching for a module.
> 
> We've talked about this before.  The main technical challenge seems to be
> backward compatibility.  You need to support both flat ("import urllib") and
> packaged namespaces ("from www import urllib"), possibly within the same
> application.  That is, postulating a www package, if I execute
> 
>     import urllib
>     from www.urllib import urlopen
> 
> the module-level code should only be executed once, and
> 
>     urlopen == urllib.urlopen
> 
> should evaluate to True.

Can't this be handled with an __init__ module? Create an inet package
and having the __init__ module pull whatever it wanted into it's own
namespace.

It would be nice to then use warnings to show deprecation messages
when the old import is used (import url instead of import inet.url),
but I don't think that is quite so easy.

-- 
David Stanek
www.roninds.net

GPG keyID #6272EDAF on http://pgp.mit.edu
Key fingerprint = 8BAA 7E11 8856 E148 6833  655A 92E2 3E00 6272 EDAF
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20050607/0d897c63/attachment.pgp

From maliger at gmail.com  Tue Jun  7 16:15:44 2005
From: maliger at gmail.com (Martin Aliger)
Date: Tue, 7 Jun 2005 16:15:44 +0200
Subject: [Python-Dev] python running in several threads
Message-ID: <63fd41d40506070715129c904a@mail.gmail.com>

Hello Python developers,

I know this is developers list and I hope I'm right here. I'm using
python as scripting engine embedded in application but issue should be
more general.

When several scripts are executed simultaneously (under several
threads) one have to acquire/release locks. This degrades performance
IMHO, mainly on server installations (many web servers use python
engine). Moreover those scripts are commonly independent chunks of
code which do not create other threats. Under those limitations, do
you think, an easy rewrite of execution engine is possible to allow
simultaneous run of all threads?

I checked the sources esp. PyThreadState object and it seems all those
variables could be held locally. Another way is using thread local
storage, but it could be a little problematic with porting issues.

How do you see it? Do you have any plans here?

Thanks and regards,
Martin

From aahz at pythoncraft.com  Tue Jun  7 17:32:53 2005
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 7 Jun 2005 08:32:53 -0700
Subject: [Python-Dev] python running in several threads
In-Reply-To: <63fd41d40506070715129c904a@mail.gmail.com>
References: <63fd41d40506070715129c904a@mail.gmail.com>
Message-ID: <20050607153253.GB2787@panix.com>

On Tue, Jun 07, 2005, Martin Aliger wrote:
>
> Under those limitations, do you think, an easy rewrite of execution
> engine is possible to allow simultaneous run of all threads?

Short answer: no

Longer answer: see previous threads about removing the GIL; if you still
have questions, please post to comp.lang.python.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From reinhold-birkenfeld-nospam at wolke7.net  Tue Jun  7 19:40:49 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Tue, 07 Jun 2005 19:40:49 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <17060.55911.922987.863770@montanaro.dyndns.org>
References: <d823u9$gs3$1@sea.gmane.org>	<9613db600506061536520abece@mail.gmail.com>
	<17060.55911.922987.863770@montanaro.dyndns.org>
Message-ID: <d84lvn$1u2$2@sea.gmane.org>

Skip Montanaro wrote:
>     Tim> On 6/6/05, Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:
> 
>     >> - Flat namespace: Should we tend to a more hierarchic library (e.g.
>     >> inet.url, inet.http, inet.nntp)? This would increase clarity when
>     >> searching for a module.
> 
>     Tim> -1. I feel the opposite way: when trying to figure out where
>     Tim> something "lives", I prefer Python's flat namespace to (for
>     Tim> example) Java's com.company.baz.bar.foo hierarchy.
> 
> I think Reinhold's suggestion (e.g., inet.*) was for a flatter namespace
> than Java's scheme, but for a slightly more structured namespace than the
> current standard library provides.  Some amount of structure helps to
> collect modules together whose names don't obviously suggest who their
> neighbors are in the functionality space.  For example, to the naive user it
> might not be obvious that these groups of modules and packages are related:
> 
>     * getopt and optparse
>     * bsddb, gdbm and anydbm
>     * email, mhlib, quopri
>     * heapq, collections
>     * user, site, sitecustomize
>     * profile, hotshot, pstats
>     * pipes, subprocess, os

Yep, exactly. Java's namespaces are ugly, that's right, but a flatter version
certainly improves readability.
"""Namespaces are one honking great idea -- let's do more of those!"""

> I wouldn't mind a stdlib that defined a set of top-level packages (some of
> which might be wholly unpopulated by modules in the standard distribution)
> It might, for example, define a gui package and gui.Tkinter and gui._tkinter
> modules, leaving the remainder of gui namespace available for 3rd party
> modules.  Such a scheme would probably work best if there was some fairly
> lightweight registration/approval process in the community to avoid needless
> collisions.  For example, it might be a bit confusing if two organizations
> began installing their packages in a subpackage named gui.widgets.  An
> unsuspecting person downloading an application that used one version of
> gui.widgets into environment with the conflicting gui.widgets would run into
> trouble.  Is the CPAN namespace wide open?  If I wanted to create a Perl
> module to fit into the HTML namespace is there some procedure involved or is
> it an example of squatters' rights?

Personally, I think that CPAN is one of the greatest strengths of Perl. The language
is a mess, and the quality of many modules is questionable, but it's incredibly
easy to find a module for handling a special problem, and the namespaces are IMHO
well thought out. Additionally, the docs

>     >> - 3rd party modules: There are many superb modules out there, some of
>     >>   which really do have a "standard" character. Examples include PIL,
>     >>   numarray, ElementTree, [wxPython - I know this is a hairy issue],
> 
>     Tim> I think the most important question for each of these is "is the
>     Tim> module's release schedule at least as stable as Python's?".  For
>     Tim> many of these, I suspect the answer is "no".
> 
> If you provide the necessary namespace structure for them to nestle into, I
> suspect most of them could be maintained outside the stdlib just fine.

Exactly! There needn't be such a big separation between stdlib and 3rd party.
Access to the net is standard nowadays, though it wouldn't be of any harm
making a big distribution with all modules available, for downloading and burning
on CD, for example.

PJE's great EasyInstall and Python Eggs will be a perfect starting point for this,
I think.

Reinhold

-- 
Mail address is perfectly valid!


From reinhold-birkenfeld-nospam at wolke7.net  Tue Jun  7 19:40:40 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Tue, 07 Jun 2005 19:40:40 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d82oi7$g22$1@sea.gmane.org>
References: <d823u9$gs3$1@sea.gmane.org>	<9613db600506061536520abece@mail.gmail.com>	<17060.55911.922987.863770@montanaro.dyndns.org>
	<d82oi7$g22$1@sea.gmane.org>
Message-ID: <d84lve$1u2$1@sea.gmane.org>

Fernando Perez wrote:
> Skip Montanaro wrote:
> 
>> I wouldn't mind a stdlib that defined a set of top-level packages (some of
>> which might be wholly unpopulated by modules in the standard distribution)
>> It might, for example, define a gui package and gui.Tkinter and gui._tkinter
>> modules, leaving the remainder of gui namespace available for 3rd party
>> modules.  Such a scheme would probably work best if there was some fairly
>> lightweight registration/approval process in the community to avoid needless
>> collisions.  For example, it might be a bit confusing if two organizations
>> began installing their packages in a subpackage named gui.widgets.  An
>> unsuspecting person downloading an application that used one version of
>> gui.widgets into environment with the conflicting gui.widgets would run into
>> trouble.
> 
> I've wondered if it wouldn't be better if the std lib were all stuffed into its
> own namespace:
> 
> from std import urllib
> 
> If a more structured approach is desired, it could be 
> 
> from std.www import urllib

One may want to look at the "py.std" approach of "the py lib", found at
http://codespeak.net/py/current/doc/misc.html#the-py-std-hook

Reinhold

-- 
Mail address is perfectly valid!


From rrr at ronadam.com  Tue Jun  7 20:31:34 2005
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 07 Jun 2005 14:31:34 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d823u9$gs3$1@sea.gmane.org>
References: <d823u9$gs3$1@sea.gmane.org>
Message-ID: <42A5E806.1060802@ronadam.com>

Reinhold Birkenfeld wrote:
> Hello,
> 
> I am currently having some thoughts about the standard library, with regard
> to Python 2.5 and 3.0. Since I don't want to withhold them from you, here
> are they ;)
> 
> - Flat namespace: Should we tend to a more hierarchic library (e.g.
>   inet.url, inet.http, inet.nntp)? This would increase clarity when
>   searching for a module.

I think putting the standard library in it's own package would be the 
first step.  "std" has been mentioned a couple of times and I think its 
a good name for it.

For backwards compatibility, maybe a "from __compatible__ import 
ver_2.x", or something similar, could be used as a way to allow future 
versions a bit more flexibility to change?

Trying to move the modules in python/lib to python/packages/stdlib to 
see what would happen revealed the following files are required to be in 
python/lib to start the python shell without errors.

	copy_reg.py
	ntpath.py
	os.py
	site.py
	stat.py
	types.py

These files are modules site.py imports.  Maybe these should be builtins 
so that the shell could start without importing any modules?

> - 3rd party modules: There are many superb modules out there, some of which
>   really do have a "standard" character. Examples include PIL, numarray,
>   ElementTree, [wxPython - I know this is a hairy issue],

I'm on the side of less in builtins and less in the standard library, 
and for having a central repository for developers to distribute their 
packages.  A python updater that gets a list of installable packages to 
install or uninstall would be fantastic. ;-)

Having less in the core distribution means smaller complete applications 
to install when py2exe is used.  There also needs to be some assurance 
that the standard library has as few bugs in it as possible.  The more 
that's added to it, the more work and testing to do that is needed.


> - User interface: Tkinter may be "the" standard, but other Toolkits deserve
>   notice, notably wxPython and PyGtk, the latter of which might be even
>   easier to package.

I think these fall into the category of optional "official" extension 
packages that could be included in the core installer, but are not part 
of the standard library package itself.  They are packages that would be 
maintained separately from the core and have their own developers 
dedicated to maintaining them.

The 'official', or other designator if appropriate, would mean that they 
are important packages that fulfill a need and efforts to make and keep 
them compatible with the current python release is made.

An "official" package designation might be something that developers 
could strive for and proudly display on their website or resume.  Some 
minimum standards would be needed... ie current with current version of 
python, fully functional, fulfilling a need, etc.  A request and voting 
process of some sort where python-dev can award "officialdom" to package 
developers might be usefull for this.

Just a few thoughts,
Cheers,
Ron





From jcarlson at uci.edu  Tue Jun  7 20:47:18 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 07 Jun 2005 11:47:18 -0700
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <42A5E806.1060802@ronadam.com>
References: <d823u9$gs3$1@sea.gmane.org> <42A5E806.1060802@ronadam.com>
Message-ID: <42A5EBB6.1020505@uci.edu>

Ron Adam wrote:
[snip]
> Having less in the core distribution means smaller complete applications 
> to install when py2exe is used.  There also needs to be some assurance 
> that the standard library has as few bugs in it as possible.  The more 
> that's added to it, the more work and testing to do that is needed.

Almost by definition, applications using py2exe are minimal Python 
installations without external dependencies.

  - Josiah


From irmen at xs4all.nl  Tue Jun  7 20:49:11 2005
From: irmen at xs4all.nl (Irmen de Jong)
Date: Tue, 07 Jun 2005 20:49:11 +0200
Subject: [Python-Dev] problem installing current cvs
Message-ID: <42A5EC27.8010409@xs4all.nl>

Hi,
I'm having 2 problems with the current cvs :

During compilation this warning occurs:
*** WARNING: renaming "dbm" since importing it failed: build/lib.linux-i686-2.5/
dbm.so: undefined symbol: dbm_firstkey

and the 'dbm' module is unavailable.
I'm running MandrakeLinux 2005 (10.2) gcc 3.4.3
(I'm also having this problem when compiling python 2.3.5 or 2.4.1)


furthermore the 'make install' of current cvs fails halfway trough
with the following errors:

   .....
   .....
Compiling /opt/python25/lib/python2.5/bsddb/test/test_associate.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
('/opt/python25/lib/python2.5/bsddb/test/test_associate.py', 97, 23, '\t 
os.mkdir(homeDir)\n'))
Compiling /opt/python25/lib/python2.5/bsddb/test/test_basics.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
('/opt/python25/lib/python2.5/bsddb/test/test_basics.py', 400, 26, '\t    if 
get_raises_error:\n'))
Compiling /opt/python25/lib/python2.5/bsddb/test/test_compare.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
('/opt/python25/lib/python2.5/bsddb/test/test_compare.py', 167, 5, '\t"""\n'))
   .....
   .....
Compiling /opt/python25/lib/python2.5/bsddb/test/test_recno.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
('/opt/python25/lib/python2.5/bsddb/test/test_recno.py', 38, 46, '\tget_returns_none = 
d.set_get_returns_none(2)\n'))
   .....
   .....
make: *** [libinstall] Error 1
$

And then it quits.
Fixing the tab indentation errors locally makes the problem go away.


Regards,

Irmen de Jong

From bob at redivi.com  Tue Jun  7 20:54:58 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 7 Jun 2005 11:54:58 -0700
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <42A5EBB6.1020505@uci.edu>
References: <d823u9$gs3$1@sea.gmane.org> <42A5E806.1060802@ronadam.com>
	<42A5EBB6.1020505@uci.edu>
Message-ID: <603C535E-BADD-4A96-9D2F-664DD6017ACA@redivi.com>


On Jun 7, 2005, at 11:47 AM, Josiah Carlson wrote:

> Ron Adam wrote:
> [snip]
>
>> Having less in the core distribution means smaller complete  
>> applications
>> to install when py2exe is used.  There also needs to be some  
>> assurance
>> that the standard library has as few bugs in it as possible.  The  
>> more
>> that's added to it, the more work and testing to do that is needed.
>>
>
> Almost by definition, applications using py2exe are minimal Python
> installations without external dependencies.

Yeah, py2exe only includes the portion of the standard library that  
you use, anyway... so the size of it is not an issue.

-bob


From rrr at ronadam.com  Tue Jun  7 21:13:06 2005
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 07 Jun 2005 15:13:06 -0400
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <603C535E-BADD-4A96-9D2F-664DD6017ACA@redivi.com>
References: <d823u9$gs3$1@sea.gmane.org> <42A5E806.1060802@ronadam.com>
	<42A5EBB6.1020505@uci.edu>
	<603C535E-BADD-4A96-9D2F-664DD6017ACA@redivi.com>
Message-ID: <42A5F1C2.4010203@ronadam.com>

Bob Ippolito wrote:

> 
> On Jun 7, 2005, at 11:47 AM, Josiah Carlson wrote:
> 
>> Ron Adam wrote:
>> [snip]
>>
>>> Having less in the core distribution means smaller complete  
>>> applications
>>> to install when py2exe is used.  There also needs to be some  assurance
>>> that the standard library has as few bugs in it as possible.  The  more
>>> that's added to it, the more work and testing to do that is needed.
>>>
>>
>> Almost by definition, applications using py2exe are minimal Python
>> installations without external dependencies.
> 
> 
> Yeah, py2exe only includes the portion of the standard library that  you 
> use, anyway... so the size of it is not an issue.
> 
> -bob

This wasn't the main point of my post in any case. Yes, as high 
bandwidth connections and disk space become more common, the 
organizational aspects are really more important than the size aspects.

Cheers,
Ron



From dstanek at dstanek.com  Tue Jun  7 21:28:32 2005
From: dstanek at dstanek.com (dstanek@dstanek.com)
Date: Tue, 7 Jun 2005 15:28:32 -0400 (EDT)
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d84lve$1u2$1@sea.gmane.org>
References: <d823u9$gs3$1@sea.gmane.org>	<9613db600506061536520abece@mail.gmail.com>	<17060.55911.922987.863770@montanaro.dyndns.org>
	<d82oi7$g22$1@sea.gmane.org> <d84lve$1u2$1@sea.gmane.org>
Message-ID: <26887.207.58.192.150.1118172512.squirrel@207.58.192.150>

> Fernando Perez wrote:
>> Skip Montanaro wrote:
>>
>> I've wondered if it wouldn't be better if the std lib were all stuffed
>> into its
>> own namespace:
>>
>> from std import urllib
>>
>> If a more structured approach is desired, it could be
>>
>> from std.www import urllib
>
> One may want to look at the "py.std" approach of "the py lib", found at
> http://codespeak.net/py/current/doc/misc.html#the-py-std-hook
>
> Reinhold

Thats where I hijacked the idea. Doing that would allow both to work.
Legacy code could still use:
  import logging
and new code could use:
  import std.logging

At some point the legacy way to do things should be deprecated.

David


From greg at electricrain.com  Tue Jun  7 23:08:16 2005
From: greg at electricrain.com (Gregory P. Smith)
Date: Tue, 7 Jun 2005 14:08:16 -0700
Subject: [Python-Dev] problem installing current cvs - TabError
In-Reply-To: <42A5EC27.8010409@xs4all.nl>
References: <42A5EC27.8010409@xs4all.nl>
Message-ID: <20050607210816.GB19337@zot.electricrain.com>

On Tue, Jun 07, 2005 at 08:49:11PM +0200, Irmen de Jong wrote:
> furthermore the 'make install' of current cvs fails halfway trough
> with the following errors:
> 
>    .....
>    .....
> Compiling /opt/python25/lib/python2.5/bsddb/test/test_associate.py ...
> Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
> ('/opt/python25/lib/python2.5/bsddb/test/test_associate.py', 97, 23, '\t 
> os.mkdir(homeDir)\n'))
> Compiling /opt/python25/lib/python2.5/bsddb/test/test_basics.py ...
> Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
> ('/opt/python25/lib/python2.5/bsddb/test/test_basics.py', 400, 26, '\t    if 
> get_raises_error:\n'))
> Compiling /opt/python25/lib/python2.5/bsddb/test/test_compare.py ...
> Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
> ('/opt/python25/lib/python2.5/bsddb/test/test_compare.py', 167, 5, '\t"""\n'))
>    .....
>    .....
> Compiling /opt/python25/lib/python2.5/bsddb/test/test_recno.py ...
> Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', 
> ('/opt/python25/lib/python2.5/bsddb/test/test_recno.py', 38, 46, '\tget_returns_none = 
> d.set_get_returns_none(2)\n'))
>    .....
>    .....
> make: *** [libinstall] Error 1
> $
> 
> And then it quits.
> Fixing the tab indentation errors locally makes the problem go away.

ugh.  this may be the result of me working on those files recently
without my usual .vimrc in place.  i'll take a look.  regardless,
where is the TabError coming from?  those files are all valid python
even if they do have an annoying mix of spaces and tabs.

major gripe to -dev: a 'make install' of the full cvs tree should not
be required to test that some changes to existing .py files that pass
their tests (in this case they -are- the tests) are ok to check in.

Greg


From martin at v.loewis.de  Wed Jun  8 00:28:29 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 08 Jun 2005 00:28:29 +0200
Subject: [Python-Dev] python running in several threads
In-Reply-To: <20050607153253.GB2787@panix.com>
References: <63fd41d40506070715129c904a@mail.gmail.com>
	<20050607153253.GB2787@panix.com>
Message-ID: <42A61F8D.2090704@v.loewis.de>

Aahz wrote:
>>Under those limitations, do you think, an easy rewrite of execution
>>engine is possible to allow simultaneous run of all threads?
> 
> 
> Short answer: no
> 
> Longer answer: see previous threads about removing the GIL; if you still
> have questions, please post to comp.lang.python.

Or, as a medium-sized answer: google for "python free threading".
Removing the GIL is easy, but then the interpreter crashes in cases of
simultaneous accesses to dictionaries, reference counters, etc.

I also disagree with Martin Aliger's premises:

# This degrades performance IMHO, mainly on server installations
# (many web servers use python engine).

This is just not true. On a single-processor machine, the GIL does
*not* degrade performance. Instead, it increases throughput (and
thus performance).

On a multi-processor machine, you often use multiple operating
system processes to serve request (e.g. in CGI or Apache mpm-prefork,
or even the typical mpm-worker configuration). If you then have
different processes running Python, they don't interfere with
each other at all.

Regards,
Martin

From bac at OCF.Berkeley.EDU  Wed Jun  8 06:28:48 2005
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue, 07 Jun 2005 21:28:48 -0700
Subject: [Python-Dev] problem installing current cvs - TabError
In-Reply-To: <20050607210816.GB19337@zot.electricrain.com>
References: <42A5EC27.8010409@xs4all.nl>
	<20050607210816.GB19337@zot.electricrain.com>
Message-ID: <42A67400.6070102@ocf.berkeley.edu>

Gregory P. Smith wrote:
[SNIP]
> major gripe to -dev: a 'make install' of the full cvs tree should not
> be required to test that some changes to existing .py files that pass
> their tests (in this case they -are- the tests) are ok to check in.
> 

You actually don't have to.  If you execute the interpreter that is built in
your cvs checkout it will use the stdlib files as found in your cvs tree.  So
if you are in your cvs checkout, ``./python.exe <whatever>`` will actually
execute using the code in the cvs checkout itself.

-Brett

From anthony at interlink.com.au  Wed Jun  8 06:49:05 2005
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 8 Jun 2005 14:49:05 +1000
Subject: [Python-Dev] problem installing current cvs - TabError
In-Reply-To: <20050607210816.GB19337@zot.electricrain.com>
References: <42A5EC27.8010409@xs4all.nl>
	<20050607210816.GB19337@zot.electricrain.com>
Message-ID: <200506081449.09254.anthony@interlink.com.au>

On Wednesday 08 June 2005 07:08, Gregory P. Smith wrote:
> ugh.  this may be the result of me working on those files recently
> without my usual .vimrc in place.  i'll take a look.  regardless,
> where is the TabError coming from?  those files are all valid python
> even if they do have an annoying mix of spaces and tabs.

There's a scripts Tools/scripts/reindent.py - put it somewhere on your
PATH and run it before checkin, like "reindent.py -r Lib". It means Tim
or I don't have to run it for you <wink>

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From fredrik at pythonware.com  Wed Jun  8 14:30:39 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 8 Jun 2005 14:30:39 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
References: <42A4C65F.6070301@v.loewis.de>
	<000b01c56ae4$8afdbe20$7105a044@oemcomputer>
Message-ID: <d86o5t$euh$1@sea.gmane.org>

Raymond Hettinger wrote:

>> > If Fred were up for it, I think ElementTree would be a wonderful,
>> > must-have addition.
>
>> I might be missing fine details of the English language here
>> (what does "to be up for something" mean?), however, I believe
>> ElementTree is an unlikely addition to the standard library.
>
> Rewritten:
>
> If Fredrik Lundh were supportive of the idea, I think the Standard
> Library would benefit greatly from incorporating ElementTree.

shipping stable versions of ElementTree/cElementTree (or PIL, or python-
doc, or exemaker, or what else you might find useful) with official Python
releases is perfectly okay.

moving the main trunk and main development over to the Python CVS is
another thing, entirely.

</F> 




From fredrik at pythonware.com  Wed Jun  8 14:40:52 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 8 Jun 2005 14:40:52 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
References: <42A4C65F.6070301@v.loewis.de><000b01c56ae4$8afdbe20$7105a044@oemcomputer>
	<d86o5t$euh$1@sea.gmane.org>
Message-ID: <d86op2$gu2$1@sea.gmane.org>

> moving the main trunk and main development over to the Python CVS is
> another thing, entirely.

(as I've said many times before, both the user community and the developer
community would benefit if the core standard library were made smaller, and
more externally maintained packages were included in the standard releases)

(the development platform could need an overhaul as well; cvs/sourceforge/tex
is a serious bottleneck.  but the former doesn't require the latter).

</F> 




From amk at amk.ca  Wed Jun  8 22:57:34 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 8 Jun 2005 16:57:34 -0400
Subject: [Python-Dev] Bug day on the 25th?
Message-ID: <20050608205734.GA13329@rogue.amk.ca>

It seems like a good idea to have another Python bug day.  Saturday
June 25th seems the most practical date (this coming weekend is too
soon, and the weekend after is a minor holiday -- Father's Day).  
We'd convene in the usual place: the #pydotorg IRC channel, on
irc.freenode.net.

Assuming no one can think of a reason why the 25th would be
unsuitable, I'll send an announcement to c.l.py.a in a few days.

--amk


From askpete at gmail.com  Thu Jun  9 05:21:20 2005
From: askpete at gmail.com (Peter Moore)
Date: Thu, 9 Jun 2005 13:21:20 +1000
Subject: [Python-Dev] FishEye on Python CVS Repository
Message-ID: <52431c5005060820217cb1f1fb@mail.gmail.com>

Greetings Python Developers,

I'm responsible for setting up free FishEye hosting for community
projects. As a long time python user I of course added Python up
front.  You can see it here:

  http://fisheye.cenqua.com/viewrep/python/

If you aren't familiar with FishEye, it is a repository browsing,
searching, analysis, monitoring..., tool for CVS (SVN coming soon).
Cool features include:

RSS feeds http://fisheye.cenqua.com/changelog/%7Erss/python/rss.xml

Synthetic changesets http://fisheye.cenqua.com/changelog/python/ and

Pretty ediffs http://fisheye.cenqua.com/viewrep/python/python/nondist/peps/pep-0343.txt?r1=1.22&r2=1.23

SQL like search http://fisheye.cenqua.com/search/python/?ql=

Note that normally FishEye is pretty close to real time (i.e. when run on
a local repository), but we can only get SF updates once a day, and they
are potentially a bit old when we grab them.  Feel free to mail me direct
pete _at_ cenqua _dot_ com if you want to tweak the python instance or
have any other questions/comments not appropriate for this list.

Cheers,
Pete.
---
+61-414-587637
www.cenqua.com/fisheye

From kbk at shore.net  Fri Jun 10 05:03:56 2005
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu, 9 Jun 2005 23:03:56 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200506100303.j5A33uIC002882@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  339 open ( -5) /  2857 closed (+12) /  3196 total ( +7)
Bugs    :  908 open ( -8) /  5036 closed (+22) /  5944 total (+14)
RFE     :  189 open ( -2) /   168 closed ( +5) /   357 total ( +3)

New / Reopened Patches
______________________

Trivial typo in unicodedata._getcode  (2005-06-02)
       http://python.org/sf/1213831  opened by  Darek Suchojad

Support non-file source/dest in marshal  (2005-06-04)
       http://python.org/sf/1214879  opened by  Skip Montanaro

file.encoding support for file.write and file.writelines  (2005-06-04)
       http://python.org/sf/1214889  opened by  Reinhold Birkenfeld

two fileinput enhancements (fileno, openhook)  (2005-06-05)
       http://python.org/sf/1215184  opened by  Reinhold Birkenfeld

Suggested Additional Material for urllib2 docs  (2005-06-08)
       http://python.org/sf/1216942  opened by  Mike Foord

proposed patch for tls wrapped ssl support added to smtplib  (2005-06-08)
       http://python.org/sf/1217246  opened by  Pete G

Patches Closed
______________

[ast] fix for 1183468: return/yield in class  (2005-04-16)
       http://python.org/sf/1184418  closed by  nascheme

[AST] throw SyntaxError in "from x import y,"  (2005-05-04)
       http://python.org/sf/1194895  closed by  nascheme

fix for trivial flatten bug in astgen  (2005-01-04)
       http://python.org/sf/1095541  closed by  nascheme

Add st_flags support to (l)stat function  (2005-06-01)
       http://python.org/sf/1212117  closed by  perky

buffer overflow in _cursesmodule.c  (2005-05-11)
       http://python.org/sf/1200134  closed by  akuchling

Typo in Curses-Function doc  (2005-04-20)
       http://python.org/sf/1186781  closed by  akuchling

binary formats for marshalling floats  (2005-04-11)
       http://python.org/sf/1180995  closed by  mwh

test_locale fix on modern linux  (2005-05-07)
       http://python.org/sf/1197218  closed by  anthonybaxter

An URL for UnicodeData File Format 3.2 has changed.  (2005-05-25)
       http://python.org/sf/1207985  closed by  perky

Patch for [ 1170331 ] Error in base64.b32decode  (2005-03-27)
       http://python.org/sf/1171487  closed by  akuchling

#1074261 gzip dies on gz files with many appended headers  (2004-11-27)
       http://python.org/sf/1074381  closed by  akuchling

asynchat does not accept 'long' terminator  (2004-08-03)
       http://python.org/sf/1002763  closed by  akuchling

New / Reopened Bugs
___________________

Queue.qsize() better info about unreliability  (2005-06-02)
       http://python.org/sf/1213475  opened by  Nikos Kouremenos

os.path.realpath() cannot handle symlinks  (2005-06-02)
CLOSED http://python.org/sf/1213894  opened by  Ilya Sandler

test_marshal.py discards marshal.load val  (2005-06-04)
CLOSED http://python.org/sf/1214662  opened by  Skip Montanaro

subprocess auto-reaps children  (2005-06-04)
       http://python.org/sf/1214859  opened by  Mattias Engdeg?rd

bsddb dbobj.DB.associate doesn't accept dbobj.DB param  (2005-06-04)
       http://python.org/sf/1215023  opened by  Gregory P. Smith

int('x',radix) puzzle  (2005-06-05)
CLOSED http://python.org/sf/1215146  opened by  elgordo

String and list methods deeply hidden  (2005-06-06)
       http://python.org/sf/1215887  opened by  Reinhold Birkenfeld

Large tarfiles cause overflow  (2005-06-06)
       http://python.org/sf/1215928  opened by  Tom Emerson

Replace MSVC memory allocator with ptmalloc2  (2005-06-07)
       http://python.org/sf/1216562  opened by  Niall Douglas

csv module sometimes raises _csv.Error  (2005-06-08)
CLOSED http://python.org/sf/1216831  opened by  Mary Gardiner

LINKCC->CXX, -ltermcap->-lncurses  (2005-06-08)
       http://python.org/sf/1216923  opened by  Niki W. Waibel

Info from man python not in docs  (2005-06-08)
       http://python.org/sf/1217152  opened by  Kent Johnson

Omission in docs for smtplib.SMTP.sendmail()  (2005-06-09)
       http://python.org/sf/1217513  opened by  Kent Johnson

make frameworkinstall fails for non-default location  (2005-06-09)
       http://python.org/sf/1217591  opened by  Mitch Chapman

pydoc includes unnecessary files for a package.  (2005-06-09)
       http://python.org/sf/1217881  opened by  Igor Belyi

Bugs Closed
___________

check for return/yield outside function is wrong  (2005-04-15)
       http://python.org/sf/1183468  closed by  nascheme

Incorrect result for regular expression - "|(hello)|(world)"  (2005-06-01)
       http://python.org/sf/1212411  closed by  tim_one

doc bug in Lock.acquire  (2005-05-27)
       http://python.org/sf/1209880  closed by  akuchling

(?(id)yes|no) only works when referencing the first group  (2005-04-06)
       http://python.org/sf/1177831  closed by  akuchling

Notation  (2005-04-30)
       http://python.org/sf/1193001  closed by  akuchling

os.path.realpath() cannot handle symlinks  (2005-06-03)
       http://python.org/sf/1213894  closed by  birkenfeld

bz2.BZ2File doesn't handle modes correctly  (2005-05-03)
       http://python.org/sf/1194181  closed by  birkenfeld

test_marshal.py discards marshal.load val  (2005-06-04)
       http://python.org/sf/1214662  closed by  montanaro

WeakValueDictionary.__init__ is backwards  (2005-05-06)
       http://python.org/sf/1196315  closed by  birkenfeld

Typo in "Differences from mimelib"  (2005-05-27)
       http://python.org/sf/1210001  closed by  birkenfeld

IDLE 1.0.5 (Python 2.3.5) crashes under Windows  (2005-05-21)
       http://python.org/sf/1206232  closed by  kbk

example broken in section 1.12 of Extending & Embedding  (2005-04-16)
       http://python.org/sf/1184380  closed by  birkenfeld

try to open /dev/null as directory  (2005-04-15)
       http://python.org/sf/1183585  closed by  birkenfeld

int('x',radix) puzzle  (2005-06-05)
       http://python.org/sf/1215146  closed by  birkenfeld

compileall doesn't notice syntax errors  (2001-03-30)
       http://python.org/sf/412436  closed by  birkenfeld

doc speaks of extensions option while it's *called* ext_modu  (2005-03-25)
       http://python.org/sf/1170422  closed by  akuchling

dumbdbm hoses index on load failure  (2005-03-29)
       http://python.org/sf/1172763  closed by  akuchling

csv module sometimes raises _csv.Error  (2005-06-07)
       http://python.org/sf/1216831  closed by  montanaro

error in documentation for splitting empty string   (2005-03-28)
       http://python.org/sf/1171994  closed by  akuchling

Error in base64.b32decode  (2005-03-24)
       http://python.org/sf/1170331  closed by  akuchling

gzip dies on gz files with many appended headers  (2004-11-27)
       http://python.org/sf/1074261  closed by  akuchling

datetime changes missing from "Porting from 2.3 to 2.4"  (2004-12-04)
       http://python.org/sf/1079134  closed by  akuchling

New / Reopened RFE
__________________

Move test_method_checksum from test_unicodedata.  (2005-06-02)
CLOSED http://python.org/sf/1213703  opened by  Darek Suchojad

module warnings lacks a remove filter function  (2005-06-04)
       http://python.org/sf/1214675  opened by  Torsten Bronger

Add Error Code Dictionary to urllib2  (2005-06-08)
       http://python.org/sf/1216944  opened by  Mike Foord

RFE Closed
__________

Move test_method_checksum from test_unicodedata.  (2005-06-02)
       http://python.org/sf/1213703  closed by  rhettinger

Prepending Text  (2005-05-31)
       http://python.org/sf/1212169  closed by  rhettinger

The array module and the buffer interface  (2005-04-26)
       http://python.org/sf/1190033  closed by  rhettinger

expm1 and log1p  (2005-03-14)
       http://python.org/sf/1163139  closed by  rhettinger

Support file path concat with &quot;/&quot;   (2002-11-27)
       http://python.org/sf/644940  closed by  birkenfeld


From kentsin at gmail.com  Thu Jun  9 10:13:31 2005
From: kentsin at gmail.com (kent sin)
Date: Thu, 9 Jun 2005 16:13:31 +0800
Subject: [Python-Dev] Example workaround classes for using Unicode with
	csv module...
Message-ID: <aa879429050609011316d1e3f3@mail.gmail.com>

The suggestion Skip is indeed very useful, however it does not work
when some of the item is not string, here is another try:

class UnicodeReader:
    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        self.reader = csv.reader(f, dialect=dialect, **kwds)
        self.encoding = encoding

    def next(self):
        row = self.reader.next()
        t = []
        for s in row:
            try:
                t.append(unicode(s,self.encoding))
            except:
                t.append(s)
        return t 
        # [unicode(s, self.encoding) for s in row] This will not work
with non string type

    def __iter__(self):
        return self

class UnicodeWriter:
    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        self.writer = csv.writer(f, dialect=dialect, **kwds)
        self.encoding = encoding

    def writerow(self, row):
        t = []
        for s in row:
            try:
                t.append(unicode(s,"utf-8"))
            except:
                t.append(s)
        self.writer.writerow(t)               
            
        #self.writer.writerow([s.encode("utf-8") for s in row]) #!
This is not working with non-string objects.
        

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)
-- 
Sin Hang Kin.

From pp64 at codelock.com  Wed Jun  8 05:23:18 2005
From: pp64 at codelock.com (Pavel Pergamenshchik)
Date: Tue, 7 Jun 2005 20:23:18 -0700
Subject: [Python-Dev] b32encode and NUL bytes
Message-ID: <20050607202318.45a459c2@dunce>

Hi.
Is this a feature? I do see b32encode padding the string with NULs first.

>>> b32decode(b32encode('\x00'))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/base64.py", line 228, in b32decode
    last = binascii.unhexlify(hex(acc)[2:-1])
TypeError: Odd-length string

From skip at pobox.com  Fri Jun 10 14:23:40 2005
From: skip at pobox.com (Skip Montanaro)
Date: Fri, 10 Jun 2005 07:23:40 -0500
Subject: [Python-Dev] [Python-checkins] python/dist/src/Doc/lib
	libtokenize.tex, 1.5, 1.6
In-Reply-To: <E1DghKC-00011r-Tq@sc8-pr-cvs1.sourceforge.net>
References: <E1DghKC-00011r-Tq@sc8-pr-cvs1.sourceforge.net>
Message-ID: <17065.34380.208609.448951@montanaro.dyndns.org>


    Raymond> Add untokenize() function to allow full round-trip tokenization.

    Raymond> Should significantly enhance the utility of the module by
    Raymond> supporting the creation of tools that modify the token stream
    Raymond> and writeback the modified result.

Raymond,

Very interesting.

Skip

From gustavo at niemeyer.net  Fri Jun 10 16:35:47 2005
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Fri, 10 Jun 2005 11:35:47 -0300
Subject: [Python-Dev] Bug day on the 25th?
In-Reply-To: <20050608205734.GA13329@rogue.amk.ca>
References: <20050608205734.GA13329@rogue.amk.ca>
Message-ID: <20050610143546.GA5929@burma.localdomain>

Greetings,

> It seems like a good idea to have another Python bug day.  Saturday
> June 25th seems the most practical date (this coming weekend is too
> soon, and the weekend after is a minor holiday -- Father's Day).  
> We'd convene in the usual place: the #pydotorg IRC channel, on
> irc.freenode.net.
> 
> Assuming no one can think of a reason why the 25th would be
> unsuitable, I'll send an announcement to c.l.py.a in a few days.

Not sure if that's a reason to prevent the event, since
fixing bugs is a good thing no matter when, but that's two
days before EuroPython, and many people might be moving to
the conference at that time.

-- 
Gustavo Niemeyer
http://niemeyer.net

From gustavo at niemeyer.net  Fri Jun 10 16:48:35 2005
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Fri, 10 Jun 2005 11:48:35 -0300
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d86op2$gu2$1@sea.gmane.org>
References: <d86o5t$euh$1@sea.gmane.org> <d86op2$gu2$1@sea.gmane.org>
Message-ID: <20050610144835.GB5929@burma.localdomain>

> > moving the main trunk and main development over to the Python CVS is
> > another thing, entirely.
> 
> (as I've said many times before, both the user community and the developer
> community would benefit if the core standard library were made smaller, and
> more externally maintained packages were included in the standard releases)

An issue to consider about this is that maintainers (not talking about
you or anyone else specifically) have different concepts of stability,
and while it may seem perfectly ok to refactor external modules between
two stable releases, doing so in the standard library would spread fear
and "python is so untrustful" feelings.

That's something a good police, reflecting practices that we learn
"by osmose" while living in that environment (python-dev) for a while,
could try to handle. Enforcing it is another issue, of course.

-- 
Gustavo Niemeyer
http://niemeyer.net

From amk at amk.ca  Fri Jun 10 17:23:00 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 10 Jun 2005 11:23:00 -0400
Subject: [Python-Dev] Bug day on the 25th?
In-Reply-To: <20050610143546.GA5929@burma.localdomain>
References: <20050608205734.GA13329@rogue.amk.ca>
	<20050610143546.GA5929@burma.localdomain>
Message-ID: <20050610152300.GA28316@rogue.amk.ca>

On Fri, Jun 10, 2005 at 11:35:47AM -0300, Gustavo Niemeyer wrote:
> Not sure if that's a reason to prevent the event, since
> fixing bugs is a good thing no matter when, but that's two
> days before EuroPython, and many people might be moving to
> the conference at that time.

Thanks for pointing that out.  It'll probably mean that most of the
participants will be North Americans.  The next available weekend
would be July 9th, though, so I think we should go ahead with the bug
day on the 25th.  I'll try to have another one in July, too.

Maybe we should adopt a standard day for bug days, e.g. second
Saturday of the month, or something like that.

--amk


From mdehoon at c2b2.columbia.edu  Fri Jun 10 18:59:03 2005
From: mdehoon at c2b2.columbia.edu (Michiel De Hoon)
Date: Fri, 10 Jun 2005 12:59:03 -0400
Subject: [Python-Dev] Five patch reviews & patch request
Message-ID: <6CA15ADD82E5724F88CB53D50E61C9AE7AC151@cgcmail.cgc.cpmc.columbia.edu>

Recently, I reviewed the following five patches:

[  827386 ] absolute paths cause problems for MSVC
[ 1161914 ] python-config
[ 1187396 ] Add const specifier to PySpam_System prototype
[ 1196895 ] in IDLE, assume new text files are python source by default
[ 1196917 ] change recall in IDLE shell to not overwrite current command

Below are the summaries of the patch reviews.

I'd like to ask your attention to this patch:

[ 1049855 ] PyOS_InputHook inconsistency on Windows

Kurt Kaiser has already looked at this patch, but was hesitant to check it in
because he wasn't able to test it on Windows. I'd be happy to send a compiled
Python for Windows to anybody willing to have a look at this patch.

Thanks!

--Michiel.


Patch review summaries:
-----------------------

[ 827386 ] absolute paths cause problems for MSVC
Solves an inconsistency between distutils' msvccompiler.py and ccompiler.py.
I am surprised by how absolute paths are handled in ccompiler.py, so I'm not
wholeheartedly in favor of copying this behavior to msvccompiler.py.

[ 1161914 ] python-config
Currently, the patch seems to be broken. Better integration with distutils is
needed. Can't recommend this patch in its current form.

[ 1187396 ] Add const specifier to PySpam_System prototype
This error is already fixed in Revision 1.33 of
python/dist/src/Doc/ext/extending.tex. Patch can be closed.

[ 1196895 ] in IDLE, assume new text files are python source by default
When going to File > New in IDLE, currently IDLE assumes that the new file is
a generic text file. With the patch, IDLE assumes that the new file contains
python source code by default, and uses syntax highlighting. Whereas
currently syntax highlighting is switched on as soon as the file is saved as
*.py, after the patch syntax highlighting is not switched off if the file is
saved as *.txt. So the patch does not seem to be perfectly OK.

[ 1196917 ] change recall in IDLE shell to not overwrite current command
I agree that this patch makes IDLE easier to use. Recommend applying this
patch.


Michiel de Hoon
Center for Computational Biology and Bioinformatics
Columbia University
1150 St Nicholas Avenue
New York, NY 10032


From theller at python.net  Fri Jun 10 19:46:00 2005
From: theller at python.net (Thomas Heller)
Date: Fri, 10 Jun 2005 19:46:00 +0200
Subject: [Python-Dev] Five patch reviews & patch request
In-Reply-To: <6CA15ADD82E5724F88CB53D50E61C9AE7AC151@cgcmail.cgc.cpmc.columbia.edu>
	(Michiel De Hoon's message of "Fri, 10 Jun 2005 12:59:03 -0400")
References: <6CA15ADD82E5724F88CB53D50E61C9AE7AC151@cgcmail.cgc.cpmc.columbia.edu>
Message-ID: <aclyqf53.fsf@python.net>

"Michiel De Hoon" <mdehoon at c2b2.columbia.edu> writes:

> I'd like to ask your attention to this patch:
>
> [ 1049855 ] PyOS_InputHook inconsistency on Windows
>
> Kurt Kaiser has already looked at this patch, but was hesitant to check it in
> because he wasn't able to test it on Windows. I'd be happy to send a compiled
> Python for Windows to anybody willing to have a look at this patch.

Comment about the patch (only from reading it):

The PyOS_StdioReadline function calls my_fgets with a file pointer
argument.  The my_fgets function in the patch assumes that
STD_INPUT_HANDLE is the handle to use - is this assumption always
correct?

Thomas


From jepler at unpythonic.net  Fri Jun 10 20:13:35 2005
From: jepler at unpythonic.net (Jeff Epler)
Date: Fri, 10 Jun 2005 13:13:35 -0500
Subject: [Python-Dev] b32encode and NUL bytes
In-Reply-To: <20050607202318.45a459c2@dunce>
References: <20050607202318.45a459c2@dunce>
Message-ID: <20050610181335.GB6998@unpythonic.net>

On Tue, Jun 07, 2005 at 08:23:18PM -0700, Pavel Pergamenshchik wrote:
> Hi.
> Is this a feature? I do see b32encode padding the string with NULs first.
> 
> >>> b32decode(b32encode('\x00'))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "/usr/lib/python2.4/base64.py", line 228, in b32decode
>     last = binascii.unhexlify(hex(acc)[2:-1])
> TypeError: Odd-length string

This also seems suspect:
>>> base64.b32encode("\0a")
'ABQQ===='
>>> base64.b32decode(_)
'a'

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20050610/36128810/attachment.pgp

From gvanrossum at gmail.com  Fri Jun 10 22:23:04 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 10 Jun 2005 13:23:04 -0700
Subject: [Python-Dev] PEP 343 - next steps
Message-ID: <ca471dc2050610132375ca7c70@mail.gmail.com>

While there's still some activity in the Wiki, nothing (to me) sounds
like signs of serious disagreement or truly better alternatives. So I
think I'd like to move forward towards acceptance soon (before
EuroPython).

Two issues brought up in the Wiki are worth considering:

- throw() is a term taken from Java & C++. We can't call the method
raise() -- but perhaps we can call it next_raising() or next_raise(),
which emphasizes the similarity with next(). Thoughts? I'm not strong
on this; I think throw() is fine too, especially since I expect that
it will be used explicitly extremely rarely -- the only customer is
the with_template decorator.

- Whether and how to keep a door open for a future extension to the
syntax that allows multiple resources to be acquired in a single
with-statement. Possible syntax could be

(a)    with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...:

or

(b)    with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...:

Variant (a) seems better and is more analogous to the use of 'as' in
import statements, and (b) has the disadvantage that if you want to
acquire several resources and not all of them have an associated
variable, you'll have to sprinkle dummy variables on the right of
'as'. So (a) would have my preference. But I would still like to start
off without this extension. The issue is: if we allow VAR to be a
comma-separated list of variables now, that cuts off the extension to
(a) in the future; so the PEP would have to be amended to state that
VAR must be a single variable or a list of variables IN PARENTHESES.
Thoughts?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Fri Jun 10 22:49:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 10 Jun 2005 16:49:13 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050610164649.0218fea0@mail.telecommunity.com>

At 01:23 PM 6/10/2005 -0700, Guido van Rossum wrote:
>- throw() is a term taken from Java & C++. We can't call the method
>raise() -- but perhaps we can call it next_raising() or next_raise(),
>which emphasizes the similarity with next(). Thoughts? I'm not strong
>on this; I think throw() is fine too, especially since I expect that
>it will be used explicitly extremely rarely -- the only customer is
>the with_template decorator.

I'm fine with throw, but if you really want to get 'raise' in there, how 
about 'raise_exc' or 'raise_exc_info'?


>The issue is: if we allow VAR to be a
>comma-separated list of variables now, that cuts off the extension to
>(a) in the future; so the PEP would have to be amended to state that
>VAR must be a single variable or a list of variables IN PARENTHESES.
>Thoughts?

Parentheses would make it clearer what's going on, so I'd be fine with that.


From python at rcn.com  Sat Jun 11 00:49:50 2005
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 10 Jun 2005 18:49:50 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <002d01c56e0e$d9699520$fa2ec797@oemcomputer>

> - throw() is a term taken from Java & C++.

This was intended.  There was much discussion about this for PEP 288 and
this was more or less a concensus choice.  The term is already
associated with exceptions in other languages and it captures the
concept of the raise occurring somewhere else (what is thrown at or
into).  It is simple, clear, and tends to suggest all the right things.

I have no doubt that someone can point out differing semantics in other
languages but I don't care.  The verbal cue is what we are after.


Raymond





From rowen at cesmail.net  Sat Jun 11 01:11:34 2005
From: rowen at cesmail.net (Russell E. Owen)
Date: Fri, 10 Jun 2005 16:11:34 -0700
Subject: [Python-Dev] PEP 343 - next steps
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <rowen-AD0A76.16113410062005@sea.gmane.org>

In article <ca471dc2050610132375ca7c70 at mail.gmail.com>,
 Guido van Rossum <gvanrossum at gmail.com> wrote:

>...
> - Whether and how to keep a door open for a future extension to the
> syntax that allows multiple resources to be acquired in a single
> with-statement. Possible syntax could be
> 
> (a)    with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...:
> 
> or
> 
> (b)    with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...:
> 
> Variant (a) seems better and is more analogous to the use of 'as' in
> import statements, and (b) has the disadvantage that if you want to
> acquire several resources and not all of them have an associated
> variable, you'll have to sprinkle dummy variables on the right of
> 'as'. So (a) would have my preference. But I would still like to start
> off without this extension. The issue is: if we allow VAR to be a
> comma-separated list of variables now, that cuts off the extension to
> (a) in the future; so the PEP would have to be amended to state that
> VAR must be a single variable or a list of variables IN PARENTHESES.
> Thoughts?

I agree that (a) sounds better.

Also, I think it is very reasonable to require that multiple variables 
be in parenthesis. You can always lift the restriction later (i.e. if 
enough folks complain and you decide never to implement (a)), but you 
can't add such a restriction later.

One nit-pick: I suggest not using the term "list of <anything> in 
parenthesis" in the PEP or other docs. It's clear here but may add to 
some people's list vs tuple confusion.

-- Russell


From gvanrossum at gmail.com  Sat Jun 11 02:05:14 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 10 Jun 2005 17:05:14 -0700
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <002d01c56e0e$d9699520$fa2ec797@oemcomputer>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
	<002d01c56e0e$d9699520$fa2ec797@oemcomputer>
Message-ID: <ca471dc205061017057349ca0a@mail.gmail.com>

On 6/10/05, Raymond Hettinger <python at rcn.com> wrote:
> > - throw() is a term taken from Java & C++.
> 
> This was intended.  There was much discussion about this for PEP 288 and
> this was more or less a concensus choice.  The term is already
> associated with exceptions in other languages and it captures the
> concept of the raise occurring somewhere else (what is thrown at or
> into).  It is simple, clear, and tends to suggest all the right things.
> 
> I have no doubt that someone can point out differing semantics in other
> languages but I don't care.  The verbal cue is what we are after.

Cool. throw() it is.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From bac at OCF.Berkeley.EDU  Sat Jun 11 07:02:20 2005
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri, 10 Jun 2005 22:02:20 -0700
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc2050610132375ca7c70@mail.gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <42AA705C.4010408@ocf.berkeley.edu>

Guido van Rossum wrote:
[SNIP - Guido already said throw() is the name to be used]
> - Whether and how to keep a door open for a future extension to the
> syntax that allows multiple resources to be acquired in a single
> with-statement. Possible syntax could be
> 
> (a)    with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...:
> 
> or
> 
> (b)    with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...:
> 
> Variant (a) seems better and is more analogous to the use of 'as' in
> import statements, and (b) has the disadvantage that if you want to
> acquire several resources and not all of them have an associated
> variable, you'll have to sprinkle dummy variables on the right of
> 'as'. So (a) would have my preference. But I would still like to start
> off without this extension. The issue is: if we allow VAR to be a
> comma-separated list of variables now, that cuts off the extension to
> (a) in the future; so the PEP would have to be amended to state that
> VAR must be a single variable or a list of variables IN PARENTHESES.
> Thoughts?
> 

I like a), so I say restrict the grammar as needed to allow it become a
possibility.  Forcing parens is a minor thing and since this is not
syntactically the same as assignment the difference is negligible.

-Brett

From amk at amk.ca  Sat Jun 11 13:08:35 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Sat, 11 Jun 2005 07:08:35 -0400
Subject: [Python-Dev] b32encode and NUL bytes
In-Reply-To: <20050610181335.GB6998@unpythonic.net>
References: <20050607202318.45a459c2@dunce>
	<20050610181335.GB6998@unpythonic.net>
Message-ID: <20050611110835.GA2217@rogue.amk.ca>

On Fri, Jun 10, 2005 at 01:13:35PM -0500, Jeff Epler wrote:
> > Is this a feature? I do see b32encode padding the string with NULs first.

This is bug #1170331, which was fixed when I applied patch #1171487
earlier this week.

> This also seems suspect:
> >>> base64.b32encode("\0a")
> 'ABQQ===='
> >>> base64.b32decode(_)
> 'a'

This case is also fixed in CVS.

--amk

From ark-mlist at att.net  Sat Jun 11 16:09:26 2005
From: ark-mlist at att.net (Andrew Koenig)
Date: Sat, 11 Jun 2005 10:09:26 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <002c01c56e8f$2d6ca240$6402a8c0@arkdesktop>

> The issue is: if we allow VAR to be a
> comma-separated list of variables now, that cuts off the extension to
> (a) in the future; so the PEP would have to be amended to state that
> VAR must be a single variable or a list of variables IN PARENTHESES.
> Thoughts?

I am not sure how relevant this is, but the syntax "x as (a, b)" sure looks
to me like (a, b) is a tuple.

Of course, I'm biased by ML, which has a feature that I wish Python had:

	fun f(x as (a, b)) = ...

This says that f's argument must be a 2-element tuple.  It binds x to the
argument and also binds a and b to the argument's components.

Of course the program could be written this way:

	fun f(x) = let val (a, b) = x in ... end

but the "as" syntax is more succinct, direct, and convenient.

If such a feature were implemented in Python, I would imagine it to allow
usages such as

	x as (a, b) = (3, 4)

which would be equivalent to

	x = (a, b) = (3, 4)

Of course, this usage shows that the syntax is unnecessary in this context,
but what I care about is

	def f(x as (a, b)):
		# ...

which has the advantage over the alternative

	def f(x):
		(a, b) = x
		# ...

that if you call f with the wrong arguments, you get a clearer diagnostic
message.

It's kind of an edge case, and I am not seriously pushing for its adoption,
but I do think it worth pointing out that when I see "x as (a, b)", that's
what it immediately brings to mind.



From tanzer at swing.co.at  Sat Jun 11 16:15:16 2005
From: tanzer at swing.co.at (tanzer@swing.co.at)
Date: Sat, 11 Jun 2005 16:15:16 +0200
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: Your message of "Sat, 11 Jun 2005 10:09:26 EDT."
	<002c01c56e8f$2d6ca240$6402a8c0@arkdesktop> 
Message-ID: <E1Dh6lY-0006Ry-E6@swing.co.at>

"Andrew Koenig" <ark-mlist at att.net> wrote:

> Of course, this usage shows that the syntax is unnecessary in this context,
> but what I care about is
> 
> 	def f(x as (a, b)):
> 		# ...
> 
> which has the advantage over the alternative
> 
> 	def f(x):
> 		(a, b) = x
> 		# ...
> 
> that if you call f with the wrong arguments, you get a clearer diagnostic
> message.

Thanks to the time machine, you can already do this:

 	def f((a, b)):
		# ...

-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From ncoghlan at gmail.com  Sat Jun 11 16:24:18 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2005 00:24:18 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <002c01c56e8f$2d6ca240$6402a8c0@arkdesktop>
References: <002c01c56e8f$2d6ca240$6402a8c0@arkdesktop>
Message-ID: <42AAF412.8060006@gmail.com>

Andrew Koenig wrote:
> Of course, this usage shows that the syntax is unnecessary in this context,
> but what I care about is
> 
> 	def f(x as (a, b)):
> 		# ...
> 
> which has the advantage over the alternative
> 
> 	def f(x):
> 		(a, b) = x
> 		# ...
> 
> that if you call f with the wrong arguments, you get a clearer diagnostic
> message.

Doesn't tuple unpacking give you what you want here already?

Py> def f((a, b)):
...   print a, b
...
Py> f(1, 2)
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: f() takes exactly 1 argument (2 given)
Py> f((1, 2))
1 2

A couple of more complex examples:

Py> def f((a, b), (c, d)):
...   print a, b, c, d
...
Py> f((1, 2), (3, 4))
1 2 3 4

Py> def f((a, b, (c, d))):
...   print a, b, c, d
...
Py> f((1, 2, (3, 4)))
1 2 3 4

About the only downside is the need to rebuild the tuple if you 
actually need it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From ncoghlan at gmail.com  Sat Jun 11 16:56:54 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2005 00:56:54 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc2050610132375ca7c70@mail.gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <42AAFBB6.2070802@gmail.com>

Guido van Rossum wrote:
> While there's still some activity in the Wiki, nothing (to me) sounds
> like signs of serious disagreement or truly better alternatives. So I
> think I'd like to move forward towards acceptance soon (before
> EuroPython).

I agree with keeping throw() as the exception injection method name, 
and the addition of required parentheses when using tuple unpacking 
when creating VAR.

I also added a new question to the Wiki regarding what, if any, 
guarantees will be made regarding when (or if) asynchronous interrupts 
will be blocked. Given that the call to __enter__() can invoke 
arbitrary Python code, is pushing the with statement setup inside a 
single opcode actually sufficient?

It also considers the possibility of using with statements in an RAII 
style by acquiring the resource in __init__ or __new__ rather than
__enter__.

This is a topic that got a brief mention during early discussion here, 
but doesn't seem to be covered in the current version of PEP 343.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com


From stephen at xemacs.org  Sat Jun 11 18:01:09 2005
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 12 Jun 2005 01:01:09 +0900
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <20050610144835.GB5929@burma.localdomain> (Gustavo Niemeyer's
	message of "Fri, 10 Jun 2005 11:48:35 -0300")
References: <d86o5t$euh$1@sea.gmane.org> <d86op2$gu2$1@sea.gmane.org>
	<20050610144835.GB5929@burma.localdomain>
Message-ID: <877jh0x4qi.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Gustavo" == Gustavo Niemeyer <gustavo at niemeyer.net> writes:

    Gustavo> An issue to consider about this is that maintainers (not
    Gustavo> talking about you or anyone else specifically) have
    Gustavo> different concepts of stability, and while it may seem
    Gustavo> perfectly ok to refactor external modules between two
    Gustavo> stable releases, doing so in the standard library would
    Gustavo> spread fear and "python is so untrustful" feelings.

This simply hasn't been a problem in XEmacs's equivalent to the
standard library.  In fact, the library modules tend to be
more stable than the core, for several reasons.  One important one is
that the modules we distribute are not necessarily the bleeding edge.
In particular, we try to keep up (within a couple weeks) with bugfixes
to stable lines, but often lag several months after a major refactoring.

This multitrack approach does involve some extra administrative work,
but the burden on the core team is minimal.  Most modules are managed
by volunteers who do not feel able to contribute to the core (and
often not to the coding of "their" module), but are very pleased to be
able to contribute in this way.  They are usually far more competent
than the core developers to judge when new features are sufficiently
attractive and stable to warrant updating to a refactored version,
too.  They tend to be more conservative than the module's lead
maintainer, too.

I will grant that XEmacs deserves (and probably has<wink>) a lower
trust rating than Python, but that is due to flaws in the _core_, not
in the package management.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From gvanrossum at gmail.com  Sat Jun 11 18:02:39 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sat, 11 Jun 2005 09:02:39 -0700
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42AAFBB6.2070802@gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
	<42AAFBB6.2070802@gmail.com>
Message-ID: <ca471dc205061109025a13c846@mail.gmail.com>

[Nick]
> I also added a new question to the Wiki regarding what, if any,
> guarantees will be made regarding when (or if) asynchronous interrupts
> will be blocked. Given that the call to __enter__() can invoke
> arbitrary Python code, is pushing the with statement setup inside a
> single opcode actually sufficient?

Not sufficient if __enter__ is written in Python; in that case,
*nothing* can be done to make it sufficient.

The single opcode makes it possible to implement __enter__ in C to
solve the issue in specific cases.

> It also considers the possibility of using with statements in an RAII
> style by acquiring the resource in __init__ or __new__ rather than
> __enter__.

Python isn't C++; you shouldn't do that.

> This is a topic that got a brief mention during early discussion here,
> but doesn't seem to be covered in the current version of PEP 343.

I haven't been infected with the RAII meme, so I can't help you here.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Sat Jun 11 21:09:59 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 11 Jun 2005 15:09:59 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc205061109025a13c846@mail.gmail.com>
References: <42AAFBB6.2070802@gmail.com>
	<ca471dc2050610132375ca7c70@mail.gmail.com>
	<42AAFBB6.2070802@gmail.com>
Message-ID: <5.1.1.6.0.20050611143158.01f73808@mail.telecommunity.com>

At 09:02 AM 6/11/2005 -0700, Guido van Rossum wrote:
>[Nick]
> > I also added a new question to the Wiki regarding what, if any,
> > guarantees will be made regarding when (or if) asynchronous interrupts
> > will be blocked. Given that the call to __enter__() can invoke
> > arbitrary Python code, is pushing the with statement setup inside a
> > single opcode actually sufficient?
>
>Not sufficient if __enter__ is written in Python; in that case,
>*nothing* can be done to make it sufficient.

I think PEP 343 would greatly benefit from some discussion of the 
signalling issue.  I've been wracking my brain a bit on it, and I'm having 
trouble seeing where the problem is.


>The single opcode makes it possible to implement __enter__ in C to
>solve the issue in specific cases.

Or if __enter__ is a no-op, that would also solve the problem.  Suitability 
for 'with' could be keyed off of the __exit__ method instead.  If there's 
no __enter__, or the __enter__ is a default no-op __enter__ written in C, 
then __exit__ is sufficient for many resources.


> > It also considers the possibility of using with statements in an RAII
> > style by acquiring the resource in __init__ or __new__ rather than
> > __enter__.
>
>Python isn't C++; you shouldn't do that.

Actually, it seems to me that it's standard procedure in today's Python: 
objects allocate resources in __init__, and then have a 'close()' or some 
similar method to release them.  That's why so many people see this:

     with open('foo') as f:
         ...

As a natural use of the 'with' statement, whether they have any familiarity 
with C++ or not.  (My own experiences with C++ are perhaps a decade out of 
date, long before RAII was a meme in any event.)

It seems to me that there are two sensible patterns for using 'with':

  1. __enter__ allocates or acquires a resource, __exit__ releases it

  2. EXPR allocates or acquires a resource, __exit__ releases it  (and 
__enter__ doesn't execute any Python, because it's a no-op)

Pattern 1 is useful for transactions, locks, redirection, decimal contexts, 
etc.
Pattern 2 is useful for resources that are acquired by creation anyway, 
like files.

But maybe my understanding is flawed, because I don't *really* understand 
the signal issue.  I've been assuming that you're talking about the fact 
that an exception can occur anywhere, due to e.g. KeyboardInterrupt or an 
exception raised by some other signal handler.  It seems to me that using 
pattern 2, one could write 'closing()' safely
by doing this:

     class closing(object):
         def __init__(self, resource):
             try:
                 self.resource = resource
             except:
                 resource.close()
                 raise

         def __exit__(self,*exc):
             self.resource.close()

but maybe I'm still fundamentally "not getting" how the signal issue 
manifests in practice.  I suppose it's true that *another* exception could 
then occur between the LOAD_ATTR and the CALL opcodes in the exception 
handler here, but it's also true that this could happen with any Python 
exception-handling code, and it could also happen  in __exit__().  So I'm 
not sure I see why this should influence the design of 'with' very much, 
other than to point out that no Python code is safe from asynchronous 
exceptions.

In any case, C-implemented types can handle this situation just fine, so a 
'closing' written in C could easily be included with the implementation.


From barry at python.org  Sat Jun 11 23:43:43 2005
From: barry at python.org (Barry Warsaw)
Date: Sat, 11 Jun 2005 17:43:43 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc2050610132375ca7c70@mail.gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <1118526223.9425.127.camel@geddy.wooz.org>

On Fri, 2005-06-10 at 16:23, Guido van Rossum wrote:
> While there's still some activity in the Wiki, nothing (to me) sounds
> like signs of serious disagreement or truly better alternatives. So I
> think I'd like to move forward towards acceptance soon (before
> EuroPython).

Well, I finally read PEP 343 in its entirety.  Having not really had the
time to follow this thread, I have to say "well done".  I like the PEP a
lot and can see many places in my own code where it would make things
more readable.

I have one comment, and I may be too late or may be repeating other
suggestions (no, I haven't poured through the wiki page yet).  I
actually don't think "with" reads as well and would propose "when".  The
word I'd really like to use here is "while" but given that we already
have while loops and "as" is optional, I don't think that's a
possibility.  If you made "as" required, it would be though, I think.

Look at the given examples:

with locking(mylock):
when locking(mylock):

with opening("/etc/passwd") as f:
when opening("/etc/passwd") as f:

with opening(filename, "w") as f:
    with redirecting_stdout(f):
        print "Hello world"

when opening(filename, "w") as f:
    when redirecting_stdout(f):
        print "Hello world"

with signal_blocking():
when signal_blocking():

> - throw() is a term taken from Java & C++. We can't call the method
> raise() -- but perhaps we can call it next_raising() or next_raise(),
> which emphasizes the similarity with next(). Thoughts? I'm not strong
> on this; I think throw() is fine too, especially since I expect that
> it will be used explicitly extremely rarely -- the only customer is
> the with_template decorator.

As others have mentioned "throw()" is fine.

> - Whether and how to keep a door open for a future extension to the
> syntax that allows multiple resources to be acquired in a single
> with-statement. Possible syntax could be
> 
> (a)    with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...:
> 
> or
> 
> (b)    with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...:

I agree that (a) is better and feel +1 on keeping the door open to it.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050611/235c0e15/attachment-0001.pgp

From ncoghlan at gmail.com  Sun Jun 12 03:18:02 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2005 11:18:02 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc205061109025a13c846@mail.gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>	
	<42AAFBB6.2070802@gmail.com>
	<ca471dc205061109025a13c846@mail.gmail.com>
Message-ID: <42AB8D4A.5020407@gmail.com>

Guido van Rossum wrote:
>>It also considers the possibility of using with statements in an RAII
>>style by acquiring the resource in __init__ or __new__ rather than
>>__enter__.
> 
> 
> Python isn't C++; you shouldn't do that.
> 
> 
>>This is a topic that got a brief mention during early discussion here,
>>but doesn't seem to be covered in the current version of PEP 343.
> 
> 
> I haven't been infected with the RAII meme, so I can't help you here.

I expected your reaction to be along these lines, and at one point 
updated the Wiki to say basically this.

Then I went back to PEP 343, and had another look at the examples. The 
one that sprang out at me was the generic 'closing()' template.

Being able to say "ensure method X of object Y is called when the 
block exits" seems like a *really* useful feature for 'with' statements.

However, when using this approach, the resource is always going to be 
acquired during evaluation of EXPR, since it needs to be passed an an 
argument to the 'closing()' (or equivalent) template.

If the guarantee only applies to __enter__ though, then such templates 
would have to be written 'correctly' using deferred evaluation of the 
argument:

   class closing(object):
     def __init__(self, resource_factory)
         self.factory = resource_factory
         self.resource = None

     def __enter__(self):
         if self.resource is not None:
             raise RuntimeException("Cannot nest use of template")
         self.resource = self.factory()
         return self.resource

     def __exit__(self, *exc_info):
       self.resource.close()
       self.resource = None


Then usage gets a bit uglier, because a lambda is needed in order to 
defer evaluation of the argument:

   with closing(lambda: open("somefile.txt", "r")) as f:
     # Do stuff


If, however, two new opcodes were added that blocked and unblocked 
asynchronous exceptions in a given thread, then not only would it be 
possible to implement 'closing()' in the natural way, then resource 
handlers written in Python would 'do the right thing' to.

Specifically, what if we added a new boolean flag 'permit_async_exc' 
to the thread state that is checked at the top of main loop in ceval.c 
where the asynchronous events are handled?

The simplest trick would be to change the line "if (--_Py_Ticker < 0)" 
  to "if ((--_Py_Ticker < 0) && (_Py_Ticker = 0, 
tstate->allow_async_exc))" (using the comma operator to keep 
_Py_Ticker from overflowing in pathological cases).

Then all the two new opcodes (e.g. ALLOW_ASYNC, BLOCK_ASYNC) would 
have to do is set the state of tstate->allow_async_exc appropriately.

Notice that _Py_Ticker keeps decrementing while the async_exceptions 
are blocked, so that any queued events should be handled immediately 
after execution of the BLOCK_ASYNC call (unless the next opcode is 
SETUP_FINALLY, in which case that opcode will be executed first).

The setting of _Py_Ticker to zero, and the check of the tstate field 
only occur when _Py_Ticker goes negative, so they only add to the cost 
of executing the ceval loop when the check interval has expired. 
Compared to the cost of Py_MakePendingCalls, and the check for thread 
context switches, these costs should be negligible.

An alternative implementation would still allow thread switches to 
happen, and prevent just the asynchronous events. Then the ALLOW_ASYNC 
opcode could just force _Py_Ticker to zero in order to ensure prompt 
evaluation of any pending events.

Either way, the code generation for the with statement could simply 
emit BLOCK_ASYNC as the first opcode, then ALLOW_ASYNC as the opcode 
immediate preceding SETUP_FINALLY. Correct behaviour in the face of 
asynchronous events is then guaranteed, even for cases where the 
resource is acquired in EXPR, or when __enter__ is written in Python.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From ncoghlan at gmail.com  Sun Jun 12 03:26:07 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2005 11:26:07 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42AB8D4A.5020407@gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>		<42AAFBB6.2070802@gmail.com>	<ca471dc205061109025a13c846@mail.gmail.com>
	<42AB8D4A.5020407@gmail.com>
Message-ID: <42AB8F2F.7060204@gmail.com>

Nick Coghlan wrote:
> Then all the two new opcodes (e.g. ALLOW_ASYNC, BLOCK_ASYNC) would 
> have to do is set the state of tstate->allow_async_exc appropriately.

Actually, to allow the use of 'with' statements inside functions 
called via EXPR and the call to __enter__, it would be necessary to 
support nesting of calls to BLOCK_ASYNC and ALLOW_ASYNC, so a better 
approach would be to make the field "tstate->block_async_exc", 
incrementing it in BLOCK_ASYNC, and decrementing it in ALLOW_ASYNC.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From ncoghlan at gmail.com  Sun Jun 12 05:26:14 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2005 13:26:14 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42AB8D4A.5020407@gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>		<42AAFBB6.2070802@gmail.com>	<ca471dc205061109025a13c846@mail.gmail.com>
	<42AB8D4A.5020407@gmail.com>
Message-ID: <42ABAB56.1050405@gmail.com>

Nick Coghlan wrote:
> Either way, the code generation for the with statement could simply 
> emit BLOCK_ASYNC as the first opcode, then ALLOW_ASYNC as the opcode 
> immediate preceding SETUP_FINALLY. Correct behaviour in the face of 
> asynchronous events is then guaranteed, even for cases where the 
> resource is acquired in EXPR, or when __enter__ is written in Python.

Coming up out of the implementation weeds, this can be described as 
follows:

"""
Atomic resource acquisition

The resource acquisition line that starts the with statement is 
executed atomically. That is, the Python interpreter prevents 
asynchronous events and automatic context switches while the resource 
is being acquired. Context switches due to the current thread becoming 
blocked (e.g. by attempting to acquire a synchronisation lock) will 
still occur, avoiding deadlock.

Automatic context switches or asynchronous events that would have 
occurred during evaluation of the resource acquisition line are 
deferred so that they occur immediately prior to the execution of the 
contained suite. This ensures that the acquired resource is still 
properly released in the face of an asynchronous exception such as 
KeyboardInterrupt.

To avoid blocking exceptions for an extended period, 'expensive' 
calculations can be carried out on the line preceding the with 
statement, and stored in a local variable for use in the resource 
acquisition:

   tmp = some_expensive_calculation()
   with tmp as rsrc:
     # Perform operations with resource

Note that, in this case, the actual resource acquisition should occur 
in tmp.__enter__, in order to enjoy the benefits of the atomic 
resource acquisition.
"""

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From greg at electricrain.com  Sun Jun 12 05:47:23 2005
From: greg at electricrain.com (Gregory P. Smith)
Date: Sat, 11 Jun 2005 20:47:23 -0700
Subject: [Python-Dev] hashlib - faster md5/sha, adds sha256/512 support
Message-ID: <20050612034723.GE19337@zot.electricrain.com>

I have finished up the hashlib work I started on in feb/march for
patch 1121611 and 935454 after some discussion on this list.  The full
patch including tests and module documentation has been posted in the
sf patch 1121611 as hashlib-008.

I believe it is done and ready and would like to commit it after a
review.  Let the reviewing begin!

For easy viewing, here's what the module documentation looks like:

 http://electricrain.com/greg/hashlib-py25-doc/module-hashlib.html

The code is in the hashlib-008.patch file:  http://redirx.com/?3e19

hashlib incorporates both the 1121611 openssl hash support and the
935454 sha256+sha512 module support into a single hashlib module that
picks the fastest implementation of each algorithm to use.  OpenSSLs
implementations of md5 and sha1 are nearly 2x as fast as the existing
python builtin versions.  The (now legacy) md5 and sha modules are
turned into simple stubs that use hashlib.

The actual sourceforge patch tracker entries:

 https://sourceforge.net/tracker/?func=detail&aid=1121611&group_id=5470&atid=305470
 https://sourceforge.net/tracker/?func=detail&aid=935454&group_id=5470&atid=305470

Greg


From ncoghlan at gmail.com  Sun Jun 12 06:10:04 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2005 14:10:04 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42AB8F2F.7060204@gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>		<42AAFBB6.2070802@gmail.com>	<ca471dc205061109025a13c846@mail.gmail.com>	<42AB8D4A.5020407@gmail.com>
	<42AB8F2F.7060204@gmail.com>
Message-ID: <42ABB59C.5050608@gmail.com>

Nick Coghlan wrote:
> Nick Coghlan wrote:
> 
>>Then all the two new opcodes (e.g. ALLOW_ASYNC, BLOCK_ASYNC) would 
>>have to do is set the state of tstate->allow_async_exc appropriately.
> 
> 
> Actually, to allow the use of 'with' statements inside functions 
> called via EXPR and the call to __enter__, it would be necessary to 
> support nesting of calls to BLOCK_ASYNC and ALLOW_ASYNC, so a better 
> approach would be to make the field "tstate->block_async_exc", 
> incrementing it in BLOCK_ASYNC, and decrementing it in ALLOW_ASYNC.

And, replying to myself yet again. . .

Such an approach would obviously break in the face of an exception 
that occurs between the BLOCK_ASYNC and ALLOW_ASYNC opcodes (async 
exceptions would remain blocked in the thread).

This can be dealt with by storing the question of whether or not async 
exceptions are permitted as part of the *frame* state, rather than the 
thread state.

That is, make the field access be "f->allow_async_events", and have 
ALLOW_ASYNC and BLOCK_ASYNC set or clear that flag, respectively.

The error handling code after the main opcode switch statement can 
then simply set "f->allow_async_exc" back to true in order to ensure 
that handling of async exceptions is resumed correctly in the event of 
try/finally or try/except blocks causing further execution within the 
frame. (Given that only expressions are possible on the resource 
acquisition line, dealing with the WHY_EXPRESSION case seems to be all 
that would be needed).

Regards,
Nick.

P.S. My suggested semantics and the above implementation outline 
should be up on the Wiki shortly.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From ncoghlan at gmail.com  Sun Jun 12 06:52:47 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2005 14:52:47 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <1118526223.9425.127.camel@geddy.wooz.org>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
	<1118526223.9425.127.camel@geddy.wooz.org>
Message-ID: <42ABBF9F.1010801@gmail.com>

Barry Warsaw wrote:
> On Fri, 2005-06-10 at 16:23, Guido van Rossum wrote:
> 
>>While there's still some activity in the Wiki, nothing (to me) sounds
>>like signs of serious disagreement or truly better alternatives. So I
>>think I'd like to move forward towards acceptance soon (before
>>EuroPython).
> 
> 
> Well, I finally read PEP 343 in its entirety.  Having not really had the
> time to follow this thread, I have to say "well done".  I like the PEP a
> lot and can see many places in my own code where it would make things
> more readable.
> 
> I have one comment, and I may be too late or may be repeating other
> suggestions (no, I haven't poured through the wiki page yet).  I
> actually don't think "with" reads as well and would propose "when".

The idea behind 'with' is that the block is executed while holding 
(i.e. 'with') the resource.

I think the '-ing' form of the example templates is a holdover from 
the days when the PEP used the 'do' keyword - I find that past tense 
or noun forms tend to read better than present tense for custom built 
with templates named after the action that occurs on entry:

   # Past tense
   with locked(my_lock):
   with opened(my_file, mode):
   with redirected_stdout(my_stream):

   # Noun forms
   with my_lock:
   with open(my_file, mode) as f:
   with extra_precision():
   with decimal.getcontext() as ctx:


The place where the '-ing' form still makes some sense is when the 
template is named after the action that will occur at the *end* of the 
block:

   with closing(some_gen()) as g:
   with releasing(some_rsrc()) as r:

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From gvanrossum at gmail.com  Sun Jun 12 18:56:13 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun, 12 Jun 2005 09:56:13 -0700
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42ABB59C.5050608@gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
	<42AAFBB6.2070802@gmail.com>
	<ca471dc205061109025a13c846@mail.gmail.com>
	<42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com>
	<42ABB59C.5050608@gmail.com>
Message-ID: <ca471dc205061209561d7c26c9@mail.gmail.com>

I can't make time for this right now, but the more I think about it,
the more I believe the worry about asynchronous exceptions is mostly
irrational. Despite your "proof" that it *can* be implemented I really
don't want to see anything like that implemented -- I doubt that
you've covered all the cases and I don't like inserting new opcodes to
handle a case that most programs don't care about. Declaring something
"atomic" that could span an arbitrary amount of Python code execution
(including calls) just doesn't seem right.

Disregarding asynchronous exceptions, you can already use
with-statements for your beloved RAII pattern, right? I'm still
skeptical how important it is in Python -- its seems something
invented very specifically for C++'s guarantee of local destructor
execution, and I don't expect to see all that many applications for it
in Python. The with-statement is about the reduction of boiler-plate
code involving try-finally, and that is typically not used for
resource release (unless you count locks, where RAII is inappropriate
-- at least in Python).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ducasse at iam.unibe.ch  Sun Jun 12 18:19:12 2005
From: ducasse at iam.unibe.ch (ducasse)
Date: Sun, 12 Jun 2005 18:19:12 +0200
Subject: [Python-Dev] [ANN] Have fun programming with your kids
Message-ID: <d8hn17$909$7@sea.gmane.org>

Hi

If you are a parent, an educator or a programmer having kids this is
for you! After 4 years of work, my new book "Squeak: Learn
programming with Robots" will be out soon.

http://smallwiki.unibe.ch/botsinc/

http://www.amazon.com/exec/obidos/tg/detail/-/1590594916/
qid=1117218524/sr=1-1/ref=sr_1_1/002-5642974-5143261?v=glance&s=books
With Bots Inc you will learn how to program robots in an interactive
environment. Bots Inc proposes three teaching approaches: direct
command of robots, scripting robots and programming robots. The book
contains 24 chapters going step by step over topics with a lot of
examples. Bots Inc is fun but it is not a toy, it teaches you 100%
real programming. Bots Inc is built on top of
the rich open-source multimedia Squeak environment that you can also
discover.

My goal is to explain key elementary programming concepts (such as
loops, abstraction, composition, and conditionals) to novices of all
ages. I believe that learning by experimenting and solving problems
with fun is central to human knowledge acquisition. Therefore, I have
presented programming concepts through simple but not trivial
problems such as drawing golden rectangles or simulating animal
behavior. The ideal reader I have in mind is an individual who wants
to have fun programming. This person may be a teenager or an adult, a
schoolteacher, or somebody teaching programming to children in some
other organization. Such an individual does not have to be fluent in
programming in any language. As a father of two young boys I also
wrote this book for all the parents that want to have fun programming
with their kids in a powerful interactive environment. Programming in
Squeak is an interactive, fun but deep experience. The present book teaches
elementary programming aspect, the following book will introduce a new fun
environment and teach object-oriented programming.

Testimonies

"I am using the version of the book on your web site to teach my
oldest daughter Becca some programming. She absolutely loves it. We
are doing the Bot graphics right. My other kids are showing interest
as well. My Fall semester schedule leaves me with almost no time free
but in the Spring I hope to bring Squeak and your book to our
elementary school's "gifted" program." C. David Shaffer

"I'm using the Bot Lab environment for three years and found it
really valuable in teaching computer science concepts for a young
audience (and even less young !). The bots commanded through balloon
(as in comic strips) is a very nice introduction for young children,
and when this aspect is well understood, you can use the Bot
Workspace to teach the notion of script, a first step in programming
languages. The Micro Browser allows children to add new behavior for
their bots, and have fun with their creation. This three-layers tool
- Balloon, Micro Workspace, Micro Browser - offers to the teacher a
fun way to introduce gently the basis of object-oriented programming
concepts. With Bots Inc, learning is playing ! ;-)" Samir Saidani -
University of Caen - France

"I recently started a course with 7th-graders (age about 13 years)
with Stephane's book --- they love it. They all know about syntactic
issues from maths --- in a way they know that an expression in a
formal language must be well formed. So they easily grasp the fact
such as "there must be a colon after the message-name if an argument
follows". Of cause they don't really read the error-messages, they
just see "there must be some error" and they remember the simple
rules. Don't underestimate Smalltalk --- it's easy understandable
because it has a simple and straight-forward design." Klaus Fuller -
Germany

Have fun...

Stef


http://www.iam.unibe.ch/~ducasse/
    "if you knew today was your last day on earth, what would you
    do different? ...  especially if,  by doing something different,
    today might not be your last day on earth" Calvin&Hobbes


From mwh at python.net  Sun Jun 12 22:17:38 2005
From: mwh at python.net (Michael Hudson)
Date: Sun, 12 Jun 2005 21:17:38 +0100
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42AAFBB6.2070802@gmail.com> (Nick Coghlan's message of "Sun,
	12 Jun 2005 00:56:54 +1000")
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
	<42AAFBB6.2070802@gmail.com>
Message-ID: <2m4qc3fhy5.fsf@starship.python.net>

Nick Coghlan <ncoghlan at gmail.com> writes:

> It also considers the possibility of using with statements in an RAII 
> style by acquiring the resource in __init__ or __new__ rather than
> __enter__.

While you can probably do this (after all, most of the time
__new__/__init__ and __enter__ will be called in rapid succession) I
really think the RAII meme doesn't apply to Python; you generally want
a resource to be acquired for a period of definite extent, and objects
in Python have indefinite extent, unlike C++ where automatic variables
have definite extent (Common Lisp terminology here).

I've gone on at length about this before:

     http://starship.python.net/crew/mwh/pep310/

Cheers,
mwh

-- 
 Have you considered downgrading your arrogance to a reasonable level?
        -- Erik Naggum, comp.lang.lisp, to yet another C++-using troll

From arigo at tunes.org  Sun Jun 12 23:10:29 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 12 Jun 2005 23:10:29 +0200
Subject: [Python-Dev] Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005
Message-ID: <20050612211029.GA26015@code1.codespeak.net>

Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005 
======================================================

The next PyPy sprint is scheduled right after EuroPython 2005
in Gothenborg, Sweden.  Its main focus is translation to 
lower level backends but there are also other possible topics. 
We'll give newcomer-friendly introductions.  To learn more
about the new PyPy Python-in-Python implementation look here: 

    http://codespeak.net/pypy 

On a side note, there are a number of sub projects that may be
interesting for participating in google's summer-of-code event
(deadline June 14th!).  The PyPy group is willing to mentor
projects that have some link with PyPy, so if you are accepted
in such a project, the sprint could also serve as a good
meeting and kick-off point.  Further down you'll find some
examples, but there are certainly more and bigger ones :-)

Goals and topics of the sprint 
------------------------------

The main, though not the only, focus of the sprint will be on
the "translation" aspect of PyPy.   The goal here is to
progress towards a completely translated PyPy.  How much will
already have been done before EuroPython is unknown; as a
guess, we will be left with:
 
- completing the "rtyper", the piece of code that assigns low-level
  C-like types to high-level RPython objects (lists, dicts, instances,
  etc.) and low-level control flow graphs to high-level ones;
   
- polish off the GenC and GenLLVM back-ends, responsible for turning
  the low-level C-like flow graphs into real C or LLVM source code.

See http://codespeak.net/pipermail/pypy-dev/2005q2/002136.html for more
information (10th of June status).
 
Non-translation-related topics are welcome too.  Here are some suggestions
from the issue tracker (https://codespeak.net/issue/pypy-dev/):
   
- integrate the parser module, possibly making it RPython
  conformant;
 
- rewrite in Python a C module you are familiar with
  (partial list of missing/incomplete modules: os, math, array,
  regular expressions, binascii...)

- implement Python 2.3's import hook extensions (zip-imports etc.)

- fix Windows-related issues, '%'-formatting rounding errors,
  add missing docstrings on app-level built-in types and functions,
  etc.

- weakrefs (but this requires discussion and planning on pypy-dev
  before the sprint! feel free to start such a discussion, though.)


Location & Accomodation  
------------------------ 

The sprint will be held in the former Math Center building
near the crossing of Gibraltargatan and Eklandagatan. Entrance
is on the middle of the side facing Gibraltargatan. The doors
to the building are normally locked, so you need the phone number
of somebody inside to get in. Instructions on whom to call will be
posted on the door.

The sprint will be co-located with several other sprints. See the 
`EuroPython Wiki`_, to find out what other sprints will be running.

Nearest, and probably cheapest is to book accomodation at SGS Veckobostäder
through the Europython website. This option will be available until about
20 June.

.. _`EuroPython special accomodation`: http://www.europython.org/sections/accomodation/special_accomodation
.. _`EuroPython Wiki`: http://www.europython.org/sections/sprints_and_wiki

Exact times 
-----------

The public Pypy sprint is held Friday 1st July - Thursday 7 July 2005.
Hours will be from 09:00 until people have had enough. It's a good 
idea to arrive a day before the sprint starts.   

(There is a sprint for people who are familiar with the Pypy codebase
before Europython as well. This will be held at Jacob & Laura's home
on Götabergsgatan 22.)


Network, Food, currency 
------------------------ 

Sweden is not part of the Euro zone. One SEK (krona in singular, kronor
in plural) is roughly 1/10th of a Euro (9.15 SEK to 1 Euro).

There are some pizzerias, kebab places and the like close to the venue.
Their food is edible and cheap, but not very good. For good food, you need to
go downtown.

You need a wireless network card to access the network. You will be 
issued a login to the Chalmers NOMAD network. This will allow you to 
use access points all over Chalmers.  However, we can likely provide 
a wireless/ethernet bridge.  

Sweden uses the same kind of plugs as Germany. 230V AC.

Registration etc.pp. 
-------------------- 

Please subscribe to the `PyPy sprint mailing list`_, introduce
yourself and post a note that you want to come.  Feel free
to ask any questions there! 

.. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint


-- 
Armin Rigo & the PyPy team


From tjreedy at udel.edu  Sun Jun 12 23:35:57 2005
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 12 Jun 2005 17:35:57 -0400
Subject: [Python-Dev] [ANN] Have fun programming with your kids
References: <d8hn17$909$7@sea.gmane.org>
Message-ID: <d8i9j2$i5u$1@sea.gmane.org>

Dear Mr. Ducasse,

The PyDev mailing list, bidirectionally gatewayed to 
gmane.comp.python.devel, which you apparently used as the insertion point, 
is a specialized list for discussion and development of the next Python 
version and occasionally beyond.  Even most posts to comp.lang.python (== 
gmane.comp.python.general) are off-topic here.

Your annoucement of a non-Python application book, which you sent numerous 
places, appears to be out of the ballpark of relevance here. Your book may 
be wonderful and I might find it useful were I to buy it, but if everyone 
imitated you, PyDev would have to be closed in order to function.  Please 
desist, both for this and future projects.

PS. Ignorance is no excuse.  Internet etiquette specifies that before 
posting, one read a group (mailing list, forum) enough to learn what it is 
about.




From skip at pobox.com  Mon Jun 13 03:35:31 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 12 Jun 2005 20:35:31 -0500
Subject: [Python-Dev] Multiple expression eval in compound if statement?
Message-ID: <17068.58083.342369.872731@montanaro.dyndns.org>


I'm horsing around with recognizing switch-like if statements like:

    if x == 1:
        print 1
    elif x == 2:
        print 2
    else:
        print "unknown"

in the compiler and generating O(1) code.  "x" can be any expression, but
must be precisely the same in each elif clause, the comparison operator must
be "==" and the RHS of the test must evaluate to a simple hashable constant
(string, float, integer - still working on None, constant tuples may be
allowed later).  I can currently recognize such constructs and am working on
code generation.

This would represent a semantic change to the Python runtime, because x
would only be evaluated once, whereas in existing usage it can potentially
be evaluated many times.  If evaluating x has side-effects, code like the
above that currently works would break.

In reading the language reference manual, it says:

    It selects exactly one of the suites by evaluating the expressions one
    by one until one is found to be true (see section 5.10 for the
    definition of true and false); then that suite is executed (and no other
    part of the if statement is executed or evaluated). If all expressions
    are false, the suite of the else clause, if present, is executed.

It says nothing about possibly caching values, which is what I'm doing
effectively, but it seems pretty clear that each full test expression will
be evaluated until one evaluates to true.

I can't imagine anyone relying on a side-effect when evaluating x in this
context, and if someone did, their code would be truly fragile.  Still, any
thought as to whether or not such a semantic change to the language might be
acceptable?

Skip

From greg.ewing at canterbury.ac.nz  Mon Jun 13 03:55:22 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 13 Jun 2005 13:55:22 +1200
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc2050610132375ca7c70@mail.gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
Message-ID: <42ACE78A.1040909@canterbury.ac.nz>

Guido van Rossum wrote:

> So (a) would have my preference.

Mine, too.
> the PEP would have to be amended to state that
> VAR must be a single variable or a list of variables IN PARENTHESES.

+1

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From ncoghlan at gmail.com  Mon Jun 13 03:58:18 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2005 11:58:18 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc205061209561d7c26c9@mail.gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>	
	<42AAFBB6.2070802@gmail.com>	
	<ca471dc205061109025a13c846@mail.gmail.com>	
	<42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com>	
	<42ABB59C.5050608@gmail.com>
	<ca471dc205061209561d7c26c9@mail.gmail.com>
Message-ID: <42ACE839.1070804@gmail.com>

Guido van Rossum wrote:
> I can't make time for this right now, but the more I think about it,
> the more I believe the worry about asynchronous exceptions is mostly
> irrational. Despite your "proof" that it *can* be implemented I really
> don't want to see anything like that implemented -- I doubt that
> you've covered all the cases and I don't like inserting new opcodes to
> handle a case that most programs don't care about. Declaring something
> "atomic" that could span an arbitrary amount of Python code execution
> (including calls) just doesn't seem right.

As I just added to the Wiki, dropping the issue completely works for 
me. I'm also far from convinced I'd covered all the corner cases 
(besides, blocking KeyboardInterrupt for an arbitrary amount of time 
made me uncomfortable).

> Disregarding asynchronous exceptions, you can already use
> with-statements for your beloved RAII pattern, right? I'm still
> skeptical how important it is in Python -- its seems something
> invented very specifically for C++'s guarantee of local destructor
> execution, and I don't expect to see all that many applications for it
> in Python.

You'd mostly convinced me that RAII was rare in Python, but two 
possible use cases remained - files, and "call method X of object Y at 
the end of the block" (e.g. "closing()" from the PEP).

Both of those would require any async guarantee to cover evaluation of 
EXPR in order to be included in the guarantee. If with statements make 
no guarantees about async exceptions, then there is no problem (as 
there is officially no difference between the two styles).

If we ever do resurrect the idea, I also realised a far simpler 
solution would involve moving evaluation of EXPR and the call to 
__enter__ inside the try/finally block, and having an internal flag to 
indicate whether or not __exit__ should be called as the block is 
exited. Far less screwing around with ceval.c, and far easier to get 
right.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From raymond.hettinger at verizon.net  Mon Jun 13 03:57:22 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun, 12 Jun 2005 21:57:22 -0400
Subject: [Python-Dev] Wishlist:  dowhile
Message-ID: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>

More than case-statement semantics or PEP343, I wish for a dowhile
statement.

The most straight-forward way is to put the conditional expression at
the beginning of the block with the understanding that a dowhile keyword
will evaluate the condition only after the block runs:

    dowhile <cond>:
        <body>

being the equivalent of;

    _firstpass = True
    while _firstpass or <cond>:
        _firstpass = False
        <body>


Raymond


From ncoghlan at gmail.com  Mon Jun 13 04:14:37 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2005 12:14:37 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <2m4qc3fhy5.fsf@starship.python.net>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>	<42AAFBB6.2070802@gmail.com>
	<2m4qc3fhy5.fsf@starship.python.net>
Message-ID: <42ACEC0D.20508@gmail.com>

Michael Hudson wrote:
> While you can probably do this (after all, most of the time
> __new__/__init__ and __enter__ will be called in rapid succession) I
> really think the RAII meme doesn't apply to Python; you generally want
> a resource to be acquired for a period of definite extent, and objects
> in Python have indefinite extent, unlike C++ where automatic variables
> have definite extent (Common Lisp terminology here).

Using the RAII term was probably a bad idea, as it brings in too many 
extra connotations (like the converse 'RRID' - 'resource release is 
destruction').

Anyway, Guido's taking the option of not providing any explicit 
guarantees at this point, which is probably the most sensible idea.

Cheers,
Nick.
-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From python at rcn.com  Mon Jun 13 04:20:31 2005
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 12 Jun 2005 22:20:31 -0400
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <17068.58083.342369.872731@montanaro.dyndns.org>
Message-ID: <000501c56fbe$79998220$4ff9a244@oemcomputer>

> I'm horsing around with recognizing switch-like if statements like:
> 
>     if x == 1:
>         print 1
>     elif x == 2:
>         print 2
>     else:
>         print "unknown"
> 
> in the compiler and generating O(1) code.  "x" can be any expression,
but
> must be precisely the same in each elif clause, the comparison
operator
> must
> be "==" and the RHS of the test must evaluate to a simple hashable
> constant
> (string, float, integer - still working on None, constant tuples may
be
> allowed later).  I can currently recognize such constructs and am
working
> on
> code generation.

I think it unwise to allow x to be any expression.  Besides altering
existing semantics, it leads to code redundancy and to a fragile
construct (where the slightest alteration of any of the expressions
triggers a silent reversion to O(n) behavior).


Raymond

From ncoghlan at gmail.com  Mon Jun 13 04:39:11 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2005 12:39:11 +1000
Subject: [Python-Dev] Wishlist:  dowhile
In-Reply-To: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
Message-ID: <42ACF1CF.5000203@gmail.com>

Raymond Hettinger wrote:
> More than case-statement semantics or PEP343, I wish for a dowhile
> statement.
> 
> The most straight-forward way is to put the conditional expression at
> the beginning of the block with the understanding that a dowhile keyword
> will evaluate the condition only after the block runs

Out-of-order code execution rarely counts as 'straightforward' ;)

With PEP 315, a do-while loop would look like:

   do:
       <body>
   while <cond>:
       pass

But then, I'm reasonably happy with the 'break out of an infinite 
loop' approach, so *shrug*.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From gvanrossum at gmail.com  Mon Jun 13 04:55:13 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun, 12 Jun 2005 19:55:13 -0700
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42ACE839.1070804@gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
	<42AAFBB6.2070802@gmail.com>
	<ca471dc205061109025a13c846@mail.gmail.com>
	<42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com>
	<42ABB59C.5050608@gmail.com>
	<ca471dc205061209561d7c26c9@mail.gmail.com>
	<42ACE839.1070804@gmail.com>
Message-ID: <ca471dc205061219551accd1fe@mail.gmail.com>

[Nick]
> As I just added to the Wiki, dropping the issue completely works for
> me. I'm also far from convinced I'd covered all the corner cases
> (besides, blocking KeyboardInterrupt for an arbitrary amount of time
> made me uncomfortable).

OK.

> You'd mostly convinced me that RAII was rare in Python, but two
> possible use cases remained - files, and "call method X of object Y at
> the end of the block" (e.g. "closing()" from the PEP).
> 
> Both of those would require any async guarantee to cover evaluation of
> EXPR in order to be included in the guarantee. If with statements make
> no guarantees about async exceptions, then there is no problem (as
> there is officially no difference between the two styles).

Any exception, async or otherwise, that happens before the 'try' of
the translation is reached, does *not* cause __exit__ to be called.

I just realized that the assignment to VAR (if present) also has to be
represented by one or more opcodes, so a special guarantee that
__enter__() called by the same opcode that sets up the try cannot work
anyway. Forget I said anything about that.

> If we ever do resurrect the idea, I also realised a far simpler
> solution would involve moving evaluation of EXPR and the call to
> __enter__ inside the try/finally block, and having an internal flag to
> indicate whether or not __exit__ should be called as the block is
> exited. Far less screwing around with ceval.c, and far easier to get
> right.

We considered this at some point during the PEP 340/343/346 discussion
(in fact I had it this way in an early version of one of the PEPs),
and in the end decided against it. I believe the argument was that any
failure *before* __enter__() returns can be handled by try/except
clauses inside __init__() or __enter__().

Changing to this style later would be a major backward incompatibility
because the guarantees made to __exit__() are different.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Mon Jun 13 05:05:59 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun, 12 Jun 2005 20:05:59 -0700
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <17068.58083.342369.872731@montanaro.dyndns.org>
References: <17068.58083.342369.872731@montanaro.dyndns.org>
Message-ID: <ca471dc205061220055eb38eb5@mail.gmail.com>

On 6/12/05, Skip Montanaro <skip at pobox.com> wrote:
> 
> I'm horsing around with recognizing switch-like if statements like:
> 
>     if x == 1:
>         print 1
>     elif x == 2:
>         print 2
>     else:
>         print "unknown"
> 
> in the compiler and generating O(1) code.  "x" can be any expression, but
> must be precisely the same in each elif clause, the comparison operator must
> be "==" and the RHS of the test must evaluate to a simple hashable constant
> (string, float, integer - still working on None, constant tuples may be
> allowed later).  I can currently recognize such constructs and am working on
> code generation.

Cool!

> This would represent a semantic change to the Python runtime, because x
> would only be evaluated once, whereas in existing usage it can potentially
> be evaluated many times.  If evaluating x has side-effects, code like the
> above that currently works would break.
> 
> In reading the language reference manual, it says:
> 
>     It selects exactly one of the suites by evaluating the expressions one
>     by one until one is found to be true (see section 5.10 for the
>     definition of true and false); then that suite is executed (and no other
>     part of the if statement is executed or evaluated). If all expressions
>     are false, the suite of the else clause, if present, is executed.
> 
> It says nothing about possibly caching values, which is what I'm doing
> effectively, but it seems pretty clear that each full test expression will
> be evaluated until one evaluates to true.
> 
> I can't imagine anyone relying on a side-effect when evaluating x in this
> context, and if someone did, their code would be truly fragile.  Still, any
> thought as to whether or not such a semantic change to the language might be
> acceptable?

I think it would be wiser if you only did this when x is a simple
local variable. For locals, we *know* that no side effect of any
expression can change the value.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From foom at fuhm.net  Mon Jun 13 05:42:45 2005
From: foom at fuhm.net (James Y Knight)
Date: Sun, 12 Jun 2005 23:42:45 -0400
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <000501c56fbe$79998220$4ff9a244@oemcomputer>
References: <000501c56fbe$79998220$4ff9a244@oemcomputer>
Message-ID: <23D1E8FC-A5DB-4CD4-9EB2-D5B637B1C552@fuhm.net>

On Jun 12, 2005, at 10:20 PM, Raymond Hettinger wrote:
> I think it unwise to allow x to be any expression.  Besides altering
> existing semantics, it leads to code redundancy and to a fragile
> construct (where the slightest alteration of any of the expressions
> triggers a silent reversion to O(n) behavior).

Yes, changing the semantics of the if statement would be pretty bad.  
Unless evaluating x provably has no side effects (e.g. accessing a  
local variable), I would say that the duplicate evaluations must not  
be eliminated.


>>     if x == 1:
>>         print 1
>>     elif x == 2:
>>         print 2
>>     else:
>>         print "unknown"

Unfortunately, I'm afraid this will not be as useful as it at first  
appears, because python has no concept of a constant variable  
binding. I hardly ever see an if/else chain being done with hardcoded  
integers. Rather, more often a variable is compared to global  
"constants", which could not be optimized away. However, this  
construction does appear reasonably often with constant strings on  
the RHS, so there may actually be a real-world gain to be had.

With a Sufficiently Smart Compiler, of course, you could make  
assumptions at runtime about values not changing, and recompile code  
depending upon that assumption if necessary, but that's more in the  
realm of a new runtime than modifications to CPython. Maybe PyPy will  
eventually provide the ability to do that kind of interesting  
optimization.

James

From gvanrossum at gmail.com  Mon Jun 13 06:01:42 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun, 12 Jun 2005 21:01:42 -0700
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <42ACF1CF.5000203@gmail.com>
References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
Message-ID: <ca471dc20506122101573e8499@mail.gmail.com>

On 6/12/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Raymond Hettinger wrote:
> > More than case-statement semantics or PEP343, I wish for a dowhile
> > statement.
> >
> > The most straight-forward way is to put the conditional expression at
> > the beginning of the block with the understanding that a dowhile keyword
> > will evaluate the condition only after the block runs
> 
> Out-of-order code execution rarely counts as 'straightforward' ;)

Right. Millions of years of programming language design have shown us
how to write a loop with the condition tested at the end -- the
condition is written after the loop body.

> With PEP 315, a do-while loop would look like:
> 
>    do:
>        <body>
>    while <cond>:
>        pass
> 
> But then, I'm reasonably happy with the 'break out of an infinite
> loop' approach, so *shrug*.

Amen.

If we have to do this, PEP 315 has my +0.

It is Pythonically minimal and the motivation rings true: I've often
written code like this in the past:

    line = f.readline()
    while line:
        <do something>
        line = f.readline()

But these days we do that using "for line in f".

I wonder if other similar use cases can't be rewritten using better iterators?

Maybe Raymond can show us some motivating use cases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Mon Jun 13 06:37:30 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun, 12 Jun 2005 21:37:30 -0700
Subject: [Python-Dev] Wishlist:  dowhile
In-Reply-To: <42ACF1CF.5000203@gmail.com>
References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
Message-ID: <20050612212924.7236.JCARLSON@uci.edu>


Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Raymond Hettinger wrote:
> > More than case-statement semantics or PEP343, I wish for a dowhile
> > statement.
> > 
> > The most straight-forward way is to put the conditional expression at
> > the beginning of the block with the understanding that a dowhile keyword
> > will evaluate the condition only after the block runs
> 
> Out-of-order code execution rarely counts as 'straightforward' ;)
> 
> With PEP 315, a do-while loop would look like:
> 
>    do:
>        <body>
>    while <cond>:
>        pass
> 
> But then, I'm reasonably happy with the 'break out of an infinite 
> loop' approach, so *shrug*.

PEP 315 offers the creation/use of an additional keyword.  It also
happens to push the condition to the end of the loop body.

I'm not sure that pushing condition evaluation to the end of the loop
body (with a trailing 'pass' as provided) is necessarily more readable
than Raymond's offered dowhile.  In fact, if I remember the discussion
over decorators correctly, putting an operation later actually reduces
readability, though at least in this case, the bare "do:" would signal
to the user "Hey, I have a condition later!"  Though we again run into,
how is that any better than having the condition at the front to begin
with, especially if a new keyword is necessary regardless of the
resulting syntax?


 - Josiah


From pje at telecommunity.com  Mon Jun 13 07:25:51 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 13 Jun 2005 01:25:51 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ca471dc20506122101573e8499@mail.gmail.com>
References: <42ACF1CF.5000203@gmail.com>
	<000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
Message-ID: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>

At 09:01 PM 6/12/2005 -0700, Guido van Rossum wrote:
>If we have to do this, PEP 315 has my +0.
>
>It is Pythonically minimal and the motivation rings true: I've often
>written code like this in the past:
>
>     line = f.readline()
>     while line:
>         <do something>
>         line = f.readline()
>
>But these days we do that using "for line in f".

Block-copying a file with read() has the same pattern (e.g. in shutil), but 
you can't make it a for loop.

By the way, whatever happened to "and while"?  i.e.:

     while True:
         data = inp.read(blocksize)
     and while data:
         out.write(data)

It seemed to me this variation of the syntax had some traction at one 
point.  Anyway, most of the times that I'm annoyed by the current while 
loop's limitations are due to the need to write things like the above as:

     while True:
         data = inp.read(blocksize)
         if not data:
             break
         out.write(data)

Which makes it a bit harder to see the control flow.  Or more precisely, 
something like the 'do/while' or 'while/and while' would make it *easier* 
to see the control flow, since really while isn't *bad*.

I seem to recall that Knuth had something to say about the ideal loop being 
one that allowed you to execute code both before and after the exit 
condition, and that many languages allow you to have code before, or code 
after, but few allow you to do both in the same loop, forcing you to either 
duplicate some code or restructure your approach in order to fit the 
limitation of the language.


From python at rcn.com  Mon Jun 13 08:43:29 2005
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 13 Jun 2005 02:43:29 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>
Message-ID: <000201c56fe3$535b9f60$5ab79d8d@oemcomputer>


> By the way, whatever happened to "and while"?

That is making something hard and weird out of something simple.
There were no shortage of odd suggestions to force the condition line
to appear lower in the block than the starting line.  All of them
smelled of rotten eggs -- they just don't fit the Python mold.

In contrast, the dowhile solution only takes about 30 seconds to get
used to.  Maybe that idea should follow the path of the genexp PEP, get
rejected now, and then get resurrected as a bright idea two years from
now.  Or maybe not.


Raymond

From bjourne at gmail.com  Mon Jun 13 11:14:24 2005
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Mon, 13 Jun 2005 11:14:24 +0200
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <000201c56fe3$535b9f60$5ab79d8d@oemcomputer>
References: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>
	<000201c56fe3$535b9f60$5ab79d8d@oemcomputer>
Message-ID: <740c3aec050613021421c8a76@mail.gmail.com>

> In contrast, the dowhile solution only takes about 30 seconds to get
> used to.  Maybe that idea should follow the path of the genexp PEP, get
> rejected now, and then get resurrected as a bright idea two years from
> now.  Or maybe not.

<nothing about foo here>
dowhile foo != 42:
    <10 lines of body>
    foo = random.randint(1, 50)

Now it becomes annoying because the name is visible long before it is
bound to something. I would like to have do-while's like this:

do:
    <body>
    until <cond>

But I'm sure that has problems too.

-- 
mvh Bj?rn

From python at rcn.com  Mon Jun 13 11:55:14 2005
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 13 Jun 2005 05:55:14 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <740c3aec050613021421c8a76@mail.gmail.com>
Message-ID: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>

[BJ?rn Lindqvist] 
> I would like to have do-while's like this:
> 
> do:
>     <body>
>     until <cond>
> 
> But I'm sure that has problems too.

That looks nice to me.



Raymond

From dmitry at targeted.org  Mon Jun 13 12:43:36 2005
From: dmitry at targeted.org (Dmitry Dvoinikov)
Date: Mon, 13 Jun 2005 16:43:36 +0600
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
References: <740c3aec050613021421c8a76@mail.gmail.com>
	<000901c56ffe$369f19e0$b8b3958d@oemcomputer>
Message-ID: <2316076.20050613164336@targeted.org>

>> do:
>>     <body>
>>     until <cond>

> That looks nice to me.

Except this puts loop entry statement (do) and loop condition
test (until) on different levels of indentation, which is
I don't quite like.

Compared to the existing loop statements,

-------------------
while Cond:
    body
else:
    exit
-------------------
for Iter:
    body
else:
    exit
-------------------

This would seem more logical:

-------------------
do:
    body
while Cond:
else:
    exit
-------------------
do:
    body
until Cond:
else:
    exit
-------------------

Although the trailing colon is still a bit unpleasant:

-------------------

do:
    body
while Cond:

# no else here

-------------------

Sincerely,
Dmitry Dvoinikov
http://www.targeted.org/

--- Original message follows ---

> [BJ?rn Lindqvist] 
>> I would like to have do-while's like this:
>> 
>> do:
>>     <body>
>>     until <cond>
>> 
>> But I'm sure that has problems too.

> That looks nice to me.



> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dmitry%40targeted.org


From skip at pobox.com  Mon Jun 13 13:14:38 2005
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 13 Jun 2005 06:14:38 -0500
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <000501c56fbe$79998220$4ff9a244@oemcomputer>
References: <17068.58083.342369.872731@montanaro.dyndns.org>
	<000501c56fbe$79998220$4ff9a244@oemcomputer>
Message-ID: <17069.27294.581932.647705@montanaro.dyndns.org>


    Raymond> I think it unwise to allow x to be any expression.  

How do you decide what's "too complex"?  Even an apparently simple variable
can have side effects, so the semantic change bit is important no matter how
complex the expression.  (Consider the builtin help object.  Type it at the
prompt with no parens.)

Like I said, "I'm horsing around with...".  If this never makes it into
Python I can accept that.  The interesting thing for me at this point is the
code generation anyway.  I figure if Guido reconsiders PEP 275 (I think it
covers all the bases), then the code generation stuff will probably be of
some use.

Skip

From mcherm at mcherm.com  Mon Jun 13 13:48:31 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon, 13 Jun 2005 04:48:31 -0700
Subject: [Python-Dev] Multiple expression eval in compound if	statement?
Message-ID: <20050613044831.kec9rksum2ec80o0@login.werra.lunarpages.com>

Skip writes:
> Even an apparently simple variable can have side effects [...] Consider
> the builtin help object.  Type it at the prompt with no parens.

I tried. As far as I can tell, the result (in the interactive interpreter)
is the same as with any other object except None: the _ variable gets set,
the interpreter prints out the __str__ of the object, sets the _ variable,
and prints a new '>>>' prompt. It happens that the __str__ of the help
object is:
'Type help() for interactive help, or help(object for help about object.'

... but if there's some side effect going on here, I don't see it. What
am I missing?

-- Michael Chermside

From mcherm at mcherm.com  Mon Jun 13 14:07:32 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon, 13 Jun 2005 05:07:32 -0700
Subject: [Python-Dev] Multiple expression eval in compound if	statement?
Message-ID: <20050613050732.gw9sc59tj6dc0804@login.werra.lunarpages.com>

A few thought's on Skip's idea here:

Skip:
> I can't imagine anyone relying on a side-effect when evaluating x in this
> context, and if someone did, their code would be truly fragile.

Yeah... people like that really annoy me. If people just wouldn't write code
that depends on the FULL dynamics possible, then it'd be a lot easier for
the VM to optimize. Unfortunately, we have to support such people.

Raymond:
I think it unwise to allow x to be any expression.  Besides altering
existing semantics, it leads to code redundancy and to a fragile
construct (where the slightest alteration of any of the expressions
triggers a silent reversion to O(n) behavior).

Skip's proposal isn't the problem here. The syntax of the if-elif-else
statement requires that when it is used in lieu of C's switch() statement
that the switch() argument be repeated in each elif clause. Repeating
it DOES violate once-and-only-once, but we've decided that that's an
acceptable price to pay to avoid needing an extra kind of flow control
statement for which there wasn't a really elegent syntax anyhow. A
reasonably tradeoff, I agree, but the cost is that we DO violate
once-and-only-once. The REAL danger from the "slightest alteration of
any of the expressions" is not that it will alter from O(1) behavior
to O(n) behavior, the REAL danger is that you may have MEANT it to be
the same everywhere and the alteration is a typo and a bug!

Guido:
> I think it would be wiser if you only did this when x is a simple
> local variable.

I agree. I think of Skip's proposal as just an optimization, of the
sort that the Python interpreter is allowed to make when there's no
possible semantic effect. It would then restrict the use to situations
where the 'switch() argument' was a local variable and the 'case values'
were string literals, integer literals, float literals (but that
shouldn't count because it's a really bad idea anyhow), None, and
perhaps (I wouldn't bother, myself) tuples of the above. It sounds
pretty restricted, which raises the question of whether it's worth it
given the harsh restrictions on when the optimization would kick in.

I write such things with local variables and literal strings quite often.
On the other hand, I don't usually have more than 4-10 values though, so
O(1) and O(n) may not be THAT different. It's one of those cases where the
only thing I'd really believe was experiments done on real code. But it's
a cool optimization if it actually pays off.

-- Michael Chermside


From ncoghlan at gmail.com  Mon Jun 13 14:07:38 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2005 22:07:38 +1000
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
Message-ID: <42AD770A.1040001@gmail.com>

Raymond Hettinger wrote:
> [BJ?rn Lindqvist] 
> 
>>I would like to have do-while's like this:
>>
>>do:
>>    <body>
>>    until <cond>
>>
>>But I'm sure that has problems too.
> 
> 
> That looks nice to me.

And this could easily be extended to allow code both before and after 
the 'until', giving a fully general loop:

   do:
       <body-1-or-more-times>
       until <cond>
       <body-0-or-more-times>
   else:
       <on-natural-loop-exit>

In fact, this would simply be giving "looks like executable 
pseudocode" syntactic sugar for the current 'do-until' workarounds:

   while 1:
     <body-1-or-more-times>
     if <cond>:
         <on-natural-loop-exit>
         break
     <body-0-or-more-times>

And:

   _exit = False:
   while not _exit:
       <body-1-or-more-times>
       _exit = <cond>
       <body-0-or-more-times>
   else:
       <on-natural-loop-exit>

The 'until' is less hidden than the 'yield' that turns a function into 
a generator, and its presence is obviously signalled by the preceding 
'do'. Its also less hidden than the 'if'/'break' construct in the 
infinite loop workaround, and about as hidden as the exit flag in the 
other style of workaround (although the 'until' can be more easily 
picked out by a syntax highlighter).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From exarkun at divmod.com  Mon Jun 13 14:14:23 2005
From: exarkun at divmod.com (Jp Calderone)
Date: Mon, 13 Jun 2005 08:14:23 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>
Message-ID: <20050613121423.5047.539927633.divmod.quotient.2056@ohm>

On Mon, 13 Jun 2005 01:25:51 -0400, "Phillip J. Eby" <pje at telecommunity.com> wrote:
>At 09:01 PM 6/12/2005 -0700, Guido van Rossum wrote:
>>If we have to do this, PEP 315 has my +0.
>>
>>It is Pythonically minimal and the motivation rings true: I've often
>>written code like this in the past:
>>
>>     line = f.readline()
>>     while line:
>>         <do something>
>>         line = f.readline()
>>
>>But these days we do that using "for line in f".
>
>Block-copying a file with read() has the same pattern (e.g. in shutil), but
>you can't make it a for loop.

Anything can be a for loop.

  for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''):
      f2.write(chunk)

Jp

From ncoghlan at gmail.com  Mon Jun 13 14:40:05 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2005 22:40:05 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <ca471dc205061219551accd1fe@mail.gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>	
	<42AAFBB6.2070802@gmail.com>	
	<ca471dc205061109025a13c846@mail.gmail.com>	
	<42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com>	
	<42ABB59C.5050608@gmail.com>	
	<ca471dc205061209561d7c26c9@mail.gmail.com>	
	<42ACE839.1070804@gmail.com>
	<ca471dc205061219551accd1fe@mail.gmail.com>
Message-ID: <42AD7EA5.2070805@gmail.com>

Guido van Rossum wrote:
> We considered this at some point during the PEP 340/343/346 discussion
> (in fact I had it this way in an early version of one of the PEPs),
> and in the end decided against it. I believe the argument was that any
> failure *before* __enter__() returns can be handled by try/except
> clauses inside __init__() or __enter__().

Hmm, you're right. Also, given implementation of PEP 343, code which 
genuinely has to care about this can do so manually:

   with async_exceptions_blocked():
       with critical_resource():
           with async_exceptions_unblocked():
               # Do the real work

This ensures acquisition of the critical resource is not interrupted. 
Even better, the above can be wrapped up in a template and still give 
the desired guarantees:

   @stmt_template
   def safe_acquisition(resource):
       with async_exceptions_blocked():
           with resource():
               with async_exceptions_unblocked():
                   yield None


   with safe_acquisition(critical_resource):
       # Do the real work

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From skip at pobox.com  Mon Jun 13 16:05:50 2005
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 13 Jun 2005 09:05:50 -0500
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <20050613044831.kec9rksum2ec80o0@login.werra.lunarpages.com>
References: <20050613044831.kec9rksum2ec80o0@login.werra.lunarpages.com>
Message-ID: <17069.37566.869168.203775@montanaro.dyndns.org>


    Michael> ... but if there's some side effect going on here, I don't see
    Michael> it. What am I missing?

Mea culpa.  I was thinking of the print as a side efefct.  Obviously
mistaken.

Skip


From anthony at interlink.com.au  Mon Jun 13 16:16:00 2005
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue, 14 Jun 2005 00:16:00 +1000
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <17069.27294.581932.647705@montanaro.dyndns.org>
References: <17068.58083.342369.872731@montanaro.dyndns.org>
	<000501c56fbe$79998220$4ff9a244@oemcomputer>
	<17069.27294.581932.647705@montanaro.dyndns.org>
Message-ID: <200506140016.02598.anthony@interlink.com.au>

On Monday 13 June 2005 21:14, Skip Montanaro wrote:
>     Raymond> I think it unwise to allow x to be any expression.
>
> How do you decide what's "too complex"?  Even an apparently simple variable
> can have side effects, so the semantic change bit is important no matter
> how complex the expression.  (Consider the builtin help object.  Type it at
> the prompt with no parens.)

Note also that saying 'just a simple local variable' doesn't cut it, either, 
because you will break on something like:

class stupid:
    v = 0
    def __eq__(self, other):
        self.v += 1
        return self.v == other

Really, you'd have to make sure you didn't optimise any LHS that defined
a comparision operator (I _think_ that covers all the cases).

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From gvanrossum at gmail.com  Mon Jun 13 16:41:45 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon, 13 Jun 2005 07:41:45 -0700
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <42AD770A.1040001@gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com>
Message-ID: <ca471dc20506130741430f8811@mail.gmail.com>

> > [BJ?rn Lindqvist]
> >
> >>I would like to have do-while's like this:
> >>
> >>do:
> >>    <body>
> >>    until <cond>
> >>
> >>But I'm sure that has problems too.
> >
[Raymond Hettinger]
> > That looks nice to me.

[Nick Coghlan]
> And this could easily be extended to allow code both before and after
> the 'until', giving a fully general loop:
> 
>    do:
>        <body-1-or-more-times>
>        until <cond>
>        <body-0-or-more-times>
>    else:
>        <on-natural-loop-exit>

Which is exactly like PEP 315 except there 'until' must be spelled
'while not' and the while is properly indented.

(I'm still not sure whether BJ?rn *meant* the 'until' to be indented
or whether he simply made a mistake; his proposal resembles a Pythonic
version of Pascal's repeat-until, which would have an unindented
until-clause.)

> The 'until' is less hidden than the 'yield' that turns a function into
> a generator, and its presence is obviously signalled by the preceding
> 'do'. Its also less hidden than the 'if'/'break' construct in the
> infinite loop workaround, and about as hidden as the exit flag in the
> other style of workaround (although the 'until' can be more easily
> picked out by a syntax highlighter).

Why are you so excited about having until indented? You didn't give
any examples with multiple occurrences. A single occurrence works just
fine unindented, as PEP 315 has already shown.

The indented until sounds like unnecessary syntactic sugar for 'if X:
break' -- not very Pythonic.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From bjourne at gmail.com  Mon Jun 13 17:53:52 2005
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Mon, 13 Jun 2005 17:53:52 +0200
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ca471dc20506130741430f8811@mail.gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com>
	<ca471dc20506130741430f8811@mail.gmail.com>
Message-ID: <740c3aec05061308536bceeead@mail.gmail.com>

> > >>do:
> > >>    <body>
> > >>    until <cond>
> > >>
> > >>But I'm sure that has problems too.
> > >
> [Raymond Hettinger]
> > > That looks nice to me.
> 
> [Nick Coghlan]
> > And this could easily be extended to allow code both before and after
> > the 'until', giving a fully general loop:
> >
> >    do:
> >        <body-1-or-more-times>
> >        until <cond>
> >        <body-0-or-more-times>
> >    else:
> >        <on-natural-loop-exit>
> 
> Which is exactly like PEP 315 except there 'until' must be spelled
> 'while not' and the while is properly indented.
> 
> (I'm still not sure whether BJ?rn *meant* the 'until' to be indented
> or whether he simply made a mistake; his proposal resembles a Pythonic
> version of Pascal's repeat-until, which would have an unindented
> until-clause.)

Sorry, I should have made that clear. I *meant* for 'until' to be
indented. I envisonaged it to be treatened similar to how the raise,
pass and return statements work. Those statements are indented in a
suite and they always end the current block. Well, technically you can
write stuff like this:

suite:
    <block>
    raise Exception
    <block2>

But 'block2' will always be dead code. Emacs, for example, always
dedents when you write any of those three statements. The reason I
suggested was not so that you could extend it to fully general loops
like Nick suggested (which I can't comment on because I never use
else: in looping constructs and always keep the condition check and
the top or bottom), but because I thought that a dedented 'until'
would be very hard to parse and in general look very unpythonic:

do:
    <block>
until <cond>

Written like this it is not very obvious that the 'unil' is part of
the do-until suite. I also imagine it to be difficult to parse and it
breaks the rule that suites end when there is a dedentation. So, IMHO
using an indented 'until' is the least evil of a number of evils.
 
> Why are you so excited about having until indented? You didn't give
> any examples with multiple occurrences. A single occurrence works just
> fine unindented, as PEP 315 has already shown.
> 
> The indented until sounds like unnecessary syntactic sugar for 'if X:
> break' -- not very Pythonic.

Yes, but grepping the stdlib produces over 300 hits for "while 1:" and
"while True:" combined. Some of those a "if <cond>: break" in the
middle and some would be better written as generators, but lots of
them would be rewritten as do-while's. So I think there is more than
enough use cases for syntactic sugar for do-while loops.

-- 
mvh Bj?rn

From steve at holdenweb.com  Mon Jun 13 18:17:08 2005
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 13 Jun 2005 12:17:08 -0400
Subject: [Python-Dev] Summer of Code: Developing complete SSL support
	for Python
In-Reply-To: <1117973125.8137.6.camel@emperor>
References: <20050604193203.03D9.FCANO@ono.com>	<42A1F267.6000504@hathawaymix.org>
	<1117973125.8137.6.camel@emperor>
Message-ID: <42ADB184.7070700@holdenweb.com>

Gustavo J. A. M. Carneiro wrote:
[...]
> 
>   4. In the socket module documentation:
> 
> 
> ssl(
> sock[, keyfile, certfile])
>         Initiate a SSL connection over the socket sock. keyfile is the
>         name of a PEM formatted file that contains your private key.
>         certfile is a PEM formatted certificate chain file. On success,
>         a new SSLObject is returned.
>         
> Warning: This does not do any certificate verification!
> 
>    I would make it a top priority to enable certificate verification in
> ssl sockets.  I don't see the point in doing SSL without certificate
> verification.  It's just false security.  Maybe adding a callback asking
> the application what to do if certificate validation fails, so that
> application writers can show a GUI dialogue or something like that...
> 
>   Best regards.
> 

I believe that SSL sockets without certificate verification will still 
retain the advantages of encryption ind integrity checking, though you 
are right to say that *authentication* is lost without certificate 
checking: the certificate is essentially the CA's assertion that they 
have applied the process described in their Certification Practices 
Statement to identify the subject.

Then you must consider which CA's will be acceptable certificate 
issuers, and build trust for their certificates into the system in some 
modifiable way - we need to be able to add CA's - by the incorporation 
of the CAs' self-signed certificates, as in the browsers.

So almost certainly if the platform has a certificate repository it 
might be good to offer an interface to that, as well as offering a 
private certificate repository.

regards
  Steve
-- 
Steve Holden        +1 703 861 4237  +1 800 494 3119
Holden Web LLC             http://www.holdenweb.com/
Python Web Programming  http://pydish.holdenweb.com/


From pje at telecommunity.com  Mon Jun 13 18:29:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 13 Jun 2005 12:29:17 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <20050613121423.5047.539927633.divmod.quotient.2056@ohm>
References: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050613122620.0339f598@mail.telecommunity.com>

At 08:14 AM 6/13/2005 -0400, Jp Calderone wrote:
>On Mon, 13 Jun 2005 01:25:51 -0400, "Phillip J. Eby" 
><pje at telecommunity.com> wrote:
> >Block-copying a file with read() has the same pattern (e.g. in shutil), but
> >you can't make it a for loop.
>
>Anything can be a for loop.
>
>   for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''):
>       f2.write(chunk)

Showoff.  ;)

More seriously, I think your translation makes an excellent argument in 
*favor* of having a do/while statement for greater clarity.  :)


From pje at telecommunity.com  Mon Jun 13 18:40:02 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 13 Jun 2005 12:40:02 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42AD7EA5.2070805@gmail.com>
References: <ca471dc205061219551accd1fe@mail.gmail.com>
	<ca471dc2050610132375ca7c70@mail.gmail.com>
	<42AAFBB6.2070802@gmail.com>
	<ca471dc205061109025a13c846@mail.gmail.com>
	<42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com>
	<42ABB59C.5050608@gmail.com>
	<ca471dc205061209561d7c26c9@mail.gmail.com>
	<42ACE839.1070804@gmail.com>
	<ca471dc205061219551accd1fe@mail.gmail.com>
Message-ID: <5.1.1.6.0.20050613123758.0339dc00@mail.telecommunity.com>

At 10:40 PM 6/13/2005 +1000, Nick Coghlan wrote:
>Hmm, you're right. Also, given implementation of PEP 343, code which
>genuinely has to care about this can do so manually:
>
>    with async_exceptions_blocked():
>        with critical_resource():
>            with async_exceptions_unblocked():
>                # Do the real work
>
>This ensures acquisition of the critical resource is not interrupted.
>Even better, the above can be wrapped up in a template and still give
>the desired guarantees:
>
>    @stmt_template
>    def safe_acquisition(resource):
>        with async_exceptions_blocked():
>            with resource():
>                with async_exceptions_unblocked():
>                    yield None
>
>
>    with safe_acquisition(critical_resource):
>        # Do the real work

Ow.  My head hurts.  :)  Seriously, though, wouldn't it make more sense to 
put the 'with async_exceptions_blocked()' in the __init__ or __enter__ of 
'critical_resource'?  Or am I still missing something?


From gvanrossum at gmail.com  Mon Jun 13 18:36:07 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon, 13 Jun 2005 09:36:07 -0700
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <740c3aec05061308536bceeead@mail.gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com>
	<ca471dc20506130741430f8811@mail.gmail.com>
	<740c3aec05061308536bceeead@mail.gmail.com>
Message-ID: <ca471dc2050613093690c1a40@mail.gmail.com>

On 6/13/05, BJ?rn Lindqvist <bjourne at gmail.com> wrote:
> do:
>     <block>
> until <cond>
> 
> Written like this it is not very obvious that the 'unil' is part of
> the do-until suite. I also imagine it to be difficult to parse and it
> breaks the rule that suites end when there is a dedentation. So, IMHO
> using an indented 'until' is the least evil of a number of evils.

Not difficult to parse at all, nor un-Pythonic. Multi-part blocks
abound in Python: if / elif / else, try / finally, etc.

> > Why are you so excited about having until indented? You didn't give
> > any examples with multiple occurrences. A single occurrence works just
> > fine unindented, as PEP 315 has already shown.
> >
> > The indented until sounds like unnecessary syntactic sugar for 'if X:
> > break' -- not very Pythonic.
> 
> Yes, but grepping the stdlib produces over 300 hits for "while 1:" and
> "while True:" combined. Some of those a "if <cond>: break" in the
> middle and some would be better written as generators, but lots of
> them would be rewritten as do-while's. So I think there is more than
> enough use cases for syntactic sugar for do-while loops.

The PEP 315 solution looks much better than an "until" that isn't what
it looks like.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mclay at python.net  Mon Jun 13 16:16:53 2005
From: mclay at python.net (Michael McLay)
Date: Mon, 13 Jun 2005 10:16:53 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <42AD770A.1040001@gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com>
Message-ID: <200506131016.53962.mclay@python.net>

On Monday 13 June 2005 08:07, Nick Coghlan wrote:
> Raymond Hettinger wrote:
> > [BJ?rn Lindqvist]
> >
> >>I would like to have do-while's like this:
> >>
> >>do:
> >>    <body>
> >>    until <cond>
> >>
> >>But I'm sure that has problems too.
> >
> > That looks nice to me.
>
> And this could easily be extended to allow code both before and after
> the 'until', giving a fully general loop:
>
>    do:
>        <body-1-or-more-times>
>        until <cond>
>        <body-0-or-more-times>
>    else:
>        <on-natural-loop-exit>
>
> In fact, this would simply be giving "looks like executable
> pseudocode" syntactic sugar for the current 'do-until' workarounds:
>
>    while 1:
>      <body-1-or-more-times>
>      if <cond>:
>          <on-natural-loop-exit>
>          break
>      <body-0-or-more-times>

Yet another way to spell this would be make the default for the while 
statement be true so the 1 could be omitted and then add a condition to 
break.  

    while:
      <body-1-or-more-times>
      break <cond>:
          <on-natural-loop-exit>
      <body-0-or-more-times>

 I think this would be feature creep. It complicates the language for a very 
small gain. While the added syntax would be intuitive, it only saves a line 
or two over the existing syntax. 


From mwh at python.net  Mon Jun 13 20:06:21 2005
From: mwh at python.net (Michael Hudson)
Date: Mon, 13 Jun 2005 19:06:21 +0100
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <200506131016.53962.mclay@python.net> (Michael McLay's message
	of "Mon, 13 Jun 2005 10:16:53 -0400")
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com> <200506131016.53962.mclay@python.net>
Message-ID: <2mwtoydtcy.fsf@starship.python.net>

Michael McLay <mclay at python.net> writes:

>  I think this would be feature creep. It complicates the language
> for a very small gain. While the added syntax would be intuitive, it
> only saves a line or two over the existing syntax.

FWIW, this is my opinion too (and I've written a bunch of while
True:...if x: break loops today, even).

It would be much nicer if people spent time worrying about the
threads in multiple subinterpreters bug, or why 

def sc(*classes):
    return type('', classes, {})

sc(sc(str), sc(str))

fails.

Cheers,
mwh

-- 
  Guido (like us!) is a bit schizophrenic here: he wants to be a
  benevolent dictator, but also wants to treat people like
  grownups. This probably worked better before Python got a large
  American audience <0.9 wink>.             -- Tim Peters, 10 Feb 2000

From mkent at webmd.net  Mon Jun 13 20:21:36 2005
From: mkent at webmd.net (Michael Kent)
Date: Mon, 13 Jun 2005 18:21:36 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?AIX_4=2E3=2C_Python_2=2E4=2E1_fails_in_tes?=
	=?utf-8?q?t=5Fexceptions_with_a_core_dump?=
Message-ID: <loom.20050613T201451-169@post.gmane.org>

I'm attempting to switch from 2.3.2 to 2.4.1 on our AIX 4.3 development system.
 I have no problems building Python 2.3.2.  I build Python 2.4.1 using
'configure --without-threads; gmake; gmake test', and always get a coredump
during the tests on 'test_exceptions'.  I've searched for any reports of this
problem on the Sourceforge bug list and here, with no success.  Has anyone seen
this problem?  


From mcherm at mcherm.com  Mon Jun 13 21:36:33 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon, 13 Jun 2005 12:36:33 -0700
Subject: [Python-Dev] Wishlist: dowhile
Message-ID: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>

Jp Calderone writes:
>  for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''):
>      f2.write(chunk)

Phillip J. Eby responds:
> Showoff.  ;)
>
> More seriously, I think your translation makes an excellent argument in
> *favor* of having a do/while statement for greater clarity.  :)

Interesting... I had the opposite reaction. I often see someone do
something "far too clever", which looks cute and impresses me, but
makes me think that one would be wiser to avoid clever tricks and
just write straightforward (if dull) code.

But when I read JP's code I immediately thought that it WAS the
straightforward way to write that code, and that I was just not
smart enough to have realized it until he showed me.

-- Michael Chermside


From ark at acm.org  Mon Jun 13 22:24:37 2005
From: ark at acm.org (Andrew Koenig)
Date: Mon, 13 Jun 2005 16:24:37 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <E1Dh78a-0006Uj-A4@swing.co.at>
Message-ID: <007301c57055$eb99bf60$6402a8c0@arkdesktop>


> > But I cannot do this:
> >
> > 	def f(x as (a, b)):
> > 		# ...

> > which is what I was suggesting.

> Yeah, I knew that. How important is that to you?

I would really have liked it for a program I was working on at one time that
had lots of small methods that passed around values of forms such as
(a,(b,c)).  Each method would pick one or more of the components out of its
compound value and then would sometimes pass the entire value down to the
next level.

Of course I could do it this way:

	def f(x):
		(a,(b,c)) = x
		# ...

but in this particular application, "as" would have offered significant
extra convenience.

I understand that this is not a real big deal, which is why I haven't
written up a PEP :-)



From python at rcn.com  Mon Jun 13 22:46:18 2005
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 13 Jun 2005 16:46:18 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <20050613121423.5047.539927633.divmod.quotient.2056@ohm>
Message-ID: <000001c57058$f30d5f60$3f21a044@oemcomputer>

[Jp Calderone]
> Anything can be a for loop.
> 
>   for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''):
>       f2.write(chunk)

It would be nice to have this encapsulated in a file method:

    for chunk in f1.iterblocks(CHUNK_SIZE):
        f2.write(chunk)


Raymond

From tdelaney at avaya.com  Tue Jun 14 00:38:03 2005
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue, 14 Jun 2005 08:38:03 +1000
Subject: [Python-Dev] PEP 342 - Enhanced Iterators
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE025205A6@au3010avexu1.global.avaya.com>

There's a thread on c.l.py at the moment ("Controlling a generator the
pythonic way") which is basically coming up with PEP 342. I've pointed
them to PEP 342, but it's made me think that the name of the PEP could
better indicate what it does.

I propose "Coroutines via Enhanced Iterators".

Tim Delaney

From aahz at pythoncraft.com  Tue Jun 14 01:25:12 2005
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 13 Jun 2005 16:25:12 -0700
Subject: [Python-Dev] AIX 4.3,
	Python 2.4.1 fails in test_exceptions with a core dump
In-Reply-To: <loom.20050613T201451-169@post.gmane.org>
References: <loom.20050613T201451-169@post.gmane.org>
Message-ID: <20050613232511.GA590@panix.com>

On Mon, Jun 13, 2005, Michael Kent wrote:
>
> I'm attempting to switch from 2.3.2 to 2.4.1 on our AIX 4.3
> development system.  I have no problems building Python 2.3.2.  I
> build Python 2.4.1 using 'configure --without-threads; gmake;
> gmake test', and always get a coredump during the tests on
> 'test_exceptions'.  I've searched for any reports of this problem on
> the Sourceforge bug list and here, with no success.  Has anyone seen
> this problem?

Your question is more on the borderline than most, but please try
comp.lang.python before posting here; although you've got more core
developers on python-dev, there's a broader spectrum on c.l.py.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From gvanrossum at gmail.com  Tue Jun 14 03:30:40 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon, 13 Jun 2005 18:30:40 -0700
Subject: [Python-Dev] PEP 342 - Enhanced Iterators
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE025205A6@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE025205A6@au3010avexu1.global.avaya.com>
Message-ID: <ca471dc205061318304ac6a092@mail.gmail.com>

Good idea, done.

On 6/13/05, Delaney, Timothy C (Timothy) <tdelaney at avaya.com> wrote:
> There's a thread on c.l.py at the moment ("Controlling a generator the
> pythonic way") which is basically coming up with PEP 342. I've pointed
> them to PEP 342, but it's made me think that the name of the PEP could
> better indicate what it does.
> 
> I propose "Coroutines via Enhanced Iterators".
> 
> Tim Delaney
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From iextream at naver.com  Tue Jun 14 04:18:32 2005
From: iextream at naver.com (=?euc-kr?B?IsGkw+K/rCI=?=)
Date: Tue, 14 Jun 2005 11:18:32 +0900 (KST)
Subject: [Python-Dev] PEP 342 - Enhanced Iterators
Message-ID: <42AE3E78.000001.05103@nhn429>



<-----Original Message----->
> From: "Guido van Rossum" <gvanrossum at gmail.com>
> To: "Delaney, Timothy C (Timothy)" <tdelaney at avaya.com>
> Cc: <python-dev at python.org>
> Sent: 2005-06-14 10:31
> Subject: Re: [Python-Dev] PEP 342 - Enhanced Iterators

Good idea, done.

On 6/13/05, Delaney, Timothy C (Timothy) <tdelaney at avaya.com> wrote:
> There's a thread on c.l.py at the moment ("Controlling a generator the
> pythonic way") which is basically coming up with PEP 342. I've pointed
> them to PEP 342, but it's made me think that the name of the PEP could
> better indicate what it does.
> 
> I propose "Coroutines via Enhanced Iterators".
> 
> Tim Delaney
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/iextream%40naver.com




------------------------------------------------------------------------
??? :: ?! ???? ?? ??? ??? ?????.
http://mail.naver.com/

From greg.ewing at canterbury.ac.nz  Tue Jun 14 04:20:26 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 14 Jun 2005 14:20:26 +1200
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>
References: <42ACF1CF.5000203@gmail.com>
	<000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
	<5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>
Message-ID: <42AE3EEA.7090704@canterbury.ac.nz>

Phillip J. Eby wrote:

> By the way, whatever happened to "and while"?  i.e.:
> 
>      while True:
>          data = inp.read(blocksize)
>      and while data:
>          out.write(data)

My favourite version of this is

   while:
     data = inp.read(blocksize)
   gives data:
     out.write(data)

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From bob at redivi.com  Tue Jun 14 04:27:31 2005
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 13 Jun 2005 22:27:31 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <42AE3EEA.7090704@canterbury.ac.nz>
References: <42ACF1CF.5000203@gmail.com>
	<000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
	<5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com>
	<42AE3EEA.7090704@canterbury.ac.nz>
Message-ID: <ABDFF7D0-8188-400A-A125-2DF71AFB0860@redivi.com>


On Jun 13, 2005, at 10:20 PM, Greg Ewing wrote:

> Phillip J. Eby wrote:
>
>
>> By the way, whatever happened to "and while"?  i.e.:
>>
>>      while True:
>>          data = inp.read(blocksize)
>>      and while data:
>>          out.write(data)
>>
>
> My favourite version of this is
>
>    while:
>      data = inp.read(blocksize)
>    gives data:
>      out.write(data)

Well, we could always just make iterator factories for the common  
cases and stuff them in itertools.  I do need to use this pattern  
from time to time, but primarily for this exact use case, so an  
"itertools.readby" or the like would probably solve this problem for  
most people most of the time.

-bob


From python at rcn.com  Tue Jun 14 07:25:38 2005
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 14 Jun 2005 01:25:38 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ABDFF7D0-8188-400A-A125-2DF71AFB0860@redivi.com>
Message-ID: <001a01c570a1$80625260$3f21a044@oemcomputer>

> >> By the way, whatever happened to "and while"?  i.e.:
> >>
> >>      while True:
> >>          data = inp.read(blocksize)
> >>      and while data:
> >>          out.write(data)
> >>
> >
> > My favourite version of this is
> >
> >    while:
> >      data = inp.read(blocksize)
> >    gives data:
> >      out.write(data)
> 
> Well, we could always just make iterator factories for the common
> cases and stuff them in itertools.  I do need to use this pattern
> from time to time, but primarily for this exact use case, so an
> "itertools.readby" or the like would probably solve this problem for
> most people most of the time.

Doesn't work.  You still need to be able to pass through the blocksize
argument and the target bound method.  To get what you want, there would
need to be a new method in the file API.  The reason is roughly similar
to why we have iteritems as part of the mapping API and not as a
standalone itertool.

    for data in inp.read(blocksize):
        . . .


Raymond

From bob at redivi.com  Tue Jun 14 07:44:31 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 14 Jun 2005 01:44:31 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <001a01c570a1$80625260$3f21a044@oemcomputer>
References: <001a01c570a1$80625260$3f21a044@oemcomputer>
Message-ID: <35C64EB9-1FCC-4169-9331-D974DBC8DCC5@redivi.com>


On Jun 14, 2005, at 1:25 AM, Raymond Hettinger wrote:

>>>> By the way, whatever happened to "and while"?  i.e.:
>>>>
>>>>      while True:
>>>>          data = inp.read(blocksize)
>>>>      and while data:
>>>>          out.write(data)
>>>>
>>>>
>>>
>>> My favourite version of this is
>>>
>>>    while:
>>>      data = inp.read(blocksize)
>>>    gives data:
>>>      out.write(data)
>>>
>>
>> Well, we could always just make iterator factories for the common
>> cases and stuff them in itertools.  I do need to use this pattern
>> from time to time, but primarily for this exact use case, so an
>> "itertools.readby" or the like would probably solve this problem for
>> most people most of the time.
>>
>
> Doesn't work.  You still need to be able to pass through the blocksize
> argument and the target bound method.  To get what you want, there  
> would
> need to be a new method in the file API.  The reason is roughly  
> similar
> to why we have iteritems as part of the mapping API and not as a
> standalone itertool.
>
>     for data in inp.read(blocksize):
>         . . .

Reality check?

def readby(inp, blocksize=1024):
     while True:
         data = inp.read(blocksize)
         if not data:
             break
         yield data

for data in readby(inp, blocksize):
     . . .

I always thought was that iteritems is part of the mapping API and  
not as a standalone itertool because it is the *most primitive* way  
to iterate over the key/value pairs.  It is simply not implementable  
without access to the private guts of the dict.  That is simply not  
the case here.

If you need an ultra-flexible (but less obvious) implementation you  
should just use the sentinel version of the iter function as JP  
demonstrated.

-bob


From fredrik at pythonware.com  Tue Jun 14 08:16:54 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 14 Jun 2005 08:16:54 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
References: <d86o5t$euh$1@sea.gmane.org> <d86op2$gu2$1@sea.gmane.org>
	<20050610144835.GB5929@burma.localdomain>
Message-ID: <d8lsfg$d4o$1@sea.gmane.org>

Gustavo Niemeyer wrote:

> > > moving the main trunk and main development over to the Python CVS is
> > > another thing, entirely.
> >
> > (as I've said many times before, both the user community and the developer
> > community would benefit if the core standard library were made smaller, and
> > more externally maintained packages were included in the standard releases)
>
> An issue to consider about this is that maintainers (not talking about
> you or anyone else specifically) have different concepts of stability,
> and while it may seem perfectly ok to refactor external modules between
> two stable releases, doing so in the standard library would spread fear
> and "python is so untrustful" feelings.

I think you're seriously underestimating the competence of other
developers, and seriously overestimating the competence of the
python core developers.

in my experience, any external library that supports more than one
Python version on more than one platform is likely to be more robust
than code in the core.  add the multilevel volunteer approach de-
described by Steven (with the right infrastructure, things like that
just appear), and you get more competent manpower contributing
to the standard distribution than you can get in any other way.

</F>




From python at rcn.com  Tue Jun 14 08:25:40 2005
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 14 Jun 2005 02:25:40 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ABDFF7D0-8188-400A-A125-2DF71AFB0860@redivi.com>
Message-ID: <000001c570a9$e37c8660$3f21a044@oemcomputer>

> def readby(inp, blocksize=1024):
>      while True:
>          data = inp.read(blocksize)
>          if not data:
>              break
>          yield data
> 
> for data in readby(inp, blocksize):
>      . . .

readby() relies on the existence of a read() method for inp. 
itertools work with generic iterators, not ones with a specific API.
Law of Demeter. 



Raymond

From mcherm at mcherm.com  Tue Jun 14 14:09:33 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue, 14 Jun 2005 05:09:33 -0700
Subject: [Python-Dev] Wishlist: dowhile
Message-ID: <20050614050933.cl98pa45q6o8k0ws@login.werra.lunarpages.com>

Jp Calderone writes:
> Anything can be a for loop.
>
>   for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''):
>       f2.write(chunk)

Raymond responds:
> It would be nice to have this encapsulated in a file method:
>
>     for chunk in f1.iterblocks(CHUNK_SIZE):
>         f2.write(chunk)

What ever happened to "Not every 3 line function needs to be a builtin"?

It's a common pattern. It's easy to do. Where's the problem?

-- Michael Chermside


From bob at redivi.com  Tue Jun 14 14:41:14 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 14 Jun 2005 08:41:14 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <000001c570a9$e37c8660$3f21a044@oemcomputer>
References: <000001c570a9$e37c8660$3f21a044@oemcomputer>
Message-ID: <327DBAAC-947F-420B-A207-425D5CC4B5AB@redivi.com>


On Jun 14, 2005, at 2:25 AM, Raymond Hettinger wrote:

>> def readby(inp, blocksize=1024):
>>      while True:
>>          data = inp.read(blocksize)
>>          if not data:
>>              break
>>          yield data
>>
>> for data in readby(inp, blocksize):
>>      . . .
>>
>
> readby() relies on the existence of a read() method for inp.
> itertools work with generic iterators, not ones with a specific API.
> Law of Demeter.

islice depends on __getitem__.

-bob


From ncoghlan at gmail.com  Tue Jun 14 15:25:11 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 14 Jun 2005 23:25:11 +1000
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <5.1.1.6.0.20050613123758.0339dc00@mail.telecommunity.com>
References: <ca471dc205061219551accd1fe@mail.gmail.com>
	<ca471dc2050610132375ca7c70@mail.gmail.com>
	<42AAFBB6.2070802@gmail.com>
	<ca471dc205061109025a13c846@mail.gmail.com>
	<42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com>
	<42ABB59C.5050608@gmail.com>
	<ca471dc205061209561d7c26c9@mail.gmail.com>
	<42ACE839.1070804@gmail.com>
	<ca471dc205061219551accd1fe@mail.gmail.com>
	<5.1.1.6.0.20050613123758.0339dc00@mail.telecommunity.com>
Message-ID: <42AEDAB7.10601@gmail.com>

Phillip J. Eby wrote:
> Ow.  My head hurts.  :)  Seriously, though, wouldn't it make more sense 
> to put the 'with async_exceptions_blocked()' in the __init__ or 
> __enter__ of 'critical_resource'?  Or am I still missing something?

Heck, I *suggested* the trick, and still had to look at it a half 
dozen times, including frontways, and backways and upside down in 
order to satisfy myself that it actually addressed all the issues 
Guido and I had been talking about.

Anyway, the kicker is that (in the rare cases where it matters), you 
want asynchronous exceptions blocked before EXPR gets evaluated, and 
then unblocked again after the try/finally has been entered, but 
before BLOCK starts evaluating.

I believe the only real way to do that deterministically is to do the 
blocking and unblocking of asynchronous exceptions inline - when the 
asynchronous exception occurs, either the critical resource hasn't 
been acquired yet, or we are inside a try/finally that will ensure its 
release.

Trying to push the blocking/unblocking inside a template like 
safe_acquisition actually results in the loss of that determinism, 
since it may be the case that safe_acquisition itself is not finalised 
properly (the asynchronous exception goes off prior to entry of the 
relevant try/finally block). However, the generator object's __del__ 
method will clean things up eventually.

Anyway, the main point is that with statements won't make any more 
guarantees about this than try/finally statements do - but they do 
provide the tool to deal with it reasonably cleanly when it matters 
(most of the time the asynchronous error will indicate program 
shutdown, and the remote possibility of misbehaviour won't really 
matter in practice).

Cheers,
Nick.

P.S. I've included this trick as part of the Wiki discussion, meaning 
my last PEP 343 question has been well and truly dealt with.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From fperez.net at gmail.com  Tue Jun 14 16:07:12 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Jun 2005 08:07:12 -0600
Subject: [Python-Dev] A bug in pyconfig.h under Linux?
Message-ID: <d8mo1s$327$1@sea.gmane.org>

Hi all,

sorry for posting to this list, but I'm not even 100% sure this is a bug.  If it
is, I'll gladly post it to SF if you folks want it there.

I use scipy a lot, and the weave.inline component in there allows dynamic
inclusion of C/C++ code in Python sources.  In particular, it supports Blitz++
array expressions for access to Numeric arrays.  However, whenever I use
blitz-based code, I get these annoying warnings:

================================================================================
In file included from /usr/include/python2.3/Python.h:8,
                 from sc_weave.cpp:5:
/usr/include/python2.3/pyconfig.h:850:1: warning: "_POSIX_C_SOURCE" redefined
In file included
from /usr/lib/gcc/i386-redhat-linux/3.4.3/../../../../include/c++/3.4.3/i386-redhat-linux/bits/os_defines.h:39,
                
from /usr/lib/gcc/i386-redhat-linux/3.4.3/../../../../include/c++/3.4.3/i386-redhat-linux/bits/c++config.h:35,
                
from /usr/lib/gcc/i386-redhat-linux/3.4.3/../../../../include/c++/3.4.3/string:45,
                
from /usr/lib/python2.3/site-packages/weave/blitz-20001213/blitz/blitz.h:153,
                
from /usr/lib/python2.3/site-packages/weave/blitz-20001213/blitz/array-impl.h:154,
                
from /usr/lib/python2.3/site-packages/weave/blitz-20001213/blitz/array.h:94,
                 from sc_weave.cpp:4:
/usr/include/features.h:150:1: warning: this is the location of the previous
definition
================================================================================

This is on a Fedora Core 3 box, using glibc-headers.i386 version 2.3.5.

The source of the problem seems to be that in
file /usr/include/python2.3/pyconfig.h, line 850, I have:

/* Define to activate features from IEEE Stds 1003.1-2001 */
#define _POSIX_C_SOURCE 200112L

But the system headers, in /usr/include/features.h, line 150 give:

# define _POSIX_C_SOURCE        199506L

Hence the double-define.  Now, I noticed that the system headers all use the
following approach to defining these constants:

# undef  _POSIX_SOURCE
# define _POSIX_SOURCE  1
# undef  _POSIX_C_SOURCE
# define _POSIX_C_SOURCE        199506L

etc.  That is, they undef everything before defining their value.  I applied the
same change manually to pyconfig.h:

/* Define to activate features from IEEE Stds 1003.1-2001 */
#undef _POSIX_C_SOURCE
#define _POSIX_C_SOURCE 200112L

and my spurious warnings went away.  But I realize that pyconfig.h is
auto-generated, so the right solution, if this is indeed a bug, has to be
applied somewhere else, at the code generation source.  I am unfortunately not
familiar enough with Python's build system and the autoconf toolset to do that. 
Furthermore, I am not even 100% sure this is really a bug, though the spurious
warning is very annoying.

If this is indeed a bug, do you folks want it reported on SF as such?  In that
case, is this explanation enough/correct?  Any advice would be much
appreciated.

Regards,

Fernando.


From martin at v.loewis.de  Tue Jun 14 17:39:29 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 14 Jun 2005 17:39:29 +0200
Subject: [Python-Dev] A bug in pyconfig.h under Linux?
In-Reply-To: <d8mo1s$327$1@sea.gmane.org>
References: <d8mo1s$327$1@sea.gmane.org>
Message-ID: <42AEFA31.40004@v.loewis.de>

Fernando Perez wrote:
> sorry for posting to this list, but I'm not even 100% sure this is a bug.  If it
> is, I'll gladly post it to SF if you folks want it there.

This is not a bug. Most likely, sc_weave.cpp fails to meet the
requirement documented in

http://docs.python.org/api/includes.html

"Warning:  Since Python may define some pre-processor definitions which
affect the standard headers on some systems, you must include Python.h
before any standard headers are included. "

If you follow this recommendation, this kind of error should not occur.

Regards,
Martin

From fperez.net at gmail.com  Tue Jun 14 17:56:47 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Jun 2005 09:56:47 -0600
Subject: [Python-Dev] A bug in pyconfig.h under Linux?
References: <d8mo1s$327$1@sea.gmane.org> <42AEFA31.40004@v.loewis.de>
Message-ID: <d8muej$qng$1@sea.gmane.org>

"Martin v. L?wis" wrote:

> Fernando Perez wrote:
>> sorry for posting to this list, but I'm not even 100% sure this is a bug. 
>> If it is, I'll gladly post it to SF if you folks want it there.
> 
> This is not a bug. Most likely, sc_weave.cpp fails to meet the
> requirement documented in
> 
> http://docs.python.org/api/includes.html
> 
> "Warning:  Since Python may define some pre-processor definitions which
> affect the standard headers on some systems, you must include Python.h
> before any standard headers are included. "

Many thanks to Martin and Jeff Epler for the feedback, and sorry for the noise,
the bug was in weave and not in pyconfig.h. 

I was able to fix scipy's weave to respect this constraint, which it didn't in
the case of blitz-enhanced array handling code.

Regards,

f


From python at rcn.com  Tue Jun 14 18:21:08 2005
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 14 Jun 2005 12:21:08 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <327DBAAC-947F-420B-A207-425D5CC4B5AB@redivi.com>
Message-ID: <003e01c570fd$12a27280$3f21a044@oemcomputer>

[Bob]
> islice depends on __getitem__.

Incorrect.  It uses the iterator protocol.

From aqua_azul_2000 at yahoo.com.mx  Tue Jun 14 18:26:55 2005
From: aqua_azul_2000 at yahoo.com.mx (Vero)
Date: Tue, 14 Jun 2005 11:26:55 -0500 (CDT)
Subject: [Python-Dev] Dynamic class inheritance  && something else
Message-ID: <20050614162655.52654.qmail@web61019.mail.yahoo.com>

Hi.  My name is Veronica, I am a master student at UNAM.  I am working on something related to Artificial Inteligence and I have been looking for the most appropriated programming language to implement my algorithms.  I found python to be very close to what I need, but there are still a couple of details that I haven't been able to solve.
 
First, (and most important) I would like to be able to dynamically modify the classes from which a class inherits.  I haven't found any way to do it with the language as it is.  If there is a way, any suggestion will be truly appreciated.  If I had to modify the source code to achieve this, I hope that you could give me some hints; I have an idea of how something like this could be achieved but since I don't know deeply the python sourcode I could get lost.
 
Second, since my program will be modifying classes during run time, I would like to have a way to write on a file the python code that would have defined the class with the functions and attributes as they were left, just as if it had been writen like that at the very begining.  I need it to be python code because I will be using that latter.  Maybe I will have to solve this second problem by myself but I just wrote in case anybody had a good idea.
 
Thank you very much for your help.


Vero
 
  La sociedad es inherentemente democr?tica: la mayor?a decide si pensar por si misma o si dejar que alguien m?s les diga qu? pensar.

 http://mx.geocities.com/aqua_azul_2000/ 






		
---------------------------------
  Do You Yahoo!? La mejor conexi?n a Internet y 2GB extra a tu correo por $100 al mes. http://net.yahoo.com.mx 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20050614/9adeafe8/attachment.htm

From aahz at pythoncraft.com  Tue Jun 14 18:29:44 2005
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 14 Jun 2005 09:29:44 -0700
Subject: [Python-Dev] Dynamic class inheritance  && something else
In-Reply-To: <20050614162655.52654.qmail@web61019.mail.yahoo.com>
References: <20050614162655.52654.qmail@web61019.mail.yahoo.com>
Message-ID: <20050614162944.GA28529@panix.com>

On Tue, Jun 14, 2005, Vero wrote:
>
> Hi.  My name is Veronica, I am a master student at UNAM.  I am working
> on something related to Artificial Inteligence and I have been looking
> for the most appropriated programming language to implement my
> algorithms.  I found python to be very close to what I need, but there
> are still a couple of details that I haven't been able to solve.

Hi Veronica,

python-dev is for future development of Python; for questions about
using Python, please go to comp.lang.python.  Thanks.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From gvanrossum at gmail.com  Tue Jun 14 18:38:31 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 14 Jun 2005 09:38:31 -0700
Subject: [Python-Dev] Dynamic class inheritance && something else
In-Reply-To: <20050614162655.52654.qmail@web61019.mail.yahoo.com>
References: <20050614162655.52654.qmail@web61019.mail.yahoo.com>
Message-ID: <ca471dc205061409384aeb185e@mail.gmail.com>

On 6/14/05, Vero <aqua_azul_2000 at yahoo.com.mx> wrote:
> Hi.  My name is Veronica, I am a master student at UNAM.  I am working on
> something related to Artificial Inteligence and I have been looking for the
> most appropriated programming language to implement my algorithms.  I found
> python to be very close to what I need, but there are still a couple of
> details that I haven't been able to solve. 
>   
> First, (and most important) I would like to be able to dynamically modify
> the classes from which a class inherits.  I haven't found any way to do it
> with the language as it is.  If there is a way, any suggestion will be truly
> appreciated.  If I had to modify the source code to achieve this, I hope
> that you could give me some hints; I have an idea of how something like this
> could be achieved but since I don't know deeply the python sourcode I could
> get lost. 

I wonder if assignment to __class__ (for instances) or to __bases__
(for classes) is what you are looking for?

>>> class C: 
     def foo(self): print "foo"
    
>>> class D:
     def foo(self): print "D.foo"
    
>>> x = C()
>>> x.__class__ = D
>>> x.foo()
D.foo
>>> x.__class__ = C
>>> x.foo()
foo
>>> 

> Second, since my program will be modifying classes during run time, I would
> like to have a way to write on a file the python code that would have
> defined the class with the functions and attributes as they were left, just
> as if it had been writen like that at the very begining.  I need it to be
> python code because I will be using that latter.  Maybe I will have to solve
> this second problem by myself but I just wrote in case anybody had a good
> idea. 

That one's not so easy; there's no standard solution for this problem.
I recommend that you follow Aahz' suggestion of asking on c.l.py
instead.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From bjourne at gmail.com  Tue Jun 14 20:24:44 2005
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Tue, 14 Jun 2005 20:24:44 +0200
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ca471dc2050613093690c1a40@mail.gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com>
	<ca471dc20506130741430f8811@mail.gmail.com>
	<740c3aec05061308536bceeead@mail.gmail.com>
	<ca471dc2050613093690c1a40@mail.gmail.com>
Message-ID: <740c3aec05061411241149a557@mail.gmail.com>

> > do:
> >     <block>
> > until <cond>
> >
> > Written like this it is not very obvious that the 'unil' is part of
> > the do-until suite. I also imagine it to be difficult to parse and it
> > breaks the rule that suites end when there is a dedentation. So, IMHO
> > using an indented 'until' is the least evil of a number of evils.
> 
> Not difficult to parse at all, nor un-Pythonic. Multi-part blocks
> abound in Python: if / elif / else, try / finally, etc.

Oh. I had the impression that the reason do-while's (or do-until's
which I like more) wasn't implemented in the language was because it
was impossible to find an acceptable looking syntax.

 > > Yes, but grepping the stdlib produces over 300 hits for "while 1:" and
> > "while True:" combined. Some of those a "if <cond>: break" in the
> > middle and some would be better written as generators, but lots of
> > them would be rewritten as do-while's. So I think there is more than
> > enough use cases for syntactic sugar for do-while loops.
> 
> The PEP 315 solution looks much better than an "until" that isn't what
> it looks like.

What do you mean by that? The proposed until would work exactly like
it do in languages that support until loops. I'm also reasonably happy
with PEP 315, except that it seems to force you to end it with a pass
if the condition is last in the loop. But is discussing do-while's
beating a dead horse or is there a possibility that do-while's will
someday make it into the language?

-- 
mvh Bj?rn

From eric.nieuwland at xs4all.nl  Tue Jun 14 20:26:37 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Tue, 14 Jun 2005 20:26:37 +0200
Subject: [Python-Dev] Wishlist:  dowhile
In-Reply-To: <42ACF1CF.5000203@gmail.com>
References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
Message-ID: <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl>

Nick Coghlan wrote:
> With PEP 315, a do-while loop would look like:
>
>    do:
>        <body>
>    while <cond>:
>        pass
>
> But then, I'm reasonably happy with the 'break out of an infinite
> loop' approach, so *shrug*.

 From Programming Languages 101 I remember this construct in Algol 68. 
It was then claimed to be *the* universal loop construct. If that is 
true __and__ it is easy to implement, I'd say +INF for PEP 315.

--eric


From gvanrossum at gmail.com  Tue Jun 14 20:46:39 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 14 Jun 2005 11:46:39 -0700
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <740c3aec05061411241149a557@mail.gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com>
	<ca471dc20506130741430f8811@mail.gmail.com>
	<740c3aec05061308536bceeead@mail.gmail.com>
	<ca471dc2050613093690c1a40@mail.gmail.com>
	<740c3aec05061411241149a557@mail.gmail.com>
Message-ID: <ca471dc205061411462aaf74df@mail.gmail.com>

On 6/14/05, BJ?rn Lindqvist <bjourne at gmail.com> wrote:
> Oh. I had the impression that the reason do-while's (or do-until's
> which I like more) wasn't implemented in the language was because it
> was impossible to find an acceptable looking syntax.

No, just because it's fairly redundant.

>  > > Yes, but grepping the stdlib produces over 300 hits for "while 1:" and
> > > "while True:" combined. Some of those a "if <cond>: break" in the
> > > middle and some would be better written as generators, but lots of
> > > them would be rewritten as do-while's. So I think there is more than
> > > enough use cases for syntactic sugar for do-while loops.
> >
> > The PEP 315 solution looks much better than an "until" that isn't what
> > it looks like.
> 
> What do you mean by that? The proposed until would work exactly like
> it do in languages that support until loops. I'm also reasonably happy
> with PEP 315, except that it seems to force you to end it with a pass
> if the condition is last in the loop. But is discussing do-while's
> beating a dead horse or is there a possibility that do-while's will
> someday make it into the language?

You are proposing 'until' that is indented. That is unlike any other
language. Translating this to curly-braces style, you're proposing
this:

  do {
    ...body...
    until (x <= 0);
  }

I find this bizarre, not " exactly like it do [sic] in languages that
support until loops."

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Tue Jun 14 20:48:03 2005
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue, 14 Jun 2005 14:48:03 -0400
Subject: [Python-Dev] Wishlist:  dowhile
In-Reply-To: <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl>
Message-ID: <001301c57111$985335e0$6402a8c0@arkdesktop>

> > With PEP 315, a do-while loop would look like:
> >
> >    do:
> >        <body>
> >    while <cond>:
> >        pass

> > But then, I'm reasonably happy with the 'break out of an infinite
> > loop' approach, so *shrug*.

>  From Programming Languages 101 I remember this construct in Algol 68.
> It was then claimed to be *the* universal loop construct. If that is
> true __and__ it is easy to implement, I'd say +INF for PEP 315.

I believe that Algol 68 loops looked like the following (in somewhat more
Python-like terms):

	[for <identifier> [from <suite>] [step <suite>] to <suite>]
	[while <suite>]
	do <suite> od

where either "for" or "while" had to be present.  Note that in Algol 68, the
value of a "suite" (which I think they called a "serial clause") is the
value of the last expression in it.



From gvanrossum at gmail.com  Tue Jun 14 20:48:53 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 14 Jun 2005 11:48:53 -0700
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl>
References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
	<7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl>
Message-ID: <ca471dc2050614114822e56963@mail.gmail.com>

On 6/14/05, Eric Nieuwland <eric.nieuwland at xs4all.nl> wrote:
>  From Programming Languages 101 I remember this construct in Algol 68.
> It was then claimed to be *the* universal loop construct. If that is
> true __and__ it is easy to implement, I'd say +INF for PEP 315.

It's true, but this both dates you (A68 has been dead for several
decades) and locates you: it was said that A68's popularity was
inversely proportional to (the square of?) the distance from
Amsterdam. It also dates and locates me. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric.nieuwland at xs4all.nl  Tue Jun 14 22:57:35 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Tue, 14 Jun 2005 22:57:35 +0200
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ca471dc2050614114822e56963@mail.gmail.com>
References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer>
	<42ACF1CF.5000203@gmail.com>
	<7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl>
	<ca471dc2050614114822e56963@mail.gmail.com>
Message-ID: <625faffa5cb24ecad85843dce2f090eb@xs4all.nl>

Guido van Rossum wrote:
> On 6/14/05, Eric Nieuwland <eric.nieuwland at xs4all.nl> wrote:
>>  From Programming Languages 101 I remember this construct in Algol 68.
>> It was then claimed to be *the* universal loop construct. If that is
>> true __and__ it is easy to implement, I'd say +INF for PEP 315.
>
> It's true, but this both dates you (A68 has been dead for several
> decades) and locates you: it was said that A68's popularity was
> inversely proportional to (the square of?) the distance from
> Amsterdam. It also dates and locates me. :-)

Lucky me ;-)


From gvanrossum at gmail.com  Tue Jun 14 23:11:47 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 14 Jun 2005 14:11:47 -0700
Subject: [Python-Dev] Compiling Python with Intel compiler?
Message-ID: <ca471dc2050614141119dacdf@mail.gmail.com>

Intel has a free (as in beer) C compiler for Linux. A friend of mine
is involved in its production and marketing. He would like to see how
"his" compiler does on Python -- does it work at all, and is there a
speedup? I promised him I'd look into this, but between PEPs and my
day job I don't seem to have the time, so now I'm making up on my
promise by trying to delegate...

The compiler is here:

  http://www.intel.com/software/products/compilers/clin/index.htm

On the right is a link to a "Free Non-Commercial Download".

That's all I know.

I think that if someone puts some time in this, they should get a free
Intel T-shirt.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Jun 14 23:40:54 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 14 Jun 2005 23:40:54 +0200
Subject: [Python-Dev] Compiling Python with Intel compiler?
In-Reply-To: <ca471dc2050614141119dacdf@mail.gmail.com>
References: <ca471dc2050614141119dacdf@mail.gmail.com>
Message-ID: <42AF4EE6.2040000@v.loewis.de>

Guido van Rossum wrote:
> Intel has a free (as in beer) C compiler for Linux. A friend of mine
> is involved in its production and marketing. He would like to see how
> "his" compiler does on Python -- does it work at all, and is there a
> speedup?

There is a bug report on this compiler: python.org/sf/1162001
There is also a patch, python.org/sf/1162023 which works around
the bug. I don't like the work-around, because it assumes that the
compiler binary is called "icc", however, it might be called just
"cc" on some installations.

With that issue resolved, the compiler apparently can build Python.
However, the resulting interpreter doesn't work for some people:

http://mail.python.org/pipermail/python-list/2005-March/270672.html

This issue apparently only exists if you use both -O3 and -xN;
if you omit one of them, it builds fine. I suspect a compiler
bug.

As for performance, Perky compared ICC (unspecified version) with
gcc 2.95 on FreeBSD 4.5:

http://groups-beta.google.com/group/sol.lists.freebsd.ports/msg/90a736b7097d9b05

At that point, the then-current version of ICC produced faster code
than the then-current version of gcc (this was in 2002).

In Dec 2004, Mark Asbach did the same thing with ICC 8 vs. gcc 3.3.4,
on Linux:

http://groups-beta.google.com/group/comp.lang.python/msg/a4af997f8f95592e?hl=en

He reported a small speedup.

I couldn't find anything on Python and ICC V9.

Regards,
Martin


From kbk at shore.net  Wed Jun 15 03:14:02 2005
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue, 14 Jun 2005 21:14:02 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200506150114.j5F1E2Zf015509@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  338 open ( -1) /  2861 closed ( +4) /  3199 total ( +3)
Bugs    :  909 open ( +1) /  5047 closed (+11) /  5956 total (+12)
RFE     :  188 open ( -1) /   170 closed ( +2) /   358 total ( +1)

New / Reopened Patches
______________________

byteorder issue in PyMac_GetOSType  (2005-06-10)
       http://python.org/sf/1218421  opened by  Ronald Oussoren

os.kill on windows  (2005-06-14)
       http://python.org/sf/1220212  opened by  Miki Tebeka

_csv.c leaks when dialect creation fails  (2005-06-14)
       http://python.org/sf/1220242  opened by  Michael Hudson

Patches Closed
______________

in IDLE, assume new text files are python source by default  (2005-05-06)
       http://python.org/sf/1196895  closed by  kbk

Add const specifier to PySpam_System prototype  (2005-04-21)
       http://python.org/sf/1187396  closed by  birkenfeld

Newline in error output of py_compile.compile  (2005-03-26)
       http://python.org/sf/1171150  closed by  birkenfeld

trivial bug in error message text  (2005-05-06)
       http://python.org/sf/1196980  closed by  kbk

New / Reopened Bugs
___________________

email.Utils.py: "'" in RFC2231 header  (2005-06-10)
       http://python.org/sf/1218081  opened by  Tokio Kikuchi

inspect.getsource doesn't update when a module is reloaded  (2005-06-10)
       http://python.org/sf/1218234  opened by  Bj?rn Lindqvist

Parser chokes on big files  (2005-06-11)
CLOSED http://python.org/sf/1218930  opened by  Alexander Schremmer

'Expression' AST Node not documented  (2005-06-12)
       http://python.org/sf/1219273  opened by  Martin Miller

copy.py typo  (2005-06-13)
CLOSED http://python.org/sf/1219342  opened by  Ori Avtalion

copy.py typo  (2005-06-13)
CLOSED http://python.org/sf/1219361  opened by  Ori Avtalion

small output bug  (2005-06-12)
CLOSED http://python.org/sf/1219448  opened by  Alex Levchuk

Need locale arg to strftime()  (2005-06-13)
       http://python.org/sf/1219840  opened by  Wilfredo Sanchez

misdocumented argument range for curses.pair_content  (2005-06-13)
CLOSED http://python.org/sf/1219862  opened by  dcrosta

tp_richcompare documentation wrong and incomplete  (2005-06-13)
       http://python.org/sf/1219903  opened by  Barry A. Warsaw

subprocess call() helper should close stdin if PIPE  (2005-06-14)
       http://python.org/sf/1220113  opened by  Stuart Bishop

Re-importing embedded thread dumps core  (2005-06-14)
       http://python.org/sf/1220756  opened by  Jay T Miller

Bugs Closed
___________

Parser chokes on big files  (2005-06-11)
       http://python.org/sf/1218930  closed by  doerwalter

color highlighting not working on EDIT windows  (2003-09-04)
       http://python.org/sf/800432  closed by  kbk

No syntax hilite on new file until saved as *.py  (2003-07-21)
       http://python.org/sf/775012  closed by  kbk

copy.py typo  (2005-06-12)
       http://python.org/sf/1219342  closed by  rhettinger

copy.py typo  (2005-06-12)
       http://python.org/sf/1219361  closed by  rhettinger

small output bug  (2005-06-12)
       http://python.org/sf/1219448  closed by  rhettinger

misdocumented argument range for curses.pair_content  (2005-06-13)
       http://python.org/sf/1219862  closed by  akuchling

lax error-checking in new-in-2.4 marshal stuff  (2005-04-11)
       http://python.org/sf/1180997  closed by  mwh

String and list methods documentation deeply hidden  (2005-06-06)
       http://python.org/sf/1215887  closed by  rhettinger

Queue.qsize() better info about unreliability  (2005-06-02)
       http://python.org/sf/1213475  closed by  rhettinger

weakref cannot handle bound methods (in contrast to docu)  (2005-05-22)
       http://python.org/sf/1206537  closed by  rhettinger

New / Reopened RFE
__________________

Create a fat build on darwin  (2005-06-10)
       http://python.org/sf/1218333  opened by  Ronald Oussoren

RFE Closed
__________

Make bisect.* functions accept an optional compare function  (2005-04-18)
       http://python.org/sf/1185383  closed by  rhettinger

sets needs an 'arbitrary element' method  (2005-05-31)
       http://python.org/sf/1212091  closed by  rhettinger


From raymond.hettinger at verizon.net  Wed Jun 15 11:09:21 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 15 Jun 2005 05:09:21 -0400
Subject: [Python-Dev] PEP 343 question
Message-ID: <000201c57189$eb273280$1c2ba044@oemcomputer>

In gen.throw(), are all three arguments required?  Or do the value and
traceback Nones need to be listed explicitly?
 
   g.throw(MyException)
 
or
 
   g.throw(MyException, None, None)
 
 
FWIW, I prefer the former.  That will make throw() as flexible as the
raise statement.
 
 
Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20050615/bc65f786/attachment.htm

From ncoghlan at gmail.com  Wed Jun 15 12:57:57 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 15 Jun 2005 20:57:57 +1000
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ca471dc20506130741430f8811@mail.gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>	
	<42AD770A.1040001@gmail.com>
	<ca471dc20506130741430f8811@mail.gmail.com>
Message-ID: <42B009B5.4040904@gmail.com>

Guido van Rossum wrote:
> Why are you so excited about having until indented? You didn't give
> any examples with multiple occurrences. A single occurrence works just
> fine unindented, as PEP 315 has already shown.

I hadn't actually thought about the temptation to try and have 
multiple 'until' statements inside the suite. Having two suites of the 
statement included in the loop, while the third is excluded, feels 
slightly odd, but other than that PEP 315 is fine by me.

The fact that PEP 315 only needs a single new keyword is a good thing, 
as is the fact that it means the 'else' clause still reads correctly.

As you said earlier: "It is Pythonically minimal and the motivation 
rings true"

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From mwh at python.net  Wed Jun 15 14:35:35 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 15 Jun 2005 13:35:35 +0100
Subject: [Python-Dev] refcounting vs PyModule_AddObject
Message-ID: <2mzmtrbxwo.fsf@starship.python.net>

I've just fixed a bug where Py_INCREF wasn't called when it should
have been before a call to PyModule_AddObject (rev. 2.62 of
Modules/threadmodule.c).

So I went looking for other instances of the same problem.  I didn't
find any (though I don't understand how _csv.c gets away with line
1579), but what I *did* find were absolutely masses of what seemed
like unnecessary increfs.  Consider this code from _hotshot.c:

        Py_INCREF(&LogReaderType);
        PyModule_AddObject(module, "LogReaderType",
                           (PyObject *)&LogReaderType);
        Py_INCREF(&ProfilerType);
        PyModule_AddObject(module, "ProfilerType",
                           (PyObject *)&ProfilerType);

        if (ProfilerError == NULL)
            ProfilerError = PyErr_NewException("hotshot.ProfilerError",
                                               NULL, NULL);
        if (ProfilerError != NULL) {
            Py_INCREF(ProfilerError);
            PyModule_AddObject(module, "ProfilerError", ProfilerError);
        }

The first two calls are fine; it was an incref like this that was
missing in threadmodule.c.  The second seems like a reference "leak":
PyErr_NewException returns a new reference, then the
Py_INCREF/PyModule_AddObject pair is refcount neutral, so control
falls off the end of the function owning a reference to ProfilerError.

I think the Py_INCREF should just be removed, but I'm wondering if I'm
missing something...

Cheers,
mwh

-- 
  MacOSX: Sort of like a pedigree persian cat. Very sleek, very
  sexy, but a little too prone to going cross-eyed, biting you on
  your thumb and then throwing up on your trousers.
                           -- Jim's pedigree of operating systems, asr

From hoffman at ebi.ac.uk  Wed Jun 15 12:13:37 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Wed, 15 Jun 2005 11:13:37 +0100
Subject: [Python-Dev] Compiling Python with Intel compiler?
In-Reply-To: <42AF4EE6.2040000@v.loewis.de>
References: <ca471dc2050614141119dacdf@mail.gmail.com>
	<42AF4EE6.2040000@v.loewis.de>
Message-ID: <Pine.LNX.4.62.0506151028410.374@qnzvnan.rov.np.hx>

On Tue, 14 Jun 2005, "Martin v. L?wis" wrote:

> Guido van Rossum wrote:
>> Intel has a free (as in beer) C compiler for Linux. A friend of mine
>> is involved in its production and marketing. He would like to see how
>> "his" compiler does on Python -- does it work at all, and is there a
>> speedup?
>
> There is a bug report on this compiler: python.org/sf/1162001
> There is also a patch, python.org/sf/1162023 which works around
> the bug. I don't like the work-around, because it assumes that the
> compiler binary is called "icc", however, it might be called just
> "cc" on some installations.

I submitted this bug, patch, and the c.l.p posting and thank Martin
for his help with them all. Martin's reservations on the workaround
are correct but I can't make more time right now to fix autoconf
"properly."

To be honest, if icc were more gcc-compatible and returned an error
code for -OPT:Olimit=0, that would obviate the need for any patch to
Python.

I never published these results before but here's what I found in
terms of performance (full results available on request):

$ python pybench.py -s icc-python -c fedora-python | tail -n 4
             Average round time:    3677.00 ms              -15.95%

*) measured against: fedora-python (rounds=10, warp=20)

$ python pybench.py -s icc-xN-python -c fedora-python | tail -n 4
             Average round time:    3911.00 ms              -10.61%

*) measured against: fedora-python (rounds=10, warp=20)

fedora-python: Python 2.4/GCC 3.4.2 20041017 from python.org yum repository
icc-python: Python 2.4/icc (8.1) -pthread -fno-strict-aliasing -DNDEBUG -O3
icc-python-xN: Python2.4/icc (8.1) -pthread -fno-strict-aliasing -DNDEBUG -O3 -xN

Intel Pentium 4 2.66GHz; 512 KB cache; 507644 kB RAM; Linux 2.6.11-1.14_FC3

While a 16% speedup is nothing to sneeze at, I felt I wasted a lot of
time dealing with incompatibilities with extension modules and
libcprts.so, and so I no longer run it on my desktop. I do use it for
heavy computes on our compute farm when I'm not using extension modules.

[Guido]
> I think that if someone puts some time in this, they should get a
> free Intel T-shirt.

I'll accept a free Intel T-shirt if one is offered but I'll probably
pass it on to our long-suffering sysadmins who have spent even more
time dealing with icc, yet remain evangelists for it.

Since this was my first message to python-dev, I think the custom is
to introduce myself. I'm a PhD student at the University of Cambridge
studying computational biology. I've been using Python for the last
three years and have loved almost every minute of it. I've written
sequence alignment algorithms in Pyrex, glue to genome database APIs
in Jython, and a whole lot of other stuff using CPython. We run Python
on a 1145-CPU compute farm that consists of Alpha/Tru64 and
Intel/Linux boxes.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From ark-mlist at att.net  Wed Jun 15 14:52:32 2005
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 15 Jun 2005 08:52:32 -0400
Subject: [Python-Dev] Wishlist:  dowhile
In-Reply-To: <77991a6494eb379e2fc724e26b4103cc@xs4all.nl>
Message-ID: <001101c571a9$1939d780$6402a8c0@arkdesktop>

> > I believe that Algol 68 loops looked like the following (in somewhat
> > more Python-like terms):
> >
> > 	[for <identifier> [from <suite>] [step <suite>] to <suite>]
> > 	[while <suite>]
> > 	do <suite> od


> As far as I remember, it was:

> do
> 	<suite>
> while <suite; ending with a boolean value>
> 	<suite>
> od

> It might be that this could be preceded by a 'for' clause, but memory
> fails me there.

To be completely sure, I dug out my copy of the "Informal Introduction to
Algol 68" by Lindsey and van der Meulen.  This book is the official Algol 68
tutorial book.  It describes the "loop clause" as follows:

	for
		some int identifier, which is hereby declared
	from
		some meek int unit
	by
		some other meek int unit
	to
		a third meek int unit
	while
		a meek bool enquiry-clause
	do
		a void serial-clause
	od

where any of these keywords and its sequel can be omitted, except for "do"
or "od".

Here, the term "meek" refers to how aggressively the compiler should try to
convert the result to int; it specifies, for example, that if after "from"
you write the name of a procedure without arguments, that procedure will be
called instead of using the procedure itself as a starting point (which
would be nonsensical).

	


From skip at pobox.com  Wed Jun 15 14:57:00 2005
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 15 Jun 2005 07:57:00 -0500
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <2mzmtrbxwo.fsf@starship.python.net>
References: <2mzmtrbxwo.fsf@starship.python.net>
Message-ID: <17072.9628.999188.106351@montanaro.dyndns.org>


    Michael> So I went looking for other instances of the same problem.  I
    Michael> didn't find any (though I don't understand how _csv.c gets away
    Michael> with line 1579)...

Same reason the Py_INCREF of ProfileError isn't necessary I think.
PyDict_New() returns a new reference which is passed onto
PyModule_AddObject().  No Py_DECREF of the dialects dict occurs, so it exits
reference-neutral w.r.t. the dialects object.

Skip

From mwh at python.net  Wed Jun 15 15:05:25 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 15 Jun 2005 14:05:25 +0100
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <17072.9628.999188.106351@montanaro.dyndns.org> (Skip
	Montanaro's message of "Wed, 15 Jun 2005 07:57:00 -0500")
References: <2mzmtrbxwo.fsf@starship.python.net>
	<17072.9628.999188.106351@montanaro.dyndns.org>
Message-ID: <2mvf4fbwiy.fsf@starship.python.net>

Skip Montanaro <skip at pobox.com> writes:

>     Michael> So I went looking for other instances of the same problem.  I
>     Michael> didn't find any (though I don't understand how _csv.c gets away
>     Michael> with line 1579)...
>
> Same reason the Py_INCREF of ProfileError isn't necessary I think.
> PyDict_New() returns a new reference which is passed onto
> PyModule_AddObject().  No Py_DECREF of the dialects dict occurs, so it exits
> reference-neutral w.r.t. the dialects object.

Oops; I meant line 1590.

Cheers,
mwh

-- 
  Windows 2000: Smaller cow. Just as much crap.
                           -- Jim's pedigree of operating systems, asr

From skip at pobox.com  Wed Jun 15 15:40:10 2005
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 15 Jun 2005 08:40:10 -0500
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <2mvf4fbwiy.fsf@starship.python.net>
References: <2mzmtrbxwo.fsf@starship.python.net>
	<17072.9628.999188.106351@montanaro.dyndns.org>
	<2mvf4fbwiy.fsf@starship.python.net>
Message-ID: <17072.12218.103715.151682@montanaro.dyndns.org>


    Michael> ... (though I don't understand how _csv.c gets away
    Michael> with line 1579)...

    Michael> Oops; I meant line 1590.

Hmmm...  Me either.  Is it possible it was just never DECREF'd?  I checked
in the obvious fix on both head and the 2.4 release branch.

Skip

From arigo at tunes.org  Wed Jun 15 15:51:10 2005
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 15 Jun 2005 15:51:10 +0200
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <2mzmtrbxwo.fsf@starship.python.net>
References: <2mzmtrbxwo.fsf@starship.python.net>
Message-ID: <20050615135110.GA2560@code1.codespeak.net>

Hi Michael,

On Wed, Jun 15, 2005 at 01:35:35PM +0100, Michael Hudson wrote:
>         if (ProfilerError == NULL)
>             ProfilerError = PyErr_NewException("hotshot.ProfilerError",
>                                                NULL, NULL);
>         if (ProfilerError != NULL) {
>             Py_INCREF(ProfilerError);
>             PyModule_AddObject(module, "ProfilerError", ProfilerError);
>         }

I think the Py_INCREF is needed here.  The ProfilerError is a global
variable that needs the extra reference.  Otherwise, a malicious user
could do "del _hotshot.ProfilerError" and have it garbage-collected
under the feet of _hotshot.c which still uses it.  What I don't get is
how ProfilerError could fail to be NULL in the first 'if' above, but
that's a different matter.

While we're at strange refcounting problems, PyModule_AddObject() only
decrefs its last argument if no error occurs.  This is probably wrong.

In general I've found that the C modules' init code is fragile.  This
might be due to the idea that it runs only once anyway, and global
C-module objects are immortal anyway, so sloppiness sneaks in.  But for
example, the following is common:

        m = Py_InitModule3("xxx", NULL, module_doc);
        Py_INCREF(&Xxx_Type);
        PyModule_AddObject(m, "Xxx", (PyObject *)&Xxx_Type);

This generates a segfault if Py_InitModule3() returns NULL (however rare
that situation is).


A bientot,

Armin

From arigo at tunes.org  Wed Jun 15 16:06:35 2005
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 15 Jun 2005 16:06:35 +0200
Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c,
	1.37, 1.38
In-Reply-To: <E1DiY2w-0001xT-LQ@sc8-pr-cvs1.sourceforge.net>
References: <E1DiY2w-0001xT-LQ@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20050615140635.GA3193@code1.codespeak.net>

Hi Skip,

On Wed, Jun 15, 2005 at 06:35:10AM -0700, montanaro at users.sourceforge.net wrote:
> Why this worked is a bit mystical.  Perhaps it never gets freed because the
> object just happens never to be DECREF'd (but that seems unlikely).
>          /* Add the Dialect type */
> +	Py_INCREF(&Dialect_Type);
>          if (PyModule_AddObject(module, "Dialect", (PyObject *)&Dialect_Type))
>                  return;

Hum, you probably don't want to know, but it works just fine to forget
a Py_INCREF before PyModule_AddObject() for the following reason:

1. the reference is stored in the module's dict, so the object is kept
alive from there.

2. after the module initialization code is completed, the import
mechanism make a copy of the dict (!) just in case some users wants to
reload() the module (!!) in which case the module's dict is simplify
overwritten with the copy again (!!!).

So there is a reference left to the object from this hidden dict, and no
way for the user to kill it -- short of using gc.getreferrers(), which
is how I figured this out, but gc.getreferrers() is officially
dangerous.  So unlike what I thought in my previous e-mail, even if the
user deletes the entry in the module's normal dict, nothing bad can
happend because of this particular feature of import...

So it just works.  Hum.


Armin

From mwh at python.net  Wed Jun 15 16:17:02 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 15 Jun 2005 15:17:02 +0100
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <20050615135110.GA2560@code1.codespeak.net> (Armin Rigo's
	message of "Wed, 15 Jun 2005 15:51:10 +0200")
References: <2mzmtrbxwo.fsf@starship.python.net>
	<20050615135110.GA2560@code1.codespeak.net>
Message-ID: <2mr7f3bt7l.fsf@starship.python.net>

Armin Rigo <arigo at tunes.org> writes:

> Hi Michael,
>
> On Wed, Jun 15, 2005 at 01:35:35PM +0100, Michael Hudson wrote:
>>         if (ProfilerError == NULL)
>>             ProfilerError = PyErr_NewException("hotshot.ProfilerError",
>>                                                NULL, NULL);
>>         if (ProfilerError != NULL) {
>>             Py_INCREF(ProfilerError);
>>             PyModule_AddObject(module, "ProfilerError", ProfilerError);
>>         }
>
> I think the Py_INCREF is needed here.  The ProfilerError is a global
> variable that needs the extra reference.  Otherwise, a malicious user
> could do "del _hotshot.ProfilerError" and have it garbage-collected
> under the feet of _hotshot.c which still uses it.  

Hmm.  Point.  But then how doesn't this apply to things like 'del
thread._local'? (after my recent fix)

> What I don't get is how ProfilerError could fail to be NULL in the
> first 'if' above, but that's a different matter.

Well, could it fail to be NULL in the multiple interpreters case?
Though I'm not at all sure that reusing is wise in that case...

> While we're at strange refcounting problems, PyModule_AddObject() only
> decrefs its last argument if no error occurs.  This is probably wrong.

Quite probably :-/

> In general I've found that the C modules' init code is fragile.  This
> might be due to the idea that it runs only once anyway, and global
> C-module objects are immortal anyway, so sloppiness sneaks in.

Oh yes.

Cheers,
mwh

-- 
  It's actually a corruption of "starling".  They used to be carried.
  Since they weighed a full pound (hence the name), they had to be
  carried by two starlings in tandem, with a line between them.
                 -- Alan J Rosenthal explains "Pounds Sterling" on asr

From mwh at python.net  Wed Jun 15 16:26:00 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 15 Jun 2005 15:26:00 +0100
Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c,
 1.37, 1.38
In-Reply-To: <20050615140635.GA3193@code1.codespeak.net> (Armin Rigo's
	message of "Wed, 15 Jun 2005 16:06:35 +0200")
References: <E1DiY2w-0001xT-LQ@sc8-pr-cvs1.sourceforge.net>
	<20050615140635.GA3193@code1.codespeak.net>
Message-ID: <2mmzprbssn.fsf@starship.python.net>

Armin Rigo <arigo at tunes.org> writes:

> Hi Skip,
>
> On Wed, Jun 15, 2005 at 06:35:10AM -0700, montanaro at users.sourceforge.net wrote:
>> Why this worked is a bit mystical.  Perhaps it never gets freed because the
>> object just happens never to be DECREF'd (but that seems unlikely).
>>          /* Add the Dialect type */
>> +	Py_INCREF(&Dialect_Type);
>>          if (PyModule_AddObject(module, "Dialect", (PyObject *)&Dialect_Type))
>>                  return;
>
> Hum, you probably don't want to know, but it works just fine to forget
> a Py_INCREF before PyModule_AddObject() for the following reason:

No, it's more complicated than that, at least in an embedded
scenario (see bug #1220756).

I don't understand how thread._local differed from _cvs.Dialect,
though.

Cheers,
mwh

-- 
  faassen: anyway, if nothing else flapping your arms running around 
           will at least give an impression of activity. :)

From maliger at gmail.com  Wed Jun 15 17:00:19 2005
From: maliger at gmail.com (Martin Aliger)
Date: Wed, 15 Jun 2005 17:00:19 +0200
Subject: [Python-Dev] python running in several threads
In-Reply-To: <42A61F8D.2090704@v.loewis.de>
References: <63fd41d40506070715129c904a@mail.gmail.com>
	<20050607153253.GB2787@panix.com> <42A61F8D.2090704@v.loewis.de>
Message-ID: <63fd41d40506150800ae6e358@mail.gmail.com>

Thanks both for answers,

I read some articles you link and I'd to say - alot discussion around!

I also found some notes that mod_python use some technique other than
GIL to allow multi-threading in Apache 2.0. Does someone know anything
closer? Any links or so?

Sorry I raise this old stuff again.

Martin


Just short notes:
> Removing the GIL is easy, but then the interpreter crashes in cases of
> simultaneous accesses to dictionaries, reference counters, etc.

I know. But under mine conditions is impossible to share dictionaty or
ref.counters so this do not happen. This dont happen neither on web
server running simmultaniously several .py scripts (same conditions)
 
> This is just not true. On a single-processor machine, the GIL does
> *not* degrade performance. Instead, it increases throughput (and
> thus performance).

Under bigger load yes. It degrades a little under small load, but its
usually not big problem.
 
> On a multi-processor machine, you often use multiple operating
> system processes to serve request (e.g. in CGI or Apache mpm-prefork,
> or even the typical mpm-worker configuration). If you then have
> different processes running Python, they don't interfere with
> each other at all.

Unfortunatelly many configurations dont do that. I run several servers
with hyperthreading processors and none of them runs two IISs or
apaches just for load-balance python scripts.

Regards,
Martin

From gvanrossum at gmail.com  Wed Jun 15 17:11:25 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Wed, 15 Jun 2005 08:11:25 -0700
Subject: [Python-Dev] PEP 343 question
In-Reply-To: <000201c57189$eb273280$1c2ba044@oemcomputer>
References: <000201c57189$eb273280$1c2ba044@oemcomputer>
Message-ID: <ca471dc2050615081126bb7965@mail.gmail.com>

On 6/15/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> 
> In gen.throw(), are all three arguments required?  Or do the value and
> traceback Nones need to be listed explicitly? 
> 
>    g.throw(MyException) 
> 
> or 
> 
>    g.throw(MyException, None, None) 
> 
> FWIW, I prefer the former.  That will make throw() as flexible as the raise
> statement. 

I don't particularly care much; the analogy with raise makes sense,
but the PEP carefully only calls it with 3 arguments, and since there
are no use cases outside the PEP, the simpler implementation
(requiring 3 arguments) makes more sense to me.

If you want to implement this part, feel free to make it the way you like!

I consider PEP 343 as accepted at this point as it ever will be;
additional tweaks will be found during implementation.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From arigo at tunes.org  Wed Jun 15 18:48:21 2005
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 15 Jun 2005 18:48:21 +0200
Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c,
	1.37, 1.38
In-Reply-To: <2mmzprbssn.fsf@starship.python.net>
References: <E1DiY2w-0001xT-LQ@sc8-pr-cvs1.sourceforge.net>
	<20050615140635.GA3193@code1.codespeak.net>
	<2mmzprbssn.fsf@starship.python.net>
Message-ID: <20050615164821.GA4757@code1.codespeak.net>

Hi Michael,

On Wed, Jun 15, 2005 at 03:26:00PM +0100, Michael Hudson wrote:
> > Hum, you probably don't want to know, but it works just fine to forget
> > a Py_INCREF before PyModule_AddObject() for the following reason:
> 
> No, it's more complicated than that, at least in an embedded
> scenario (see bug #1220756).

Uh, yes, indeed.  I guess that my explanations don't survive an
interpreter shutdown, after which the reference counter of the object
effectively drops to zero.  I don't see how this doesn't crash Python
when you exit.  It could be that the gc is not called after the
reference from the import mechanism was effectively removed -- type
objects after a PyType_Ready() are full of ref cycles.  But it's all
quite obscure.


Armin

From skip at pobox.com  Wed Jun 15 19:10:29 2005
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 15 Jun 2005 12:10:29 -0500
Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c,
 1.37, 1.38
In-Reply-To: <20050615164821.GA4757@code1.codespeak.net>
References: <E1DiY2w-0001xT-LQ@sc8-pr-cvs1.sourceforge.net>
	<20050615140635.GA3193@code1.codespeak.net>
	<2mmzprbssn.fsf@starship.python.net>
	<20050615164821.GA4757@code1.codespeak.net>
Message-ID: <17072.24837.306667.831660@montanaro.dyndns.org>


    Armin> But it's all quite obscure.

If this was comp.lang.python I'd nominate this for QOTW...

Skip

From martin at v.loewis.de  Wed Jun 15 20:24:29 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 15 Jun 2005 20:24:29 +0200
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <2mzmtrbxwo.fsf@starship.python.net>
References: <2mzmtrbxwo.fsf@starship.python.net>
Message-ID: <42B0725D.9040200@v.loewis.de>

Michael Hudson wrote:

>         if (ProfilerError == NULL)
>             ProfilerError = PyErr_NewException("hotshot.ProfilerError",
>                                                NULL, NULL);
>         if (ProfilerError != NULL) {
>             Py_INCREF(ProfilerError);
>             PyModule_AddObject(module, "ProfilerError", ProfilerError);
>         }
> 
> 
> I think the Py_INCREF should just be removed, but I'm wondering if I'm
> missing something...

It may be me who is missing something, but...

On reference is added to the dictionary, this is the one the explicit
INCREF creates. The other reference is held in the C variable
ProfilerError; this is the one that creating the exception object
creates.

It is convention that C variables which are explicitly used also
hold their own references, even if they are global, and even if there
is no procedure to clear them. The reason is that they would become
stale if the module object went away. As there is no way to protect
against this case, they just keep a garbage reference.

Regards,
Martin

From nidoizo at yahoo.com  Wed Jun 15 20:37:05 2005
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Wed, 15 Jun 2005 14:37:05 -0400
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <ca471dc20506130741430f8811@mail.gmail.com>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>	<42AD770A.1040001@gmail.com>
	<ca471dc20506130741430f8811@mail.gmail.com>
Message-ID: <d8ps6s$gfk$1@sea.gmane.org>

Guido van Rossum wrote:
> Why are you so excited about having until indented? You didn't give
> any examples with multiple occurrences. A single occurrence works just
> fine unindented, as PEP 315 has already shown.

FWIW, I must say I disagree (about "works just fine").  I find PEP 315 
counter-intuitive.  There's multi-part blocks in Python, but they all 
behave like a step-sequence where you never come back to a previous 
step.  What I mean is that after entering a finally/except/elif/else, 
there's no coming back in the try/if/for/while.  My first impression 
when looking at:

do:
     <block1>
while condition:
     <block2>

is that <block1> is executed only once.  Yes, it would not make any 
sense since you would write instead:

<block1>
while condition:
     <block2>

But, FWIW, that's still my impression.  I remember reading Stroustrup 
saying he was never using do-while loops because he find them 
counter-intuitive.  I feel the same way.  I think this is important, 
because it means that even if do-while is available in C++, some 
programmers, even the creator himself, are still always using while 
loops.  Even if different languages, I don't see why that would be 
different in Python.

My mental image of a do-while loop looks more like a while-True-if-break 
Python loop than a PEP315-like loop.  For one thing, since the loop 
content is executed at least once, I think about the condition breaking 
the loop, not the one maintaining it alive.  The keyword 'until' has 
been proposed; I guess it would be as in Perl where it is an equivalent 
of "while not".  Also, the "do" keyword means nothing to me; maybe 
something like "loop" would be better.  Still, my feeling is that Python 
is just getting it right with the status quo.

Regards,
Nicolas


From python at discworld.dyndns.org  Wed Jun 15 21:03:52 2005
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Wed, 15 Jun 2005 13:03:52 -0600
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <d8ps6s$gfk$1@sea.gmane.org>
References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer>
	<42AD770A.1040001@gmail.com>
	<ca471dc20506130741430f8811@mail.gmail.com>
	<d8ps6s$gfk$1@sea.gmane.org>
Message-ID: <20050615190352.GA28330@discworld.dyndns.org>

Nicolas Fleury <nidoizo at yahoo.com> wrote:
> Guido van Rossum wrote:
> > Why are you so excited about having until indented? You didn't give
> > any examples with multiple occurrences. A single occurrence works just
> > fine unindented, as PEP 315 has already shown.
> 
> FWIW, I must say I disagree (about "works just fine").  I find PEP 315 
> counter-intuitive.  There's multi-part blocks in Python, but they all 
> behave like a step-sequence where you never come back to a previous 
> step.  What I mean is that after entering a finally/except/elif/else, 
> there's no coming back in the try/if/for/while.  My first impression 
> when looking at:
> 
> do:
>      <block1>
> while condition:
>      <block2>
> 
> is that <block1> is executed only once. 

Indeed.  The original poster seems to want something that would work (not
necessarily look) like this:

  do:
      <block>
  while <condition>

with <block> executed once prior to <condition> first being tested.  But the
above is ugly, and you can get much the same effect with Python today:

  firsttime = True
  while firsttime or <condition>:
      <block>
      firsttime = False

Seems fairly Pythonic to me.  YMMV.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python at discworld.dyndns.org>
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

From jcarlson at uci.edu  Wed Jun 15 21:38:16 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 15 Jun 2005 12:38:16 -0700
Subject: [Python-Dev] Wishlist: dowhile
In-Reply-To: <20050615190352.GA28330@discworld.dyndns.org>
References: <d8ps6s$gfk$1@sea.gmane.org>
	<20050615190352.GA28330@discworld.dyndns.org>
Message-ID: <20050615122018.726B.JCARLSON@uci.edu>


Charles Cazabon <python at discworld.dyndns.org> wrote:
> Indeed.  The original poster seems to want something that would work (not
> necessarily look) like this:
> 
>   do:
>       <block>
>   while <condition>
> 
> with <block> executed once prior to <condition> first being tested.  But the
> above is ugly, and you can get much the same effect with Python today:
> 
>   firsttime = True
>   while firsttime or <condition>:
>       <block>
>       firsttime = False
> 
> Seems fairly Pythonic to me.  YMMV.

I agree.  It also handles 'else' clauses very well.  Heck, even Michael
McLay's offering of ...

while True:
    <body-1-or-more-times>
    if <condition>:
        <on-natural-loop-exit>
        break
    <body-0-or-more-times>

... doesn't look too bad, and supports the full range of pre and post
condition code. Just like the indented "until", it may not be obvious
where the breaking condition lies.  However, a syntax highlighting
editor and a properly located comment or two would solve the problem
with "until" and the breaking "if" above.

I'm of the opinion that the "do while" and "loop and a half" variants of
the while loop should be offered as a recipe somewhere in the
documentation.  Perhaps not directly in the tutorial (maybe a link),
describing quickly how you can use while loops to emulate the full
spectrum of looping constructs.

 - Josiah


From steven.bethard at gmail.com  Thu Jun 16 03:17:53 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 15 Jun 2005 19:17:53 -0600
Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was:
	Wishlist: dowhile)
In-Reply-To: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>
References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>
Message-ID: <d11dcfba05061518171ee7c71a@mail.gmail.com>

Jp Calderone:
>  for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''):
>      f2.write(chunk)

Phillip J. Eby:
> More seriously, I think your translation makes an excellent argument in
> *favor* of having a do/while statement for greater clarity.  :)

Michael Chermside
> Interesting... I had the opposite reaction.

I, too, thought that Jp's solution was quite easy to read, though
every few months I forget about iter()'s alternate form again. ;-)

Perhaps one of the things that makes Jp's solution slightly hard to
read is the fact that f1.read must be wrapped in a function (in Jp's
example, a lambda).  It reminds me of one of my minor gripes about the
standard lib -- a number of functions that take another function as an
argument don't take *args and **kwargs to be passed to that function
when it's called.  The iter() alternate form is a common example of
this.  I would prefer that the alternate iter() form was broken off
into another separate function, say, iterfunc(), that would let me
write Jp's solution something like:

for chunk in iterfunc('', f1.read, CHUNK_SIZE):
    f2.write(chunk)

Even better would be to add the *args and **kwargs to iter() directly,
but of course, that would be backwards incompatible.

Anyway, in general, I'd like to see the standard lib in Python 3K
convert functions like iter()'s alternate form to functions like
iterfunc(), i.e. provide *args and **kwargs whenever possible.

Steve
-- 
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From benji at benjiyork.com  Thu Jun 16 05:07:05 2005
From: benji at benjiyork.com (Benji York)
Date: Wed, 15 Jun 2005 23:07:05 -0400
Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was:
 Wishlist: dowhile)
In-Reply-To: <d11dcfba05061518171ee7c71a@mail.gmail.com>
References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>
	<d11dcfba05061518171ee7c71a@mail.gmail.com>
Message-ID: <42B0ECD9.3060207@benjiyork.com>

Steven Bethard wrote:
> It reminds me of one of my minor gripes about the standard lib -- a
> number of functions that take another function as an argument don't
> take *args and **kwargs to be passed to that function when it's
> called. The iter() alternate form is a common example of this. I
> would prefer that the alternate iter() form was broken off into
> another separate function, say, iterfunc(), that would let me write
> Jp's solution something like:
> 
> for chunk in iterfunc('', f1.read, CHUNK_SIZE):
>     f2.write(chunk)

How about 2.5's "partial":

for chunk in iter(partial(f1.read, CHUNK_SIZE), ''):
     f2.write(chunk)
--
Benji York


From mwh at python.net  Thu Jun 16 11:51:13 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 16 Jun 2005 10:51:13 +0100
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <42B0725D.9040200@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Wed,
	15 Jun 2005 20:24:29 +0200")
References: <2mzmtrbxwo.fsf@starship.python.net> <42B0725D.9040200@v.loewis.de>
Message-ID: <2m4qbybpf2.fsf@starship.python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Michael Hudson wrote:
>
>>         if (ProfilerError == NULL)
>>             ProfilerError = PyErr_NewException("hotshot.ProfilerError",
>>                                                NULL, NULL);
>>         if (ProfilerError != NULL) {
>>             Py_INCREF(ProfilerError);
>>             PyModule_AddObject(module, "ProfilerError", ProfilerError);
>>         }
>> 
>> 
>> I think the Py_INCREF should just be removed, but I'm wondering if I'm
>> missing something...
>
> It may be me who is missing something, but...

Well, quite possibly not.

> On reference is added to the dictionary, this is the one the explicit
> INCREF creates. The other reference is held in the C variable
> ProfilerError; this is the one that creating the exception object
> creates.
>
> It is convention that C variables which are explicitly used also
> hold their own references, even if they are global, and even if there
> is no procedure to clear them. The reason is that they would become
> stale if the module object went away. As there is no way to protect
> against this case, they just keep a garbage reference.

This means two things, as I see it:

1) Py_Initialize()/Py_Finalize() loops are going to leak quite a lot.
   Maybe we don't care about this.

2) In the case of the init_hotshot code above and such a loop, the
   ProfilerError object from the first interpreter will be reused by
   the second, which doesn't seem like a good idea (won't it be
   inheriting from the wrong PyExc_Exception?).

Currently running Demo/embed/loop 'import gc' crashes for a similar
kind of reason -- the gc.garbage object is shared between
interpreters, but the only reference to it is in the module's
__dict__ (well, if the module exists...).

I've been looking at this area partly to try and understand this bug:

    [ 1163563 ] Sub threads execute in restricted mode

but I'm not sure the whole idea of multiple interpreters isn't
inherently doomed :-/

Cheers,
mwh

-- 
  Premature optimization is the root of all evil.
       -- Donald E. Knuth, Structured Programming with goto Statements

From faassen at infrae.com  Thu Jun 16 12:49:44 2005
From: faassen at infrae.com (Martijn Faassen)
Date: Thu, 16 Jun 2005 12:49:44 +0200
Subject: [Python-Dev] Thoughts on stdlib evolvement
In-Reply-To: <d8lsfg$d4o$1@sea.gmane.org>
References: <d86o5t$euh$1@sea.gmane.org>
	<d86op2$gu2$1@sea.gmane.org>	<20050610144835.GB5929@burma.localdomain>
	<d8lsfg$d4o$1@sea.gmane.org>
Message-ID: <42B15948.9060605@infrae.com>

Fredrik Lundh wrote:
[snip]
> in my experience, any external library that supports more than one
> Python version on more than one platform is likely to be more robust
> than code in the core.  add the multilevel volunteer approach de-
> described by Steven (with the right infrastructure, things like that
> just appear), and you get more competent manpower contributing
> to the standard distribution than you can get in any other way.

In this context PEP 2 might be useful to look at again:

http://www.python.org/peps/pep-0002.html

It separates between library integrators (into the Python standard 
library) and library maintainers, and tries to ensure maintenance 
happens on a continuing basis.

A multi-level setup to develop the Python standard library could take 
other forms, of course. I sometimes feel the Python-dev community is 
more focused on the development of the interpreter than of the library, 
and that this priority tends to be reversed outside the Python-dev 
community. So, it might be nice if the Python standard library 
development integrators and maintainers could be more separate from the 
Python core developers. A python-library-dev, say.

Then again, this shouldn't result in large changes in the standard 
library, as old things should continue to work for the forseeable 
future. So for larger reorganizations and refactorings, such development 
should likely take place entirely outside the scope of the core 
distribution, at least for the time being.

Finally, I haven't really seen much in the way of effort by developers 
to actually do such a large-scale cleanup. Nobody seems to have stepped 
up, taking the standard library, and made it undergo a radical 
refactoring (and just releasing it separately). That this hasn't 
happened seems to indicate the priority is not very high in the mind of 
people, so the problem might not be high either. :)

Regards,

Martijn

From barry at python.org  Thu Jun 16 15:37:35 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 16 Jun 2005 09:37:35 -0400
Subject: [Python-Dev] PEP 343 - next steps
In-Reply-To: <42ABBF9F.1010801@gmail.com>
References: <ca471dc2050610132375ca7c70@mail.gmail.com>
	<1118526223.9425.127.camel@geddy.wooz.org> <42ABBF9F.1010801@gmail.com>
Message-ID: <1118929055.22208.19.camel@presto.wooz.org>

On Sun, 2005-06-12 at 00:52, Nick Coghlan wrote:

> The idea behind 'with' is that the block is executed while holding 
> (i.e. 'with') the resource.
> 
> I think the '-ing' form of the example templates is a holdover from 
> the days when the PEP used the 'do' keyword - I find that past tense 
> or noun forms tend to read better than present tense for custom built 
> with templates named after the action that occurs on entry:
> 
>    # Past tense
>    with locked(my_lock):
>    with opened(my_file, mode):
>    with redirected_stdout(my_stream):

Work for me.  Thanks.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050616/44debb85/attachment.pgp

From steven.bethard at gmail.com  Thu Jun 16 16:50:42 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 16 Jun 2005 08:50:42 -0600
Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was:
	Wishlist: dowhile)
In-Reply-To: <42B0ECD9.3060207@benjiyork.com>
References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>
	<d11dcfba05061518171ee7c71a@mail.gmail.com>
	<42B0ECD9.3060207@benjiyork.com>
Message-ID: <d11dcfba05061607501c433c74@mail.gmail.com>

On 6/15/05, Benji York <benji at benjiyork.com> wrote:
> Steven Bethard wrote:
> > I would prefer that the alternate iter() form was broken off into
> > another separate function, say, iterfunc(), that would let me write
> > Jp's solution something like:
> >
> > for chunk in iterfunc('', f1.read, CHUNK_SIZE):
> >     f2.write(chunk)
> 
> How about 2.5's "partial":
> 
> for chunk in iter(partial(f1.read, CHUNK_SIZE), ''):
>     f2.write(chunk)

Yeah, there are a number of workarounds.  Using partial, def-ing a
function, or using a lambda will all work.  My point was that, with
the right API, these workarounds wouldn't be necessary.  Look at
unittest.TestCase.assertRaises to see another example of the kind of
API I think should be supported (when possible, of course).

Steve
-- 
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From mwh at python.net  Thu Jun 16 17:04:53 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 16 Jun 2005 16:04:53 +0100
Subject: [Python-Dev] iter alternate form and *args and **kwargs
In-Reply-To: <d11dcfba05061607501c433c74@mail.gmail.com> (Steven Bethard's
	message of "Thu, 16 Jun 2005 08:50:42 -0600")
References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>
	<d11dcfba05061518171ee7c71a@mail.gmail.com>
	<42B0ECD9.3060207@benjiyork.com>
	<d11dcfba05061607501c433c74@mail.gmail.com>
Message-ID: <2m8y1a9wbu.fsf@starship.python.net>

Steven Bethard <steven.bethard at gmail.com> writes:

> On 6/15/05, Benji York <benji at benjiyork.com> wrote:
>> Steven Bethard wrote:
>> > I would prefer that the alternate iter() form was broken off into
>> > another separate function, say, iterfunc(), that would let me write
>> > Jp's solution something like:
>> >
>> > for chunk in iterfunc('', f1.read, CHUNK_SIZE):
>> >     f2.write(chunk)
>> 
>> How about 2.5's "partial":
>> 
>> for chunk in iter(partial(f1.read, CHUNK_SIZE), ''):
>>     f2.write(chunk)
>
> Yeah, there are a number of workarounds.  Using partial, def-ing a
> function, or using a lambda will all work.  My point was that, with
> the right API, these workarounds wouldn't be necessary. 

Well, I dunno.  I can see where you're coming from, but I think you
could make the argument that the form using partial is clearer to read
-- it's not absolutely clear that the CHUNK_SIZE argument is intended
to be passed to f1.read.  Also, the partial approach works better when
there is more than one callable.

Cheers,
mwh

-- 
  Like most people, I don't always agree with the BDFL (especially
  when he wants to change things I've just written about in very 
  large books), ... 
         -- Mark Lutz, http://python.oreilly.com/news/python_0501.html

From foom at fuhm.net  Thu Jun 16 17:43:34 2005
From: foom at fuhm.net (James Y Knight)
Date: Thu, 16 Jun 2005 11:43:34 -0400
Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was:
	Wishlist: dowhile)
In-Reply-To: <d11dcfba05061607501c433c74@mail.gmail.com>
References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>
	<d11dcfba05061518171ee7c71a@mail.gmail.com>
	<42B0ECD9.3060207@benjiyork.com>
	<d11dcfba05061607501c433c74@mail.gmail.com>
Message-ID: <081482E7-D65B-445A-8CC3-59C3A185029C@fuhm.net>


On Jun 16, 2005, at 10:50 AM, Steven Bethard wrote:

> On 6/15/05, Benji York <benji at benjiyork.com> wrote:
>
>> Steven Bethard wrote:
>>
>>> I would prefer that the alternate iter() form was broken off into
>>> another separate function, say, iterfunc(), that would let me write
>>> Jp's solution something like:
>>>
>>> for chunk in iterfunc('', f1.read, CHUNK_SIZE):
>>>     f2.write(chunk)
>>>
>>
>> How about 2.5's "partial":
>>
>> for chunk in iter(partial(f1.read, CHUNK_SIZE), ''):
>>     f2.write(chunk)
>>
>
> Yeah, there are a number of workarounds.  Using partial, def-ing a
> function, or using a lambda will all work.  My point was that, with
> the right API, these workarounds wouldn't be necessary.  Look at
> unittest.TestCase.assertRaises to see another example of the kind of
> API I think should be supported (when possible, of course).

I think it's really the other way around. Forcing every API that  
takes a callable to also take a *args, **kwargs is a workaround for  
not having partial.

James

From mwh at python.net  Thu Jun 16 19:00:14 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 16 Jun 2005 18:00:14 +0100
Subject: [Python-Dev] Multiple interpreters not compatible with current
 thread module
In-Reply-To: <93dc9c320505170425392e3d80@mail.gmail.com> (Jeremy Maxfield's
	message of "Tue, 17 May 2005 13:25:32 +0200")
References: <93dc9c320505170425392e3d80@mail.gmail.com>
Message-ID: <2m4qby9qzl.fsf@starship.python.net>

Jeremy Maxfield <anothermax at gmail.com> writes:

> The current threadmodule.c does not seem to correctly support multiple
> (sub) interpreters.

This would seem to be an accurate statement.

A short history:

The GILState functions were implemented.

The way they work is that when you call PyGILState_Ensure,
(essentially) a thread local variable is checked to see if a thread
state is known for this thread.  If one is found, fine, it is used.
If not, one is created (using the interpreter state what was passed to
_PyGILState_Init() by Py_Initialize()) and stored in the thread local
variable.

This had a (pretty obvious in hindsight) problem (bug #1010677): when
you create a thread via thread.start_new_thread a thread state is
created, but not stored in the thread local variable consulted by
PyGILState_Ensure.  So if you call PyGILState_Ensure another thread
state is created for the thread (generally a no-no) and if you already
hold the GIL PyGILState_Ensure will attempt to acquire it again -- a
deadlock.

This was fixed by essentially using PyGILState_Ensure to create the
threadstate.  This has a (pretty obvious in hindsight) problem (bug
#1163563): PyGILState_Ensure always uses the interpreter state created
by Py_Initialize, ignoring the interpreter state carefully supplied to
t_bootstrap.  Hilarity ensues.

So, what's the fix?  Well, the original problem was only the lack of
association between a C level thread and a thread state.  This can be
fixed by setting up this association in t_bootstrap (and I've posted a
patch that does this to the report of #1163563).  This suffices for
all known problems, but it's a bit hackish.  Another approach is to
set up this association PyThreadState_New(), which is possibly a bit
more elegant, but carries a risk: is PyThreadState_New always called
from the new thread?  ISTM that it is, but I'm not sure.

I'm not expecting anyone else to think hard about this on recent form,
so I'll think about it for a bit and then fix it in the way that seems
best after that.  Feel free to surprise me.

Cheers,
mwh

-- 
  I would hereby duly point you at the website for the current pedal
  powered submarine world underwater speed record, except I've lost
  the URL.                                         -- Callas, cam.misc

From gvanrossum at gmail.com  Thu Jun 16 20:41:05 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 16 Jun 2005 11:41:05 -0700
Subject: [Python-Dev] Please spread the word about OSCON early reg deadline
In-Reply-To: <25fced5c5f9467929cacbd05a06c7438@oreilly.com>
References: <49e0655542c64ebcabaf05533f0bfa48@oreilly.com>
	<200506160944.27095.josh@agliodbs.com>
	<20050616171341.GA1857@panix.com> <42B1A5F4.1090203@shiflett.org>
	<m3u0jyyzae.wl_rs@pobox.com>
	<25fced5c5f9467929cacbd05a06c7438@oreilly.com>
Message-ID: <ca471dc20506161141562b3b4d@mail.gmail.com>

FYI. Check it out, the Python program is really good!

---------- Forwarded message ----------
From: Gina Blaber <gina at oreilly.com>
Date: Jun 16, 2005 11:31 AM
Subject: [Oscon] please spread the word about OSCON early reg deadline
To: OSCON Committee Mailing List <oscon at labs.oreilly.com>
Cc: Gina Blaber <gina at oreilly.com>


Can all of you on the OSCON program committee please take a moment
today to either blog OSCON or mention it on some relevant lists?
Timing is important because the Early Registration period (with the
early reg discount) ends on Monday June 20 (end of day).  It would be
great if you could mention the early reg deadline.

Here are few relevant links to include, if you're so inclined:

http://conferences.oreillynet.com/os2005/index_new.csp  (OSCON home
page)
http://conferences.oreillynet.com/pub/w/38/speakers.html (OSCON
speakers)
http://conferences.oreillynet.com/cs/os2005/create/ord_os05   (OSCON
attendee registration page)

Thanks,

-Gina

------------------------------------------------------------------------
-------
Gina Blaber,  Director of Conferences
O'Reilly Media, Inc.
1005 Gravenstein Highway North   Sebastopol, CA  95472
gina at oreilly.com  (707) 827-7185  http://conferences.oreilly.com

_______________________________________________
Oscon mailing list
Oscon at labs.oreilly.com
http://labs.oreilly.com/mailman/listinfo/oscon


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Thu Jun 16 21:28:46 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 16 Jun 2005 13:28:46 -0600
Subject: [Python-Dev] iter alternate form and *args and **kwargs
In-Reply-To: <2m8y1a9wbu.fsf@starship.python.net>
References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com>
	<d11dcfba05061518171ee7c71a@mail.gmail.com>
	<42B0ECD9.3060207@benjiyork.com>
	<d11dcfba05061607501c433c74@mail.gmail.com>
	<2m8y1a9wbu.fsf@starship.python.net>
Message-ID: <d11dcfba050616122819fd7b9e@mail.gmail.com>

Steven Bethard wrote:
> I would prefer that the alternate iter() form was broken off into
> another separate function, say, iterfunc(), that would let me write
> Jp's solution something like:
>
> for chunk in iterfunc('', f1.read, CHUNK_SIZE):
>     f2.write(chunk)

Benji York wrote:
> for chunk in iter(partial(f1.read, CHUNK_SIZE), ''):
>     f2.write(chunk)

Steven Bethard wrote:
> Yeah, there are a number of workarounds.  Using partial, def-ing a
> function, or using a lambda will all work.  My point was that, with
> the right API, these workarounds wouldn't be necessary.

Michael Hudson wrote: 
> Well, I dunno.  I can see where you're coming from, but I think you
> could make the argument that the form using partial is clearer to read
> -- it's not absolutely clear that the CHUNK_SIZE argument is intended
> to be passed to f1.read.

True, but the same argument could be made for
unittest.TestCase.assertRaises, and I don't find that confusing at
all.  YMMV, of course.

> Also, the partial approach works better when there is more than one
> callable.

Yeah, I thought of that.  But how often are there multiple callables? 
Just scanning through the builtin functions I see that filter, iter,
map, and reduce all take functions to be called, and none of them take
multiple functions.  (Note that classmethod, property and staticmethod
are not good examples because they don't take callables to be
*called*, they take callables to be *wrapped*.)  OTOH, filter, map and
reduce are all basically deprecated at this point thanks to list
comprehensions and generator expressions, so I guess YMMV.

Steve
-- 
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From raymond.hettinger at verizon.net  Fri Jun 17 02:24:49 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 16 Jun 2005 20:24:49 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
Message-ID: <003601c572d2$f9933540$bb07a044@oemcomputer>

PEP 288 is now withdrawn.  The generator exceptions portion is subsumed
by PEP 343, and the generator attributes portion never garnered any
support.

The fate of generator attributes is interesting v?s-a-v?s PEP 342.  The
motivation was always related to supporting advanced generator uses such
as emulating coroutines and writing generator based data consumer
functions.  At the time, Guido and everyone else found those use cases
to be less than persuasive.  Also, people countered that that
functionality could be easily simulated with class based iterators,
global variables, or passing a mutable argument to a generator.
Amazingly, none of those objections seem to be directed toward 342 which
somehow seems on the verge of acceptance even without use cases, clear
motivation, examples, or a draft implementation.

Looking back at the history of 288, generator attributes surfaced only
in later drafts.  In the earlier drafts, the idea for passing arguments
to and from running generators used an argument to next() and a return
value for yield.  If this sounds familiar, it is because it is not much
different from the new PEP 342 proposal.  However, generator argument
passing via next() was shot down early-on.  The insurmountable concept
flaw was an off-by-one issue.  The very first call to next() does not
correspond to a yield statement; instead, it corresponds to the first
lines of a generator (those run *before* the first yield).  All of the
proposed use cases needed to have the data passed in earlier.
  
With the death of that idea, generator attributes were born as a way of
being able to pass in data before the first yield was encountered and to
receive data after the yield.  This was workable and satisfied the use
cases.  Coroutine simulations such as those in Dr Mertz's articles were
easily expressed with generator attributes.  As a further benefit, using
attributes was a natural approach because that same technique has long
been used with classes (so no new syntax was needed and the learning
curve was zero).

In contrast to PEP 288's low impact approach, PEP 342 changes the
implementation of the for-loop, alters the semantics of "continue",
introduces new and old-style iterators, and creates a new magic method.
Meanwhile, it hasn't promised any advantages over the dead PEP 288
proposals.

IOW, I don't follow how 342 got this far, how 342 intends to overcome
the off-by-one issue, how it addresses all of the other objections
leveled at the now dead PEP 288, and why no one appears concerned about
introducing yet another new-style/old-style issue that will live in
perpetuity.


Raymond



Sidenote: generator attributes also failed because generators lacked a
sufficiently elegant way to refer to running instances of themselves
(there is no self argument so we would need an access function or have a
dynamic function attribute accessible only from within a running
generator).


From raymond.hettinger at verizon.net  Fri Jun 17 03:03:57 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 16 Jun 2005 21:03:57 -0400
Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries by
	Value
Message-ID: <004301c572d8$711919e0$bb07a044@oemcomputer>

May I suggest rejecting PEP 265.

As of Py2.4, its use case is easily solved with:

>>> sorted(d.iteritems(), key=itemgetter(1), reverse=True)
[('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)]

Further, Py2.5 offers a parallel solution to the more likely use case of
wanting the access only the largest counts:

>>> nlargest(2, d.iteritems(), itemgetter(1))
[('b', 23), ('d', 17)]


Raymond


From raymond.hettinger at verizon.net  Fri Jun 17 03:13:31 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 16 Jun 2005 21:13:31 -0400
Subject: [Python-Dev] Propose to reject PEP 281 -- Loop Counter Iteration
 with range and xrange
Message-ID: <004601c572d9$c7578ac0$bb07a044@oemcomputer>

The need for the indices() proposal was mostly met by PEP 279's
enumerate() builtin.

Commenting on 279 before it was accepted for Py2.3, PEP 281's author,
Magnus Lie Hetland, wrote, "I'm quite happy to have it make PEP 281
obsolete."


Raymond


From skip at pobox.com  Fri Jun 17 03:38:08 2005
From: skip at pobox.com (Skip Montanaro)
Date: Thu, 16 Jun 2005 20:38:08 -0500
Subject: [Python-Dev] PEP 304 "Controlling Generation of Bytecode Files" -
	patch updated
Message-ID: <17074.10624.578497.392072@montanaro.dyndns.org>


I updated the patch that supports PEP 304, "Controlling Generation of
Bytecode Files" to apply cleanly against current CVS.  I've tested it on Mac
OS X (straight Unix build only).  I'd appreciate it if some Linux, Windows
and Mac framework folks could apply the patch, rebuild, then run the tests
(there is a "testbcb" target in the Makefile that should give Windows people
an idea what to do).  The patch is attached to

    http://python.org/sf/677103

Now that I think about it, there is probably a file in the Windows build
tree (equivalent of pyconfig.h?) that still needs to be updated.

The text of the PEP has not been updated in awhile.  I will try to look at
that in the next couple of days.  I'd appreciate some critical review by
people with Windows filesystem experience.  There was a comment ages ago
about problems with this scheme due to Windows multi-rooted directory tree
that I can no longer find (and failed to record in the PEP at the time).
I'd like to see if the problem can be resurrected then addressed.

Thanks,

Skip


From steven.bethard at gmail.com  Fri Jun 17 03:39:11 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 16 Jun 2005 19:39:11 -0600
Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries
	by Value
In-Reply-To: <004301c572d8$711919e0$bb07a044@oemcomputer>
References: <004301c572d8$711919e0$bb07a044@oemcomputer>
Message-ID: <d11dcfba05061618391f88ffa5@mail.gmail.com>

Raymond Hettinger wrote:
> May I suggest rejecting PEP 265.
> 
> As of Py2.4, its use case is easily solved with:
> 
> >>> sorted(d.iteritems(), key=itemgetter(1), reverse=True)
> [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)]

+1.

I find that usually when I want something like this, I use:
   sorted(d, key=d.__getitem__, reverse=True)
because it doesn't require the operator module and most of the time I
just need the keys anyway.

py> sorted(d, key=d.__getitem__, reverse=True)
['b', 'd', 'c', 'a', 'e']

Steve
-- 
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From pje at telecommunity.com  Fri Jun 17 03:53:11 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 16 Jun 2005 21:53:11 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <003601c572d2$f9933540$bb07a044@oemcomputer>
Message-ID: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>

At 08:24 PM 6/16/2005 -0400, Raymond Hettinger wrote:
>Looking back at the history of 288, generator attributes surfaced only
>in later drafts.  In the earlier drafts, the idea for passing arguments
>to and from running generators used an argument to next() and a return
>value for yield.  If this sounds familiar, it is because it is not much
>different from the new PEP 342 proposal.  However, generator argument
>passing via next() was shot down early-on.  The insurmountable concept
>flaw was an off-by-one issue.  The very first call to next() does not
>correspond to a yield statement; instead, it corresponds to the first
>lines of a generator (those run *before* the first yield).  All of the
>proposed use cases needed to have the data passed in earlier.

Huh?  I don't see why this is a problem.  PEP 342 says:

"""When the *initial* call to __next__() receives an argument that is not 
None, TypeError is raised; this is likely caused by some logic error."""


>With the death of that idea, generator attributes were born as a way of
>being able to pass in data before the first yield was encountered and to
>receive data after the yield.  This was workable and satisfied the use
>cases.  Coroutine simulations such as those in Dr Mertz's articles were
>easily expressed with generator attributes.  As a further benefit, using
>attributes was a natural approach because that same technique has long
>been used with classes (so no new syntax was needed and the learning
>curve was zero).

Ugh.  Having actually emulated co-routines using generators, I have to tell 
you that I don't find generator attributes natural for this at all; 
returning a value or error (via PEP 343's throw()) from a yield expression 
as in PEP 342 is just what I've been wanting.


>In contrast to PEP 288's low impact approach, PEP 342 changes the
>implementation of the for-loop, alters the semantics of "continue",
>introduces new and old-style iterators, and creates a new magic method.

I could definitely go for dropping __next__ and the next() builtin from PEP 
342, as they don't do anything extra.  I also personally don't care about 
the new continue feature, so I could do without for-loop alteration 
too.  I'd be perfectly happy passing arguments to next() explicitly; I just 
want yield expressions.


>Meanwhile, it hasn't promised any advantages over the dead PEP 288
>proposals.

Reading the comments in PEP 288's revision history, it sounds like the 
argument was to postpone implementation of next(arg) and yield expressions 
to a later version of Python, after more community experience with 
generators.  We've had that experience now.


>IOW, I don't follow how 342 got this far, how 342 intends to overcome
>the off-by-one issue,

It explicitly addresses it already.


>  how it addresses all of the other objections
>leveled at the now dead PEP 288

Arguments for waiting aren't the same thing as arguments for never 
doing.  I interpret the comments in 288's history as ranging from -0 to +0 
on the yield expr/next(arg) issue, and didn't see any -1's except on the 
generator attribute concept.


>and why no one appears concerned about
>introducing yet another new-style/old-style issue that will live in
>perpetuity.

I believe it has been brought up before, and I also believe I pointed out 
once or twice that __next__ wasn't needed.  I think Guido even mentioned 
something to that effect himself, but everybody was busy with PEP 
340-inspired ideas at the time.  342 was split off in part to avoid losing 
the ideas that were in it.


From python at rcn.com  Fri Jun 17 04:26:19 2005
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 16 Jun 2005 22:26:19 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
Message-ID: <005301c572e3$f2518aa0$bb07a044@oemcomputer>

 [Phillip]
> I could definitely go for dropping __next__ and the next() builtin
from
> PEP
> 342, as they don't do anything extra.  I also personally don't care
about
> the new continue feature, so I could do without for-loop alteration
> too.  I'd be perfectly happy passing arguments to next() explicitly; I
> just
> want yield expressions.

That's progress!  Please do what you can to get the non-essential
changes out of 342.




> >Meanwhile, it hasn't promised any advantages over the dead PEP 288
> >proposals.
> 
> Reading the comments in PEP 288's revision history, it sounds like the
> argument was to postpone implementation of next(arg) and yield
expressions
> to a later version of Python, after more community experience with
> generators.  We've had that experience now.

288 was brought out of retirement a few months ago.  Guido hated every
variation of argument passing and frequently quipped that data passing
was trivially accomplished though mutable arguments to a generator,
through class based iterators, or via a global variable.  I believe all
of those comments were made recently and they all apply equally to 342.


Raymond

From pje at telecommunity.com  Fri Jun 17 05:05:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 16 Jun 2005 23:05:17 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <005301c572e3$f2518aa0$bb07a044@oemcomputer>
References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050616225814.01e1ba98@mail.telecommunity.com>

At 10:26 PM 6/16/2005 -0400, Raymond Hettinger wrote:
>288 was brought out of retirement a few months ago.  Guido hated every
>variation of argument passing and frequently quipped that data passing
>was trivially accomplished though mutable arguments to a generator,
>through class based iterators, or via a global variable.  I believe all
>of those comments were made recently and they all apply equally to 342.

Clearly, then, he's since learned the error of his ways.  :)

More seriously, I would say that data passing is not the same thing as 
coroutine suspension, and that PEP 340 probably gave Guido a much better 
look at at least one use case for the latter.

In the meantime, I applaud your foresight in having invented significant 
portions of PEP 343 years ahead of time.  Now give Guido back his time 
machine, please.  :)  If you hadn't borrowed it to write the earlier PEP, 
he could have seen for himself that all this would happen, and neatly 
avoided it by just approving PEP 288 to start with.  :)


From gvanrossum at gmail.com  Fri Jun 17 05:03:49 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 16 Jun 2005 20:03:49 -0700
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <005301c572e3$f2518aa0$bb07a044@oemcomputer>
References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
	<005301c572e3$f2518aa0$bb07a044@oemcomputer>
Message-ID: <ca471dc2050616200360b5276@mail.gmail.com>

On 6/16/05, Raymond Hettinger <python at rcn.com> wrote:
>  [Phillip]
> > I could definitely go for dropping __next__ and the next() builtin from PEP
> > 342, as they don't do anything extra.  I also personally don't care about
> > the new continue feature, so I could do without for-loop alteration
> > too.  I'd be perfectly happy passing arguments to next() explicitly; I just
> > want yield expressions.
> 
> That's progress!  Please do what you can to get the non-essential
> changes out of 342.

Here's my current position: instead of g.__next__(arg) I'd like to use
g.next(arg). The next() builtin then isn't needed.

I do like "continue EXPR" but I have to admit I haven't even tried to
come up with examples -- it may be unnecessary. As Phillip says, yield
expressions and g.next(EXPR) are the core -- and also incidentally
look like they will cause the most implementation nightmares. (If
someone wants to start implementing these two now, go right ahead!)

> > >Meanwhile, it hasn't promised any advantages over the dead PEP 288
> > >proposals.
> >
> > Reading the comments in PEP 288's revision history, it sounds like the
> > argument was to postpone implementation of next(arg) and yield expressions
> > to a later version of Python, after more community experience with
> > generators.  We've had that experience now.
> 
> 288 was brought out of retirement a few months ago.  Guido hated every
> variation of argument passing and frequently quipped that data passing
> was trivially accomplished though mutable arguments to a generator,
> through class based iterators, or via a global variable.  I believe all
> of those comments were made recently and they all apply equally to 342.

That was all before I (re-)discovered yield-expressions (in Ruby!),
and mostly in response to the most recent version of PEP 288, with its
problem of accessing the generator instance. I now strongly feel that
g.next(EXPR) and yield-expressions are the way to go.

Making g.next(EXPR) an error when this is the *initial* resumption of
the frame was also a (minor) breakthrough. Any data needed by the
generator at this point can be passed in as an argument to the
generator.

Someone should really come up with some realistic coroutine examples
written using PEP 342 (with or without "continue EXPR").

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Fri Jun 17 06:00:49 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 16 Jun 2005 21:00:49 -0700
Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries
	by Value
In-Reply-To: <d11dcfba05061618391f88ffa5@mail.gmail.com>
References: <004301c572d8$711919e0$bb07a044@oemcomputer>
	<d11dcfba05061618391f88ffa5@mail.gmail.com>
Message-ID: <ca471dc2050616210053431c22@mail.gmail.com>

Agreed. I don't want to add sorting abilities (with all its infinite
variants) to every data structure -- or even one or two common data
structures. You want something sorted that's not already a list? Use
the sorted() method.

On 6/16/05, Steven Bethard <steven.bethard at gmail.com> wrote:
> Raymond Hettinger wrote:
> > May I suggest rejecting PEP 265.
> >
> > As of Py2.4, its use case is easily solved with:
> >
> > >>> sorted(d.iteritems(), key=itemgetter(1), reverse=True)
> > [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)]
> 
> +1.
> 
> I find that usually when I want something like this, I use:
>    sorted(d, key=d.__getitem__, reverse=True)
> because it doesn't require the operator module and most of the time I
> just need the keys anyway.
> 
> py> sorted(d, key=d.__getitem__, reverse=True)
> ['b', 'd', 'c', 'a', 'e']
> 
> Steve
> --
> You can wordify anything if you just verb it.
>         --- Bucky Katt, Get Fuzzy
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Fri Jun 17 06:01:38 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 16 Jun 2005 21:01:38 -0700
Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries
	by Value
In-Reply-To: <ca471dc2050616210053431c22@mail.gmail.com>
References: <004301c572d8$711919e0$bb07a044@oemcomputer>
	<d11dcfba05061618391f88ffa5@mail.gmail.com>
	<ca471dc2050616210053431c22@mail.gmail.com>
Message-ID: <ca471dc2050616210156b927cb@mail.gmail.com>

On 6/16/05, Guido van Rossum <gvanrossum at gmail.com> wrote:
> Agreed. I don't want to add sorting abilities (with all its infinite
> variants) to every data structure -- or even one or two common data
> structures. You want something sorted that's not already a list? Use
> the sorted() method.

I meant the sorted() function, of course. Java on my mind. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Fri Jun 17 06:07:22 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 17 Jun 2005 00:07:22 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <ca471dc2050616200360b5276@mail.gmail.com>
References: <005301c572e3$f2518aa0$bb07a044@oemcomputer>
	<5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
	<005301c572e3$f2518aa0$bb07a044@oemcomputer>
Message-ID: <5.1.1.6.0.20050616233011.01e26378@mail.telecommunity.com>

At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote:
>Someone should really come up with some realistic coroutine examples
>written using PEP 342 (with or without "continue EXPR").

How's this?

    def echo(sock):
        while True:
            try:
                data = yield nonblocking_read(sock)
                yield nonblocking_write(sock, data)
            except ConnectionLost:
                pass

    def run_server(sock, handler):
        while True:
            connected_socket = yield nonblocking_accept(sock)
            schedule_coroutine(handler(connected_socket))

    schedule_coroutine(
        run_server(
            setup_listening_socket("localhost","echo"),
            echo
    )

Of course, I'm handwaving a lot here, but this is a much clearer example 
than anything I tried to pull out of the coroutines I've written for actual 
production use.  That is, I originally started this email with a real 
routine from a complex multiprocess application doing lots of IPC, and 
quickly got bogged down in explaining all the details of things like 
yielding to semaphores and whatnot.  But I can give you that example too, 
if you like.

Anyway, the handwaving above is only in explanation of details, not in 
their implementability.    It would be pretty straightforward to use 
Twisted's callback facilities to trigger next() or throw() calls to resume 
the coroutine in progress.  In fact, schedule_coroutine is probably 
implementable as something like this in Twisted:

     def schedule_coroutine(geniter, *arg):
         def resume():
             value = geniter.next(*arg)
             if value is not None:
                 schedule_coroutine(value)
         reactor.callLater(0, resume)

This assumes, of course, that you only yield between coroutines.  A better 
implementation would need to be more like the events.Task class in 
peak.events, which can handle yielding to Twisted's "Deferreds" and various 
other kinds of things that can provide callbacks.  But this snippet is 
enough to show that yield expressions let you write event-driven code 
without going crazy writing callback functions.

And of course, you can do this without yield expressions today, with a 
suitably magic function, but it doesn't read as well:

            yield nonblocking_accept(sock); connected_socket = events.resume()

This is how I actually do this stuff today.  'events.resume()' is a magic 
function that uses sys._getframe() to peek at the argument passed to the 
equivalent of 'next()' on the Task that wraps the 
generator.  events.resume() can also raise an error if the equivalent of 
'throw()' was called instead.

With yield expressions, the code in those Task methods would just do 
next(arg) or throw(*sys.exc_info()) on the generator-iterator, and 
'events.resume()' and its stack hackery could go away.


From gvanrossum at gmail.com  Fri Jun 17 06:03:51 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 16 Jun 2005 21:03:51 -0700
Subject: [Python-Dev] Propose to reject PEP 281 -- Loop Counter
	Iteration with range and xrange
In-Reply-To: <004601c572d9$c7578ac0$bb07a044@oemcomputer>
References: <004601c572d9$c7578ac0$bb07a044@oemcomputer>
Message-ID: <ca471dc205061621034718c526@mail.gmail.com>

On 6/16/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> The need for the indices() proposal was mostly met by PEP 279's
> enumerate() builtin.
> 
> Commenting on 279 before it was accepted for Py2.3, PEP 281's author,
> Magnus Lie Hetland, wrote, "I'm quite happy to have it make PEP 281
> obsolete."

Yes please. These examples are especially jarring:

    >>> range(range(5), range(10), range(2))
    [5, 7, 9]
    (etc.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Fri Jun 17 06:41:43 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 17 Jun 2005 00:41:43 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <5.1.1.6.0.20050616233011.01e26378@mail.telecommunity.com>
References: <ca471dc2050616200360b5276@mail.gmail.com>
	<005301c572e3$f2518aa0$bb07a044@oemcomputer>
	<5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
	<005301c572e3$f2518aa0$bb07a044@oemcomputer>
Message-ID: <5.1.1.6.0.20050617000918.01e1b840@mail.telecommunity.com>

At 12:07 AM 6/17/2005 -0400, Phillip J. Eby wrote:
>      def schedule_coroutine(geniter, *arg):
>          def resume():
>              value = geniter.next(*arg)
>              if value is not None:
>                  schedule_coroutine(value)
>          reactor.callLater(0, resume)

Oops.  I just realized that this is missing a way to return a value back to 
a calling coroutine, and that I also forgot to handle exceptions:

      def schedule_coroutine(coroutine, stack=(), *args):
          def resume():
              try:
                  if len(args)==3:
                      value = coroutine.throw(*args)
                  else:
                      value = coroutine.next(*args)
              except:
                  if stack:
                      # send the error back to the "calling" coroutine
                      schedule_coroutine(stack[0], stack[1], *sys.exc_info())
                  else:
                      # Nothing left in this pseudothread, let the
                      # event loop handle it
                      raise

              if isinstance(value,types.GeneratorType):
                  # Yielded to a specific coroutine, push the current
                  # one on the stack, and call the new one with no args
                  schedule_coroutine(value, (coroutine,stack))

              elif stack:
                  # Yielded a result, pop the stack and send the
                  # value to the caller
                  schedule_coroutine(stack[0], stack[1], value)

              # else: this pseudothread has ended

          reactor.callLater(0, resume)


There, that's better.  Now, if a coroutine yields a coroutine, the yielding 
coroutine is pushed on a stack.  If a coroutine yields a non-coroutine 
value, the stack is popped and the value returned to the 
previously-suspended coroutine.  If a coroutine raises an exception, the 
stack is popped and the exception is thrown to the previously-suspended 
coroutine.

This little routine basically replaces a whole bunch of code in peak.events 
that manages a similar coroutine stack right now, but is complicated by the 
absence of throw() and next(arg); the generators have to be wrapped by 
objects that add equivalent functionality, and the whole thing gets a lot 
more complicated as a result.

Note that we could add a version of the above to the standard library 
without using Twisted.  A simple loop class could have a deque of 
"callbacks to invoke", and the reactor.callLater() could be replaced by 
appending the 'resume' closure to the deque.  A main loop function would 
then just peel items off the deque and call them, looping until an 
unhandled exception (such as SystemExit) occurs, or some other way of 
indicating an exit occurs.


From python at rcn.com  Fri Jun 17 06:43:32 2005
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 17 Jun 2005 00:43:32 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <ca471dc2050616200360b5276@mail.gmail.com>
Message-ID: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer>

[Phillip]
> > I also personally don't care about the new continue feature, 
> > so I could do without for-loop alteration too.  

[Guido]
> I do like "continue EXPR" but I have to admit I haven't even tried to
> come up with examples -- it may be unnecessary. As Phillip says, yield
> expressions and g.next(EXPR) are the core -- and also incidentally
> look like they will cause the most implementation nightmares. 

Let me go on record as a strong -1 for "continue EXPR".  The for-loop is
our most basic construct and is easily understood in its present form.
The same can be said for "continue" and "break" which have the added
advantage of a near zero learning curve for people migrating from other
languages.

Any urge to complicate these basic statements should be seriously
scrutinized and held to high standards of clarity, explainability,
obviousness, usefulness, and necessity.  IMO, it fails most of those
tests. 

I would not look forward to explaining "continue EXPR" in the tutorial
and think it would stand out as an anti-feature.



Raymond

From raymond.hettinger at verizon.net  Fri Jun 17 07:19:51 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri, 17 Jun 2005 01:19:51 -0400
Subject: [Python-Dev] Propose to reject PEP 276 -- Simple iterator for ints
Message-ID: <000201c572fc$30d94e80$bb07a044@oemcomputer>

The principal use case was largely met by enumerate().  From PEP 276's
rationale section:

"""
A common programming idiom is to take a collection of objects and apply
some operation to each item in the collection in some established
sequential order.  Python provides the "for in" looping control
structure for handling this common idiom.  Cases arise, however, where
it is necessary (or more convenient) to access each item in an "indexed"
collection by iterating through each index and accessing each item in
the collection using the corresponding index.
"""

Also, while some nice examples are provided, the proposed syntax allows
and encourages some horrid examples as well:

    >>> for i in 3: print i
    0
    1
    2

The backwards compatability section lists another problematic
consequence; the following would stop being a syntax error and would
become valid:

   x, = 1

The proposal adds iterability to all integers but silently does nothing
for negative values.

A minor additional concern is that floats are not given an equivalent
capability (for obvious reasons) but this breaks symmetry with
range/xrange which still accept float args.


Raymond


From gvanrossum at gmail.com  Fri Jun 17 08:09:54 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 16 Jun 2005 23:09:54 -0700
Subject: [Python-Dev] Propose to reject PEP 276 -- Simple iterator for
	ints
In-Reply-To: <000201c572fc$30d94e80$bb07a044@oemcomputer>
References: <000201c572fc$30d94e80$bb07a044@oemcomputer>
Message-ID: <ca471dc2050616230953ddeb5e@mail.gmail.com>

I've never liked that idea. Down with it!

On 6/16/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> The principal use case was largely met by enumerate().  From PEP 276's
> rationale section:
> 
> """
> A common programming idiom is to take a collection of objects and apply
> some operation to each item in the collection in some established
> sequential order.  Python provides the "for in" looping control
> structure for handling this common idiom.  Cases arise, however, where
> it is necessary (or more convenient) to access each item in an "indexed"
> collection by iterating through each index and accessing each item in
> the collection using the corresponding index.
> """
> 
> Also, while some nice examples are provided, the proposed syntax allows
> and encourages some horrid examples as well:
> 
>     >>> for i in 3: print i
>     0
>     1
>     2
> 
> The backwards compatability section lists another problematic
> consequence; the following would stop being a syntax error and would
> become valid:
> 
>    x, = 1
> 
> The proposal adds iterability to all integers but silently does nothing
> for negative values.
> 
> A minor additional concern is that floats are not given an equivalent
> capability (for obvious reasons) but this breaks symmetry with
> range/xrange which still accept float args.
> 
> 
> Raymond
> 
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From raymond.hettinger at verizon.net  Fri Jun 17 08:39:22 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri, 17 Jun 2005 02:39:22 -0400
Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral
	Literals to Python
Message-ID: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>

While the majority of Python users deem this to be a nice-to-have
feature, the community has been unable to reach a consensus on the
proper syntax after more than two years of intensive debate (the PEP was
introduced in early April 2003).

Most agree that there should be only-one-way-to-do-it; however, the
proponents are evenly split into two camps, with the modernists
preferring IX for nine and the classicists preferring VIIII which was
the most likely spelling in ancient Rome.

The classicists not only rely on set-in-stone tradition, they point to
pragmatic issues such as avoidance of subtraction, ease of coding,
easier mental parsing (much less error prone), and ease of teaching to
beginners.  They assert that the modernists have introduced unnecessary
algorithmic complexity just to save two keystrokes.

The modernists point to compatible Java implementations and current
grade school textbooks.  They believe that users from other languages
will expect the IX form.  Note however, not all the modernists agree on
whether MXM would be a well-formed spelling of 1990; most, but not all
prefer MCMXC despite its likelihood of being mis-parsed on a first
reading.

There is also a small but vocal user group demanding that lowercase
forms be allowed.  Their use cases fall into four categories:  (i)
academia, (ii) the legal profession, (iii) research paper writing, and
(iv) powerpoint slideshows.  Reportedly, this is also a common
convention among Perl programmers.

Links:
 
http://hrsbstaff.ednet.ns.ca/waymac/History%20A/A%20Term%201/1.%20Rome/R
oman_Numerals.htm
    http://www.sizes.com/numbers/roman_numerals.htm


Raymond 


From gvanrossum at gmail.com  Fri Jun 17 08:43:12 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 16 Jun 2005 23:43:12 -0700
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer>
References: <ca471dc2050616200360b5276@mail.gmail.com>
	<006e01c572f7$1ec8eca0$bb07a044@oemcomputer>
Message-ID: <ca471dc2050616234366fb33c2@mail.gmail.com>

[Raymond]
> Let me go on record as a strong -1 for "continue EXPR".  The for-loop is
> our most basic construct and is easily understood in its present form.
> The same can be said for "continue" and "break" which have the added
> advantage of a near zero learning curve for people migrating from other
> languages.
> 
> Any urge to complicate these basic statements should be seriously
> scrutinized and held to high standards of clarity, explainability,
> obviousness, usefulness, and necessity.  IMO, it fails most of those
> tests.
> 
> I would not look forward to explaining "continue EXPR" in the tutorial
> and think it would stand out as an anti-feature.

You sometimes seem to compound a rational argument with too much rhetoric.

The correct argument against "continue EXPR" is that there are no use
cases yet; if there were a good use case, the explanation would follow
easily.

The original use case (though not presented in PEP 340) was to serve
as the equivalent to "return EXPR" in a Ruby block. In Ruby you have
something like this (I probably get the syntax wrong):

  a.foreach() { |x| ...some code... }

This executes the block for each item in a, with x (a formal parameter
to the block) set to each consecutive item. In Python we would write
it like this of course:

  for x in a:
      ...some code...

In Ruby, the block is an anonymous procedure (a thunk) and foreach() a
method that receives a margic (anonymous) parameter which is the
thunk. Inside foreach(), you write "yield EXPR" which calls the block
with x set to EXPR. When the block contains a return statement, the
return value is delivered to the foreach() method as the return value
of yield, which can be assigned like this:

  VAR = yield EXPR

Note that Ruby's yield is just a magic call syntax that calls the thunk!

But this means that the thunks can be used for other purposes as well.
One common use is to have the block act as a Boolean function that
selects items from a list; this way you could write filter() with an
inline selection, for example (making this up):

  a1 = a.filter() { |x| return x > 0 }

might set a1 to the list of a's elements that are > 0. (Not saying
that this is a built-in array method in Ruby, but I think you could
write one.)

This particular example doesn't translate well into Python because a
for-loop doesn't have a return value. Maybe that would be a future
possibility if yield-expressions become accepted (just kidding:-).

However, I can see other uses for looping over a sequence using a
generator and telling the generator something interesting about each
of the sequence's items, e.g. whether they are green, or should be
printed, or which dollar value they represent if any (to make up a
non-Boolean example).

Anyway, "continue EXPR" was born as I was thinking of a way to do this
kind of thing in Python, since I didn't want to give up return as a
way of breaking out of a loop (or several!) and returning from a
function.

But I'm the first to admit that the use case is still very much
hypothetical -- unlike that for g.next(EXPR) and VAR = yield.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From raymond.hettinger at verizon.net  Fri Jun 17 08:55:26 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri, 17 Jun 2005 02:55:26 -0400
Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable
Message-ID: <000b01c57309$8b347d20$bb07a044@oemcomputer>

After nine months, no support has grown beyond the original poster.  The
PEP did however generate some negative responses when brought-up on
comp.lang.python (it made some people's stomach churn).

The PEP fails the tests of obviousness and necessity.  The PEP's switch
example is easily handled by replacing the callable None with a simple,
null lambda:

   def __call__(self, input):
      return {1 : self.a,
              2 : self.b,
              3 : self.c
             }.get(input, lambda *args: 0)(input)

The PEP does not address conflicts with other uses of None (such as
default arguments or indicating values that are not applicable).
Mysteriously, the PEP consumes only *args but not **kwargs.

It also fails with respect to clarity and explicitness.  Defining a
short, explicit default function is a hands-down winner in these two
important measures of merit.


Raymond


From raymond.hettinger at verizon.net  Fri Jun 17 10:36:01 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri, 17 Jun 2005 04:36:01 -0400
Subject: [Python-Dev] Propose rejection of PEPs 239 and 240 -- a builtin
 rational type and rational literals
Message-ID: <000e01c57317$984f40e0$bb07a044@oemcomputer>

These PEPs are four years old.  Nothing is intrinsically wrong with
them, but they have garnered little enthusiasm, discussion, or support,
suggesting that the original need was somewhat modest.
 
In addition, the principal (but not only) use cases for a builtin
rational type and corresponding rational literal have already been
addressed by the decimal module (and the expected future integration of
decimal literals).  From rationale section of the PEPs:
 
"""
Rational numbers are useful for exact and unsurprising arithmetic.  They
give the correct results people have been taught in various math
classes.  Making the "obvious" non-integer type one with more
predictable semantics will surprise new programmers less than using
floating point numbers. As quite a few posts on c.l.py and on
tutor at python.org have shown, people often get bit by strange semantics
of floating point numbers: for example, round(0.98, 2) still gives
0.97999999999999998.
"""
 
The future direction of the decimal module likely entails literals in
the form of 123.45d with binary floats continuing to have the form
123.45.  This conflicts with the rational literal proposal of having
123.45 interpreted as 123 + 45/100.
 
There may still be a use for a rational module in the standard library,
but builtin support is no longer needed or desirable.
 
The PEPs also touch on division ambiguities which were largely resolved
by the acceptance of PEP 238 which introduced the floor division
operator and from __future__ import division.
 
The rational proposal also has an intrinsic weakness shared with Java's
prior versions of BigDecimal which they found to be unworkable in
practice.  The weakness was that repeated operations caused the internal
number of digits to grow beyond reason.  For this reason, the PEP
proposes some arbitrary level of truncation which conflicts with the
goals of having "obvious and exact" arithmetic.  The proposed truncation
methodology was undeveloped and made no proposal for the same fine level
of control as its counterpart in the decimal module where issues of
variable precision, multiple contexts, and alternate rounding modes have
been fully thought out.
 
 
 
Raymond
 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20050617/8367e132/attachment.htm

From Michaels at rd.bbc.co.uk  Fri Jun 17 11:12:25 2005
From: Michaels at rd.bbc.co.uk (Michael Sparks)
Date: Fri, 17 Jun 2005 10:12:25 +0100
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
Message-ID: <200506171012.26036.Michaels@rd.bbc.co.uk>

At 08:24 PM 6/16/2005 -0400, Raymond Hettinger wrote:
> As a further benefit, using
>attributes was a natural approach because that same technique has long
>been used with classes (so no new syntax was needed and the learning
>curve was zero).

On Friday 17 Jun 2005 02:53, Phillip J. Eby wrote:
> Ugh.  Having actually emulated co-routines using generators, I have to tell
> you that I don't find generator attributes natural for this at all;
> returning a value or error (via PEP 343's throw()) from a yield expression
> as in PEP 342 is just what I've been wanting.

We've been essentially emulating co-routines using generators embedded
into a class to give us the equivalent of generator attributes. We've found
this very natural for system composition. (Essentially it's a CSP type system, 
though with an aim of ease of use)

I've written up my talk from ACCU/Python UK this year, and it's available
here: http://www.bbc.co.uk/rd/pubs/whp/whp113.shtml

I'll also be talking about it at Europython later this month.

At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote:
>Someone should really come up with some realistic coroutine examples
>written using PEP 342 (with or without "continue EXPR").

On Friday 17 Jun 2005 05:07:22, Phillip J. Eby wrote:
> How's this?
> 
>    def echo(sock):
>        while True:
>            try:
>                data = yield nonblocking_read(sock)
>                yield nonblocking_write(sock, data)
... snip ...

For comparison, our version of this would be:

from Axon.Component import component
from Kamaelia.SimpleServerComponent import SimpleServer
class Echo(component):
   def mainBody(self):
      while True:
         if self.dataReady("inbox"):
            self.send(data,"outbox")
         yield1

SimpleServer(protocol=EchoProtocol, port=1501).run()


For more interesting pipelines we have:

pipeline(TCPClient("127.0.0.1",1500),
         VorbisDecode(),
         AOAudioPlaybackAdaptor()
        ).run()

Which works in the same way as a Unix pipeline. I haven't written the 
"pipegraph" or similar component yet that could allow this:

graph(A=SingleServer("0.0.0.0", 1500),
           B=Echo(),
           layout = { "A:outbox": "B:inbox", "B:outbox" : "A:inbox" } )

(Still undecided on API for that really, currently the above is a lot more 
verbose -)

By contrast I really can't see how passing attributes in via .next() helps 
this approach in any way (Not that that's a problem for us :).

I CAN see though it helps if you're taking the approach for generator
composition if you're using twisted.flow (though I'll defer a good example
for that to someone else since although I've been asked for a comparison in 
the past, I don't think I'm sufficiently twisted to do so!). 


Michael.
-- 
Michael Sparks, Senior R&D Engineer, Digital Media Group
Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This e-mail may contain personal views which are not the views of the BBC.

From raymond.hettinger at verizon.net  Fri Jun 17 11:57:22 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri, 17 Jun 2005 05:57:22 -0400
Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod() for
	Multiple Divisors
Message-ID: <001d01c57322$f5af0ee0$bb07a044@oemcomputer>

This PEP has been open for two and half years without generating
discussion or support.

Its primary case (converting cumulative seconds into a tuple days,
hours, minutes, and seconds) is a bit wanting because it doesn't
generalize to months and years.  That need is already met in a more
robust and flexible way by date and time specific modules.

The use case is also somewhat unique.  Recalling 25 years of
programming, almost every real case of repeated divmod() calls have been
in a loop with a single constant denominator (i.e. dividing by a base in
a radix conversion).  The PEP does suggest other applications but they
are all narrow offerings (gallon/quart/pint/ounce, miles/yards/feet,
pound/shilling/pence) that are extremely rare in real apps.

More importantly, the gain in succinctness is offset by a loss of
clarity.  Consider:

   dist, inches = divmod(dist, 12)
   yards, feet = divmod(dist, 3)

versus a proposed:

   yards, feet, inches = divmod(dist, 3, 12)

The latter form makes it difficult to visually confirm the correct
number of target variables.  Likewise, it is not at all obvious that the
order of the 3 and 12 are correct.  Users from other languages will tend
to automatically understand the current form and be mystified by the
second (especially given how infrequently it will arise).

The PEP draws its inspiration from an APL operator.  APL imbues many of
its operators with scalar/scalar, scalar/array, and array/array
capabilities.  But in Python (not numeric) we tend to leave the
operators in simple form and abstract the vectorization into operator
independent primitives (i.e. map and reduce).  Mathematica takes a
similar approach by offering functions like NestList().  So, instead of
a vectorized divmod(), it is wiser for someone to post a single
accumulation recipe that would work with a variety of binary operators.


Executive summary:  cute, but unpersuasive and unnecessary, not worth
the time to code, test, document, maintain, and explain.



Raymond 


From gjc at inescporto.pt  Fri Jun 17 12:29:49 2005
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Fri, 17 Jun 2005 11:29:49 +0100
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <200506171012.26036.Michaels@rd.bbc.co.uk>
References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
	<200506171012.26036.Michaels@rd.bbc.co.uk>
Message-ID: <1119004190.648.13.camel@localhost>

  Hello,

  I found your paper very interesting.  I have also written a very
minimalistic white paper, mostly aimed at the PyGTK community, with a
small module for pseudo-threads using python generators:

	http://www.gnome.org/~gjc/gtasklet/gtasklets.html

  I don't have time to follow this whole discussion, but I leave it here
as another example of python pseudo-threads.  I also am very much in
favour of having yield receive return values or exceptions, as this
would make pseudo-threads much more elegant.  And I very much wish
python had this builtin or in std library.

  In conjunction with pseudo-threads, I think a "python main loop"
implementation is fundamental. Such main loop with permit the programmer
to  register callbacks for events, such as timeouts, IO conditions, idle
tasks, etc., such as one found glib (gtk+'s underlying library).  I
already pointed out one such implementation that I use for one of my
projects, and it already has unit tests to prove that it works.

  This is also related to the "deprecate asyncore/asynchat" discussions
going on earlier.  IMHO, they should really be deprecated, and a
pseudo-threads solution could be used instead.

  Anyway, I'd love to help more in this area, but unfortunately I don't
have time for these endless discussions... :P

  Best regards.


On Fri, 2005-06-17 at 10:12 +0100, Michael Sparks wrote:
> At 08:24 PM 6/16/2005 -0400, Raymond Hettinger wrote:
> > As a further benefit, using
> >attributes was a natural approach because that same technique has long
> >been used with classes (so no new syntax was needed and the learning
> >curve was zero).
> 
> On Friday 17 Jun 2005 02:53, Phillip J. Eby wrote:
> > Ugh.  Having actually emulated co-routines using generators, I have to tell
> > you that I don't find generator attributes natural for this at all;
> > returning a value or error (via PEP 343's throw()) from a yield expression
> > as in PEP 342 is just what I've been wanting.
> 
> We've been essentially emulating co-routines using generators embedded
> into a class to give us the equivalent of generator attributes. We've found
> this very natural for system composition. (Essentially it's a CSP type system, 
> though with an aim of ease of use)
> 
> I've written up my talk from ACCU/Python UK this year, and it's available
> here: http://www.bbc.co.uk/rd/pubs/whp/whp113.shtml
> 
> I'll also be talking about it at Europython later this month.
> 
> At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote:
> >Someone should really come up with some realistic coroutine examples
> >written using PEP 342 (with or without "continue EXPR").
> 
> On Friday 17 Jun 2005 05:07:22, Phillip J. Eby wrote:
> > How's this?
> > 
> >    def echo(sock):
> >        while True:
> >            try:
> >                data = yield nonblocking_read(sock)
> >                yield nonblocking_write(sock, data)
> ... snip ...
> 
> For comparison, our version of this would be:
> 
> from Axon.Component import component
> from Kamaelia.SimpleServerComponent import SimpleServer
> class Echo(component):
>    def mainBody(self):
>       while True:
>          if self.dataReady("inbox"):
>             self.send(data,"outbox")
>          yield1
> 
> SimpleServer(protocol=EchoProtocol, port=1501).run()
> 
> 
> For more interesting pipelines we have:
> 
> pipeline(TCPClient("127.0.0.1",1500),
>          VorbisDecode(),
>          AOAudioPlaybackAdaptor()
>         ).run()
> 
> Which works in the same way as a Unix pipeline. I haven't written the 
> "pipegraph" or similar component yet that could allow this:
> 
> graph(A=SingleServer("0.0.0.0", 1500),
>            B=Echo(),
>            layout = { "A:outbox": "B:inbox", "B:outbox" : "A:inbox" } )
> 
> (Still undecided on API for that really, currently the above is a lot more 
> verbose -)
> 
> By contrast I really can't see how passing attributes in via .next() helps 
> this approach in any way (Not that that's a problem for us :).
> 
> I CAN see though it helps if you're taking the approach for generator
> composition if you're using twisted.flow (though I'll defer a good example
> for that to someone else since although I've been asked for a comparison in 
> the past, I don't think I'm sufficiently twisted to do so!). 
> 
> 
> Michael.
-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic.


From ncoghlan at gmail.com  Fri Jun 17 12:54:22 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 17 Jun 2005 20:54:22 +1000
Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod()
 for	Multiple Divisors
In-Reply-To: <001d01c57322$f5af0ee0$bb07a044@oemcomputer>
References: <001d01c57322$f5af0ee0$bb07a044@oemcomputer>
Message-ID: <42B2ABDE.60608@gmail.com>

Raymond Hettinger wrote:
> Executive summary:  cute, but unpersuasive and unnecessary, not worth
> the time to code, test, document, maintain, and explain.

Plus, it fails the "not every 3-line function has to be a builtin" 
guideline:

  def extended_divmod(numerator, *denominators):
     remainders = []
     for denominator in reversed(denominators):
         numerator, remainder = divmod(numerator, denominator)
         remainders.insert(0, remainder)
     return tuple(remainders)

OK, 5 lines. Anyway, not very hard to write for anyone with a genuine 
use case - and, like you, I've never used divmod for anything other 
than extracting digits (or groups of digits) from numbers.

I also don't buy the 'tedious and easy to get wrong each time you need 
it' justification in the PEP. Getting the argument order to the 
extended divmod wrong seems to be even easier.

For each of the cited use cases, a well-named function, or a proper 
class seems like a much cleaner solution.

e.g.

   class Declination(object):
     def __init__(self, value):
       try:
         # Copy a duck-typed declination
         self.degrees = value.degrees
         self.minutes = value.minutes
         self.seconds = value.seconds
       except AttributeError:
         try:
           # Allow any three-value sequence
           self.degrees, self.minutes, self.seconds = value
         except TypeError:
           # Divide a number
           value, self.seconds = divmod(value, 60)
           value, self.minutes = divmod(value, 60)
           value, self.degrees = divmod(value, 360)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From skip at pobox.com  Fri Jun 17 13:29:03 2005
From: skip at pobox.com (Skip Montanaro)
Date: Fri, 17 Jun 2005 06:29:03 -0500
Subject: [Python-Dev] PEP 304 "Controlling Generation of Bytecode Files"
	- patch updated
In-Reply-To: <vf4dwg97.fsf@python.net>
References: <mailman.559.1118972240.10512.python-list@python.org>
	<vf4dwg97.fsf@python.net>
Message-ID: <17074.46079.101393.509174@montanaro.dyndns.org>

    Skip> http://python.org/sf/677103

    Thomas> There's no patch attached.

*sigh*

Thanks for noticing the problem.  Apparently, since I last updated the
patch, SF implemented a 250kbyte limit on file uploads.  This one is big
because it includes a suitably modified configure script that was generated
with a slightly different version of autoconf than what's in the current
Python distro so people don't need to have autoconf installed to work with
the patch.  I've attached a gzipped version of the patch (bcb.diffs.gz).

Skip


From amk at amk.ca  Fri Jun 17 14:30:23 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 17 Jun 2005 08:30:23 -0400
Subject: [Python-Dev] Request to rewrite PEP 206
Message-ID: <20050617123023.GA15532@rogue.amk.ca>

Just a note, sparked by Raymond's recent work cleaning up old PEPs:
I'd like to take over PEP 206, the "Batteries Included" PEP, and
rewrite it to describe a "Python Advanced Library", a set of
third-party packages to complement the standard library.

I've written to Moshe, the original author, to check that this is OK
with him, and am waiting for a response.

--amk

From barry at python.org  Fri Jun 17 15:29:00 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 17 Jun 2005 09:29:00 -0400
Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman
	Numeral	Literals to Python
In-Reply-To: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>
References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>
Message-ID: <1119014940.21565.7.camel@geddy.wooz.org>

On Fri, 2005-06-17 at 02:39, Raymond Hettinger wrote:
> While the majority of Python users deem this to be a nice-to-have
> feature

Really?  Where's the supporting poll data?  In over 10 years of Python
programming, I've never once needed a Roman number literal.  Worse, I
don't buy the compatibility argument.  I'm as anal as anyone about PEP 8
style, but that's still no reason to break code like

>>> MIX = True

I wouldn't be opposed to a library that provided a function to convert
to and from Romans but I don't think Python needs Roman numeral literal
support.

+1 for rejecting.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/9b09348e/attachment.pgp

From barry at python.org  Fri Jun 17 15:34:35 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 17 Jun 2005 09:34:35 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer>
References: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer>
Message-ID: <1119015274.21570.10.camel@geddy.wooz.org>

On Fri, 2005-06-17 at 00:43, Raymond Hettinger wrote:

> Let me go on record as a strong -1 for "continue EXPR".  The for-loop is
> our most basic construct and is easily understood in its present form.
> The same can be said for "continue" and "break" which have the added
> advantage of a near zero learning curve for people migrating from other
> languages.
> 
> Any urge to complicate these basic statements should be seriously
> scrutinized and held to high standards of clarity, explainability,
> obviousness, usefulness, and necessity.  IMO, it fails most of those
> tests. 
> 
> I would not look forward to explaining "continue EXPR" in the tutorial
> and think it would stand out as an anti-feature.

I'm sympathetic to this argument.  I also find yield expressions
jarring.  I don't have any better suggestions though.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/1d15dc92/attachment.pgp

From barry at python.org  Fri Jun 17 15:36:25 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 17 Jun 2005 09:36:25 -0400
Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable
In-Reply-To: <000b01c57309$8b347d20$bb07a044@oemcomputer>
References: <000b01c57309$8b347d20$bb07a044@oemcomputer>
Message-ID: <1119015384.21571.12.camel@geddy.wooz.org>

On Fri, 2005-06-17 at 02:55, Raymond Hettinger wrote:
> After nine months, no support has grown beyond the original poster.  The
> PEP did however generate some negative responses when brought-up on
> comp.lang.python (it made some people's stomach churn).
> 
> The PEP fails the tests of obviousness and necessity.

I agree.  The fact that None is not callable is a /feature/ IMO.  +1 for
rejecting this PEP.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/e79027f4/attachment.pgp

From barry at python.org  Fri Jun 17 15:50:38 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 17 Jun 2005 09:50:38 -0400
Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend
	divmod()	for Multiple Divisors
In-Reply-To: <001d01c57322$f5af0ee0$bb07a044@oemcomputer>
References: <001d01c57322$f5af0ee0$bb07a044@oemcomputer>
Message-ID: <1119016238.21566.18.camel@geddy.wooz.org>

On Fri, 2005-06-17 at 05:57, Raymond Hettinger wrote:
> This PEP has been open for two and half years without generating
> discussion or support.

Interesting.  Just yesterday I wrote a simple stopwatch-like timer
script and I found that I needed three divmod calls to convert from
seconds into a datetime.time object.  This PEP might have come in handy
there, but OTOH, I'm not so sure that's enough justification to accept
the PEP.

> Its primary case (converting cumulative seconds into a tuple days,
> hours, minutes, and seconds) is a bit wanting because it doesn't
> generalize to months and years.  That need is already met in a more
> robust and flexible way by date and time specific modules.

Actually, no, because datetime.time(seconds=50227) throws an exception. 
But in my specific case, I didn't find the need for three divmod calls
nearly as frustrating as the lack of a datetime.time.fromseconds() call.

> More importantly, the gain in succinctness is offset by a loss of
> clarity.  Consider:
> 
>    dist, inches = divmod(dist, 12)
>    yards, feet = divmod(dist, 3)
> 
> versus a proposed:
> 
>    yards, feet, inches = divmod(dist, 3, 12)
> 
> The latter form makes it difficult to visually confirm the correct
> number of target variables.  Likewise, it is not at all obvious that the
> order of the 3 and 12 are correct.

I agree.  My three divmod solution is perfectly readable and simple to
write.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/a6d19c01/attachment.pgp

From tim.peters at gmail.com  Fri Jun 17 16:41:12 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 17 Jun 2005 10:41:12 -0400
Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral
	Literals to Python
In-Reply-To: <1119014940.21565.7.camel@geddy.wooz.org>
References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>
	<1119014940.21565.7.camel@geddy.wooz.org>
Message-ID: <1f7befae05061707413b58bf94@mail.gmail.com>

[Raymond Hettinger]
>> While the majority of Python users deem this to be a nice-to-have
>> feature

[Barry Warsaw]
> Really?  Where's the supporting poll data?

We've run IV polls since this PEP was introduced, and the geometric
mean of those shows LXVIII% of Python users strongly in favor (+I),
and an additional XXI% not opposed (+roman(0), really -- I'm not sure
how to spell zero in Roman numerals, but trust that the new `roman()`
builtin will handle it correctly).

> In over 10 years of Python programming, I've never once needed a Roman
> number literal.

Who cares what someone still stuck on Python Challenge #XVII thinks? 
Input from real Python programmers would be appreciated, though.  For
example, I'm eager to switch ZODB's object ids to Roman numerals.  For
example, as Raymond acknowledged, that would make it much easier to
produce ZODB PowerPoint slides.

> Worse, I don't buy the compatibility argument.  I'm as anal as anyone about
> PEP 8 style, but that's still no reason to break code like
>
> >>> MIX = True

And you care nothing for the possibility that an ancient Roman
explorer found frozen in the Arctic may be revived, and find such code
incomprehensibly obscure?  Python should be for everyone, not just the
living.

BTW, you might want to reread Raymond's post -- he was having an
indecently good time at, well, someone's expense <wink>.

From gvanrossum at gmail.com  Fri Jun 17 16:48:57 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 17 Jun 2005 07:48:57 -0700
Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral
	Literals to Python
In-Reply-To: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>
References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>
Message-ID: <ca471dc20506170748109ae6a4@mail.gmail.com>

+M to reject.

On 6/16/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> While the majority of Python users deem this to be a nice-to-have
> feature, the community has been unable to reach a consensus on the
> proper syntax after more than two years of intensive debate (the PEP was
> introduced in early April 2003).
> 
> Most agree that there should be only-one-way-to-do-it; however, the
> proponents are evenly split into two camps, with the modernists
> preferring IX for nine and the classicists preferring VIIII which was
> the most likely spelling in ancient Rome.
> 
> The classicists not only rely on set-in-stone tradition, they point to
> pragmatic issues such as avoidance of subtraction, ease of coding,
> easier mental parsing (much less error prone), and ease of teaching to
> beginners.  They assert that the modernists have introduced unnecessary
> algorithmic complexity just to save two keystrokes.
> 
> The modernists point to compatible Java implementations and current
> grade school textbooks.  They believe that users from other languages
> will expect the IX form.  Note however, not all the modernists agree on
> whether MXM would be a well-formed spelling of 1990; most, but not all
> prefer MCMXC despite its likelihood of being mis-parsed on a first
> reading.
> 
> There is also a small but vocal user group demanding that lowercase
> forms be allowed.  Their use cases fall into four categories:  (i)
> academia, (ii) the legal profession, (iii) research paper writing, and
> (iv) powerpoint slideshows.  Reportedly, this is also a common
> convention among Perl programmers.
> 
> Links:
> 
> http://hrsbstaff.ednet.ns.ca/waymac/History%20A/A%20Term%201/1.%20Rome/R
> oman_Numerals.htm
>     http://www.sizes.com/numbers/roman_numerals.htm
> 
> 
> Raymond
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Fri Jun 17 17:33:22 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 17 Jun 2005 11:33:22 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <1119004190.648.13.camel@localhost>
References: <200506171012.26036.Michaels@rd.bbc.co.uk>
	<5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
	<200506171012.26036.Michaels@rd.bbc.co.uk>
Message-ID: <5.1.1.6.0.20050617111758.01e021e0@mail.telecommunity.com>

At 11:29 AM 6/17/2005 +0100, Gustavo J. A. M. Carneiro wrote:
>   In conjunction with pseudo-threads, I think a "python main loop"
>implementation is fundamental. Such main loop with permit the programmer
>to  register callbacks for events, such as timeouts, IO conditions, idle
>tasks, etc., such as one found glib (gtk+'s underlying library).  I
>already pointed out one such implementation that I use for one of my
>projects, and it already has unit tests to prove that it works.

I think it's important to point out that such a "main loop" needs to be 
defined as an interface, rather than an implementation, because there are 
many such "main loops" out there as far as GTK, wx, OS X, etc. that have 
different implementation details as to how timeouts and I/O have to be managed.

Since I see from your web page that you've looked at peak.events, I would 
refer you to the IEventLoop interface as an example of such an interface; 
any Twisted reactor can be adapted to provide most of the IEventLoop 
features (and vice versa) which means an interface like it should be usable 
on a variety of platforms.

Of course, I also think that before proposing an event loop facility for 
the stdlib, we should actually succeed in implementing next(arg), yield 
expressions, and throw().  I'll probably take a look at next()/throw() this 
weekend to see if there's anything I can contribute to the ceval.c part; 
I'm a bit less likely to be able to help on the compilation side of things, 
though.  (Apart from maybe adding a POP_TOP after existing yield statements.)


From ncoghlan at gmail.com  Fri Jun 17 17:50:02 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jun 2005 01:50:02 +1000
Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod()
 for	Multiple Divisors
In-Reply-To: <000001c57330$28e13740$bb07a044@oemcomputer>
References: <000001c57330$28e13740$bb07a044@oemcomputer>
Message-ID: <42B2F12A.2020809@gmail.com>

Raymond Hettinger wrote:
>>Plus, it fails the "not every 3-line function has to be a builtin"
>>guideline:
> 
> 
> Not to pick, but I hope this doesn't become a recurring refrain.  That
> isn't a real guideline, it's more of a snipe.  It also runs counter to
> Zen about proposals not being complex and being easy to explain.

I guess I really mean "the use case is obscure enough that simply 
writing the 5-line function is better than cluttering the API of a 
builtin", but that doesn't have quite the same ring to it ;)

>  There
> are tons of exceptions.  Guido's new any() and all() replace only a
> single line.  The sorted() builtin was very successful and it only
> replaced a couple of lines.  The key= argument is also successful but
> replaces a simple, three-line Schwarzian transform.  Reading DictMixin
> shows that most dictionary methods are trivially expressed in terms of a
> few primitives; however, we've learned that the mapping API is
> excellent, expressive, and useful (except for setdefault which I've
> grown to hate ;-).  IOW, the quip is food for thought but not
> necessarily either a positive or negative point about a proposal.

That's basically what I meant - and what I take the phrase to mean 
when someone else uses it.

If something is simple to write, but the use case is obscure, then 
that's an argument *against* making it a builtin, since half the time 
you'll have the function written before you remember there's a builtin 
for it. On the other hand, if the use case is common enough, rewriting 
it every time you need it is just a pain.

The 'not every . . .' comment just tries to say all that using only 
ten words.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From gvanrossum at gmail.com  Fri Jun 17 17:55:48 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 17 Jun 2005 08:55:48 -0700
Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable
In-Reply-To: <1119015384.21571.12.camel@geddy.wooz.org>
References: <000b01c57309$8b347d20$bb07a044@oemcomputer>
	<1119015384.21571.12.camel@geddy.wooz.org>
Message-ID: <ca471dc20506170855ba3b777@mail.gmail.com>

Yes please. That one will never get past me as long as I'm alive.

(At PyCon I met one person who proposed an even more radical approach;
I think he wanted getattr(None, <anything>) to return None. But he was
certified insane. :-)

On 6/17/05, Barry Warsaw <barry at python.org> wrote:
> On Fri, 2005-06-17 at 02:55, Raymond Hettinger wrote:
> > After nine months, no support has grown beyond the original poster.  The
> > PEP did however generate some negative responses when brought-up on
> > comp.lang.python (it made some people's stomach churn).
> >
> > The PEP fails the tests of obviousness and necessity.
> 
> I agree.  The fact that None is not callable is a /feature/ IMO.  +1 for
> rejecting this PEP.
> 
> -Barry
> 
> 
> 
> BodyID:129997985.2.n.logpart (stored separately)
> 
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.peters at gmail.com  Fri Jun 17 18:16:10 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 17 Jun 2005 12:16:10 -0400
Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable
In-Reply-To: <000b01c57309$8b347d20$bb07a044@oemcomputer>
References: <000b01c57309$8b347d20$bb07a044@oemcomputer>
Message-ID: <1f7befae050617091673caabe8@mail.gmail.com>

[Raymond Hettinger]
> After nine months, no support has grown beyond the original poster.

Never will, either -- even Roman numeral literals are more Pythonic
than this one.

More Pythonic:  make integers callable:  i(arglist) returns the i'th
argument.  So, e.g., people who find it inconvenient to index a list
like this:

    x[i]

could index it like this instead:

    i(*x)

Punchline:  I didn't make this up -- that's how integers work in Icon!  Kinda.

    y := 2
    y(x, y, z) := 3

also works to bind `y` to 3.  Python is falling _way_ behind <wink>.

From tim.peters at gmail.com  Fri Jun 17 18:33:32 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 17 Jun 2005 12:33:32 -0400
Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod()
	for Multiple Divisors
In-Reply-To: <1119016238.21566.18.camel@geddy.wooz.org>
References: <001d01c57322$f5af0ee0$bb07a044@oemcomputer>
	<1119016238.21566.18.camel@geddy.wooz.org>
Message-ID: <1f7befae0506170933636c2dc@mail.gmail.com>

About PEP 303, I use divmod for lots (and lots) of things, but I've
got no real use for an extended divmod() either.  -1:  it would be
low-use, confusing clutter.

[Barry]
> Interesting.  Just yesterday I wrote a simple stopwatch-like timer
> script and I found that I needed three divmod calls to convert from
> seconds into a datetime.time object.

You don't need any divmods for that ...

...
> Actually, no, because datetime.time(seconds=50227) throws an exception.

That's right:  the time class models a time of day, not "seconds from
Barry's implied midnight epoch" (or something like that).  What you
wanted was a timedelta instead.  Converting that to a time is then
simplicity itself <wink>:

>>> print (datetime(1, 1, 1) + timedelta(seconds=50227)).time()
13:57:07

You have to go thru a datetime object first because time objects don't
support arithmetic either.  That isn't all bad.  By going thru a
datetime first, it's clear what happens if the number of seconds you
feed in exceeds a day's worth.  You can check for that or ignore it
then, depending on what your app wants (it may or may not be "an
error", depending on the app).

From gvanrossum at gmail.com  Fri Jun 17 20:23:05 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 17 Jun 2005 11:23:05 -0700
Subject: [Python-Dev] Request to rewrite PEP 206
In-Reply-To: <20050617123023.GA15532@rogue.amk.ca>
References: <20050617123023.GA15532@rogue.amk.ca>
Message-ID: <ca471dc20506171123c0e1a9b@mail.gmail.com>

Sounds great! If Moshe doesn't respond within a reasonable time, you
can take it over by default IMO.

On 6/17/05, A.M. Kuchling <amk at amk.ca> wrote:
> Just a note, sparked by Raymond's recent work cleaning up old PEPs:
> I'd like to take over PEP 206, the "Batteries Included" PEP, and
> rewrite it to describe a "Python Advanced Library", a set of
> third-party packages to complement the standard library.
> 
> I've written to Moshe, the original author, to check that this is OK
> with him, and am waiting for a response.
> 
> --amk
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Fri Jun 17 20:28:57 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 17 Jun 2005 11:28:57 -0700
Subject: [Python-Dev] Propose rejection of PEPs 239 and 240 -- a builtin
	rational type and rational literals
In-Reply-To: <000e01c57317$984f40e0$bb07a044@oemcomputer>
References: <000e01c57317$984f40e0$bb07a044@oemcomputer>
Message-ID: <ca471dc20506171128793fba9d@mail.gmail.com>

Agreed. Rational arithmetic was the default "exact" arithmetic in ABC
and it did not work out as expected.

On 6/17/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
>  
> These PEPs are four years old.  Nothing is intrinsically wrong with them,
> but they have garnered little enthusiasm, discussion, or support, suggesting
> that the original need was somewhat modest.
>  
> In addition, the principal (but not only) use cases for a builtin rational
> type and corresponding rational literal have already been addressed by the
> decimal module (and the expected future integration of decimal literals). 
> From rationale section of the PEPs:
>  
> """
> Rational numbers are useful for exact and unsurprising arithmetic.  They
> give the correct results people have been taught in various math classes. 
> Making the "obvious" non-integer type one with more predictable semantics
> will surprise new programmers less than using floating point numbers. As
> quite a few posts on c.l.py and on tutor at python.org have shown, people often
> get bit by strange semantics of floating point numbers: for example,
> round(0.98, 2) still gives 0.97999999999999998.
> """
>  
> The future direction of the decimal module likely entails literals in the
> form of 123.45d with binary floats continuing to have the form 123.45.  This
> conflicts with the rational literal proposal of having 123.45 interpreted as
> 123 + 45/100.
>  
> There may still be a use for a rational module in the standard library, but
> builtin support is no longer needed or desirable.
>  
> The PEPs also touch on division ambiguities which were largely resolved by
> the acceptance of PEP 238 which introduced the floor division operator and
> from __future__ import division.
>  
> The rational proposal also has an intrinsic weakness shared with Java's
> prior versions of BigDecimal which they found to be unworkable in practice. 
> The weakness was that repeated operations caused the internal number of
> digits to grow beyond reason.  For this reason, the PEP proposes some
> arbitrary level of truncation which conflicts with the goals of having
> "obvious and exact" arithmetic.  The proposed truncation methodology was
> undeveloped and made no proposal for the same fine level of control as its
> counterpart in the decimal module where issues of variable precision,
> multiple contexts, and alternate rounding modes have been fully thought out.
>  
>  
>  
> Raymond
>  
>  
> 
>   
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 
> 
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Fri Jun 17 20:29:52 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri, 17 Jun 2005 11:29:52 -0700
Subject: [Python-Dev] Propose rejection of PEPs 239 and 240 -- a builtin
	rational type and rational literals
In-Reply-To: <000e01c57317$984f40e0$bb07a044@oemcomputer>
References: <000e01c57317$984f40e0$bb07a044@oemcomputer>
Message-ID: <20050617103216.7277.JCARLSON@uci.edu>


"Raymond Hettinger" <raymond.hettinger at verizon.net> wrote:
> There may still be a use for a rational module in the standard library,
> but builtin support is no longer needed or desirable.

Sounds good.

> The rational proposal also has an intrinsic weakness shared with Java's
> prior versions of BigDecimal which they found to be unworkable in
> practice.  The weakness was that repeated operations caused the internal
> number of digits to grow beyond reason.  For this reason, the PEP
> proposes some arbitrary level of truncation which conflicts with the
> goals of having "obvious and exact" arithmetic.  The proposed truncation
> methodology was undeveloped and made no proposal for the same fine level
> of control as its counterpart in the decimal module where issues of
> variable precision, multiple contexts, and alternate rounding modes have
> been fully thought out.

Here is an option: exploit all of the thought and effort that has gone
into decimals.  Offer a "context" argument which handles the precision
and rounding of the rational numerator and denominator as necessary,
defaulting to the current decimal context, with automatic rounding
(using the same kinds of semantics that decimals currently uses). Offer
a mechanism by which one can use exact rationals, which use longs rather
than decimals.

The idea obviously needs fleshing out, but perhaps it is a start for
someone who wants rationals in the standard library.

 - Josiah


From martin at v.loewis.de  Fri Jun 17 23:34:27 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 17 Jun 2005 23:34:27 +0200
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <2m4qbybpf2.fsf@starship.python.net>
References: <2mzmtrbxwo.fsf@starship.python.net> <42B0725D.9040200@v.loewis.de>
	<2m4qbybpf2.fsf@starship.python.net>
Message-ID: <42B341E3.5000609@v.loewis.de>

Michael Hudson wrote:
> I've been looking at this area partly to try and understand this bug:
> 
>     [ 1163563 ] Sub threads execute in restricted mode
> 
> but I'm not sure the whole idea of multiple interpreters isn't
> inherently doomed :-/

That's what Tim asserts, saying that people who want to use the
feature should fix it themselves.

Regards,
Martin

From raymond.hettinger at verizon.net  Fri Jun 17 23:49:37 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri, 17 Jun 2005 17:49:37 -0400
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda
Message-ID: <000001c57386$75c23180$bb07a044@oemcomputer>

This PEP is an excellent example of improving readability and usability
by omitting a keyword and simplifying syntax.  It neither provides nor
takes away functionality; instead, it is a bit of a beautification
effort.

Essentially it creates a lighter-weight, more usable syntax for
specifying deferred function arguments.  Like decorators, this helps out
programmers who understand and use this technique, and it is harmless
and invisible for the rest.  It is especially handy in the absence of an
if-then-else expression.  The PEP is backwards compatible.

Recommend accepting just the basic PEP which only targets simple,
obvious cases.  The discussed extensions are unattractive and should be
skipped.



Raymond


From gvanrossum at gmail.com  Fri Jun 17 23:59:58 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 17 Jun 2005 14:59:58 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <000001c57386$75c23180$bb07a044@oemcomputer>
References: <000001c57386$75c23180$bb07a044@oemcomputer>
Message-ID: <ca471dc20506171459b54789d@mail.gmail.com>

[Raymond Hettinger]
> This PEP is an excellent example of improving readability and usability
> by omitting a keyword and simplifying syntax.  It neither provides nor
> takes away functionality; instead, it is a bit of a beautification
> effort.
> 
> Essentially it creates a lighter-weight, more usable syntax for
> specifying deferred function arguments.  Like decorators, this helps out
> programmers who understand and use this technique, and it is harmless
> and invisible for the rest.  It is especially handy in the absence of an
> if-then-else expression.  The PEP is backwards compatible.
> 
> Recommend accepting just the basic PEP which only targets simple,
> obvious cases.  The discussed extensions are unattractive and should be
> skipped.

-1. The "unary colon" looks unPythonic to me.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From raymond.hettinger at verizon.net  Sat Jun 18 00:25:07 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri, 17 Jun 2005 18:25:07 -0400
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers
Message-ID: <000301c5738b$6afc5fa0$bb07a044@oemcomputer>

IIRC, there was a decision to not implement phase C and to keep the
trailing L in representations of long integers.

If so, I believe the PEP can be marked as final.  We've done all we're
going to do.


Raymond


From gvanrossum at gmail.com  Sat Jun 18 00:33:47 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri, 17 Jun 2005 15:33:47 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <000301c5738b$6afc5fa0$bb07a044@oemcomputer>
References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer>
Message-ID: <ca471dc20506171533147714a8@mail.gmail.com>

On 6/17/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> IIRC, there was a decision to not implement phase C and to keep the
> trailing L in representations of long integers.

Actually, the PEP says phase C will be implemented in Python 3.0 and
that's still my plan.

> If so, I believe the PEP can be marked as final.  We've done all we're
> going to do.

For 2.x, yes. I'm fine with marking it as Final and adding this to PEP
3000 instead.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From Scott.Daniels at Acm.Org  Sat Jun 18 01:23:36 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Fri, 17 Jun 2005 16:23:36 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <ca471dc20506171533147714a8@mail.gmail.com>
References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer>
	<ca471dc20506171533147714a8@mail.gmail.com>
Message-ID: <d8vlna$i36$1@sea.gmane.org>

Guido van Rossum wrote:
> On 6/17/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
>>IIRC, there was a decision to not implement phase C and to keep the
>>trailing L in representations of long integers.
> For 2.x, yes. I'm fine with marking it as Final and adding this to PEP
> 3000 instead.

Since PEP 313 has been rejected, the trailing L no longer introduces
ambiguity in the representation of roman(40) vs. roman(10L).

--Scott David Daniels
Scott.Daniels at Acm.Org


From ncoghlan at gmail.com  Sat Jun 18 01:33:28 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jun 2005 09:33:28 +1000
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple
	Implicit	Lambda
In-Reply-To: <ca471dc20506171459b54789d@mail.gmail.com>
References: <000001c57386$75c23180$bb07a044@oemcomputer>
	<ca471dc20506171459b54789d@mail.gmail.com>
Message-ID: <42B35DC8.6040308@gmail.com>

Guido van Rossum wrote:
>>Recommend accepting just the basic PEP which only targets simple,
>>obvious cases.  The discussed extensions are unattractive and should be
>>skipped.
> 
> 
> -1. The "unary colon" looks unPythonic to me.
> 

Step 1 would be to require parentheses around the whole thing (ala 
generator expressions) to make it easier to see where the deferred 
expression ends.

But all my use cases that I can think off the top of my head involve 
'sorted', where it wouldn't help at all because of the need for an 
argument.

So I'd rather see a serious discussion regarding giving lambdas a more 
Pythonic syntax in general, rather than one that only applied to the 
'no-argument' case [1]

Cheers,
Nick.

[1] http://wiki.python.org/moin/AlternateLambdaSyntax
The 'expression-before-args' version using just the 'from' keyword is 
still my favourite.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From pje at telecommunity.com  Sat Jun 18 02:54:22 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 17 Jun 2005 20:54:22 -0400
Subject: [Python-Dev] Implementing PEP 342 (was Re: Withdrawn PEP 288 and
 thoughts on PEP 342)
In-Reply-To: <ca471dc2050616200360b5276@mail.gmail.com>
References: <005301c572e3$f2518aa0$bb07a044@oemcomputer>
	<5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
	<005301c572e3$f2518aa0$bb07a044@oemcomputer>
Message-ID: <5.1.1.6.0.20050617155758.035955d8@mail.telecommunity.com>

At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote:
>I do like "continue EXPR" but I have to admit I haven't even tried to
>come up with examples -- it may be unnecessary. As Phillip says, yield
>expressions and g.next(EXPR) are the core -- and also incidentally
>look like they will cause the most implementation nightmares. (If
>someone wants to start implementing these two now, go right ahead!)

FYI, I've started work on a patch.  I've got argument passing into 
generators working, and compiling of parenthesized yield expressions, in 
both the C and Python compilers (although the output of the compiler 
package isn't tested yet).  I haven't implemented no-argument yields yet, 
either, or unparenthesized yields on the far RHS of an assignment.  I do 
plan to implement throw() as part of the same patch.  Much of what remains 
is expanding the test suite and writing documentation, though.

It turns out that making 'next(EXPR)' work is a bit tricky; I was going to 
use METH_COEXIST and METH_VARARGS, but then it occurred to me that 
METH_VARARGS adds overhead to normal Python calls to 'next()', so I 
implemented a separate 'send(EXPR)' method instead, and left 'next()' a 
no-argument call.  Whether this is the way it should really work or not is 
a PEP discussion, of course, but it does seem to me that making send(ob) 
and throw(typ,val,tb) separate methods from the iterator protocol is a 
reasonable thing to do.

Anyway, the patch isn't ready yet, but I hope to be able to post something 
for review before the weekend is out.


From joachim.koenig-baltes at emesgarten.de  Fri Jun 17 22:53:32 2005
From: joachim.koenig-baltes at emesgarten.de (Joachim Koenig-Baltes)
Date: Fri, 17 Jun 2005 22:53:32 +0200
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <ca471dc2050616234366fb33c2@mail.gmail.com>
References: <ca471dc2050616200360b5276@mail.gmail.com>	<006e01c572f7$1ec8eca0$bb07a044@oemcomputer>
	<ca471dc2050616234366fb33c2@mail.gmail.com>
Message-ID: <42B3384C.7080908@emesgarten.de>

Guido van Rossum wrote:

>
>However, I can see other uses for looping over a sequence using a
>generator and telling the generator something interesting about each
>of the sequence's items, e.g. whether they are green, or should be
>printed, or which dollar value they represent if any (to make up a
>non-Boolean example).
>
>Anyway, "continue EXPR" was born as I was thinking of a way to do this
>kind of thing in Python, since I didn't want to give up return as a
>way of breaking out of a loop (or several!) and returning from a
>function.
>
>But I'm the first to admit that the use case is still very much
>hypothetical -- unlike that for g.next(EXPR) and VAR = yield.
>
>  
>
My use case for this is a directory tree walking generator that
yields all the files including the directories in a depth first manner.
If a directory satisfies a condition (determined by the caller) the
generator shall not descend into it.

Something like:

DONOTDESCEND=1
for path in mywalk("/usr/src"):
    if os.path.isdir(path) and os.path.basename(path) == "CVS":
        continue DONOTDESCEND
    # do something with path

Of course there are different solutions to this problem with callbacks
or filters but i like this one as the most elegant.

Joachim






From abo at minkirri.apana.org.au  Sat Jun 18 03:18:07 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Fri, 17 Jun 2005 18:18:07 -0700
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <42B3384C.7080908@emesgarten.de>
References: <ca471dc2050616200360b5276@mail.gmail.com>
	<006e01c572f7$1ec8eca0$bb07a044@oemcomputer>
	<ca471dc2050616234366fb33c2@mail.gmail.com>
	<42B3384C.7080908@emesgarten.de>
Message-ID: <1119057487.15054.196.camel@warna.corp.google.com>

On Fri, 2005-06-17 at 13:53, Joachim Koenig-Baltes wrote:
[...]
> My use case for this is a directory tree walking generator that
> yields all the files including the directories in a depth first manner.
> If a directory satisfies a condition (determined by the caller) the
> generator shall not descend into it.
> 
> Something like:
> 
> DONOTDESCEND=1
> for path in mywalk("/usr/src"):
>     if os.path.isdir(path) and os.path.basename(path) == "CVS":
>         continue DONOTDESCEND
>     # do something with path
> 
> Of course there are different solutions to this problem with callbacks
> or filters but i like this one as the most elegant.

I have implemented almost exactly this use-case using the standard
Python generators, and shudder at the complexity something like this
would introduce.

For me, the right solution would be to either write your own generator
that "wraps" the other generator and filters it, or just make the
generator with additional (default value) parameters that support the
DONOTDECEND filtering.

FWIW, my usecase is a directory comparison generator that walks two
directorys producing tuples of corresponding files. It optionally will
not decend directories in either tree that do not have a corresponding
directory in the other tree. See;

http://minkirri.apana.org.au/~abo/projects/utils/

-- 
Donovan Baarda <abo at minkirri.apana.org.au>


From python at rcn.com  Sat Jun 18 06:58:06 2005
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 18 Jun 2005 00:58:06 -0400
Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342
In-Reply-To: <1119057487.15054.196.camel@warna.corp.google.com>
Message-ID: <003801c573c2$5145b660$bb07a044@oemcomputer>

[Joachim Koenig-Baltes]
> > My use case for this is a directory tree walking generator that
> > yields all the files including the directories in a depth first
manner.
> > If a directory satisfies a condition (determined by the caller) the
> > generator shall not descend into it.
> >
> > Something like:
> >
> > DONOTDESCEND=1
> > for path in mywalk("/usr/src"):
> >     if os.path.isdir(path) and os.path.basename(path) == "CVS":
> >         continue DONOTDESCEND
> >     # do something with path
> >
> > Of course there are different solutions to this problem with
callbacks
> > or filters but i like this one as the most elegant.


[Donovan Baarda]
> I have implemented almost exactly this use-case using the standard
> Python generators, and shudder at the complexity something like this
> would introduce.
> 
> For me, the right solution would be to either write your own generator
> that "wraps" the other generator and filters it, or just make the
> generator with additional (default value) parameters that support the
> DONOTDECEND filtering.
> 
> FWIW, my usecase is a directory comparison generator that walks two
> directorys producing tuples of corresponding files. It optionally will
> not decend directories in either tree that do not have a corresponding
> directory in the other tree. See;
> 
> http://minkirri.apana.org.au/~abo/projects/utils/


Thank both of you for the excellent posts.  This is exactly kind of
feedback and analysis that will show whether continue EXPR is worth it.


Raymond

From simonwittber at gmail.com  Sat Jun 18 08:19:34 2005
From: simonwittber at gmail.com (Simon Wittber)
Date: Sat, 18 Jun 2005 14:19:34 +0800
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
Message-ID: <4e4a11f805061723192888915e@mail.gmail.com>

Hello Chaps,

The attached PEP (pep.txt) is for RFE 46738, which you can view here:

http://sourceforge.net/tracker/index.php?func=detail&aid=467384&group_id=5470&atid=355470

It provides a safe, documented class for serialization of simple python types.

A sample implementation is also attached (gherkin.py).

Critcism and comments on the PEP and the implementation are appreciated.

Simon Wittber.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pep.txt
Url: http://mail.python.org/pipermail/python-dev/attachments/20050618/b5fa6bf6/pep.txt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gherkin.py
Type: text/x-python
Size: 7225 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20050618/b5fa6bf6/gherkin.py

From kay.schluehr at gmx.net  Sat Jun 18 09:16:59 2005
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sat, 18 Jun 2005 09:16:59 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <42B35DC8.6040308@gmail.com>
References: <000001c57386$75c23180$bb07a044@oemcomputer>	<ca471dc20506171459b54789d@mail.gmail.com>
	<42B35DC8.6040308@gmail.com>
Message-ID: <d90hes$17d$1@sea.gmane.org>

Nick Coghlan wrote:
> Guido van Rossum wrote:
> 
>>>Recommend accepting just the basic PEP which only targets simple,
>>>obvious cases.  The discussed extensions are unattractive and should be
>>>skipped.
>>
>>
>>-1. The "unary colon" looks unPythonic to me.
>>
> 
> 
> Step 1 would be to require parentheses around the whole thing (ala 
> generator expressions) to make it easier to see where the deferred 
> expression ends.
> 
> But all my use cases that I can think off the top of my head involve 
> 'sorted', where it wouldn't help at all because of the need for an 
> argument.
> 
> So I'd rather see a serious discussion regarding giving lambdas a more 
> Pythonic syntax in general, rather than one that only applied to the 
> 'no-argument' case [1]
> 
> Cheers,
> Nick.
> 
> [1] http://wiki.python.org/moin/AlternateLambdaSyntax
> The 'expression-before-args' version using just the 'from' keyword is 
> still my favourite.
> 

Maybe anonymus function closures should be pushed forward right now not 
only syntactically? Personally I could live with lambda or several
of the alternative syntaxes listed on the wiki page.

But asking for a favourite syntax I would skip the "def" keyword from 
your def-arrow syntax proposal and use:

    ((a, b, c) -> f(a) + o(b) - o(c))
    ((x) -> x * x)
    (() -> x)
    ((*a, **k) -> x.bar(*a, **k))
    ( ((x=x, a=a, k=k) -> x(*a, **k)) for x, a, k in funcs_and_args_list)

The arrow is a straightforward punctuation for function definitions. 
Reusing existing keywords for different semantics seems to me as a kind 
of inbreeding.

For pushing anymus functions forward I propose to enable explizit 
partial evaluation as a programming technique:

Example 1:

 >>> ((x,y) -> (x+1)*y**2)
((x,y) -> (x+1)*y**2)

 >>> ((x,y) -> (x+1)*y**2)(x=5)
((y) -> 6*y**2)


Example 2:

def f(x):
     return x**2

 >>> ((x,y) -> f(x)+f(y))(x=2)
((y) -> 4 + f(y))


Example 3:

 >>> ((f,x,y) -> f(x)+f(y))(f=((x)-> x**2), y=3)
((x) -> ((x)-> x**2))(x)+9)

The keyword style argument passing can be omitted in case of complete
evaluation where pattern matching on the argument tuple is applied.

Regards,
Kay












From jcarlson at uci.edu  Sat Jun 18 10:34:30 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat, 18 Jun 2005 01:34:30 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <d90hes$17d$1@sea.gmane.org>
References: <42B35DC8.6040308@gmail.com> <d90hes$17d$1@sea.gmane.org>
Message-ID: <20050618011502.728D.JCARLSON@uci.edu>


Kay Schluehr <kay.schluehr at gmx.net> wrote:
> Maybe anonymus function closures should be pushed forward right now not 
> only syntactically? Personally I could live with lambda or several
> of the alternative syntaxes listed on the wiki page.

> But asking for a favourite syntax I would skip the "def" keyword from 
> your def-arrow syntax proposal and use:
> 
>     ((a, b, c) -> f(a) + o(b) - o(c))
...

> The arrow is a straightforward punctuation for function definitions. 
> Reusing existing keywords for different semantics seems to me as a kind 
> of inbreeding.

That's starting to look like the pseudocode from old algorithms
textbooks, which is very similar to bad pseudocode from modern CS theory
papers.  Punctuation as a replacement for words does not always win
(perfect examples being 'and' vs. &&, 'or' vs. ||, 'not' vs. !, ...)

-1 on the syntax offering.

> For pushing anymus functions forward I propose to enable explizit 
> partial evaluation as a programming technique:

If I remember correctly, we've got rightcurry and leftcurry for that (or
rightpartial and leftpartial, or something).

>  >>> ((x,y) -> (x+1)*y**2)
> ((x,y) -> (x+1)*y**2)
> 
>  >>> ((x,y) -> (x+1)*y**2)(x=5)
> ((y) -> 6*y**2)

I'll assume that you don't actually want it to rewrite the source, or
actually return the source representation of the anonymous function
(those are almost non-starters).

As for all anonymous functions allowing partial evaluation via keywords:
it would hide errors.  Right now, if you forget an argument or add too
many arguments, you get a TypeError.  Your proposal would make
forgetting an argument in certain ways return a partially evaluated
function.

-1 on partial evaluation.


 - Josiah


From raymond.hettinger at verizon.net  Sat Jun 18 10:45:32 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat, 18 Jun 2005 04:45:32 -0400
Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops
Message-ID: <000401c573e2$16cd8ce0$bb07a044@oemcomputer>

I recommend that the proposed syntax be altered to be more parallel with
the existing for-loop syntax to make it more parsable for both humans
and for the compiler.  Like existing for-statements, the target
expression should immediately follow the 'for' keyword.  Since this is
known to be a range assignment, only an expression is needed, not a full
expression list.  Immediately following should be a token to distinguish
the new and old syntaxes.  Putting that distinction early in the
statement prepares readers (who scan left-to-right) for what follows.

IOW, augment the existing syntax:

  for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]

with an alternative syntax in the form:

  for_stmt: 'for' expr 'tok' rangespec ':' suite ['else' ':' suite]

Given that form, the PEP authors can choose the best options for 'tok'.
Basically, anything will do as long as it is not 'in'.  Likewise, they
can choose any rangespec format.  Within that framework, there are many
possibilities:

    for i between 2 < i <= 10: ...
    for i over 2 < i <= 10: ...     # chained comparison style
    for i over [2:11]: ...          # Slice style
    for i = 3 to 10:  ...           # Basic style

The rangespecs with comparison operators offer more flexibility in terms
of open/closed intervals.  In contrast, the slice notation version and
the Basic versions can admit a step argument.

The key changes are stating the target variable first and then using a
specific token to distinguish between the two forms.

Also, I recommend tightening the PEP's motivation.  There are only two
advantages, encoding and readability.  The former is only a minor gain
because all it saves is a function call, an O(1) savings in an O(n)
context.  The latter is where the real benefits lay.

The PEP authors should also put to rest the issues section:

* Decide once and for all on the simplest approach of immediately
evaluating the whole rangespec prior to execution. This best parallels
the range() approach and it is the least surprising.

* Note that new proposal works equally well with list comps and genexps.

* Decide to always return an iterator rather than a list as there is
never an advantage to doing otherwise.

* If you go for a chained comparison style rangespec, then drop the
issue of a step argument.  If the slice or Basic style rangespecs are
chosen, then there is no reason not to allow a step argument.

* Competition with PEP 276 is no longer an issue.

* Simply disallow floating point bounds.  We've already got deprecation
warnings in place for Py2.5.  There is no need to exacerbate the
problem.  Alternately, simplify the issue by declaring that the values
will be handled as if by xrange().

The above recommendations should get the PEP ready for judgement day.
Good luck.



Raymond


From kdart at kdart.com  Sat Jun 18 11:06:34 2005
From: kdart at kdart.com (Keith Dart)
Date: Sat, 18 Jun 2005 02:06:34 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers
	and	Integers
In-Reply-To: <ca471dc20506171533147714a8@mail.gmail.com>
References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer>
	<ca471dc20506171533147714a8@mail.gmail.com>
Message-ID: <42B3E41A.2020608@kdart.com>

Guido van Rossum wrote:

>On 6/17/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
>  
>
>>IIRC, there was a decision to not implement phase C and to keep the
>>trailing L in representations of long integers.
>>    
>>
>
>Actually, the PEP says phase C will be implemented in Python 3.0 and
>that's still my plan.
>
>  
>
>>If so, I believe the PEP can be marked as final.  We've done all we're
>>going to do.
>>    
>>
>
>For 2.x, yes. I'm fine with marking it as Final and adding this to PEP
>3000 instead.
>
>  
>
I am very concernced about something. The following code breaks with 2.4.1:

fcntl.ioctl(self.rtc_fd, RTC_RD_TIME, ...)

Where RTC_RD_TIME = 2149871625L

In Python 2.3 it is -2145095671.

Actually, this is supposed to be an unsigned int, and it was construced
with hex values and shifts.

Now, with the integer unification, how is ioctl() supposed to work? I
cannot figure out how to make it work in this case.

I suppose the best thing is to introduce an "unsignedint" type for this
purpose. As it is right now, I cannot use 2.4 at all.



-- 

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   Keith Dart <kdart at kdart.com>
   public key: ID: F3D288E4
   =====================================================================

-------------- next part --------------
A non-text attachment was scrubbed...
Name: kdart.vcf
Type: text/x-vcard
Size: 179 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20050618/ec966cbe/kdart.vcf

From ncoghlan at gmail.com  Sat Jun 18 12:37:41 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jun 2005 20:37:41 +1000
Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops
In-Reply-To: <000401c573e2$16cd8ce0$bb07a044@oemcomputer>
References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer>
Message-ID: <42B3F975.5010601@gmail.com>

Raymond Hettinger wrote:
> Also, I recommend tightening the PEP's motivation.  There are only two
> advantages, encoding and readability.  The former is only a minor gain
> because all it saves is a function call, an O(1) savings in an O(n)
> context.  The latter is where the real benefits lay.

The introduction of 'enumerate', and the proliferation of better 
iterators that reduce the need for pure-integer loops should be 
adressed in the revised motivation. I know my use of 'range' drops 
close to zero when I'm working with Python versions that supply 
'enumerate'.

Even when I do have a pure integer loop, I'll often assign the result 
of the range/xrange call to a local variable, just so I can give it a 
name that indicates the *significance* of that particular bunch of 
numbers.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From kay.schluehr at gmx.net  Sat Jun 18 13:43:27 2005
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sat, 18 Jun 2005 13:43:27 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <20050618011502.728D.JCARLSON@uci.edu>
References: <42B35DC8.6040308@gmail.com> <d90hes$17d$1@sea.gmane.org>
	<20050618011502.728D.JCARLSON@uci.edu>
Message-ID: <d9112d$vm2$1@sea.gmane.org>

Josiah Carlson wrote:

 > Kay Schluehr <kay.schluehr at gmx.net> wrote:
 >
 >
 >> Maybe anonymus function closures should be pushed forward right now 
not only syntactically? Personally I could live with lambda or several
 >> of the alternative syntaxes listed on the wiki page.
 >>
 >
 >
 >
 >
 >> But asking for a favourite syntax I would skip the "def" keyword 
from your def-arrow syntax proposal and use:
 >>
 >>    ((a, b, c) -> f(a) + o(b) - o(c))
 >>
 >
 > ...
 >
 >
 >
 >> The arrow is a straightforward punctuation for function definitions. 
Reusing existing keywords for different semantics seems to me as a kind 
of inbreeding.
 >>
 >
 >
 > That's starting to look like the pseudocode from old algorithms
 > textbooks, which is very similar to bad pseudocode from modern CS theory
 > papers.  Punctuation as a replacement for words does not always win
 > (perfect examples being 'and' vs. &&, 'or' vs. ||, 'not' vs. !, ...)
 >
 >

Writing functions as arrows is very convenient not only in CS but also 
in mathematics. Looking like pseudo-code was not one of Guidos Python 
regrets if I remember them correctly.

 > -1 on the syntax offering.
 >
 >
 >
 >> For pushing anymus functions forward I propose to enable explizit 
partial evaluation as a programming technique:
 >>
 >
 >
 > If I remember correctly, we've got rightcurry and leftcurry for that (or
 > rightpartial and leftpartial, or something).
 >
 >

Currying usually does not perform a function evaluation in order to 
create another more special function. Partial evaluation is a dynamic 
programming and optimization technique. Psyco uses specialization and
caching implicitely. I propose to use it explicitely but in a more 
restricted context.

 >> >>> ((x,y) -> (x+1)*y**2)
 >> ((x,y) -> (x+1)*y**2)
 >>
 >> >>> ((x,y) -> (x+1)*y**2)(x=5)
 >> ((y) -> 6*y**2)
 >>
 >
 >
 > I'll assume that you don't actually want it to rewrite the source, or
 > actually return the source representation of the anonymous function
 > (those are almost non-starters).
 >
 >

Computer algebra systems store expressions in internal tree form, 
manipulate them efficiently and pretty-print them in textual or latex 
output on demand. There would be much more involved than a tree to tree 
translation starting with Pythons internal parse tree and an evaluator 
dedicated to it.

 > As for all anonymous functions allowing partial evaluation via keywords:
 > it would hide errors. Right now, if you forget an argument or add too
 > many arguments, you get a TypeError. Your proposal would make
 > forgetting an argument in certain ways return a partially evaluated
 > function.


That's why I like to dump the function in a transparent mode. Personally 
I could dispense a little security in favor for cheap metainformation.

Regards,
Kay





From mwh at python.net  Sat Jun 18 14:50:57 2005
From: mwh at python.net (Michael Hudson)
Date: Sat, 18 Jun 2005 13:50:57 +0100
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <42B341E3.5000609@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Fri,
	17 Jun 2005 23:34:27 +0200")
References: <2mzmtrbxwo.fsf@starship.python.net>
	<42B0725D.9040200@v.loewis.de> <2m4qbybpf2.fsf@starship.python.net>
	<42B341E3.5000609@v.loewis.de>
Message-ID: <2macln7rri.fsf@starship.python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Michael Hudson wrote:
>> I've been looking at this area partly to try and understand this bug:
>> 
>>     [ 1163563 ] Sub threads execute in restricted mode
>> 
>> but I'm not sure the whole idea of multiple interpreters isn't
>> inherently doomed :-/
>
> That's what Tim asserts, saying that people who want to use the
> feature should fix it themselves.

Well, they've tried, and I think I've worked out a proper fix (life
would be easier if people didn't check in borken code :).

Cheers,
mwh

-- 
  Premature optimization is the root of all evil.
       -- Donald E. Knuth, Structured Programming with goto Statements

From mwh at python.net  Sat Jun 18 14:56:53 2005
From: mwh at python.net (Michael Hudson)
Date: Sat, 18 Jun 2005 13:56:53 +0100
Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops
In-Reply-To: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> (Raymond
	Hettinger's message of "Sat, 18 Jun 2005 04:45:32 -0400")
References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer>
Message-ID: <2m64wb7rhm.fsf@starship.python.net>

"Raymond Hettinger" <raymond.hettinger at verizon.net> writes:

> I recommend that the proposed syntax be altered to be more parallel
> with the existing for-loop syntax to make it more parsable for both
> humans and for the compiler.

Although all your suggestions are improvments, I'm still -1 on the PEP.

Cheers,
mwh

-- 
  Windows installation day one.  Getting rid of the old windows 
  was easy - they fell apart quite happily, and certainly wont 
  be re-installable anywhere else.   -- http://www.linux.org.uk/diary/
                                       (not *that* sort of windows...)

From mwh at python.net  Sat Jun 18 15:10:57 2005
From: mwh at python.net (Michael Hudson)
Date: Sat, 18 Jun 2005 14:10:57 +0100
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers
	and	Integers
In-Reply-To: <42B3E41A.2020608@kdart.com> (Keith Dart's message of "Sat, 18
	Jun 2005 02:06:34 -0700")
References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer>
	<ca471dc20506171533147714a8@mail.gmail.com>
	<42B3E41A.2020608@kdart.com>
Message-ID: <2m1x6z7qu6.fsf@starship.python.net>

Keith Dart <kdart at kdart.com> writes:

> I am very concernced about something. The following code breaks with 2.4.1:
>
> fcntl.ioctl(self.rtc_fd, RTC_RD_TIME, ...)
>
> Where RTC_RD_TIME = 2149871625L
>
> In Python 2.3 it is -2145095671.

Well, you could always use "-2145095671"...

> Actually, this is supposed to be an unsigned int, and it was construced
> with hex values and shifts.

But well, quite.

> Now, with the integer unification, how is ioctl() supposed to work? I
> cannot figure out how to make it work in this case.

The shortest way I know of going from 2149871625L to -2145095671 is
the still-fairly-gross:

>>> v = 2149871625L
>>> ~int(~v&0xFFFFFFFF)
-2145095671

> I suppose the best thing is to introduce an "unsignedint" type for this
> purpose. 

Or some kind of bitfield type, maybe.

C uses integers both as bitfields and to count things, and at least in
my opinion the default assumption in Python should be that this is
what an integer is being used for, but when you need a bitfield it can
all get a bit horrible.

That said, I think in this case we can just make fcntl_ioctl use the
(new-ish) 'I' format argument to PyArg_ParseTuple and then you'll just
be able to use 2149871625L and be happy (I think, haven't tried this).

> As it is right now, I cannot use 2.4 at all.

/Slightly/ odd place to mae this report!  Hope this mail helped.

Cheers,
mwh

-- 
  I'm okay with intellegent buildings, I'm okay with non-sentient
  buildings. I have serious reservations about stupid buildings.
     -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year)

From oren.tirosh at gmail.com  Sat Jun 18 15:44:02 2005
From: oren.tirosh at gmail.com (Oren Tirosh)
Date: Sat, 18 Jun 2005 16:44:02 +0300
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <4e4a11f805061723192888915e@mail.gmail.com>
References: <4e4a11f805061723192888915e@mail.gmail.com>
Message-ID: <7168d65a0506180644367753b9@mail.gmail.com>

Please don't invent new serialization formats. I think we have enough
of those already.

The RFE suggests that "the protocol is specified in the documentation,
precisely enough to write interoperating implementations in other
languages". If interoperability with other languages is really the
issue, use an existing format like JSON.

If you want an efficient binary format you can use a subset of the
pickle protocol supporting only basic types. I tried this once. I
ripped out all the fancy parts from pickle.py and left only binary
pickling (protocol version 2) of basic types. It took less than hour
and I was left with something only marginally more complex than your
new proposed protocol.

  Oren

From simonwittber at gmail.com  Sat Jun 18 16:12:16 2005
From: simonwittber at gmail.com (Simon Wittber)
Date: Sat, 18 Jun 2005 22:12:16 +0800
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <7168d65a0506180644367753b9@mail.gmail.com>
References: <4e4a11f805061723192888915e@mail.gmail.com>
	<7168d65a0506180644367753b9@mail.gmail.com>
Message-ID: <4e4a11f805061807127f39700b@mail.gmail.com>

> The RFE suggests that "the protocol is specified in the documentation,
> precisely enough to write interoperating implementations in other
> languages". If interoperability with other languages is really the
> issue, use an existing format like JSON.

JSON is slow (this is true of the python version, at least). Whether
it is slow because of the implementation, or because of its textual
nature, I do not know. The implementation I tested also failed to
encode {1:2}. I am not sure if this is a problem with JSON or the
implementation.

> If you want an efficient binary format you can use a subset of the
> pickle protocol supporting only basic types. I tried this once. I
> ripped out all the fancy parts from pickle.py and left only binary
> pickling (protocol version 2) of basic types. It took less than hour
> and I was left with something only marginally more complex than your
> new proposed protocol.

I think you are missing the point. Is your pickle hack available for
viewing? If it, or JSON is a better choice, then so be it. The point
of the PEP is not the protocol, but the need for a documented,
efficient, _safe_ serializion module in the standard library.

Do you disagree?


Simon Wittber.

From pje at telecommunity.com  Sat Jun 18 16:55:19 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 18 Jun 2005 10:55:19 -0400
Subject: [Python-Dev] Implementing PEP 342 (was Re: Withdrawn PEP 288
 and thoughts on PEP 342)
In-Reply-To: <7168d65a05061806553bad04b3@mail.gmail.com>
References: <5.1.1.6.0.20050617155758.035955d8@mail.telecommunity.com>
	<5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com>
	<005301c572e3$f2518aa0$bb07a044@oemcomputer>
	<ca471dc2050616200360b5276@mail.gmail.com>
	<5.1.1.6.0.20050617155758.035955d8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050618104339.02bbf650@mail.telecommunity.com>

At 04:55 PM 6/18/2005 +0300, Oren Tirosh wrote:
>On 6/18/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote:
> > It turns out that making 'next(EXPR)' work is a bit tricky; I was going to
> > use METH_COEXIST and METH_VARARGS, but then it occurred to me that
> > METH_VARARGS adds overhead to normal Python calls to 'next()', so I
> > implemented a separate 'send(EXPR)' method instead, and left 'next()' a
> > no-argument call.
>
>Please use the name "feed", not "send". That would make the enhanced
>generators already compatible "out of the box" with existing code
>expecting the de-facto consumer interface (see
>http://effbot.org/zone/consumer.htm).  The other part of the consumer
>interface (the close() method) is already being added in PEP 343.

Hm.  Do you want reset() as well?  :)

More seriously, I'm not sure that PEP 343's close() does something 
desirable for the consumer interface.  Overall, this sounds like something 
that should be added to the PEPs though.  I hadn't really thought of using 
inbound generator communication for parsing; it's an interesting use case.

However, looking more closely at the consumer interface, it seems to me the 
desired semantics for feed() are different than for send(), because of the 
"just-started generator can't receive data" problem.  Also, the consumer 
interface doesn't include handling for StopIteration.

Maybe feed() should prime the generator if it's just started, and throw 
away the yield result as long as it's None?  Maybe it should ignore 
StopIteration?  Perhaps it should raise an error if the generator yields 
anything but None in response?  These seem like questions worth discussing.




From gerrit at nl.linux.org  Sat Jun 18 17:59:07 2005
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Sat, 18 Jun 2005 17:59:07 +0200
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <000501c56fbe$79998220$4ff9a244@oemcomputer>
References: <17068.58083.342369.872731@montanaro.dyndns.org>
	<000501c56fbe$79998220$4ff9a244@oemcomputer>
Message-ID: <20050618155907.GA8136@topjaklont.student.utwente.nl>

Hi,

Raymond Hettinger wrote:
> I think it unwise to allow x to be any expression.  Besides altering
> existing semantics, it leads to code redundancy and to a fragile
> construct (where the slightest alteration of any of the expressions
> triggers a silent reversion to O(n) behavior).

What would happen if 'x' were to be an object whose class has a __eq__
that is defined in an odd way, e.g. having side effects?
Might this mean behaviour change even if 'x' is a local variable?

yours,
Gerrit Holl.

-- 
Weather in Twenthe, Netherlands 18/06 17:25:
	24.0?C   wind 3.1 m/s NE (57 m above NAP)
-- 
In the councils of government, we must guard against the acquisition of
unwarranted influence, whether sought or unsought, by the
military-industrial complex. The potential for the disastrous rise of
misplaced power exists and will persist.
    -Dwight David Eisenhower, January 17, 1961

From python at rcn.com  Sat Jun 18 18:09:27 2005
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 18 Jun 2005 12:09:27 -0400
Subject: [Python-Dev] Multiple expression eval in compound if statement?
In-Reply-To: <20050618155907.GA8136@topjaklont.student.utwente.nl>
Message-ID: <000a01c57420$1b234940$0636c797@oemcomputer>

[Raymond Hettinger]
> > I think it unwise to allow x to be any expression.  Besides altering
> > existing semantics, it leads to code redundancy and to a fragile
> > construct (where the slightest alteration of any of the expressions
> > triggers a silent reversion to O(n) behavior).

[Gerrit Holl]
> What would happen if 'x' were to be an object whose class has a __eq__
> that is defined in an odd way, e.g. having side effects?

Every molecule in your body would simultaneously implode at the speed of
light.


Raymond

From skip at pobox.com  Sat Jun 18 19:02:45 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sat, 18 Jun 2005 12:02:45 -0500
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <4e4a11f805061807127f39700b@mail.gmail.com>
References: <4e4a11f805061723192888915e@mail.gmail.com>
	<7168d65a0506180644367753b9@mail.gmail.com>
	<4e4a11f805061807127f39700b@mail.gmail.com>
Message-ID: <17076.21429.501499.914085@montanaro.dyndns.org>


Why this discussion of yet another serialization format?  The wire-encoding
for XML-RPC is quite stable, handles all the basic Python types proposed in
the proto-PEP, and is highly interoperable.  If performance is an issue,
make sure you have a C-based accelerator module like sgmlop installed.  If
size is an issue, gzip it before sending it over the wire or to a file.

Skip

From gvanrossum at gmail.com  Sat Jun 18 21:54:00 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sat, 18 Jun 2005 12:54:00 -0700
Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops
In-Reply-To: <2m64wb7rhm.fsf@starship.python.net>
References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer>
	<2m64wb7rhm.fsf@starship.python.net>
Message-ID: <ca471dc205061812542175c3da@mail.gmail.com>

On 6/18/05, Michael Hudson <mwh at python.net> wrote:
> "Raymond Hettinger" <raymond.hettinger at verizon.net> writes:
> 
> > I recommend that the proposed syntax be altered to be more parallel
> > with the existing for-loop syntax to make it more parsable for both
> > humans and for the compiler.
> 
> Although all your suggestions are improvments, I'm still -1 on the PEP.

Same here. The whole point (15 years ago) of range() was to *avoid*
needing syntax to specify a loop over numbers. I think it's worked out
well and there's nothing that needs to be fixed (except range() needs
to become an interator, which it will in Python 3.0).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Sat Jun 18 22:34:20 2005
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 18 Jun 2005 16:34:20 -0400
Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops
In-Reply-To: <ca471dc205061812542175c3da@mail.gmail.com>
Message-ID: <001901c57445$1b4b1cc0$0636c797@oemcomputer>

 [Raymond Hettinger]
> > > I recommend that the proposed syntax be altered to be more
parallel
> > > with the existing for-loop syntax to make it more parsable for
both
> > > humans and for the compiler.

[Michael Hudson]
> > Although all your suggestions are improvments, I'm still -1 on the
PEP.

[Guido]
> Same here. The whole point (15 years ago) of range() was to *avoid*
> needing syntax to specify a loop over numbers. I think it's worked out
> well and there's nothing that needs to be fixed (except range() needs
> to become an interator, which it will in Python 3.0).

I concur.

Saying that no form of the idea is viable will save the PEP authors from
another round or two of improvements.

Marking as rejected and noting why.



Raymond

From gustavo at niemeyer.net  Sat Jun 18 22:39:42 2005
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Sat, 18 Jun 2005 17:39:42 -0300
Subject: [Python-Dev] PyPI: no space left on device
Message-ID: <20050618203942.GA15154@burma.localdomain>

PyPI seems to be out of space:

% ./setup.py register --show-response
running register
Using PyPI login from /home/niemeyer/.pypirc
---------------------------------------------------------------------------
Error...

There's been a problem with your request

psycopg.ProgrammingError: ERROR:  could not extend relation "releases":
No space left on device
HINT:  Check free disk space.

-- 
Gustavo Niemeyer
http://niemeyer.net

From aahz at pythoncraft.com  Sat Jun 18 23:09:01 2005
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 18 Jun 2005 14:09:01 -0700
Subject: [Python-Dev] PyPI: no space left on device
In-Reply-To: <20050618203942.GA15154@burma.localdomain>
References: <20050618203942.GA15154@burma.localdomain>
Message-ID: <20050618210901.GB848@panix.com>

On Sat, Jun 18, 2005, Gustavo Niemeyer wrote:
>
> PyPI seems to be out of space:

FYI, python-dev is not a good place to send messages like this.  Please
use webmaster at python.org.  (I've already notified the appropriate
parties.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From gustavo at niemeyer.net  Sat Jun 18 23:20:21 2005
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Sat, 18 Jun 2005 18:20:21 -0300
Subject: [Python-Dev] PyPI: no space left on device
In-Reply-To: <20050618210901.GB848@panix.com>
References: <20050618203942.GA15154@burma.localdomain>
	<20050618210901.GB848@panix.com>
Message-ID: <20050618212021.GA23867@burma.localdomain>

> > PyPI seems to be out of space:
> 
> FYI, python-dev is not a good place to send messages like this.  Please
> use webmaster at python.org.  (I've already notified the appropriate
> parties.)

Before sending the message I thought, "let's see how much time it
takes until someone mentions the right place to deliver the message".
Adding that address to the PyPI page itself would be valueable, and
will probably save python-dev from further misinformed reporters.

Thanks for forwarding it this time,

-- 
Gustavo Niemeyer
http://niemeyer.net

From pje at telecommunity.com  Sat Jun 18 23:50:36 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 18 Jun 2005 17:50:36 -0400
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects
Message-ID: <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>

Working on the PEP 342/343 generator enhancements, I've got working 
send/throw/close() methods, but am not sure how to deal with getting 
__del__ to invoke close().  Naturally, I can add a "__del__" entry to its 
methods list easily enough, but the 'has_finalizer()' function in 
gcmodule.c only checks for a __del__ attribute on instance objects, and for 
tp_del on heap types.

It looks to me like the correct fix would be to check for tp_del always, 
not just on heap types.  However, when I tried this, I started getting 
warnings from the tests, saying that 22 uncollectable objects were being 
created (all generators, in test_generators).

It seems that the tests create cycles via globals(), since they define a 
bunch of generator functions and then call them, saving the generator 
iterators (or objects that reference them) in global variables

after investigating this a bit, it seems to me that either has_finalizer() 
needs to 


From pje at telecommunity.com  Sat Jun 18 23:56:54 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 18 Jun 2005 17:56:54 -0400
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator
 objects
In-Reply-To: <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com>

At 05:50 PM 6/18/2005 -0400, Phillip J. Eby wrote:
>Working on the PEP 342/343 generator enhancements, I've got working
>send/throw/close() methods, but am not sure how to deal with getting
>__del__ to invoke close().  Naturally, I can add a "__del__" entry to its
>methods list easily enough, but the 'has_finalizer()' function in
>gcmodule.c only checks for a __del__ attribute on instance objects, and for
>tp_del on heap types.
>
>It looks to me like the correct fix would be to check for tp_del always,
>not just on heap types.  However, when I tried this, I started getting
>warnings from the tests, saying that 22 uncollectable objects were being
>created (all generators, in test_generators).
>
>It seems that the tests create cycles via globals(), since they define a
>bunch of generator functions and then call them, saving the generator
>iterators (or objects that reference them) in global variables
>
>after investigating this a bit, it seems to me that either has_finalizer()
>needs to

Whoops.  I hit send by accident.  Anyway, the issue seems to mostly be that 
the tests create generator-iterators in global variables.  With a bit of 
effort, I've been able to stomp most of the cycles.


>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com


From pje at telecommunity.com  Sun Jun 19 00:00:46 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 18 Jun 2005 18:00:46 -0400
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator
 objects
In-Reply-To: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com>
References: <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050618175716.01e413a8@mail.telecommunity.com>

Argh!  My email client's shortcut for Send is Ctrl-E, which is the same as 
end-of-line in the editor I've been using all day.  Anyway, the problem is 
that it seems to me as though actually checking for tp_del is too 
aggressive (conservative?) for generators, because sometimes a generator 
object is finished or un-started, and therefore can't resurrect objects 
during close().  However, I don't really know how to implement another 
strategy; gcmodule isn't exactly my forte.  :)  Any input from the GC gurus 
would be appreciated.  Thanks!

At 05:56 PM 6/18/2005 -0400, Phillip J. Eby wrote:
>At 05:50 PM 6/18/2005 -0400, Phillip J. Eby wrote:
> >Working on the PEP 342/343 generator enhancements, I've got working
> >send/throw/close() methods, but am not sure how to deal with getting
> >__del__ to invoke close().  Naturally, I can add a "__del__" entry to its
> >methods list easily enough, but the 'has_finalizer()' function in
> >gcmodule.c only checks for a __del__ attribute on instance objects, and for
> >tp_del on heap types.
> >
> >It looks to me like the correct fix would be to check for tp_del always,
> >not just on heap types.  However, when I tried this, I started getting
> >warnings from the tests, saying that 22 uncollectable objects were being
> >created (all generators, in test_generators).
> >
> >It seems that the tests create cycles via globals(), since they define a
> >bunch of generator functions and then call them, saving the generator
> >iterators (or objects that reference them) in global variables
> >
> >after investigating this a bit, it seems to me that either has_finalizer()
> >needs to
>
>Whoops.  I hit send by accident.  Anyway, the issue seems to mostly be that
>the tests create generator-iterators in global variables.  With a bit of
>effort, I've been able to stomp most of the cycles.


From pje at telecommunity.com  Sun Jun 19 01:16:25 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 18 Jun 2005 19:16:25 -0400
Subject: [Python-Dev] Generator enhancements patch available
Message-ID: <5.1.1.6.0.20050618191302.03424200@mail.telecommunity.com>

I've just submitted patch 1223381 (http://python.org/sf/1223381), which 
implements code and test changes for:

* yield expressions
* bare yield (short for yield None)
* yield in try/finally
* generator.send(value) (send value into generator; substituted for PEP 
342's next(arg))
* generator.throw(typ[,val[,tb]]) (raise error in generator)
* generator.close()
* GeneratorExit built-in exception type
* generator.__del__ (well, the C equivalent)
* All necessary mods to the compiler, parser module, and Python 'compiler' 
package to support these changes.

It was necessary to change a small part of the eval loop (well, the 
initialization, not the loop) and the gc module's has_finalizer() logic in 
order to support a C equivalent to __del__.  Specialists in these areas 
should probably scrutinize this patch!

There is one additional implementation detail that was not contemplated in 
either PEP. in order to prevent used-up generators from retaining 
unnecessary references to their frame's contents, I set the generator's 
gi_frame member to None whenever the generator finishes normally or with an 
error.  Thus, an exhausted generator cannot be part of a cycle, and it 
releases its frame object sooner than in previous Python versions.  For 
generators used only in a direct "for" loop, this makes no difference, but 
for generators used with the iterator protocol (i.e. "gen.next()") from 
Python, this avoids stranding the generator's frame in a traceback cycle.

Anyway, your comments/questions/feedback/bug reports are welcome.


From bjourne at gmail.com  Sun Jun 19 01:12:11 2005
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Sun, 19 Jun 2005 01:12:11 +0200
Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral
	Literals to Python
In-Reply-To: <ca471dc20506170748109ae6a4@mail.gmail.com>
References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>
	<ca471dc20506170748109ae6a4@mail.gmail.com>
Message-ID: <740c3aec05061816127acb2469@mail.gmail.com>

*cough*
Would it also be possible for the PEP-maintainers not to accept PEPs
that are obvious jokes unless thedate is April I?
*uncough*

-- 
mvh Bj?rn

From pje at telecommunity.com  Sun Jun 19 00:24:48 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 18 Jun 2005 18:24:48 -0400
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator
 objects
In-Reply-To: <5.1.1.6.0.20050618175716.01e413a8@mail.telecommunity.com>
References: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com>
	<5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>

One more note; I tried changing generators to set their gi_frame to None 
whenever the generator finishes normally or with an error; this eliminated 
most of the reference cycles, and I was able to make test_generators work 
correctly with only 3 explicit close() calls, for the "fun" tests that use 
objects which hold references to generators that in turn reference the object.

So, I think I've got this sorted out, assuming that I'm not doing something 
hideously insane by having 'has_finalizer()' always check tp_del even for 
non-heap types, and defining a tp_del slot for generators to call close() in.

I ended up having to copy a bunch of stuff from typeobject.c in order to 
make this work, as there doesn't appear to be any way to share stuff like 
subtype_del and subtype_dealloc in a meaningful way with the generator code.


At 06:00 PM 6/18/2005 -0400, Phillip J. Eby wrote:
>Argh!  My email client's shortcut for Send is Ctrl-E, which is the same as
>end-of-line in the editor I've been using all day.  Anyway, the problem is
>that it seems to me as though actually checking for tp_del is too
>aggressive (conservative?) for generators, because sometimes a generator
>object is finished or un-started, and therefore can't resurrect objects
>during close().  However, I don't really know how to implement another
>strategy; gcmodule isn't exactly my forte.  :)  Any input from the GC gurus
>would be appreciated.  Thanks!
>
>At 05:56 PM 6/18/2005 -0400, Phillip J. Eby wrote:
> >At 05:50 PM 6/18/2005 -0400, Phillip J. Eby wrote:
> > >Working on the PEP 342/343 generator enhancements, I've got working
> > >send/throw/close() methods, but am not sure how to deal with getting
> > >__del__ to invoke close().  Naturally, I can add a "__del__" entry to its
> > >methods list easily enough, but the 'has_finalizer()' function in
> > >gcmodule.c only checks for a __del__ attribute on instance objects, 
> and for
> > >tp_del on heap types.
> > >
> > >It looks to me like the correct fix would be to check for tp_del always,
> > >not just on heap types.  However, when I tried this, I started getting
> > >warnings from the tests, saying that 22 uncollectable objects were being
> > >created (all generators, in test_generators).
> > >
> > >It seems that the tests create cycles via globals(), since they define a
> > >bunch of generator functions and then call them, saving the generator
> > >iterators (or objects that reference them) in global variables
> > >
> > >after investigating this a bit, it seems to me that either has_finalizer()
> > >needs to
> >
> >Whoops.  I hit send by accident.  Anyway, the issue seems to mostly be that
> >the tests create generator-iterators in global variables.  With a bit of
> >effort, I've been able to stomp most of the cycles.
>
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com


From martin at v.loewis.de  Sun Jun 19 01:42:20 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 19 Jun 2005 01:42:20 +0200
Subject: [Python-Dev] PyPI: no space left on device
In-Reply-To: <20050618212021.GA23867@burma.localdomain>
References: <20050618203942.GA15154@burma.localdomain>	<20050618210901.GB848@panix.com>
	<20050618212021.GA23867@burma.localdomain>
Message-ID: <42B4B15C.1030206@v.loewis.de>

Gustavo Niemeyer wrote:
> Before sending the message I thought, "let's see how much time it
> takes until someone mentions the right place to deliver the message".
> Adding that address to the PyPI page itself would be valueable, and
> will probably save python-dev from further misinformed reporters.
> 
> Thanks for forwarding it this time,

Unfortunately, the "right place" depends on the nature of the problem:
could be a PyPI problem, could be a pydotorg problem, could be a
distutils problem.

As for "adding (that) address to the PyPI page itself": How did you
miss the "Get help" and "Bug reports" links below "Contact Us"
on the PyPI page? They would have brought you to the PyPI SF trackers,
but that would also have been the right place.

Regards,
Martin


From martin at v.loewis.de  Sun Jun 19 01:47:45 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 19 Jun 2005 01:47:45 +0200
Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral
 Literals to Python
In-Reply-To: <740c3aec05061816127acb2469@mail.gmail.com>
References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer>	<ca471dc20506170748109ae6a4@mail.gmail.com>
	<740c3aec05061816127acb2469@mail.gmail.com>
Message-ID: <42B4B2A1.5090903@v.loewis.de>

BJ?rn Lindqvist wrote:
> Would it also be possible for the PEP-maintainers not to accept PEPs
> that are obvious jokes unless thedate is April I?

I believe this is the current policy. Why do you think the PEP editor
works differently?

Regards,
Martin

From abo at minkirri.apana.org.au  Sun Jun 19 09:48:19 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Sun, 19 Jun 2005 00:48:19 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple
	Implicit	Lambda
In-Reply-To: <d9112d$vm2$1@sea.gmane.org>
References: <42B35DC8.6040308@gmail.com>
	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>
	<d9112d$vm2$1@sea.gmane.org>
Message-ID: <42B52343.3050902@minkirri.apana.org.au>

Kay Schluehr wrote:
> Josiah Carlson wrote:
> 
>  > Kay Schluehr <kay.schluehr at gmx.net> wrote:
>  >
>  >
>  >> Maybe anonymus function closures should be pushed forward right now 
> not only syntactically? Personally I could live with lambda or several
>  >> of the alternative syntaxes listed on the wiki page.

I must admit I ended up deleting most of the "alternative to lambda" 
threads after they flooded my in box. So it is with some dread I post 
this, contributing to it...

As I see it, a lambda is an anonymous function. An anonymous function is 
a function without a name. We already have a syntax for a function... 
why not use it. ie:

  f = filter(def (a): return a > 1, [1,2,3])

The implications of this are that both functions and procedures can be 
anonymous. This also implies that unlike lamba's, anonymous functions 
can have statements, not just expressions. You can even do compound 
stuff like;

   f = filter(def (a): b=a+1; return b>1, [1,2,3])

or if you want you can use indenting;

   f = filter(def (a):
     b=a+1
     return b>1, [1,2,3])

It also means the following becomes valid syntax;

f = def (a,b):
   return a>b

I'm not sure if there are syntactic ambiguities to this. I'm not sure if 
  the CS boffins are disturbed by "side effects" from statements. 
Perhaps both can be resolved by limiting annonymous functions to 
expressions. Or require use of brackets or ";" to resolve ambiguity.

This must have been proposed already and shot down in flames... sorry 
for re-visiting old stuff and contributing noise.

--
Donovan Baarda

From simonwittber at gmail.com  Sun Jun 19 02:49:02 2005
From: simonwittber at gmail.com (Simon Wittber)
Date: Sun, 19 Jun 2005 08:49:02 +0800
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <17076.21429.501499.914085@montanaro.dyndns.org>
References: <4e4a11f805061723192888915e@mail.gmail.com>
	<7168d65a0506180644367753b9@mail.gmail.com>
	<4e4a11f805061807127f39700b@mail.gmail.com>
	<17076.21429.501499.914085@montanaro.dyndns.org>
Message-ID: <4e4a11f805061817495428952d@mail.gmail.com>

> Why this discussion of yet another serialization format?

Pickle is stated to be unsafe. Marshal is also stated to be unsafe.
XML can be bloated, and XML+gzip is quite slow.

Do size,speed, and security features have to mutually exclusive? No,
that possibly is why people have had to invent their own formats. I
can list four off the top of my head:

bencode (bittorrent)
jelly (twisted)
banana (twisted)
tofu (soya3d, looks like it is using twisted now... hmmm)

XML is simply not suitable for database appplications, real time data
capture and game/entertainment applications.

I'm sure other people have noticed this... or am I alone on this issue? :-)

Have a look at this contrived example:

import time

value = (("this is a record",1,2,1000001,"(08)123123123","some more
text")*10000)

import gherkin
t = time.clock()
s = gherkin.dumps(value)
print 'Gherkin encode', time.clock() - t, 'seconds'
t = time.clock()
gherkin.loads(s)
print 'Gherkin decode', time.clock() - t, 'seconds'

import xmlrpclib
t = time.clock()
s = xmlrpclib.dumps(value)
print 'XMLRPC encode', time.clock() - t, 'seconds'
t = time.clock()
xmlrpclib.loads(s)
print 'XMLRPC decode', time.clock() - t, 'seconds'

Which produces the output:

>pythonw -u "bench.py"
Gherkin encode 0.120689361357 seconds
Gherkin decode 0.395871262968 seconds
XMLRPC encode 0.528666352847 seconds
XMLRPC decode 9.01307819849 seconds

From ncoghlan at gmail.com  Sun Jun 19 02:49:56 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 19 Jun 2005 10:49:56 +1000
Subject: [Python-Dev] Recommend accepting PEP 312 --
	Simple	Implicit	Lambda
In-Reply-To: <42B52343.3050902@minkirri.apana.org.au>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>
	<42B52343.3050902@minkirri.apana.org.au>
Message-ID: <42B4C134.8010308@gmail.com>

Donovan Baarda wrote:
> As I see it, a lambda is an anonymous function. An anonymous function is 
> a function without a name.

And here we see why I'm such a fan of the term 'deferred expression' 
instead of 'anonymous function'.

Python's lambda expressions *are* the former, but they are 
emphatically *not* the latter.

Anyway, the AlternateLambdaSyntax Wiki page has a couple of relevant 
entries under 'real closures'.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From nas at arctrix.com  Sun Jun 19 02:50:24 2005
From: nas at arctrix.com (Neil Schemenauer)
Date: Sat, 18 Jun 2005 18:50:24 -0600
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator
	objects
In-Reply-To: <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
References: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com>
	<5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>
	<5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
Message-ID: <20050619005024.GA14709@mems-exchange.org>

On Sat, Jun 18, 2005 at 06:24:48PM -0400, Phillip J. Eby wrote:
> So, I think I've got this sorted out, assuming that I'm not doing
> something hideously insane by having 'has_finalizer()' always
> check tp_del even for non-heap types, and defining a tp_del slot
> for generators to call close() in.

That sounds like the right thing to do.

I suspect the "uncollectable cycles" problem will not be completely
solvable.  With this change, all generators become objects with
finalizers.  In reality, a 'file' object, for example, has a
finalizer as well but it gets away without telling the GC that
because its finalizer doesn't do anything "evil".  Since generators
can do arbitrary things, the GC must assume the worst.

Most cycles involving enhanced generators can probably be broken by
the GC because the generator is not in the strongly connected part
of cycle.  The GC will have to work a little harder to figure that
out but that's probably not too significant.

The real problem is that some cycles involving enhanced generators
will not be breakable by the GC.  I think some programs that used to
work okay are now going to start leaking memory because objects will
accumulate in gc.garbage.

Now, I could be wrong about all this.  I've have not been following
the PEP 343 discussion too closely.  Maybe Guido has some clever
idea.  Also, I find it difficult to hold in my head a complete model
of how the GC now works.  It's an incredibly subtle piece of code.
Perhaps Tim can comment.

  Neil

From jcarlson at uci.edu  Sun Jun 19 02:51:19 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat, 18 Jun 2005 17:51:19 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <d9112d$vm2$1@sea.gmane.org>
References: <20050618011502.728D.JCARLSON@uci.edu> <d9112d$vm2$1@sea.gmane.org>
Message-ID: <20050618171515.72A5.JCARLSON@uci.edu>


Kay Schluehr <kay.schluehr at gmx.net> wrote:
> Josiah Carlson wrote:
> 
>  > Kay Schluehr <kay.schluehr at gmx.net> wrote:
>  >> The arrow is a straightforward punctuation for function definitions. 
>  >> Reusing existing keywords for different semantics seems to me as a kind 
>  >> of inbreeding.
>  >
>  > That's starting to look like the pseudocode from old algorithms
>  > textbooks, which is very similar to bad pseudocode from modern CS theory
>  > papers.  Punctuation as a replacement for words does not always win
>  > (perfect examples being 'and' vs. &&, 'or' vs. ||, 'not' vs. !, ...)
> 
> Writing functions as arrows is very convenient not only in CS but also 
> in mathematics. Looking like pseudo-code was not one of Guidos Python 
> regrets if I remember them correctly.

Apparently you weren't reading what I typed.  I don't find '->' notation
for functions to be readable, either in code, papers, or otherwise.  In
fact, when I teach my CS courses, I use Python, and in the case where I
need to describe mathematical functions, I use a standard mathematical
notation for doing so: 'f(x) = ...'.


>  >> For pushing anymus functions forward I propose to enable explizit 
>  >> partial evaluation as a programming technique:
>  >
>  > If I remember correctly, we've got rightcurry and leftcurry for that (or
>  > rightpartial and leftpartial, or something).
>  >
> Currying usually does not perform a function evaluation in order to 
> create another more special function. Partial evaluation is a dynamic 
> programming and optimization technique. Psyco uses specialization and
> caching implicitely. I propose to use it explicitely but in a more 
> restricted context.

I never claimed that currying was partial function evaluation.  In fact,
if I remember correctly (I don't use curried functions), the particular
currying that was implemented and shipped for Python 2.4 was a simple
function that kept a list of args and kwargs that were already passed to
the curried function. When it got enough arguments, it would go ahead
and evaluate it.


>  > I'll assume that you don't actually want it to rewrite the source, or
>  > actually return the source representation of the anonymous function
>  > (those are almost non-starters).
>  >
> 
> Computer algebra systems store expressions in internal tree form, 
> manipulate them efficiently and pretty-print them in textual or latex 
> output on demand. There would be much more involved than a tree to tree 
> translation starting with Pythons internal parse tree and an evaluator 
> dedicated to it.

Right, and Python is not a CAS.  You want a CAS?  Use Mathematica, Maple,
or some CAS addon to Python.  Keeping the parse tree around when it is
not needed is a waste. In the code that I write, that would be 100% of
the time.


>  > As for all anonymous functions allowing partial evaluation via keywords:
>  > it would hide errors. Right now, if you forget an argument or add too
>  > many arguments, you get a TypeError. Your proposal would make
>  > forgetting an argument in certain ways return a partially evaluated
>  > function.
> 
> 
> That's why I like to dump the function in a transparent mode. Personally 
> I could dispense a little security in favor for cheap metainformation.

It's not about security, it's about practicality and backwards
compatibility.

Here is an example:
>>> x = lambda a,b: a+b+1
>>> x(a=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: <lambda>() takes exactly 2 non-keyword arguments (1 given)
>>>

According to you, that should instead produce...
  lambda b: 1+b+1

New and old users alike have gotten used to the fact that calling a
function without complete arguments is an error.  You are proposing to
make calling a function with less than the required number of arguments
not be an error.

For the sake of sanity, I have to be -1000 on this.


I'm also not aware of an automatic mechanism for code compilation (in
Python) that both attaches the parse tree to the compiled function, or
allows for the conversion of that parse tree back to a source form, or
allows one to replace variable references with constants.  If you would
care to write the pieces necessary, I'm sure you will find a use for it.

 - Josiah


From jcarlson at uci.edu  Sun Jun 19 03:03:58 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat, 18 Jun 2005 18:03:58 -0700
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <4e4a11f805061817495428952d@mail.gmail.com>
References: <17076.21429.501499.914085@montanaro.dyndns.org>
	<4e4a11f805061817495428952d@mail.gmail.com>
Message-ID: <20050618175824.72A8.JCARLSON@uci.edu>


Simon Wittber <simonwittber at gmail.com> wrote:
> > Why this discussion of yet another serialization format?
> 
> Pickle is stated to be unsafe. Marshal is also stated to be unsafe.
> XML can be bloated, and XML+gzip is quite slow.
> 
> Do size,speed, and security features have to mutually exclusive? No,
> that possibly is why people have had to invent their own formats. I
> can list four off the top of my head:

...


Looks to me like the eval(repr(obj)) loop spanks XMLRPC.  It likely also
spanks Gherkin, but I'm not one to run untrusted code.  Give it a shot
on your own machine.  As for parsing, repr() of standard Python objects
is pretty easy to parse, and if you want something a bit easier to read,
there's always pprint (though it may not be quite as fast).


 - Josiah

Python 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> value = (("this is a record",1,2,1000001,"(08)123123123","some moretext")*10000)
>>> import time
>>> t = time.time();y = repr(value);time.time()-t
0.030999898910522461
>>> t = time.time();z = eval(y);time.time()-t
0.26600003242492676
>>> import xmlrpclib
>>> t = time.time();n = xmlrpclib.dumps(value);time.time()-t
0.42100000381469727
>>> t = time.time();m = xmlrpclib.loads(n);time.time()-t
4.4529998302459717
>>>


From python at rcn.com  Sun Jun 19 04:09:24 2005
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 18 Jun 2005 22:09:24 -0400
Subject: [Python-Dev] Recommend accepting PEP 312 --
	SimpleImplicit	Lambda
In-Reply-To: <42B52343.3050902@minkirri.apana.org.au>
Message-ID: <000901c57473$ea3610c0$7c03a044@oemcomputer>

[Donovan Baarda]
> As I see it, a lambda is an anonymous function. An anonymous function
is
> a function without a name. We already have a syntax for a function...
> why not use it. ie:
> 
>   f = filter(def (a): return a > 1, [1,2,3])

This approach is entirely too obvious.  If we want to be on the leading
edge, we can't be copying what was done years ago in Lua. ;-)


Raymond

From pje at telecommunity.com  Sun Jun 19 04:15:54 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 18 Jun 2005 22:15:54 -0400
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator
 objects
In-Reply-To: <20050619005024.GA14709@mems-exchange.org>
References: <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
	<5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com>
	<5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>
	<5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050618212627.01e22658@mail.telecommunity.com>

At 06:50 PM 6/18/2005 -0600, Neil Schemenauer wrote:
>On Sat, Jun 18, 2005 at 06:24:48PM -0400, Phillip J. Eby wrote:
> > So, I think I've got this sorted out, assuming that I'm not doing
> > something hideously insane by having 'has_finalizer()' always
> > check tp_del even for non-heap types, and defining a tp_del slot
> > for generators to call close() in.
>
>That sounds like the right thing to do.
>
>I suspect the "uncollectable cycles" problem will not be completely
>solvable.  With this change, all generators become objects with
>finalizers.  In reality, a 'file' object, for example, has a
>finalizer as well but it gets away without telling the GC that
>because its finalizer doesn't do anything "evil".  Since generators
>can do arbitrary things, the GC must assume the worst.

Yep.  It's too bad that there's no simple way to guarantee that the 
generator won't resurrect anything.  On the other hand, close() is 
guaranteed to give the generator at most one chance to do this.  So, 
perhaps there's some way we could have the GC close() generators in 
unreachable cycles.  No, wait, that would mean they could resurrect things, 
right?  Argh.


>Most cycles involving enhanced generators can probably be broken by
>the GC because the generator is not in the strongly connected part
>of cycle.  The GC will have to work a little harder to figure that
>out but that's probably not too significant.

Yep; by setting the generator's frame to None, I was able to significantly 
reduce the number of generator cycles in the tests.


>The real problem is that some cycles involving enhanced generators
>will not be breakable by the GC.  I think some programs that used to
>work okay are now going to start leaking memory because objects will
>accumulate in gc.garbage.

Yep, unless we .close() generators after adding them to gc.garbage(), which 
*might* be an option.  Although, I suppose if it *were* an option, then why 
doesn't GC already have some sort of ability to do this?  (i.e. run __del__ 
methods on items in gc.garbage, then remove them if their refcount drops to 
1 as a result).

[...pause to spend 5 minutes working it out in pseudocode...]

Okay, I think I see why you can't do it.  You could guarantee that all 
relevant __del__ methods get called, but it's bloody difficult to end up 
with only unreachable items in gc.garbage afterwards.   I think gc would 
have to keep a new list for items reachable from finalizers, that don't 
themselves have finalizers.  Then, before creating gc.garbage, you walk the 
finalizers and call their finalization (__del__) methods.  Then, you put 
any remaining items that are in either the finalizer list or the 
reachable-from-finalizers list into gc.garbage.

This approach might need a new type slot, but it seems like it would let us 
guarantee that finalizers get called, even if the object ends up in garbage 
as a result.  In the case of generators, however, close() guarantees that 
the generator releases all its references, and so can no longer be part of 
a cycle.  Thus, it would guarantee eventual cleanup of all 
generators.  And, it would lift the general limitation on __del__ methods.

Hm.  Sounds too good to be true.  Surely if this were possible, Uncle Timmy 
would've thought of it already, no?  Guess we'll have to wait and see what 
he thinks.


>Now, I could be wrong about all this.  I've have not been following
>the PEP 343 discussion too closely.  Maybe Guido has some clever
>idea.  Also, I find it difficult to hold in my head a complete model
>of how the GC now works.  It's an incredibly subtle piece of code.
>Perhaps Tim can comment.

I'm hoping Uncle Timmy can work his usual algorithmic magic here and 
provide us with a brilliant but impossible-for-mere-mortals-to-understand 
solution.  (The impossible-to-understand part being optional, of course. :) )


From python at rcn.com  Sun Jun 19 05:28:51 2005
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 18 Jun 2005 23:28:51 -0400
Subject: [Python-Dev] Recommend accepting PEP 312 --SimpleImplicit	Lambda
In-Reply-To: <000901c57473$ea3610c0$7c03a044@oemcomputer>
Message-ID: <000001c5747f$040c22e0$7c03a044@oemcomputer>

> [Donovan Baarda]
> > As I see it, a lambda is an anonymous function. An anonymous
function
> is
> > a function without a name. We already have a syntax for a
function...
> > why not use it. ie:
> >
> >   f = filter(def (a): return a > 1, [1,2,3])

[Me]
> This approach is entirely too obvious.  If we want to be on the
leading
> edge, we can't be copying what was done years ago in Lua. ;-)

Despite the glib comment, the idea as presented doesn't work because it
mixes statement and expression semantics (the inner 'return' is at best
unpleasant).  Also, the naming is off -- 'def' defines some variable
name. In Lua, the 'def' is called 'function' which doesn't imply a
variable assignment.


Raymond

From skip at pobox.com  Sun Jun 19 06:07:57 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sat, 18 Jun 2005 23:07:57 -0500
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <4e4a11f805061817462238af59@mail.gmail.com>
References: <4e4a11f805061723192888915e@mail.gmail.com>
	<7168d65a0506180644367753b9@mail.gmail.com>
	<4e4a11f805061807127f39700b@mail.gmail.com>
	<17076.21429.501499.914085@montanaro.dyndns.org>
	<4e4a11f805061817462238af59@mail.gmail.com>
Message-ID: <17076.61341.558478.28695@montanaro.dyndns.org>


    Simon> XML is simply not suitable for database appplications, real time
    Simon> data capture and game/entertainment applications.

I use XML-RPC as the communications protocol between an Apache web server
and a middleware piece that talks to a MySQL database.  The web server
contains a mixture of CGI scripts written in Python and two websites written
in Mason (Apache+mod_perl).  Performance is fine.  Give either of these a
try:

    http://www.mojam.com/
    http://www.musi-cal.com/

    Simon> I'm sure other people have noticed this... or am I alone on this
    Simon> issue? :-)

Probably not.  XML-RPC is commonly thought of as slow, and if you operate
with it in a completely naive fashion, I don't doubt that it can be.  It
doesn't have to be though.  Do you have to be intelligent about caching
frequently used information and returning results in reasonably sized
chunks?  Sure, but that's probably true of any database-backed application.

    Simon> Have a look at this contrived example:

    ...

    Simon> Which produces the output:

    >> pythonw -u "bench.py"
    Simon> Gherkin encode 0.120689361357 seconds
    Simon> Gherkin decode 0.395871262968 seconds
    Simon> XMLRPC encode 0.528666352847 seconds
    Simon> XMLRPC decode 9.01307819849 seconds

That's fine, so XML-RPC is slower than Gherkin.  I can't run the Gherkin
code, but my XML-RPC numbers are a bit different than yours:

    XMLRPC encode 0.65 seconds
    XMLRPC decode 2.61 seconds

That leads me to believe you're not using any sort of C XML decoder.  (I
mentioned sgmlop in my previous post.  I'm sure /F has some other
super-duper accelerator that's even faster.)

I'm not saying that XML-RPC is the fastest thing on Earth.  I'd be willing
to bet it's a lot more interoperable than Gherkin is though:

    http://xmlrpc.scripting.com/directory/1568/implementations

and probably will be for the forseeable future.

Also, as you indicated, your example was a bit contrived.  XML-RPC seems to
be fast enough for many real-world applications.  Here's a somewhat
less-contrived example from the above websites:

    >>> orca
    <ServerProxy for orca.mojam.com:5007/RPC2>
    >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 200, 50, 10000) ; print time.time() - t
    1.28429102898
    >>> len(x[1])
    200
    >>> x[1][0]
    ['MusicEntry', {'venue': 'Jazz Showcase', 'address': '', 'price': '', 'keywords': ['.jz.1369'], 'event': '', 'city': 'Chicago', 'end': datetime.datetime(2005, 6, 19, 0, 0), 'zip': '', 'start': datetime.datetime(2005, 6, 14, 0, 0), 'state': 'IL', 'program': '', 'email': 'skip at mojam.com', 'info': '', 'update_time': datetime.datetime(2005, 6, 8, 0, 0), 'address1': '59 West Grand Avenue', 'address2': '', 'address3': '', 'venueid': 2630, 'key': 816875, 'submit_time': datetime.datetime(2005, 6, 8, 0, 0), 'active': 1, 'merchandise': '', 'tickets': '', 'name': '', 'addressid': 17056, 'performers': ['Cedar Walton Quartet'], 'country': '', 'venueurl': '', 'time': '', 'performerids': [174058]}]
    >>> orca.checkpoint()
    'okay'
    >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 200, 50, 10000) ; print time.time() - t
    1.91681599617

orca is an xmlrpclib proxy to the aforementioned middleware component.
(These calls are being made to a production server.)  The search() call gets
the first 200 concert listings within a 50-mile radius of Chicago.  x[1][0]
is the first item returned.  All 200 returned items are of the same
complexity.  1.28 seconds certainly isn't Earth-shattering performance.
Even worse is the 1.92 seconds after the checkpoint (which flushes the
caches forcing the MySQL database to be queried for everything).  Returning
200 items is also contrived.  Real users can't ask for that many items at a
time through the web interface.  Cutting it down to 20 items (which is what
users get by default) shows a different story:

    >>> orca.checkpoint()
    'okay'
    >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 20, 50, 10000) ; print time.time() - t
    0.29478096962
    >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 20, 50, 10000) ; print time.time() - t
    0.0978591442108

The first query after a checkpoint is slow because we have to go to MySQL a
few times, but once everything's cached, things are fast.  The 0.1 second
time for the last call is going to be almost all XML-RPC overhead, because
all the data's already been cached.  I find that acceptable.  If you go to
the Mojam website and click "Chicago", the above query is pretty much what's
performed (several other queries are also run to get corollary info).

I still find it hard to believe that yet another serialization protocol is
necessary.  XML is certainly overkill for almost everything.  I'll be the
first to admit that.  (I have to struggle with it as a configuration file
format at work.)  However, it is certainly widely available.

Skip

From simonwittber at gmail.com  Sun Jun 19 06:37:56 2005
From: simonwittber at gmail.com (Simon Wittber)
Date: Sun, 19 Jun 2005 12:37:56 +0800
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <17076.61341.558478.28695@montanaro.dyndns.org>
References: <4e4a11f805061723192888915e@mail.gmail.com>
	<7168d65a0506180644367753b9@mail.gmail.com>
	<4e4a11f805061807127f39700b@mail.gmail.com>
	<17076.21429.501499.914085@montanaro.dyndns.org>
	<4e4a11f805061817462238af59@mail.gmail.com>
	<17076.61341.558478.28695@montanaro.dyndns.org>
Message-ID: <4e4a11f805061821378cd84b6@mail.gmail.com>

> I use XML-RPC as the communications protocol between an Apache web server
> and a middleware piece that talks to a MySQL database.  The web server
> contains a mixture of CGI scripts written in Python and two websites written
> in Mason (Apache+mod_perl).  Performance is fine.  Give either of these a
> try:

I've also implemented some middleware between Apache and SQL Server,
using XMLRPC. Unfortunately, large queries were the order of the day,
and things started moving like treacle. I profiled the code, and
discovered that 30 seconds of the 35 second response time was being
chewed up in building and parsing XML. I hacked things a bit, and
instead of sending XML, sent pickles inside the XML response. This
dropped response times to around the 8 second mark. This was still not
good enough, as multiple queries could still choke up the system.

Typical queries were about 1000-10000 records of 3-15 fields of
varying datatypes, including unicode. Don't ask me why people needed
these kind of reports, I'm just the programmer, and don't always get
to control design decisions. :-) This demonstrates that sometimes, in
real world apps, large amounts of data need to be shipped around, and
it needs to happen fast.

I guess when interoperability counts, XMLRPC is the best solution.
When controlling both ends of the connection, pickle _should_ be best,
except it is stated as being unsecure.

> That's fine, so XML-RPC is slower than Gherkin.  I can't run the Gherkin
> code, but my XML-RPC numbers are a bit different than yours:
> 
>     XMLRPC encode 0.65 seconds
>     XMLRPC decode 2.61 seconds
> 
> That leads me to believe you're not using any sort of C XML decoder.  

I used the standard library xmlrpclib only.

> I'm not saying that XML-RPC is the fastest thing on Earth.  I'd be willing
> to bet it's a lot more interoperable than Gherkin is though:

Agreed. Interoperability was a request in the aforementioned RFE,
though it was not an explicit goal of mine. I needed a very fast
serialization routine, which could safely unserialize messages from
untrusted sources, for sending big chunks of data over TCP. I also
needed it to be suitable for use in real-time multiplayer games (my
part time hobby). XMLRPC doesn't suit this application. When I control
the server, and the client applications, I shouldn't have to use XML.

Thanks for the feedback.

Simon.

From raymond.hettinger at verizon.net  Sun Jun 19 07:43:27 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun, 19 Jun 2005 01:43:27 -0400
Subject: [Python-Dev] Propose to reject PEP 294 -- Type Names in the types
	Module
Message-ID: <000001c57491$d1dc8fe0$25fccc97@oemcomputer>

Introducing a new set of duplicate type names and deprecating old ones
causes a certain amount of disruption.  Given the age of the types
module, the disruption is likely to be greater than any potential
benefit that could be realized.  Plenty of people will have to incur the
transition costs, but no one will likely find the benefit to be
perceptible.

Suggest rejecting this PEP and making a note for Py3.0 to either sync-up
the type names or abandon the types module entirely.



Raymond


From raymond.hettinger at verizon.net  Sun Jun 19 08:50:54 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun, 19 Jun 2005 02:50:54 -0400
Subject: [Python-Dev] Question about PEP 330 -- Python Bytecode Verification
Message-ID: <000101c5749b$3d5a2d00$25fccc97@oemcomputer>

Do we have *any* known use cases where we would actually run bytecode
that was suspicious enough to warrant running a well-formedness check?

In assessing security risks, the PEP notes, "Practically, it would be
difficult for a malicious user to 'inject' invalid bytecode into a PVM
for the purposes of exploitation, but not impossible."

Can that ever occur without there being a far greater risk of malicious,
but well-formed bytecode?

If you download a file, foo.pyc, from an untrusted source and run it in
a susceptible environment, does its well-formedness give you *any*
feeling of security.  I think not.

There isn't anything wrong with having a verifier module, but I can't
think of any benefit that would warrant changing the bytecode semantics
just to facilitate one of the static stack checks.



Raymond


From kay.schluehr at gmx.net  Sun Jun 19 08:52:19 2005
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sun, 19 Jun 2005 08:52:19 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <42B52343.3050902@minkirri.apana.org.au>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>
	<42B52343.3050902@minkirri.apana.org.au>
Message-ID: <d934cd$qn9$1@sea.gmane.org>

Donovan Baarda wrote:

> I must admit I ended up deleting most of the "alternative to lambda" 
> threads after they flooded my in box. So it is with some dread I post 
> this, contributing to it...

I must admit you are right. And I will stop defending my proposal 
because it seems to create nothing more than pointless polemics. 
Suggesting elements of FP to python-dev may have the same chances to be 
accepted than raising a Buddha statue in a catholic church.

> As I see it, a lambda is an anonymous function. An anonymous function is 
> a function without a name. We already have a syntax for a function... 
> why not use it. ie:
> 
>   f = filter(def (a): return a > 1, [1,2,3])

You mix expressions with statements. This is a no-go in Python. 
Permitting those constructs is a radical language change not just a 
simple syntax replacement.

Kay


From abo at minkirri.apana.org.au  Sun Jun 19 16:55:29 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Sun, 19 Jun 2005 07:55:29 -0700
Subject: [Python-Dev] Recommend accepting PEP 312
	--	Simple	Implicit	Lambda
In-Reply-To: <42B4C134.8010308@gmail.com>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>
	<42B4C134.8010308@gmail.com>
Message-ID: <42B58761.7090701@minkirri.apana.org.au>

Nick Coghlan wrote:
> Donovan Baarda wrote:
> 
>>As I see it, a lambda is an anonymous function. An anonymous function is 
>>a function without a name.
> 
> 
> And here we see why I'm such a fan of the term 'deferred expression' 
> instead of 'anonymous function'.

But isn't a function just a deferred expression with a name :-)

As a person who started out writing assembler where every "function" I 
wrote was a macro that got expanded inline, the distiction is kinda 
blurry to me.

> Python's lambda expressions *are* the former, but they are 
> emphatically *not* the latter.

Isn't that because lambda's have the limitation of not allowing 
statements, only expressions? I know this limitation avoids side-effects 
and has significance in some formal (functional?) languages... but is 
that what Python is? In the Python I use, lambda's are always used where 
you are too lazy to define a function to do it's job.

To me, anonymous procedures/functions would be a superset of "deferred 
expressions", and if the one stone fits perfectly in the slingshot we 
have and can kill multiple birds... why hunt for another stone?

Oh yeah Raymond: on the "def defines some variable name"... are you 
joking? You forgot the smiley :-)

I don't get what the problem is with mixing statement and expression 
semantics... from a practial point of view, statements just offer a 
superset of expression functionality.

If there really is a serious practical reason why they must be limited 
to expressions, why not just raise an exception or something if the 
"anonymous function" is too complicated...

I did some fiddling and it seems lambda's can call methods and stuff 
that can have side effects, which kinda defeats what I thought was the 
point of "statements vs expressions"... I guess I just don't 
understand... maybe I'm just thick :-)

> Anyway, the AlternateLambdaSyntax Wiki page has a couple of relevant 
> entries under 'real closures'.

Where is that wiki BTW? I remember looking at it ages ago but can't find 
the link anymore.

--
Donovan Baarda

From raymond.hettinger at verizon.net  Sun Jun 19 09:03:50 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun, 19 Jun 2005 03:03:50 -0400
Subject: [Python-Dev] Propose to close PEP 254 -- Making Classes Look More
	Like Types
Message-ID: <000201c5749d$0c133500$25fccc97@oemcomputer>

This PEP is an empty stub that is unlikely to ever get filled-out in a
way that adds anything beyond what is already implemented and
documented.


Raymond


From ncoghlan at gmail.com  Sun Jun 19 09:32:39 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 19 Jun 2005 17:32:39 +1000
Subject: [Python-Dev] Recommend accepting PEP 312
	--	Simple	Implicit	Lambda
In-Reply-To: <42B58761.7090701@minkirri.apana.org.au>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>
	<42B4C134.8010308@gmail.com>
	<42B58761.7090701@minkirri.apana.org.au>
Message-ID: <42B51F97.3080206@gmail.com>

Donovan Baarda wrote:
> Nick Coghlan wrote:
>> And here we see why I'm such a fan of the term 'deferred expression' 
>> instead of 'anonymous function'.
> 
> But isn't a function just a deferred expression with a name :-)

According to the specific meaning of 'expression' in the Python 
grammar? No.

> Isn't that because lambda's have the limitation of not allowing 
> statements, only expressions? I know this limitation avoids side-effects 
> and has significance in some formal (functional?) languages... but is 
> that what Python is?

Personally, I believe it has to do with two things, one practical and 
one more stylistic/philosophical.

Firstly, Python uses significant whitespace between statements, and to 
delineate sections of compound statements. Within expressions however, 
whitespace is essentially insignificant (except where it affects the 
tokenisation). Trying to embed suites (where whitespace can be 
significant) inside expressions (where whitespace is generally 
insiginificant) tends to result in a horrible mess, or else some 
rather unPythonic concepts (e.g. using something other than 
indentation to demarcate the suite).

Secondly, trying to cram too much functionality into a single line is 
a questionable activity. The 'deferred expression' approach taken with 
lambdas is a hint that if the thing you're trying to defer is more 
complex than a single expression, it may be worth giving it a proper 
name via 'def'.

> In the Python I use, lambda's are always used where 
> you are too lazy to define a function to do it's job.

In simple cases, naming the expression to be deferred results in it 
gaining a prominence it doesn't deserve, possibly obscuring the 
purpose of the main statement.

In such cases, deferred expressions are a better solution - it's 
merely unfortunate that Python's current syntax for the practice 
anything to write home about.

> To me, anonymous procedures/functions would be a superset of "deferred 
> expressions", and if the one stone fits perfectly in the slingshot we 
> have and can kill multiple birds... why hunt for another stone?

Because there is already a solution for the large birds - *named* 
functions.

Unfortunately, using a named function for a simple deferred expression 
is like using a catapult to kill a canary.

> If there really is a serious practical reason why they must be limited 
> to expressions, why not just raise an exception or something if the 
> "anonymous function" is too complicated...

You'd still need a definition of 'too complicated' - which we already 
have for deferred expressions. Specifically "unable to be expressed as 
a single Python expression".

>> Anyway, the AlternateLambdaSyntax Wiki page has a couple of relevant 
>> entries under 'real closures'.
> 
> Where is that wiki BTW? I remember looking at it ages ago but can't find 
> the link anymore.

http://wiki.python.org/moin/AlternateLambdaSyntax

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From kdart at kdart.com  Sun Jun 19 09:45:27 2005
From: kdart at kdart.com (Keith Dart)
Date: Sun, 19 Jun 2005 00:45:27 -0700 (PDT)
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
 Integers
In-Reply-To: <2m1x6z7qu6.fsf@starship.python.net>
References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer>
	<ca471dc20506171533147714a8@mail.gmail.com>
	<42B3E41A.2020608@kdart.com> <2m1x6z7qu6.fsf@starship.python.net>
Message-ID: <Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>

On Sat, 18 Jun 2005, Michael Hudson wrote:

>
> The shortest way I know of going from 2149871625L to -2145095671 is
> the still-fairly-gross:
>
>>>> v = 2149871625L
>>>> ~int(~v&0xFFFFFFFF)
> -2145095671
>
>> I suppose the best thing is to introduce an "unsignedint" type for this
>> purpose.
>
> Or some kind of bitfield type, maybe.
>
> C uses integers both as bitfields and to count things, and at least in
> my opinion the default assumption in Python should be that this is
> what an integer is being used for, but when you need a bitfield it can
> all get a bit horrible.
>
> That said, I think in this case we can just make fcntl_ioctl use the
> (new-ish) 'I' format argument to PyArg_ParseTuple and then you'll just
> be able to use 2149871625L and be happy (I think, haven't tried this).

Thanks for the reply. I think I will go ahead and add some extension types 
to Python. Thankfully, Python is extensible with new objects.

It is also useful (to me, anyway) to be able to map, one to one,
external primitives from other systems to Python primitives. For
example, CORBA and SNMP have a set of types (signed ints, unsigned ints,
etc.) defined that I would like to interface to Python (actually I have
already done this to some degree). But Python makes it a bit more
difficult without that one-to-one mapping of basic types.  Having an
unsigned int type, for example, would make it easier to interface Python
to SNMP or even some C libraries.

In other words, Since the "Real World" has these types that I must
sometimes interface to, it is useful to have these same (predictable)
types in Python.

So, it is worth extending the basic set of data types, and I will add it
to my existing collection of Python extensions.

Therefore, I would like to ask here if anyone has already started
something like this? If not, I will go ahead and do it (if I have time).


-- 

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Keith Dart <kdart at kdart.com>
    public key: ID: F3D288E4
    =====================================================================

From jcarlson at uci.edu  Sun Jun 19 10:01:47 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun, 19 Jun 2005 01:01:47 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 --Simple Implicit
	Lambda
In-Reply-To: <42B58761.7090701@minkirri.apana.org.au>
References: <42B4C134.8010308@gmail.com>
	<42B58761.7090701@minkirri.apana.org.au>
Message-ID: <20050619002931.72AB.JCARLSON@uci.edu>


Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> Nick Coghlan wrote:
> > Donovan Baarda wrote:
> > 
> >>As I see it, a lambda is an anonymous function. An anonymous function is 
> >>a function without a name.
> > 
> > 
> > And here we see why I'm such a fan of the term 'deferred expression' 
> > instead of 'anonymous function'.
> 
> But isn't a function just a deferred expression with a name :-)

A function in Python is actually a deferred sequence of statements and
expressions. An anonymous function in Python (a lambda) is a deferred
expression.


> > Python's lambda expressions *are* the former, but they are 
> > emphatically *not* the latter.
> 
> Isn't that because lambda's have the limitation of not allowing 
> statements, only expressions? I know this limitation avoids side-effects 
> and has significance in some formal (functional?) languages... but is 
> that what Python is? In the Python I use, lambda's are always used where 
> you are too lazy to define a function to do it's job.

I've generally seen people use lambdas for things that don't require
names in the current context; i.e. callbacks with simple executions.


> To me, anonymous procedures/functions would be a superset of "deferred 
> expressions", and if the one stone fits perfectly in the slingshot we 
> have and can kill multiple birds... why hunt for another stone?

Are "deferred expressions" perhaps another way of spelling "function
closure"?


> Oh yeah Raymond: on the "def defines some variable name"... are you 
> joking? You forgot the smiley :-)

'def' happens to bind the name that follows the def to the function with
the arguments and body following the name.


> I don't get what the problem is with mixing statement and expression 
> semantics... from a practial point of view, statements just offer a 
> superset of expression functionality.

Statements don't have a return value.  To be more precise, what is the
value of "for i in xrange(10): z.append(...)"?  Examine the selection of
statements available to Python, and ask that question.  The only one
that MAY have a return value, is 'return' itself, which really requires
an expression to the right (which passes the expression to the right to
the caller's frame).  When you have statements that ultimately need a
'return' for a return value; you may as well use a standard function
definition.


> If there really is a serious practical reason why they must be limited 
> to expressions, why not just raise an exception or something if the 
> "anonymous function" is too complicated...

Define "too complicated"?


> I did some fiddling and it seems lambda's can call methods and stuff 
> that can have side effects, which kinda defeats what I thought was the 
> point of "statements vs expressions"... I guess I just don't 
> understand... maybe I'm just thick :-)

There is nothing stopping anyone from modifying anything in a lambda...

a = list(...)
pop = lambda:a.pop()
lcycle = lambda: a and a.append(a.pop(0))
rcycle = lambda: a and a.insert(0, a.pop())
popany = lambda: a and a.pop(random.randrange(len(a)))


 - Josiah


From jcarlson at uci.edu  Sun Jun 19 10:07:57 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun, 19 Jun 2005 01:07:57 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
Message-ID: <20050619010354.72AE.JCARLSON@uci.edu>


Keith Dart <kdart at kdart.com> wrote:

> Therefore, I would like to ask here if anyone has already started
> something like this? If not, I will go ahead and do it (if I have time).

If all you need to do is read or write C-like types to or from memory,
you should spend some time looking through the 'struct' module if you
haven't already.

 - Josiah


From kdart at kdart.com  Sun Jun 19 10:25:55 2005
From: kdart at kdart.com (Keith Dart)
Date: Sun, 19 Jun 2005 01:25:55 -0700 (PDT)
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
 Integers
In-Reply-To: <20050619010354.72AE.JCARLSON@uci.edu>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>
Message-ID: <Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>

On Sun, 19 Jun 2005, Josiah Carlson wrote:

>
> Keith Dart <kdart at kdart.com> wrote:
>
>> Therefore, I would like to ask here if anyone has already started
>> something like this? If not, I will go ahead and do it (if I have time).
>
> If all you need to do is read or write C-like types to or from memory,
> you should spend some time looking through the 'struct' module if you
> haven't already.

I know about 'struct'. However, it will just convert to Python "native" 
types. C unsigned become Python longs.

  u = struct.pack("I", 0xfffffffe)
  struct.unpack("I", u)
(4294967294L,)

In SNMP, for example, a Counter32 is basically an unsigned int, defined
as "IMPLICIT INTEGER (0..4294967295)". One cannot efficiently translate
and use that type in native Python. Currently, I have defined an
"unsigned" type as a subclass of long, but I don't think that would be
speed or storage efficient.

On the other hand, adding my own type won't help with the ioctl()
problem, since it won't know about it.




-- 

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Keith Dart <kdart at kdart.com>
    public key: ID: F3D288E4
    =====================================================================

From abo at minkirri.apana.org.au  Sun Jun 19 18:29:21 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Sun, 19 Jun 2005 09:29:21 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 --Simple Implicit
	Lambda
In-Reply-To: <20050619002931.72AB.JCARLSON@uci.edu>
References: <42B4C134.8010308@gmail.com>
	<42B58761.7090701@minkirri.apana.org.au>
	<20050619002931.72AB.JCARLSON@uci.edu>
Message-ID: <42B59D61.1000009@minkirri.apana.org.au>

Josiah Carlson wrote:
> Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> 
>>Nick Coghlan wrote:
>>
>>>Donovan Baarda wrote:
[...]
>>But isn't a function just a deferred expression with a name :-)
> 
> 
> A function in Python is actually a deferred sequence of statements and
> expressions. An anonymous function in Python (a lambda) is a deferred
> expression.

in the end though, a sequence of statements that completes with a 
"return value" is, when treated as a black box, indistinguishable from 
an expression. Originally I thought that this also had to be qualified 
with "and has no side-effects", but I see now that is not the case.

[...]
>>Oh yeah Raymond: on the "def defines some variable name"... are you 
>>joking? You forgot the smiley :-)
> 
> 
> 'def' happens to bind the name that follows the def to the function with
> the arguments and body following the name.

Yeah, but we don't use "def" to bind arbitary variables, only 
functions/procedures. So in python, they are intimately identified with 
functions and procedures.

>>I don't get what the problem is with mixing statement and expression 
>>semantics... from a practial point of view, statements just offer a 
>>superset of expression functionality.
> 
> 
> Statements don't have a return value.  To be more precise, what is the
> value of "for i in xrange(10): z.append(...)"?  Examine the selection of
> statements available to Python, and ask that question.  The only one
> that MAY have a return value, is 'return' itself, which really requires
> an expression to the right (which passes the expression to the right to
> the caller's frame).  When you have statements that ultimately need a
> 'return' for a return value; you may as well use a standard function
> definition.

Hmmm. For some reason I thought that these kind of things would have a 
return value of None, the same as a function without an explicit return. 
I see now that this is not true...

>>If there really is a serious practical reason why they must be limited 
>>to expressions, why not just raise an exception or something if the 
>>"anonymous function" is too complicated...
> 
> 
> Define "too complicated"?

I was thinking that this is up to the interpreter... depending on what 
the practical limitations are that cause the limitation in the first 
place. For example... if it can't be reduced to an "expression" through 
simple transforms.

But look... I've gone and created another monster thread on 
"alternatives to lambda"... I'm going to shut up now.

--
Donovan Baarda

From kay.schluehr at gmx.net  Sun Jun 19 13:47:54 2005
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sun, 19 Jun 2005 13:47:54 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <42B58761.7090701@minkirri.apana.org.au>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>	<42B4C134.8010308@gmail.com>
	<42B58761.7090701@minkirri.apana.org.au>
Message-ID: <d93lmi$ot7$1@sea.gmane.org>

Donovan Baarda wrote:

> I don't get what the problem is with mixing statement and expression 
> semantics... from a practial point of view, statements just offer a 
> superset of expression functionality.
> 
> If there really is a serious practical reason why they must be limited 
> to expressions, why not just raise an exception or something if the 
> "anonymous function" is too complicated...
> 
> I did some fiddling and it seems lambda's can call methods and stuff 
> that can have side effects, which kinda defeats what I thought was the 
> point of "statements vs expressions"... I guess I just don't 
> understand... maybe I'm just thick :-)

The whole point is that you are able to do all the basic control flow 
operations like IF, FOR and WHILE using simple expressions ( like in 
Pythons lambda ), the lambda statement itself and recursion. Therefore 
lambda expressions constitute a Turing complete language and they also 
do so in Python. Different to many FP languages lambda plays no central
role in Python because statements won't be reduced to lambda expressions 
( or some kind of ). They are merely an add-on.

Reduction provides often the advantage to make expressions/statements 
scriptable what they are not in Python. Python is strong in scripting 
classes/objects ( a big plus of the language ) but you can't simply use 
the language to prove that

                lambda x,y: x+y*y
                lambda x,y: y**2+x

are essentialy the same functions with different implementations [1]. I 
think this is a severe lack of expressibility and has nothing to do with 
the silly objection that one has to write one more line for a simple 
callback - o.k. I admit that I'm lazy too ;)

Regards,
Kay


[1] Not without hacking the parse tree. Doing so one might finally end 
up accessing the expression in a simple modifieable manner:

 >>> (lambda x,y: x+y*y).expr
('+',(x,'*',(y,y)))

 >>> (lambda x,y: y**2+x).expr
('+',(('**',(y,2)),x))





From reinhold-birkenfeld-nospam at wolke7.net  Sun Jun 19 14:44:32 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Sun, 19 Jun 2005 14:44:32 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <d93lmi$ot7$1@sea.gmane.org>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>	<42B4C134.8010308@gmail.com>	<42B58761.7090701@minkirri.apana.org.au>
	<d93lmi$ot7$1@sea.gmane.org>
Message-ID: <d93p0o$vgd$1@sea.gmane.org>

Kay Schluehr wrote:

> Reduction provides often the advantage to make expressions/statements 
> scriptable what they are not in Python. Python is strong in scripting 
> classes/objects ( a big plus of the language ) but you can't simply use 
> the language to prove that
> 
>                 lambda x,y: x+y*y
>                 lambda x,y: y**2+x
> 
> are essentialy the same functions with different implementations [1].

Except that they are not. Think of __pow__, think of __add__ and __radd__.

Reinhold


-- 
Mail address is perfectly valid!


From skip at pobox.com  Sun Jun 19 15:39:48 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 19 Jun 2005 08:39:48 -0500
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
In-Reply-To: <4e4a11f805061821378cd84b6@mail.gmail.com>
References: <4e4a11f805061723192888915e@mail.gmail.com>
	<7168d65a0506180644367753b9@mail.gmail.com>
	<4e4a11f805061807127f39700b@mail.gmail.com>
	<17076.21429.501499.914085@montanaro.dyndns.org>
	<4e4a11f805061817462238af59@mail.gmail.com>
	<17076.61341.558478.28695@montanaro.dyndns.org>
	<4e4a11f805061821378cd84b6@mail.gmail.com>
Message-ID: <17077.30116.594374.702068@montanaro.dyndns.org>


    Simon> I hacked things a bit, and instead of sending XML, sent pickles
    Simon> inside the XML response.

I've done the same thing (I think I may have used marshal).  It works fine
as long as you know both ends are Python.

Skip

From skip at pobox.com  Sun Jun 19 15:40:46 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 19 Jun 2005 08:40:46 -0500
Subject: [Python-Dev] Propose to reject PEP 294 -- Type Names in the
 types Module
In-Reply-To: <000001c57491$d1dc8fe0$25fccc97@oemcomputer>
References: <000001c57491$d1dc8fe0$25fccc97@oemcomputer>
Message-ID: <17077.30174.836873.22778@montanaro.dyndns.org>

    Raymond> Suggest rejecting this PEP and making a note for Py3.0 to
    Raymond> either sync-up the type names or abandon the types module
    Raymond> entirely.

I thought the types module was already deprecated, at least verbally if not
officially.

Skip


From skip at pobox.com  Sun Jun 19 15:45:24 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 19 Jun 2005 08:45:24 -0500
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <d934cd$qn9$1@sea.gmane.org>
References: <42B35DC8.6040308@gmail.com> <d90hes$17d$1@sea.gmane.org>
	<20050618011502.728D.JCARLSON@uci.edu> <d9112d$vm2$1@sea.gmane.org>
	<42B52343.3050902@minkirri.apana.org.au>
	<d934cd$qn9$1@sea.gmane.org>
Message-ID: <17077.30452.331966.648854@montanaro.dyndns.org>


    >> As I see it, a lambda is an anonymous function. An anonymous function
    >> is a function without a name. We already have a syntax for a
    >> function...  why not use it. ie:
    >> 
    >> f = filter(def (a): return a > 1, [1,2,3])

    Kay> You mix expressions with statements. 

You could remove the "return" and restrict the body of the def to an
expression: 

    f = filter(def (a): a > 1, [1,2,3])

That looks almost exactly like a lambda, but uses "def" and parenthesizes
the argument list. It seems to me that would remind people "this is a
function".

Skip

From kay.schluehr at gmx.net  Sun Jun 19 16:45:07 2005
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sun, 19 Jun 2005 16:45:07 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <d93p0o$vgd$1@sea.gmane.org>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>	<42B4C134.8010308@gmail.com>	<42B58761.7090701@minkirri.apana.org.au>	<d93lmi$ot7$1@sea.gmane.org>
	<d93p0o$vgd$1@sea.gmane.org>
Message-ID: <d9402q$e62$1@sea.gmane.org>

Reinhold Birkenfeld wrote:

>>
>>                lambda x,y: x+y*y
>>                lambda x,y: y**2+x
>>
>> are essentialy the same functions with different implementations [1].
> 
> 
> Except that they are not. Think of __pow__, think of __add__ and __radd__.

You know the difference between the concept of a function and it's 
infinitely many representations? That's why formal definitions exist.

> 
> Reinhold
> 
> 

Just for refresh:

"Formally, a function f from a set X of input values to a set Y of 
possible output values (written as f : X -> Y) is a relation between X 
and Y which satisfies:

    1. f is total, or entire: for all x in X, there exists a y in Y such 
that x f y (x is f-related to y), i.e. for each input value, there is at 
least one output value in Y.

    2. f is many-to-one, or functional: if x f y and x f z, then y = z. 
i.e., many input values can be related to one output value, but one 
input value cannot be related to many output values.

A more concise expression of the above definition is the following: a 
function from X to Y is a subset f of the cartesian product X ? Y, such 
that for each x in X, there is a unique y in Y such that the ordered 
pair (x, y) is in f."

http://en.wikipedia.org/wiki/Function_%28mathematics%29

Kay




From kay.schluehr at gmx.net  Sun Jun 19 17:22:47 2005
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sun, 19 Jun 2005 17:22:47 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <17077.30452.331966.648854@montanaro.dyndns.org>
References: <42B35DC8.6040308@gmail.com>
	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>
	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>	<d934cd$qn9$1@sea.gmane.org>
	<17077.30452.331966.648854@montanaro.dyndns.org>
Message-ID: <d9429f$iou$1@sea.gmane.org>

Skip Montanaro wrote:

>     >> As I see it, a lambda is an anonymous function. An anonymous function
>     >> is a function without a name. We already have a syntax for a
>     >> function...  why not use it. ie:
>     >> 
>     >> f = filter(def (a): return a > 1, [1,2,3])
> 
>     Kay> You mix expressions with statements. 
> 
> You could remove the "return" and restrict the body of the def to an
> expression: 
> 
>     f = filter(def (a): a > 1, [1,2,3])
> 
> That looks almost exactly like a lambda, but uses "def" and parenthesizes
> the argument list. It seems to me that would remind people "this is a
> function".

Yes, but skipping the name of a function ( anonymizing it ) is not a 
strong reason to disallow statements in the anonymus function body. The 
crucial issue is the notation of callable expressions that are not 
statements but can be defined inside of other expressions ( e.g. inside 
a filter() call as in the example ). That's why I prefer notations that 
emphasize the expression character. Using the arrow notation
( (args) -> expr ) would be fine for me but I would not deselect Python 
in favor for Java if ( expr from (args) ) is used instead. To me it's a 
"judean popular front" vs "popular front of judea" kind of thing.

Kay



From rrr at ronadam.com  Sun Jun 19 18:21:40 2005
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 19 Jun 2005 12:21:40 -0400
Subject: [Python-Dev] Recommend accepting PEP 312
	--	Simple	Implicit	Lambda
In-Reply-To: <42B4C134.8010308@gmail.com>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>
	<42B4C134.8010308@gmail.com>
Message-ID: <42B59B94.5070906@ronadam.com>

Nick Coghlan wrote:
> Donovan Baarda wrote:
> 
>>As I see it, a lambda is an anonymous function. An anonymous function is 
>>a function without a name.
> 
> 
> And here we see why I'm such a fan of the term 'deferred expression' 
> instead of 'anonymous function'.
 >
> Python's lambda expressions *are* the former, but they are 
> emphatically *not* the latter.

Lambda's have inputs and outputs like a function, so they can be used in 
place of functions.

A simple deferred expression wouldn't be the same thing since it would 
use local names directly, but not at the time the expression is created, 
and couldn't (easily) be used in place of a function call.

addxy = defer x+y          # do this later
for x in range(10):
     for y in range(10):
         print addxy        # prints sums of x,y

This is basically a limited single expression macro. (Which I believe 
isn't wanted.)  A better example might be as used in a dictionary in 
place of a case statement.  ie..  case['add'] = defer x+y, etc...

Regards, Ron

(Reply to this in python-list if it would be better discussed there.)


From jcarlson at uci.edu  Sun Jun 19 18:45:00 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun, 19 Jun 2005 09:45:00 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 --Simple Implicit
	Lambda
In-Reply-To: <42B59D61.1000009@minkirri.apana.org.au>
References: <20050619002931.72AB.JCARLSON@uci.edu>
	<42B59D61.1000009@minkirri.apana.org.au>
Message-ID: <20050619082907.72B1.JCARLSON@uci.edu>


Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> Josiah Carlson wrote:
> > Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> > 
> >>Nick Coghlan wrote:
> >>
> >>>Donovan Baarda wrote:
> [...]
> >>But isn't a function just a deferred expression with a name :-)
> > 
> > 
> > A function in Python is actually a deferred sequence of statements and
> > expressions. An anonymous function in Python (a lambda) is a deferred
> > expression.
> 
> in the end though, a sequence of statements that completes with a 
> "return value" is, when treated as a black box, indistinguishable from 
> an expression. Originally I thought that this also had to be qualified 
> with "and has no side-effects", but I see now that is not the case.

Lambdas/anonymous functions are generally used when they are simple and
are either to be applied _now_, or passed as a callback to someone else
(sometimes people use the x = lambda..., but that is laziness,
def/return is clearer).  Are either of the following two examples easy
to read?

rslt = (def (arg):
            #body that handles arg
            ...
       )(argument_passed_to_fcn)

foo((def (arg):
        #body that handles arg
        ...
     ), foo_arg, ...)

I personally think they are horrible.  Both of the above would be easier
to read as...

def _throw_away(arg):
    #body that handles arg
    ...

rslt = _throw_away(argument_passed_to_fcn)
foo(_throw_away, ...)


> [...]
> >>Oh yeah Raymond: on the "def defines some variable name"... are you 
> >>joking? You forgot the smiley :-)
> > 
> > 
> > 'def' happens to bind the name that follows the def to the function with
> > the arguments and body following the name.
> 
> Yeah, but we don't use "def" to bind arbitary variables, only 
> functions/procedures. So in python, they are intimately identified with 
> functions and procedures.

Being that this entire thread is about functions, perhaps there was an
implied "function" in there?  Also, with the right decorator, it becomes
an arbitrary assignment method in Python 2.4 ...

>>> def assign(val):
...     return lambda a:val
...
>>> @assign(5)
... def foo(arg): pass
...
>>> foo
5
>>>


> >>If there really is a serious practical reason why they must be limited 
> >>to expressions, why not just raise an exception or something if the 
> >>"anonymous function" is too complicated...
> > 
> > 
> > Define "too complicated"?
> 
> I was thinking that this is up to the interpreter... depending on what 
> the practical limitations are that cause the limitation in the first 
> place. For example... if it can't be reduced to an "expression" through 
> simple transforms.

Care to write the transformation library for all Python statements? Care
to see that multi-line anonymous function I described above? I
certainly hope not.


 - Josiah


From reinhold-birkenfeld-nospam at wolke7.net  Sun Jun 19 19:33:11 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Sun, 19 Jun 2005 19:33:11 +0200
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <d9402q$e62$1@sea.gmane.org>
References: <42B35DC8.6040308@gmail.com>	<d90hes$17d$1@sea.gmane.org>	<20050618011502.728D.JCARLSON@uci.edu>	<d9112d$vm2$1@sea.gmane.org>	<42B52343.3050902@minkirri.apana.org.au>	<42B4C134.8010308@gmail.com>	<42B58761.7090701@minkirri.apana.org.au>	<d93lmi$ot7$1@sea.gmane.org>	<d93p0o$vgd$1@sea.gmane.org>
	<d9402q$e62$1@sea.gmane.org>
Message-ID: <d949ts$3e7$1@sea.gmane.org>

Kay Schluehr wrote:
> Reinhold Birkenfeld wrote:
> 
>>>
>>>                lambda x,y: x+y*y
>>>                lambda x,y: y**2+x
>>>
>>> are essentialy the same functions with different implementations [1].
>> 
>> 
>> Except that they are not. Think of __pow__, think of __add__ and __radd__.
> 
> You know the difference between the concept of a function and it's 
> infinitely many representations? That's why formal definitions exist.

I must say that I don't understand what you're after.

Reinhold

-- 
Mail address is perfectly valid!


From facundobatista at gmail.com  Sun Jun 19 20:31:54 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Sun, 19 Jun 2005 15:31:54 -0300
Subject: [Python-Dev] Propose to reject PEP 276 -- Simple iterator for
	ints
In-Reply-To: <000201c572fc$30d94e80$bb07a044@oemcomputer>
References: <000201c572fc$30d94e80$bb07a044@oemcomputer>
Message-ID: <e04bdf3105061911311f05f0b4@mail.gmail.com>

On 6/17/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:

> The principal use case was largely met by enumerate().  From PEP 276's

+1 for reject it.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From pje at telecommunity.com  Sun Jun 19 22:16:30 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 19 Jun 2005 16:16:30 -0400
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator
 objects
In-Reply-To: <5.1.1.6.0.20050618212627.01e22658@mail.telecommunity.com>
References: <20050619005024.GA14709@mems-exchange.org>
	<5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
	<5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com>
	<5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>
	<5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050618222502.01e444e8@mail.telecommunity.com>

At 10:15 PM 6/18/2005 -0400, Phillip J. Eby wrote:
>Okay, I think I see why you can't do it.  You could guarantee that all
>relevant __del__ methods get called, but it's bloody difficult to end up
>with only unreachable items in gc.garbage afterwards.   I think gc would
>have to keep a new list for items reachable from finalizers, that don't
>themselves have finalizers.  Then, before creating gc.garbage, you walk the
>finalizers and call their finalization (__del__) methods.  Then, you put
>any remaining items that are in either the finalizer list or the
>reachable-from-finalizers list into gc.garbage.
>
>This approach might need a new type slot, but it seems like it would let us
>guarantee that finalizers get called, even if the object ends up in garbage
>as a result.  In the case of generators, however, close() guarantees that
>the generator releases all its references, and so can no longer be part of
>a cycle.  Thus, it would guarantee eventual cleanup of all
>generators.  And, it would lift the general limitation on __del__ methods.
>
>Hm.  Sounds too good to be true.  Surely if this were possible, Uncle Timmy
>would've thought of it already, no?  Guess we'll have to wait and see what
>he thinks.

Or maybe not.  After sleeping on it, I realized that the problems are all 
in when and how often __del__ is called.  The idea I had above would end up 
calling __del__ twice on non-generator objects.  For generators it's not a 
problem because the first call ends up ensuring that the second call is a 
no-op.

However, the *order* of __del__ calls makes a difference, even for 
generators.  What good is a finally: clause if all the objects reachable 
from it have been finalized already, anyway?

Ultimately, I'm thinking that maybe we were right not to allow try-finally 
to cross yield boundaries in generators.  It doesn't seem like you can 
guarantee anything about the behavior in the presence of cycles, so what's 
the point?

For a while I played around with the idea that maybe we could still support 
'with:' in generators, though, because to implement that we could make 
frames call __exit__ on any pending 'with' blocks as part of their tp_clear 
operation.  This would only work, however, if the objects with __exit__ 
methods don't have any references back to the frame.  In essence, you'd 
need a way to put the __exit__ objects on a GC-managed list that wouldn't 
run until after all the tp_clear calls had finished.

But even that is tough to make guarantees about.  For example, can you 
guarantee in that case that a generator's 'with:' blocks are __exit__-ed in 
the proper order?

Really, if we do allow 'with' and 'try-finally' to surround yield, I think 
we're going to have to tell people that it only works if you use a with or 
try-finally in some non-generator code to ensure that the generator.close() 
gets called, and that if you end up creating a garbage cycle, we either 
have to let it end up in gc.garbage, or just not execute its finally clause 
or __exit__ methods.

Of course, this sort of happens right now for other things with __del__; if 
it's part of a cycle the __del__ method never gets called.  The only 
difference is that it hangs around in gc.garbage, doing nothing useful.  If 
it's garbage, it's not reachable from anywhere else, so it does nobody any 
good to have it around.  So, maybe we should just say, "sucks to be you" 
and tp_clear anything that we'd otherwise have put in gc.garbage.  :)

In other words, since we're not going to call those __del__ methods anyway, 
maybe it just needs to be part of the language semantics that __del__ isn't 
guaranteed to be called, and a garbage collector that can't find a safe way 
to call it, doesn't have to.  tp_dealloc for classic classes and heap types 
could then just skip calling __del__ if they've already been cleared... oh 
wait, how do you know you've been cleared?  Argh.  Another nice idea runs 
up on the rocks of reality.

On the other hand, if you go ahead and run __del__ after tp_clear, the 
__del__ method will quickly run afoul of an AttributeError and die with 
only a minor spew to sys.stderr, thus encouraging people to get rid of 
their silly useless __del__ methods on objects that normally end up in 
cycles.  :)


From pje at telecommunity.com  Sun Jun 19 23:04:48 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 19 Jun 2005 17:04:48 -0400
Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator
 objects
In-Reply-To: <5.1.1.6.0.20050618222502.01e444e8@mail.telecommunity.com>
References: <5.1.1.6.0.20050618212627.01e22658@mail.telecommunity.com>
	<20050619005024.GA14709@mems-exchange.org>
	<5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
	<5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com>
	<5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com>
	<5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050619163017.01e189c8@mail.telecommunity.com>

Sigh.  Looks like Guido already used the time machine to bring up these 
ideas five years ago:

    http://mail.python.org/pipermail/python-dev/2000-March/002514.html

And apparently you went back with him:

    http://mail.python.org/pipermail/python-dev/2000-March/002478.html

So I give up, 'cause there's no way I can compete with you time travellers.  :)

Although I do wonder -- why was __cleanup__ never implemented?  The only 
clue seems to be Guido's comment that he "[finds] having a separate 
__cleanup__ protocol cumbersome."  It certainly seems to me that having a 
__cleanup__ that allows an object to handle itself being garbage would be 
handy, although it's only meaningful to have a __cleanup__ if you also have 
a __del__; otherwise, there would never be a reason to call it.  Maybe 
that's the reason it was considered cumbersome.


At 04:16 PM 6/19/2005 -0400, Phillip J. Eby wrote:
>At 10:15 PM 6/18/2005 -0400, Phillip J. Eby wrote:
> >Okay, I think I see why you can't do it.  You could guarantee that all
> >relevant __del__ methods get called, but it's bloody difficult to end up
> >with only unreachable items in gc.garbage afterwards.   I think gc would
> >have to keep a new list for items reachable from finalizers, that don't
> >themselves have finalizers.  Then, before creating gc.garbage, you walk the
> >finalizers and call their finalization (__del__) methods.  Then, you put
> >any remaining items that are in either the finalizer list or the
> >reachable-from-finalizers list into gc.garbage.
> >
> >This approach might need a new type slot, but it seems like it would let us
> >guarantee that finalizers get called, even if the object ends up in garbage
> >as a result.  In the case of generators, however, close() guarantees that
> >the generator releases all its references, and so can no longer be part of
> >a cycle.  Thus, it would guarantee eventual cleanup of all
> >generators.  And, it would lift the general limitation on __del__ methods.
> >
> >Hm.  Sounds too good to be true.  Surely if this were possible, Uncle Timmy
> >would've thought of it already, no?  Guess we'll have to wait and see what
> >he thinks.
>
>Or maybe not.  After sleeping on it, I realized that the problems are all
>in when and how often __del__ is called.  The idea I had above would end up
>calling __del__ twice on non-generator objects.  For generators it's not a
>problem because the first call ends up ensuring that the second call is a
>no-op.
>
>However, the *order* of __del__ calls makes a difference, even for
>generators.  What good is a finally: clause if all the objects reachable
>from it have been finalized already, anyway?
>
>Ultimately, I'm thinking that maybe we were right not to allow try-finally
>to cross yield boundaries in generators.  It doesn't seem like you can
>guarantee anything about the behavior in the presence of cycles, so what's
>the point?
>
>For a while I played around with the idea that maybe we could still support
>'with:' in generators, though, because to implement that we could make
>frames call __exit__ on any pending 'with' blocks as part of their tp_clear
>operation.  This would only work, however, if the objects with __exit__
>methods don't have any references back to the frame.  In essence, you'd
>need a way to put the __exit__ objects on a GC-managed list that wouldn't
>run until after all the tp_clear calls had finished.
>
>But even that is tough to make guarantees about.  For example, can you
>guarantee in that case that a generator's 'with:' blocks are __exit__-ed in
>the proper order?
>
>Really, if we do allow 'with' and 'try-finally' to surround yield, I think
>we're going to have to tell people that it only works if you use a with or
>try-finally in some non-generator code to ensure that the generator.close()
>gets called, and that if you end up creating a garbage cycle, we either
>have to let it end up in gc.garbage, or just not execute its finally clause
>or __exit__ methods.
>
>Of course, this sort of happens right now for other things with __del__; if
>it's part of a cycle the __del__ method never gets called.  The only
>difference is that it hangs around in gc.garbage, doing nothing useful.  If
>it's garbage, it's not reachable from anywhere else, so it does nobody any
>good to have it around.  So, maybe we should just say, "sucks to be you"
>and tp_clear anything that we'd otherwise have put in gc.garbage.  :)
>
>In other words, since we're not going to call those __del__ methods anyway,
>maybe it just needs to be part of the language semantics that __del__ isn't
>guaranteed to be called, and a garbage collector that can't find a safe way
>to call it, doesn't have to.  tp_dealloc for classic classes and heap types
>could then just skip calling __del__ if they've already been cleared... oh
>wait, how do you know you've been cleared?  Argh.  Another nice idea runs
>up on the rocks of reality.
>
>On the other hand, if you go ahead and run __del__ after tp_clear, the
>__del__ method will quickly run afoul of an AttributeError and die with
>only a minor spew to sys.stderr, thus encouraging people to get rid of
>their silly useless __del__ methods on objects that normally end up in
>cycles.  :)
>
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com


From nicksjacobson at yahoo.com  Mon Jun 20 01:22:04 2005
From: nicksjacobson at yahoo.com (Nick Jacobson)
Date: Sun, 19 Jun 2005 16:22:04 -0700 (PDT)
Subject: [Python-Dev] misplaced PEP
Message-ID: <20050619232204.21175.qmail@web53906.mail.yahoo.com>

At the www.python.org/peps page, PEP 281 is
erroneously listed in the "Finished PEPs (done,
implemented in CVS)" section.


		
____________________________________________________ 
Yahoo! Sports 
Rekindle the Rivalries. Sign up for Fantasy Football 
http://football.fantasysports.yahoo.com

From luisa.valbusa at tiscali.it  Sun Jun 19 12:46:00 2005
From: luisa.valbusa at tiscali.it (Luisa)
Date: Sun, 19 Jun 2005 12:46:00 +0200
Subject: [Python-Dev]  Problem with embedded python
Message-ID: <002101c574bc$15c6f040$0100a8c0@LUISA>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20050619/1f1fdaae/attachment.htm

From nicksjacobson at yahoo.com  Mon Jun 20 06:36:45 2005
From: nicksjacobson at yahoo.com (Nick Jacobson)
Date: Sun, 19 Jun 2005 21:36:45 -0700 (PDT)
Subject: [Python-Dev] misplaced PEP
Message-ID: <20050620043645.56407.qmail@web53901.mail.yahoo.com>

Well, it's fixed now.  Thanks to whomever took care of
it.

--- Nick Jacobson <nicksjacobson at yahoo.com> wrote:

> At the www.python.org/peps page, PEP 281 is
> erroneously listed in the "Finished PEPs (done,
> implemented in CVS)" section.
> 
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

From gvanrossum at gmail.com  Mon Jun 20 12:10:26 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon, 20 Jun 2005 03:10:26 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <42B4C134.8010308@gmail.com>
References: <42B35DC8.6040308@gmail.com> <d90hes$17d$1@sea.gmane.org>
	<20050618011502.728D.JCARLSON@uci.edu> <d9112d$vm2$1@sea.gmane.org>
	<42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com>
Message-ID: <ca471dc205062003102595e0c7@mail.gmail.com>

[Nick Coghlan]
> And here we see why I'm such a fan of the term 'deferred expression'
> instead of 'anonymous function'.
> 
> Python's lambda expressions *are* the former, but they are
> emphatically *not* the latter.

Let me emphatically disagree. Your POV is entirely syntactical, which
IMO is a shallow perspective. Semantically, lambdas ARE anonymous
functions, and that's what counts.

Since "deferred expression" is not defined in Python, you can't
reasonably argue about whether that's what lambdas are, but
intuitively for me the term refers to something more lightweight than
lambdas.

Now, whether the use cases for lambdas are better served by anonymous
functions or by something else that might be called deferred
expression, I don't know yet. But as long as we are describing the
present state we should call a spade a spade, etc.

(Alas, no time to follow the rest of this thread -- vacationing with
little connectivity until EuroPython starts next week.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Mon Jun 20 16:06:22 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon, 20 Jun 2005 07:06:22 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
Message-ID: <ca471dc205062007066ee2883e@mail.gmail.com>

[Keith Dart]
> In SNMP, for example, a Counter32 is basically an unsigned int, defined
> as "IMPLICIT INTEGER (0..4294967295)". One cannot efficiently translate
> and use that type in native Python. Currently, I have defined an
> "unsigned" type as a subclass of long, but I don't think that would be
> speed or storage efficient.

In my experience you can just use Python longs whenever a C API needs
an "unsigned" long. There's no need to subtype, and your assumption
that it would not be efficient enough is mistaken (unless you are
manipulating arrays with millions of them, in which case you should be
using Numeric, which has its own types for this purpose). (Want to
argue about the efficiency? Write a typical use case and time it.)

By far the easiest way to do arithmetic mod 2**32 is to just add "&
0xFFFFFFFF" to the end of your expression. For example, simulating the
effect of multiplying an unsigned long by 3 would be x = (x * 3) &
0xFFFFFFFF.

If there is a problem with ioctl() not taking long ints, that would be
a bug in ioctl, not a lacking data type or a problem with long ints.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mcherm at mcherm.com  Mon Jun 20 17:31:58 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon, 20 Jun 2005 08:31:58 -0700
Subject: [Python-Dev] Multiple expression eval in compound if	statement?
Message-ID: <20050620083158.kqfxwkg63qhccc80@login.werra.lunarpages.com>

Gerrit Holl writes:
> What would happen if...

Raymond replies:
> Every molecule in your body would simultaneously implode at the speed of
> light.

So you're saying it triggers C-language "undefined behavior"?

-- Michael Chermside


From mwh at python.net  Mon Jun 20 18:59:19 2005
From: mwh at python.net (Michael Hudson)
Date: Mon, 20 Jun 2005 17:59:19 +0100
Subject: [Python-Dev] Multiple interpreters not compatible with current
 thread module
In-Reply-To: <2m4qby9qzl.fsf@starship.python.net> (Michael Hudson's message
	of "Thu, 16 Jun 2005 18:00:14 +0100")
References: <93dc9c320505170425392e3d80@mail.gmail.com>
	<2m4qby9qzl.fsf@starship.python.net>
Message-ID: <2mzmtl6k2g.fsf@starship.python.net>

Michael Hudson <mwh at python.net> writes:

> I'm not expecting anyone else to think hard about this on recent form,
> so I'll think about it for a bit and then fix it in the way that seems
> best after that.  Feel free to surprise me.

And so that's what I did.  I think I got this right, but threads are
tricky buggers so x-platform testing of CVS HEAD would be appreciated
(heck, that would be appreciated anyway, I don't get the impression
much of it is happening at the moment...).

Cheers,
mwh

-- 
  <moshez> glyph: I don't know anything about reality.
                                                -- from Twisted.Quotes

From jimjjewett at gmail.com  Mon Jun 20 19:42:09 2005
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 20 Jun 2005 13:42:09 -0400
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda
Message-ID: <fb6fbf56050620104219bebec6@mail.gmail.com>

>>>                lambda x,y: x+y*y
>>>                lambda x,y: y**2+x

>>> are essentialy the same functions with different implementations [1].

>> Except that they are not. Think of __pow__, think of __add__ and __radd__.

> You know the difference between the concept of a function and it's 
> infinitely many representations? That's why formal definitions exist.

To clear up the archives, the problem is that Kay and Reinhold were 
talking about different things when they said "function".  

In arithmetic (and most sane extensions), those two lines are 
essentially the same function written different ways.

In python those two lines represent two different functions, because 
x and y might not be (bound to) sane numbers.  The variables could
be bound to objects that redefine the addition, multiplication, and 
exponentiation operators.  So y*y might not be the same as y**2, 
and x+y might not be the same as y+x.  

-jJ

From kdart at kdart.com  Tue Jun 21 05:49:24 2005
From: kdart at kdart.com (Keith Dart)
Date: Mon, 20 Jun 2005 20:49:24 -0700 (PDT)
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
 Integers
In-Reply-To: <ca471dc205062007066ee2883e@mail.gmail.com>
References: <2m1x6z7qu6.fsf@starship.python.net> 
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com> 
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
	<ca471dc205062007066ee2883e@mail.gmail.com>
Message-ID: <Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>

On Mon, 20 Jun 2005, Guido van Rossum wrote:

> [Keith Dart]
>> In SNMP, for example, a Counter32 is basically an unsigned int, defined
>> as "IMPLICIT INTEGER (0..4294967295)". One cannot efficiently translate
>> and use that type in native Python. Currently, I have defined an
>> "unsigned" type as a subclass of long, but I don't think that would be
>> speed or storage efficient.
>
> In my experience you can just use Python longs whenever a C API needs
> an "unsigned" long. There's no need to subtype, and your assumption
> that it would not be efficient enough is mistaken (unless you are
> manipulating arrays with millions of them, in which case you should be
> using Numeric, which has its own types for this purpose). (Want to
> argue about the efficiency? Write a typical use case and time it.)

Ok, I'll take your word for it. I don't have any performance problems now, 
in my usage, but I wanted to make sure that Python "shows well" in certain 
"bake offs" ;-)


> By far the easiest way to do arithmetic mod 2**32 is to just add "&
> 0xFFFFFFFF" to the end of your expression. For example, simulating the
> effect of multiplying an unsigned long by 3 would be x = (x * 3) &
> 0xFFFFFFFF.

But then I wouldn't know if it overflowed 32 bits. In my usage, the 
integer will be translated to an unsigned (32 bit) integer in another 
system (SNMP). I want to know if it will fit, and I want to know early if 
there will be a problem, rather than later (at conversion time).

One of the "selling points" of Python in previous versions was that you 
would get an OverFlowError on overflow, where other languages did not 
(they overflowed silently). So I subclassed long in 2.3, to get the same 
overflow exception:

class unsigned(long):
 	floor = 0L
 	ceiling = 4294967295L
 	bits = 32
 	_mask = 0xFFFFFFFFL
 	def __new__(cls, val):
 		return long.__new__(cls, val)
 	def __init__(self, val):
 		if val < self.floor or val > self.ceiling:
 			raise OverflowError, "value %s out of range for
                                 type %s" % (val, self.__class__.__name__)
 	def __repr__(self):
 		return "%s(%sL)" % (self.__class__.__name__, self)
 	def __add__(self, other):
 		return self.__class__(long.__add__(self, other))
         ...


Again, because I want to catch the error early, before conversion to the 
external type.

BTW, the conversion is done in pure Python (to a BER representation), so 
using a C type (via ctypes, or pyrex, or whatever) is not possible.


> If there is a problem with ioctl() not taking long ints, that would be
> a bug in ioctl, not a lacking data type or a problem with long ints.

That must be it, then. Shall I file a bug somewhere?



-- 

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Keith Dart <kdart at kdart.com>
    public key: ID: F3D288E4
    =====================================================================

From kdart at kdart.com  Tue Jun 21 06:58:24 2005
From: kdart at kdart.com (Keith Dart)
Date: Mon, 20 Jun 2005 21:58:24 -0700 (PDT)
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
 Integers
In-Reply-To: <Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net> 
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com> 
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
	<ca471dc205062007066ee2883e@mail.gmail.com>
	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
Message-ID: <Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>

On Mon, 20 Jun 2005, Keith Dart wrote:

> But then I wouldn't know if it overflowed 32 bits. In my usage, the
> integer will be translated to an unsigned (32 bit) integer in another
> system (SNMP). I want to know if it will fit, and I want to know early if
> there will be a problem, rather than later (at conversion time).
>
> class unsigned(long):

I guess I just clarify this more. My "unsigned" type really is an object
that represents a type of number from the external system. Previously,
there was a nice, clean mapping between external types and Python types.
Now there is not so clean a mapping. Not that that makes it a problem
with Python itself.

However, since it is sometimes necessary to interface to other systems
with Python, I see no reason why Python should not have a full set of
built in numeric types corresponding to the machine types and, in turn,
other system types. Then it would be easier (and probaby a bit faster)
to interface to them. Perhaps Python could have an "integer" type for
long/int unified types, and just "int" type as "normal" integers?




-- 

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Keith Dart <kdart at kdart.com>
    public key: ID: F3D288E4
    =====================================================================

From dmitry at targeted.org  Tue Jun 21 07:25:18 2005
From: dmitry at targeted.org (Dmitry Dvoinikov)
Date: Tue, 21 Jun 2005 11:25:18 +0600
Subject: [Python-Dev] Explicitly declaring expected exceptions for a block
Message-ID: <1355231971.20050621112518@targeted.org>

Excuse me if I couldn't find that in the existing PEPs, but
wouldn't that be useful to have a construct that explicitly
tells that we know an exception of specific type could happen
within a block, like:

----------------------------------

ignore TypeError:
    do stuff
[else:
    do other stuff]

being essintially identical to

try:
    do stuff
except TypeError:
    pass
[else:
    do other stuff]
    
----------------------------------

The reason for that being self-tests with lots and lots of
little code snippets like this:

try:
    c().foo()
except TypeError:
    pass

which could be written as

ignore TypeError: c().foo()

Sincerely,
Dmitry Dvoinikov
http://www.targeted.org/


From paul.dubois at gmail.com  Tue Jun 21 07:56:10 2005
From: paul.dubois at gmail.com (Paul Du Bois)
Date: Mon, 20 Jun 2005 22:56:10 -0700
Subject: [Python-Dev] Explicitly declaring expected exceptions for a
	block
In-Reply-To: <1355231971.20050621112518@targeted.org>
References: <1355231971.20050621112518@targeted.org>
Message-ID: <85f6a31f05062022564619f4f6@mail.gmail.com>

On 6/20/05, Dmitry Dvoinikov <dmitry at targeted.org> wrote:
> Excuse me if I couldn't find that in the existing PEPs, but
> wouldn't that be useful to have a construct that explicitly
> tells that we know an exception of specific type could happen
> within a block, like:

> ignore TypeError:
>     do stuff
> [else:
>     do other stuff]

If I understand PEP 343 correctly, it allows for easy implementation
of part of your request. It doesn't implement the else: clause, but
you don't give a use case for it either.

class ignored_exceptions(object):
   def __init__(self, *exceptions):
       self.exceptions = exceptions
   def __enter__(self):
       return None
   def __exit__(self, type, value, traceback):
       try:
           raise type, value, traceback
       except self.exceptions:
           pass

with ignored_exceptions(SomeError):
   do stuff

I don't see the use, but it's possible.

> The reason for that being self-tests with lots and lots of
> little code snippets like this:

If you're trying to write tests, perhaps a better use-case would be
something like:

with required_exception(SomeError):
   do something that should cause SomeError

paul

From dmitry at targeted.org  Tue Jun 21 08:22:54 2005
From: dmitry at targeted.org (Dmitry Dvoinikov)
Date: Tue, 21 Jun 2005 12:22:54 +0600
Subject: [Python-Dev] Explicitly declaring expected exceptions for a
	block
In-Reply-To: <85f6a31f0506202255128b39e2@mail.gmail.com>
References: <1355231971.20050621112518@targeted.org>
	<85f6a31f0506202255128b39e2@mail.gmail.com>
Message-ID: <1341515918.20050621122254@targeted.org>

> If you're trying to write tests, perhaps a better use-case would be
> something like:

> with required_exception(SomeError):
>     do something that should cause SomeError

Yes, you are right, that'd be a better and more flexible way,
thank you.

Sincerely,
Dmitry Dvoinikov
http://www.targeted.org/

--- Original message follows ---

> On 6/20/05, Dmitry Dvoinikov <dmitry at targeted.org> wrote:
>> Excuse me if I couldn't find that in the existing PEPs, but
>> wouldn't that be useful to have a construct that explicitly
>> tells that we know an exception of specific type could happen
>> within a block, like:

>> ignore TypeError:
>>     do stuff
>> [else:
>>     do other stuff]

> If I understand PEP 343 correctly, it allows for easy implementation
> of part of your request. It doesn't implement the else: clause, but
> you don't give a use case for it either.

> class ignored_exceptions(object):
>     def __init__(self, *exceptions):
>         self.exceptions = exceptions
>     def __enter__(self):
>         return None
>     def __exit__(self, type, value, traceback):
>         try:
>             raise type, value, traceback
>         except self.exceptions:
>             pass

> with ignored_exceptions(SomeError):
>     do stuff

> I don't see the use, but it's possible.

>> The reason for that being self-tests with lots and lots of
>> little code snippets like this:

> If you're trying to write tests, perhaps a better use-case would be
> something like:

> with required_exception(SomeError):
>     do something that should cause SomeError

> paul


From shane at hathawaymix.org  Tue Jun 21 09:49:19 2005
From: shane at hathawaymix.org (Shane Hathaway)
Date: Tue, 21 Jun 2005 01:49:19 -0600
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>	<ca471dc205062007066ee2883e@mail.gmail.com>	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
	<Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>
Message-ID: <42B7C67F.1060304@hathawaymix.org>

Keith Dart wrote:
> I guess I just clarify this more. My "unsigned" type really is an object
> that represents a type of number from the external system. Previously,
> there was a nice, clean mapping between external types and Python types.
> Now there is not so clean a mapping. Not that that makes it a problem
> with Python itself.
> 
> However, since it is sometimes necessary to interface to other systems
> with Python, I see no reason why Python should not have a full set of
> built in numeric types corresponding to the machine types and, in turn,
> other system types. Then it would be easier (and probaby a bit faster)
> to interface to them. Perhaps Python could have an "integer" type for
> long/int unified types, and just "int" type as "normal" integers?

For your purposes, would it work to use the struct module to detect
overflows early?

>>> import struct
>>> struct.pack('i', 2 ** 33)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to int

Another possibility would be to add to the "struct" module a full set of
integer types with a fixed width: int8, uint8, int16, uint16, int32,
uint32, int64, and uint64.  Code that's focused on integration with
other languages might benefit.

Shane

From gvanrossum at gmail.com  Tue Jun 21 12:16:17 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 21 Jun 2005 03:16:17 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
	<ca471dc205062007066ee2883e@mail.gmail.com>
	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
Message-ID: <ca471dc2050621031618063e94@mail.gmail.com>

On 6/20/05, Keith Dart <kdart at kdart.com> wrote:
> On Mon, 20 Jun 2005, Guido van Rossum wrote:
[...]
> > By far the easiest way to do arithmetic mod 2**32 is to just add "&
> > 0xFFFFFFFF" to the end of your expression. For example, simulating the
> > effect of multiplying an unsigned long by 3 would be x = (x * 3) &
> > 0xFFFFFFFF.
> 
> But then I wouldn't know if it overflowed 32 bits.

Huh? C unsigned ints don't flag overflow either -- they perform
perfect arithmetic mod 2**32.

> In my usage, the
> integer will be translated to an unsigned (32 bit) integer in another
> system (SNMP). I want to know if it will fit, and I want to know early if
> there will be a problem, rather than later (at conversion time).

So check if it is >= 2**32 (or < 0, of course).

> One of the "selling points" of Python in previous versions was that you
> would get an OverFlowError on overflow, where other languages did not
> (they overflowed silently). So I subclassed long in 2.3, to get the same
> overflow exception:
>          ...
> Again, because I want to catch the error early, before conversion to the
> external type.

This is a very specialized application. Your best approach is to check
for overflow before passing into the external API -- ideally the
wrappers for that API should do so.

> > If there is a problem with ioctl() not taking long ints, that would be
> > a bug in ioctl, not a lacking data type or a problem with long ints.
> 
> That must be it, then. Shall I file a bug somewhere?

SourceForge. (python.org/dev for more info)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvanrossum at gmail.com  Tue Jun 21 12:19:04 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 21 Jun 2005 03:19:04 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
	<ca471dc205062007066ee2883e@mail.gmail.com>
	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
	<Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>
Message-ID: <ca471dc2050621031946b09bad@mail.gmail.com>

On 6/20/05, Keith Dart <kdart at kdart.com> wrote:
> However, since it is sometimes necessary to interface to other systems
> with Python, I see no reason why Python should not have a full set of
> built in numeric types corresponding to the machine types and, in turn,
> other system types. Then it would be easier (and probaby a bit faster)
> to interface to them. Perhaps Python could have an "integer" type for
> long/int unified types, and just "int" type as "normal" integers?

Strongly disagree.

(a) Stop worrying about speed. The overhead of a single Python
bytecode execution is probably more than the cost of an integer
operation in most cases.

(b) I don't know what you call a "normal" integer any more; to me,
unified long/int is as normal as they come. Trust me, that's the case
for most users. Worrying about 32 bits becomes less and less normal.

(c) The right place to do the overflow checks is in the API wrappers,
not in the integer types.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mcherm at mcherm.com  Tue Jun 21 13:12:23 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue, 21 Jun 2005 04:12:23 -0700
Subject: [Python-Dev] Explicitly declaring expected exceptions for	ablock
Message-ID: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com>

Dmitry Dvoinikov writes:
> The reason for that being self-tests with lots and lots of
> little code snippets like this:
>
> try:
>     c().foo()
> except TypeError:
>     pass

Paul Du Boise already responded explaining that PEP 343 probably handles
the task you want. I just wanted to mention that you may need to
reconsider the task. The above snippet is almost certainly incorrect.
I suspect that you wanted either:

   try:
       c().foo()
       fail('Should have raised TypeError')
   except TypeError:
       pass  # expected

or perhaps this:

    try:
        c().foo()
    except TypeError:
        fail('Should not have raised TypeError')

There ARE situations when you want to allow an exception (but not
necessarily expect it) and do nothing when it occurs, but I don't
find them all that common, and I certainly don't find them arising
in unit tests.

-- Michael Chermside


From dmitry at targeted.org  Tue Jun 21 13:30:00 2005
From: dmitry at targeted.org (Dmitry Dvoinikov)
Date: Tue, 21 Jun 2005 17:30:00 +0600
Subject: [Python-Dev] Explicitly declaring expected exceptions for ablock
In-Reply-To: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com>
References: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com>
Message-ID: <315080598.20050621173000@targeted.org>

> I suspect that you wanted either:

>    try:
>        c().foo()
>        fail('Should have raised TypeError')
>    except TypeError:
>        pass  # expected

Right, of course I use something along these lines:

try:
    c().foo()
except TypeError:
    pass
else:
    assert False, "c.foo() should have thrown TypeError"

and so if foo throws anything but TypeError I get traceback,
otherwise it's an assertion error and is reported as such.
This is probably just one of the many possible wording of the
same thing though.

Sincerely,
Dmitry Dvoinikov
http://www.targeted.org/

--- Original message follows ---

> Dmitry Dvoinikov writes:
>> The reason for that being self-tests with lots and lots of
>> little code snippets like this:
>>
>> try:
>>     c().foo()
>> except TypeError:
>>     pass

> Paul Du Boise already responded explaining that PEP 343 probably handles
> the task you want. I just wanted to mention that you may need to
> reconsider the task. The above snippet is almost certainly incorrect.
> I suspect that you wanted either:

>    try:
>        c().foo()
>        fail('Should have raised TypeError')
>    except TypeError:
>        pass  # expected

> or perhaps this:

>     try:
>         c().foo()
>     except TypeError:
>         fail('Should not have raised TypeError')

> There ARE situations when you want to allow an exception (but not
> necessarily expect it) and do nothing when it occurs, but I don't
> find them all that common, and I certainly don't find them arising
> in unit tests.

> -- Michael Chermside


From dmitry at targeted.org  Tue Jun 21 13:43:16 2005
From: dmitry at targeted.org (Dmitry Dvoinikov)
Date: Tue, 21 Jun 2005 17:43:16 +0600
Subject: [Python-Dev] Explicitly declaring expected exceptions for ablock
In-Reply-To: <315080598.20050621173000@targeted.org>
References: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com>
	<315080598.20050621173000@targeted.org>
Message-ID: <7810152903.20050621174316@targeted.org>

> try:
>     c().foo()
> except TypeError:
>     pass
> else:
>     assert False, "c.foo() should have thrown TypeError"


In fact, the above code actually expects foo to throw
particular exception, not exactly the same as the original
requirement. More of

expect TypeError:
       c().foo()

and still can be implemented under PEP 343, that's what
I meant under "more flexible".

Sincerely,
Dmitry Dvoinikov
http://www.targeted.org/

--- Original message follows ---

>> I suspect that you wanted either:

>>    try:
>>        c().foo()
>>        fail('Should have raised TypeError')
>>    except TypeError:
>>        pass  # expected

> Right, of course I use something along these lines:

> try:
>     c().foo()
> except TypeError:
>     pass
> else:
>     assert False, "c.foo() should have thrown TypeError"

> and so if foo throws anything but TypeError I get traceback,
> otherwise it's an assertion error and is reported as such.
> This is probably just one of the many possible wording of the
> same thing though.

> Sincerely,
> Dmitry Dvoinikov
> http://www.targeted.org/

> --- Original message follows ---

>> Dmitry Dvoinikov writes:
>>> The reason for that being self-tests with lots and lots of
>>> little code snippets like this:
>>>
>>> try:
>>>     c().foo()
>>> except TypeError:
>>>     pass

>> Paul Du Boise already responded explaining that PEP 343 probably handles
>> the task you want. I just wanted to mention that you may need to
>> reconsider the task. The above snippet is almost certainly incorrect.
>> I suspect that you wanted either:

>>    try:
>>        c().foo()
>>        fail('Should have raised TypeError')
>>    except TypeError:
>>        pass  # expected

>> or perhaps this:

>>     try:
>>         c().foo()
>>     except TypeError:
>>         fail('Should not have raised TypeError')

>> There ARE situations when you want to allow an exception (but not
>> necessarily expect it) and do nothing when it occurs, but I don't
>> find them all that common, and I certainly don't find them arising
>> in unit tests.

>> -- Michael Chermside

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dmitry%40targeted.org


From ncoghlan at gmail.com  Tue Jun 21 16:51:07 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jun 2005 00:51:07 +1000
Subject: [Python-Dev] Explicitly declaring expected exceptions for
	a	block
In-Reply-To: <85f6a31f05062022564619f4f6@mail.gmail.com>
References: <1355231971.20050621112518@targeted.org>
	<85f6a31f05062022564619f4f6@mail.gmail.com>
Message-ID: <42B8295B.8030107@gmail.com>

Paul Du Bois wrote:
> If I understand PEP 343 correctly, it allows for easy implementation
> of part of your request. It doesn't implement the else: clause, but
> you don't give a use case for it either.
> 
> class ignored_exceptions(object):
>    def __init__(self, *exceptions):
>        self.exceptions = exceptions
>    def __enter__(self):
>        return None
>    def __exit__(self, type, value, traceback):
>        try:
>            raise type, value, traceback
>        except self.exceptions:
>            pass
> 
> with ignored_exceptions(SomeError):
>    do stuff
> 
> I don't see the use, but it's possible.

It was possible in PEP 340 and in early drafts of PEP 346, but it 
isn't possible in PEP 343.

In PEP 343, the statement template *cannot* suppress exceptions - it 
can react to them, and it can turn them into different exceptions, but 
that's all.

And yes, this is deliberate - the control flow is too murky otherwise:

   with some_template():
       raise TypeError
   print "Hi!"

Does that code print "Hi!" or not? Under PEP 343, you know it doesn't, 
because the TypeError isn't trapped. If templates could actually 
suppress exceptions, you'd have no idea what the code does without 
looking up the definition of 'some_template' - this is a bad thing, 
which is why PEP 343 defines the semantics the way it does.

However, I'm with Michael - every time I've needed something like 
this, I have had non-trivial code in either the 'except' or the 'else' 
blocks of the try statement.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From ncoghlan at gmail.com  Tue Jun 21 17:13:01 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jun 2005 01:13:01 +1000
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <ca471dc205062003102595e0c7@mail.gmail.com>
References: <42B35DC8.6040308@gmail.com> <d90hes$17d$1@sea.gmane.org>	
	<20050618011502.728D.JCARLSON@uci.edu>
	<d9112d$vm2$1@sea.gmane.org>	
	<42B52343.3050902@minkirri.apana.org.au>
	<42B4C134.8010308@gmail.com>
	<ca471dc205062003102595e0c7@mail.gmail.com>
Message-ID: <42B82E7D.4070906@gmail.com>

Guido van Rossum wrote:
> [Nick Coghlan]
> 
>>And here we see why I'm such a fan of the term 'deferred expression'
>>instead of 'anonymous function'.
>>
>>Python's lambda expressions *are* the former, but they are
>>emphatically *not* the latter.
> 
> Let me emphatically disagree. Your POV is entirely syntactical, which
> IMO is a shallow perspective. Semantically, lambdas ARE anonymous
> functions, and that's what counts.

Interesting. Knowing this, I think I better understand your desire to 
get rid of lambda expressions for Py3K.

> Since "deferred expression" is not defined in Python, you can't
> reasonably argue about whether that's what lambdas are, but
> intuitively for me the term refers to something more lightweight than
> lambdas.

As you say, it is a matter of looking at lambdas based on what the 
current syntax restricts them to (i.e. single expressions), rather 
than what the underlying machinery is capable of (i.e. full-fledged 
functions).

> Now, whether the use cases for lambdas are better served by anonymous
> functions or by something else that might be called deferred
> expression, I don't know yet. 

Like you (judging by your stated goals for Py3K), I don't have any use 
cases for full anonymous functions - named functions serve that role 
quite happily for me.

Where I find lambda expressions useful is the role that Python's 
current syntax restricts them too - functions which consist of a 
single (usually simple) expression. For those, pulling the expression 
out and naming it would just end up hiding the meaningful sections of 
code behind a few pieces of boilerplate

 > But as long as we are describing the
 > present state we should call a spade a spade, etc.

I guess I take a syntactic view of the status quo, because, while 
lambdas may be implemented as anonymous functions, the current syntax 
doesn't let me *write* an arbitrary function as a lambda.

Regardless, I believe the balance will eventually tip in some 
direction - either lambdas disappear entirely, become able support 
full anonymous functions, or else the idea of a 'deferred expression' 
becomes a defining characteristic, rather than a syntactic quirk.

I figure it's all Py3K type stuff though, without any great urgency 
associated with it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From rrr at ronadam.com  Tue Jun 21 18:14:49 2005
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 21 Jun 2005 12:14:49 -0400
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>	<ca471dc205062007066ee2883e@mail.gmail.com>	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
	<Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>
Message-ID: <42B83CF9.2010908@ronadam.com>

Keith Dart wrote:
> On Mon, 20 Jun 2005, Keith Dart wrote:
> 
> 
>>But then I wouldn't know if it overflowed 32 bits. In my usage, the
>>integer will be translated to an unsigned (32 bit) integer in another
>>system (SNMP). I want to know if it will fit, and I want to know early if
>>there will be a problem, rather than later (at conversion time).
>>
>>class unsigned(long):
> 
> 
> I guess I just clarify this more. My "unsigned" type really is an object
> that represents a type of number from the external system. Previously,
> there was a nice, clean mapping between external types and Python types.
> Now there is not so clean a mapping. Not that that makes it a problem
> with Python itself.
> 
> However, since it is sometimes necessary to interface to other systems
> with Python, I see no reason why Python should not have a full set of
> built in numeric types corresponding to the machine types and, in turn,
> other system types. Then it would be easier (and probaby a bit faster)
> to interface to them. Perhaps Python could have an "integer" type for
> long/int unified types, and just "int" type as "normal" integers?

It seems to me, that maybe a single "byte_buffer" type, that can be 
defined to the exact needed byte lengths and have possible other 
characteristics to aid in interfacing to other languages or devices, 
would be a better choice.

Then pythons ints, floats, etc... can uses what ever internal lengths is 
most efficient for the system it's compiled on and then the final result 
can be stored in the 'byte_buffer' for interfacing purposes.

It would also be a good choice for bit manipulation when someone needs 
that, instead of trying to do it in an integer.

Would something like that fulfill your need?

Regards, Ron


From python at rcn.com  Wed Jun 22 00:08:07 2005
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 21 Jun 2005 18:08:07 -0400
Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops
In-Reply-To: <7592058.1119361866@dhcp31-171.ics.uci.edu>
Message-ID: <007b01c576ad$b46d2a00$d922c797@oemcomputer>

[Raymond]
> > The above recommendations should get the PEP ready for judgement
day.

[David Eppstein]
> I thought judgement day already happened for this PEP in the "Parade
of
> PEPs".  No?

The parade's text said the main gripe was having the index in the
middle, rather than right after the keyword.

However, even after addressed the gripe, Guido still hated it.  So, the
PEP is now dead as a doornail.


Raymond

From kdart at kdart.com  Wed Jun 22 00:18:13 2005
From: kdart at kdart.com (Keith Dart)
Date: Tue, 21 Jun 2005 15:18:13 -0700 (PDT)
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
 Integers
In-Reply-To: <ca471dc2050621031618063e94@mail.gmail.com>
References: <2m1x6z7qu6.fsf@starship.python.net> 
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com> 
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
	<ca471dc205062007066ee2883e@mail.gmail.com>
	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
	<ca471dc2050621031618063e94@mail.gmail.com>
Message-ID: <Pine.LNX.4.63.0506211442360.22846@leviathan.kdart.com>

On Tue, 21 Jun 2005, Guido van Rossum wrote:
  [two messages mixed]

> Huh? C unsigned ints don't flag overflow either -- they perform
> perfect arithmetic mod 2**32.

I was talking about signed ints. Sorry about the confusion. Other
scripting languages (e.g. perl) do not error on overflow.

>
>> In my usage, the
>> integer will be translated to an unsigned (32 bit) integer in another
>> system (SNMP). I want to know if it will fit, and I want to know early if
>> there will be a problem, rather than later (at conversion time).
>
> So check if it is >= 2**32 (or < 0, of course).

That's exactly what I do. ;-) The "unsigned" subclass of long is part of
the API, and checks the range when it is created (and they get created
implicitly when operated on).


> (a) Stop worrying about speed. The overhead of a single Python
> bytecode execution is probably more than the cost of an integer
> operation in most cases.

I am not thinking of the integer operation, but the extra Python
bytecode necessary to implement the extra checks for overflow.


>> Again, because I want to catch the error early, before conversion to the
>> external type.
>
> This is a very specialized application. Your best approach is to check
> for overflow before passing into the external API -- ideally the
> wrappers for that API should do so.

> (c) The right place to do the overflow checks is in the API wrappers,
> not in the integer types.

That would be the "traditional" method.

I was trying to keep it an object-oriented API. What should "know" the 
overflow condition is the type object itself. It raises OverFlowError any 
time this occurs, for any operation, implicitly. I prefer to catch errors 
earlier, rather than later.


> (b) I don't know what you call a "normal" integer any more; to me,
> unified long/int is as normal as they come. Trust me, that's the case
> for most users. Worrying about 32 bits becomes less and less normal.

By "normal" integer I mean the mathematical definition. Most Python users 
don't have to worry about 32 bits now, that is a good thing when you are 
dealing only with Python. However, if one has to interface to other 
systems that have definite types with limits, then one must "hack around" 
this feature. I was just thinking how nice it would be if Python had, in 
addition to unified ("real", "normal") integers it also had built-in
types that could be more easily mapped to external types (the typical
set of signed, unsigned, short, long, etc.).Yes, you can check it at
conversion time, but that would mean extra Python bytecode. It seems you
think this is a special case, but I think Python may be used as a "glue
language" fairly often, and some of us would benefit from having those
extra types as built-ins.



-- 

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Keith Dart <kdart at kdart.com>
    public key: ID: F3D288E4
    =====================================================================

From kdart at kdart.com  Wed Jun 22 00:23:20 2005
From: kdart at kdart.com (Keith Dart)
Date: Tue, 21 Jun 2005 15:23:20 -0700 (PDT)
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
 Integers
In-Reply-To: <42B83CF9.2010908@ronadam.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
	<ca471dc205062007066ee2883e@mail.gmail.com>
	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
	<Pine.LNX.4.63.0506202148070.12294@leviathan.kdart.com>
	<42B83CF9.2010908@ronadam.com>
Message-ID: <Pine.LNX.4.63.0506211518440.24098@leviathan.kdart.com>

On Tue, 21 Jun 2005, Ron Adam wrote:

> It seems to me, that maybe a single "byte_buffer" type, that can be
> defined to the exact needed byte lengths and have possible other
> characteristics to aid in interfacing to other languages or devices,
> would be a better choice.
>
> Then pythons ints, floats, etc... can uses what ever internal lengths is
> most efficient for the system it's compiled on and then the final result
> can be stored in the 'byte_buffer' for interfacing purposes.
>
> It would also be a good choice for bit manipulation when someone needs
> that, instead of trying to do it in an integer.
>
> Would something like that fulfill your need?



Sounds interresting. Not exactly stright-forward. What i have now is 
functional, but if speed  becomes a problem then this might be useful.





-- 

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Keith Dart <kdart at kdart.com>
    public key: ID: F3D288E4
    =====================================================================

From dmitry at targeted.org  Wed Jun 22 06:09:49 2005
From: dmitry at targeted.org (Dmitry Dvoinikov)
Date: Wed, 22 Jun 2005 10:09:49 +0600
Subject: [Python-Dev] Explicitly declaring expected exceptions for a
	block
In-Reply-To: <42B8295B.8030107@gmail.com>
References: <1355231971.20050621112518@targeted.org>
	<85f6a31f05062022564619f4f6@mail.gmail.com>
	<42B8295B.8030107@gmail.com>
Message-ID: <1259444531.20050622100949@targeted.org>

> It was possible in PEP 340 and in early drafts of PEP 346, but it
> isn't possible in PEP 343.

> In PEP 343, the statement template *cannot* suppress exceptions - it
> can react to them, and it can turn them into different exceptions, but
> that's all.

...doing homework...

The following code, combined from PEP 343 and Mr. Du Bois suggestion:

--------------------------------------------------------------

from sys import exc_info

class ignored_exceptions(object):
        def __init__(self, *exceptions):
                self.exceptions = exceptions
        def __enter__(self):
                return None
        def __exit__(self, type, value, traceback):
                try:
                        raise type, value, traceback
                except self.exceptions:
                        pass

try:
        with = ignored_exceptions(TypeError)
        with.__enter__()
        try:
                try:
                        raise TypeError()
                except:
                        exc = exc_info()
                        raise
        finally:
                with.__exit__(*exc)
except:
        print exc_info()

--------------------------------------------------------------

still yields exceptions.TypeError.

Now, back to original question then, do you think it'd be
beneficial to have some sort of exception ignoring or expecting
statement ?

Sincerely,
Dmitry Dvoinikov
http://www.targeted.org/

--- Original message follows ---

> Paul Du Bois wrote:
>> If I understand PEP 343 correctly, it allows for easy implementation
>> of part of your request. It doesn't implement the else: clause, but
>> you don't give a use case for it either.
>> 
>> class ignored_exceptions(object):
>>    def __init__(self, *exceptions):
>>        self.exceptions = exceptions
>>    def __enter__(self):
>>        return None
>>    def __exit__(self, type, value, traceback):
>>        try:
>>            raise type, value, traceback
>>        except self.exceptions:
>>            pass
>> 
>> with ignored_exceptions(SomeError):
>>    do stuff
>> 
>> I don't see the use, but it's possible.

> It was possible in PEP 340 and in early drafts of PEP 346, but it 
> isn't possible in PEP 343.

> In PEP 343, the statement template *cannot* suppress exceptions - it
> can react to them, and it can turn them into different exceptions, but
> that's all.

> And yes, this is deliberate - the control flow is too murky otherwise:

>    with some_template():
>        raise TypeError
>    print "Hi!"

> Does that code print "Hi!" or not? Under PEP 343, you know it doesn't,
> because the TypeError isn't trapped. If templates could actually 
> suppress exceptions, you'd have no idea what the code does without 
> looking up the definition of 'some_template' - this is a bad thing, 
> which is why PEP 343 defines the semantics the way it does.

> However, I'm with Michael - every time I've needed something like 
> this, I have had non-trivial code in either the 'except' or the 'else'
> blocks of the try statement.

> Regards,
> Nick.



From kbk at shore.net  Wed Jun 22 06:55:15 2005
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed, 22 Jun 2005 00:55:15 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200506220455.j5M4tFZf007346@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  338 open ( +0) /  2866 closed ( +5) /  3204 total ( +5)
Bugs    :  914 open ( +5) /  5060 closed (+13) /  5974 total (+18)
RFE     :  188 open ( +0) /   170 closed ( +0) /   358 total ( +0)

New / Reopened Patches
______________________

update the binhex module for Mach-O  (2005-06-14)
       http://python.org/sf/1220874  opened by  Bob Ippolito

ftplib storbinary callback  (2005-06-15)
       http://python.org/sf/1221598  opened by  Phil Schwartz

C++ compilation support for distutils  (2005-06-17)
       http://python.org/sf/1222585  opened by  Antti Honkela

PEP 342/343 Generator enhancements  (2005-06-18)
       http://python.org/sf/1223381  opened by  Phillip J. Eby

HP-UX ia64 64-bit Python executable  (2005-06-21)
       http://python.org/sf/1225212  opened by  Peter Kropf

Patches Closed
______________

_csv.c leaks when dialect creation fails  (2005-06-14)
       http://python.org/sf/1220242  closed by  montanaro

test_curses bugs  (2004-08-09)
       http://python.org/sf/1005892  closed by  akuchling

no html file for modulefinder  (2005-04-10)
       http://python.org/sf/1180012  closed by  birkenfeld

Restore GC support to set objects  (2005-05-11)
       http://python.org/sf/1200018  closed by  rhettinger

change recall in IDLE shell to not overwrite current command  (2005-05-06)
       http://python.org/sf/1196917  closed by  kbk

New / Reopened Bugs
___________________

buffer() object broken.  (2005-06-15)
CLOSED http://python.org/sf/1221424  opened by  Ray Schumacher

Incorrect documentation for InPlace Number operators  (2005-06-15)
CLOSED http://python.org/sf/1221477  opened by  Daniel Stutzbach

float() not accurate  (2005-06-16)
CLOSED http://python.org/sf/1222098  opened by  Brian Dols

Bad optparse help wrapping with multiple paragraphs  (2005-06-16)
       http://python.org/sf/1222235  opened by  Barry A. Warsaw

Typo in section 5.3 of tutorial  (2005-06-17)
CLOSED http://python.org/sf/1222459  opened by  uman

tk + setlocale problems...  (2005-06-17)
       http://python.org/sf/1222721  opened by  Alex A. Naanou

SimpleXMLRPCServer does not set FD_CLOEXEC  (2005-06-17)
       http://python.org/sf/1222790  opened by  Winfried Harbecke

it's -> its  (2005-06-17)
CLOSED http://python.org/sf/1222928  opened by  Ed Swierk

2.4.1 .msi file won't install to different disk  (2005-06-17)
CLOSED http://python.org/sf/1222978  opened by  Ray Trent

race in os.makedirs()  (2005-06-18)
       http://python.org/sf/1223238  opened by  Mattias Engdeg?rd

subprocess.py abuse of errno  (2005-06-20)
       http://python.org/sf/1223937  reopened by  astrand

subprocess.py abuse of errno  (2005-06-20)
       http://python.org/sf/1223937  opened by  Oren Tirosh

error locale.getlocale() with LANGUAGE=eu_ES  (2005-06-20)
       http://python.org/sf/1223976  opened by  Zunbeltz Izaola

Len too large with national characters  (2005-06-20)
CLOSED http://python.org/sf/1224047  opened by  Henrik Winther Jensen

int/long unification and hex()  (2005-06-20)
       http://python.org/sf/1224347  opened by  Josiah Carlson

tokenize module does not detect inconsistent dedents  (2005-06-21)
CLOSED http://python.org/sf/1224621  opened by  Danny Yoo

rsplit introduces spurious CR  (2005-06-21)
       http://python.org/sf/1224672  opened by  Richard Prosser

Line endings problem with Python 2.4.1 cause imports to fail  (2005-06-21)
       http://python.org/sf/1225059  opened by  Nicolas Lehuen

inspect.isclass() fails with custom __getattr__  (2005-06-21)
       http://python.org/sf/1225107  opened by  Jeremy Kloth

Bugs Closed
___________

float issue for NaN type in .pyc file  (2004-12-07)
       http://python.org/sf/1080440  closed by  mwh

buffer() object broken.  (2005-06-15)
       http://python.org/sf/1221424  closed by  nascheme

Incorrect documentation for InPlace Number operators  (2005-06-15)
       http://python.org/sf/1221477  closed by  rhettinger

Replace MSVC memory allocator with ptmalloc2  (2005-06-07)
       http://python.org/sf/1216562  closed by  loewis

Sub threads execute in restricted mode  (2005-03-15)
       http://python.org/sf/1163563  closed by  mwh

float() not accurate  (2005-06-16)
       http://python.org/sf/1222098  closed by  nascheme

Typo in section 5.3 of tutorial  (2005-06-17)
       http://python.org/sf/1222459  closed by  rhettinger

it's -> its  (2005-06-17)
       http://python.org/sf/1222928  closed by  nascheme

2.4.1 .msi file won't install to different disk  (2005-06-17)
       http://python.org/sf/1222978  closed by  hacksoncode

spurious blank page in dist.pdf  (2005-05-27)
       http://python.org/sf/1209560  closed by  birkenfeld

subprocess.py abuse of errno  (2005-06-20)
       http://python.org/sf/1223937  closed by  astrand

Len too large with national characters  (2005-06-20)
       http://python.org/sf/1224047  closed by  mwh

tokenize module does not detect inconsistent dedents  (2005-06-21)
       http://python.org/sf/1224621  closed by  rhettinger

subprocess call() helper should close stdin if PIPE  (2005-06-14)
       http://python.org/sf/1220113  closed by  astrand


From jcarlson at uci.edu  Wed Jun 22 07:16:34 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 21 Jun 2005 22:16:34 -0700
Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit
	Lambda
In-Reply-To: <42B82E7D.4070906@gmail.com>
References: <ca471dc205062003102595e0c7@mail.gmail.com>
	<42B82E7D.4070906@gmail.com>
Message-ID: <20050621221018.44F9.JCARLSON@uci.edu>


Nick Coghlan wrote:
> 
> Guido van Rossum wrote:
> > But as long as we are describing the
> > present state we should call a spade a spade, etc.
> 
> I guess I take a syntactic view of the status quo, because, while 
> lambdas may be implemented as anonymous functions, the current syntax 
> doesn't let me *write* an arbitrary function as a lambda.

You can write anything as a lambda, but it may not be easy.


> Regardless, I believe the balance will eventually tip in some 
> direction - either lambdas disappear entirely, become able support 
> full anonymous functions, or else the idea of a 'deferred expression' 
> becomes a defining characteristic, rather than a syntactic quirk.

I would put my money on the latter rather than the former.  The moment
functions start moving beyond a line or so is when they usually start
begging for a name.

 - Josiah


From stuart at stuartbishop.net  Wed Jun 22 08:20:02 2005
From: stuart at stuartbishop.net (Stuart Bishop)
Date: Wed, 22 Jun 2005 16:20:02 +1000
Subject: [Python-Dev] subprocess.call() and stdin
Message-ID: <42B90312.5070809@stuartbishop.net>

Redirecting to python-dev for discussion.

When I invoke subprocess.call(), I often want to ensure that the subprocess'
stdin is closed. This ensures it will die if the subprocess attempts to read
from stdin rather than block.

This could be done if the subprocess.call() helper closes the input if
stdin=subprocess.PIPE, which may be the only sane way to handle this
argument (I can't think of any use cases for spawning a subprocess with an
input stream that nothing can write too).

It could also be done by adding a subprocess.CLOSED constant, which if
passed to Popen causes a new closed file descriptor to be given to the
subprocess.


-------- Original Message --------

Bugs item #1220113, was opened at 2005-06-14 15:04
Message generated for change (Comment added) made by zenzen
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Closed
Resolution: Rejected
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Peter ?strand (astrand)
Summary: subprocess call() helper should close stdin if PIPE

Initial Comment:
The following code snippet should die instead of hang.

>>> from subprocess import call, PIPE
>>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE,
stdin=PIPE)

It makes no sense not to close stdin if it is PIPE
because the stream cannot be accessed.

The use case for this is ensuring a subprocess that
detects if it is connected to a terminal or not runs in
'batch' mode, and that it will die instead of hang if
it unexpectidly attempts to read from stdin.

Workaround is to use Popen instead.

----------------------------------------------------------------------

>Comment By: Stuart Bishop (zenzen)
Date: 2005-06-22 16:12

Message:
Logged In: YES
user_id=46639

I can't think of any uses cases for wanting to create an
inaccessible pipe and give it to the child.

Wanting to pass a closed file handle is common. It is needed
when calling a program that behaves differently if its stdin
is a terminal or not. Or when you simply would prefer the
subprocess to die if it attempts to read from its stdin
rather than block.

Using Popen instead of call is s simpler workaround than
creating and closing a file descriptor and passing it in.

Perhaps what is needed is a new constant, subprocess.CLOSED
which creates a new file descriptor and closes it? This
would be useful for Popen too, allowing call() to remain a
think and trivially documented wrapper?

----------------------------------------------------------------------

Comment By: Peter ?strand (astrand)
Date: 2005-06-22 02:08

Message:
Logged In: YES
user_id=344921

>It makes no sense not to close stdin if it is PIPE
>because the stream cannot be accessed

True, but what if you actually *want* to create an
inaccessible pipe, and give it to the child?

Currently, the call() wrapper is *very* short and simple. I
think this is very good. For example, it allows us to
document it in two sentences. You get what you ask for: If
you use call() with strange arguments, you'll get a somewhat
strange behav?our. I see no point in introducing lots of
sanity checks in the wrapper functions.

>The use case for this is ensuring a subprocess that
>detects if it is connected to a terminal or not runs in
>batch' mode, and that it will die instead of hang if
>it unexpectidly attempts to read from stdin

I'm not sure I understand what you want, but if you want to
have stdin connected to a closed file descriptor, just pass one:

>>> from subprocess import call, PIPE
>>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE,
stdin=4711)

(Of course, you should verify that 4711 is unused.)

If you don't agree with me, post to python-dev for discussion.

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_id=5470

-- 
Stuart Bishop <stuart at stuartbishop.net>
http://www.stuartbishop.net/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20050622/8173968d/signature-0001.pgp

From fredrik at pythonware.com  Wed Jun 22 10:48:47 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 22 Jun 2005 10:48:47 +0200
Subject: [Python-Dev] PEP for RFE 46738 (first draft)
References: <4e4a11f805061723192888915e@mail.gmail.com><7168d65a0506180644367753b9@mail.gmail.com><4e4a11f805061807127f39700b@mail.gmail.com><17076.21429.501499.914085@montanaro.dyndns.org><4e4a11f805061817462238af59@mail.gmail.com>
	<17076.61341.558478.28695@montanaro.dyndns.org>
Message-ID: <d9b89s$2v6$1@sea.gmane.org>

Skip Montanaro wrote:

> That's fine, so XML-RPC is slower than Gherkin.  I can't run the Gherkin
> code, but my XML-RPC numbers are a bit different than yours:
>
>     XMLRPC encode 0.65 seconds
>     XMLRPC decode 2.61 seconds
>
> That leads me to believe you're not using any sort of C XML decoder.  (I
> mentioned sgmlop in my previous post.  I'm sure /F has some other
> super-duper accelerator that's even faster.)

the CET/iterparse-based decoder described here

     http://effbot.org/zone/element-iterparse.htm

is 3-4 times faster (but I don't recall if I used sgmlop or just plain
expat in those tests).

</F>




From t-meyer at ihug.co.nz  Wed Jun 22 12:54:26 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Wed, 22 Jun 2005 22:54:26 +1200
Subject: [Python-Dev] python-dev Summary for 2005-05-16 through 2005-05-31
	[draft]
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E801B0F788@its-xchg4.massey.ac.nz>

You may have noticed that the summaries have been absent for the last month
- apologies for that; Steve has been dutifully doing his part, but I've been
caught up with other things.

Anyway, Steve will post the May 01-15 draft shortly, and here's May 16-31.
We should be able to get the first June one done fairly shortly, too.

If anyone has time to flick over this and let me/Steve/Tim know if you have
corrections, that would be great; thanks!

=Tony.Meyer

=============
Announcements
=============

----
QOTF
----

We have our first ever Quote of the Fortnight (QOTF), thanks to
the wave of discussion over `PEP 343`_ and Jack Diederich:

    I still haven't gotten used to Guido's heart-attack inducing early
    enthusiasm for strange things followed later by a simple
    proclamation I like.  Some day I'll learn that the sound of
    fingernails on the chalkboard is frequently followed by candy for
    the whole class.

See, even threads about anonymous block statements can end happily! ;)

.. _PEP 343: http://www.python.org/peps/pep-0343.html

Contributing thread:

- `PEP 343 - Abstract Block Redux
<http://mail.python.org/pipermail/python-dev/2005-May/053828.html>`__

[SJB]

------------------
First PyPy Release
------------------

The first release of `PyPy`_, the Python implementation of Python, is
finally available. The PyPy team has made impressive progress, and
the current release of PyPy now passes around 90% of the Python
language regression tests that do not depend deeply on C-extensions.
The PyPy interpreter still runs on top of a CPython interpreter
though, so it is still quite slow due to the double-interpretation
penalty.

.. _PyPy: http://codespeak.net/pypy

Contributing thread:

- `First PyPy (preview) release
<http://mail.python.org/pipermail/python-dev/2005-May/053964.html>`__


[SJB]

--------------------------------
Thesis: Type Inference in Python
--------------------------------

Brett C. successfully defended his masters thesis `Localized Type
Inference of Atomic Types in Python`_, which investigates some of the
issues of applying type inference to the current Python language, as
well as to the Python language augmented with type annotations.
Congrats Brett!

.. _Localized Type Inference of Atomic Types in Python:
http://www.drifty.org/thesis.pdf

Contributing thread:

- `Localized Type Inference of Atomic Types in Python
<http://mail.python.org/pipermail/python-dev/2005- May/053993.html>`__


[SJB]

=========
Summaries
=========

---------------------------
PEP 343 and With Statements
---------------------------

The discussion on "anonymous block statements" brought itself closer
to a real conclusion this fortnight, with the discussion around
`PEP 343`_ and `PEP 3XX`_ converging not only on the semantics for
"with statements", but also on semantics for using generators as
with-statement templates.

To aid in the adaptation of generators to with-statements, Guido
proposed adding close() and throw() methods to generator objects,
similar to the ones suggested by `PEP 325`_ and `PEP 288`_. The
throw() method would cause an exception to be raised at the point
where the generator is currently suspended, and the close() method
would use throw() to signal the generator to clean itself up by
raising a GeneratorExit exception.

People seemed generally happy with this proposal and -- believe it or
not -- we actually went an entire eight days without an email about
anonymous block statements!! It looked as if an updated `PEP 343`_,
including the new generator functionality, would be coming early the
next month. So stay tuned. ;)

.. _PEP 288: http://www.python.org/peps/pep-0288.html

.. _PEP 325: http://www.python.org/peps/pep-0325.html

.. _PEP 343: http://www.python.org/peps/pep-0343.html

.. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html

Contributing threads:

- `PEP 343 - Abstract Block Redux
<http://mail.python.org/pipermail/python-dev/2005-May/053731.html>`__
- `Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
<http://mail.python.org/pipermail/python -dev/2005-May/053812.html>`__
- `Example for PEP 343
<http://mail.python.org/pipermail/python-dev/2005-May/053855.html>`__
- `Combining the best of PEP 288 and PEP 325: generator exceptions and
cleanup
<http://mail.python.org/pipermail/python-dev/2005-May/053885.html>`__
- `PEP 343 - New kind of yield statement?
<http://mail.python.org/pipermail/python-dev/2005-May/053905.html>`__
- `PEP 342/343 status?
<http://mail.python.org/pipermail/python-dev/2005-May/054007.html>`__
- `PEP 346: User defined statements (formerly known as PEP 3XX)
<http://mail.python.org/pipermail/python-dev/2005- May/054014.html>`__

[SJB]

-----------
Decimal FAQ
-----------

Raymond Hettinger suggested that a decimal FAQ would shorten the module's
learning curve, and drafted one.  There  were no objections, but few
adjustments (to the list, at least).  Raymond will probably make the FAQ
available at  some point.

Contributing thread:

- `Decimal FAQ
<http://mail.python.org/pipermail/python-dev/2005-May/053982.html>`__

[TAM]

---------------------
Constructing Decimals
---------------------

A long discussion took place regarding whether the decimal constructor
should or should not respect context  settings, and whether matching the
standard (and what the standard says) should be a priority.  Raymond
Hettinger  took the lead in the status-quo (does not) corner, with Tim
Peters leading the opposition.  Tim and Guido eventually  called in the
standard's expert, Mike Cowlishaw.  He gave a very thorough explanation of
the history behind his  decisions in this matter, and eventually weighed in
on Raymond's side.  As such, it seems that the status-quo has  won (not that
it was a competition, of course <wink>).

For those that need to know, the unary plus operation, as strange as it
looks, forces a rounding using the current  context.  As such, context-aware
construction can be written::

    val = +Decimal(string_repr)

Contributing threads:

- `Adventures with Decimal
<http://mail.python.org/pipermail/python-dev/2005-May/053879.html>`__
- `Decimal construction
<http://mail.python.org/pipermail/python-dev/2005-May/053882.html>`__
- `[Python-checkins] python/nondist/peps pep-0343.txt, 1.8, 1.9
<http://mail.python.org/pipermail/python-dev/2005- May/053766.html>`__

[TAM]

------------------------
Handling old bug reports
------------------------

Facundo Batista continued with his progress checking the open bug reports,
looking for bugs that are specific to  2.2.1 or 2.2.2.  The aim is to verify
whether these bugs exist in current CVS, or are old-of-date.  There are no
longer any bugs in the 2.1.x or 2.2.x categories, and Facundo wondered
whether removing those categories would be a  good idea.  The consensus was
that there was no harm in leaving the categories there, but that changing
the text to  indicate that those versions are unmaintained would be a good
idea.

Raymond Hettinger reminded us that care needs to be taken in closing old bug
reports.  Particularly, a bug report  should only be closed if (a) there are
no means of reproducing the error, (b) it is impossible to tell what the
poster meant, and they are no longer contactable, or (c) the bug is no
longer present in current CVS.


Contributing threads:

- `Deprecating old bugs, now from 2.2.2
<http://mail.python.org/pipermail/python-dev/2005-May/054019.html>`__
- `Closing old bugs
<http://mail.python.org/pipermail/python-dev/2005-May/054031.html>`__
- `Old Python version categories in Bug Tracker
<http://mail.python.org/pipermail/python-dev/2005- May/054020.html>`__

[TAM]

------------------
Exception chaining
------------------

Ka-Ping Yee has submitted `PEP 344`_, which is a concrete proposal for
exception chaining.  It proposes three  standard attributes on trackback
objects: __context__ for implicit chaining, __cause__ for explicit chaining,
and  __traceback__ to point to the traceback.  Guido likes the motivation
and rationale, but feels that the  specification needs more work.  A lot of
discussion about the specifics of the PEP took place, and Ka-Ping is
working these into a revised version.

One of the major questions was whether there is no need for both __context__
and __cause__ (to differentiate between  explicit and implicit chaining).
Guido didn't feel that there was, but others disagreed.

Discussion branched off into whether which attributes should be
double-underscored, or not.  Guido's opinion is that  it depends who "owns"
the namespace, and with "magic" behaviour caused (or indicated) by the
presence of the  attribute.  He felt that the underscores in the proposed
exception attributes should remain.
        

.. _PEP 344: http://www.python.org/peps/pep-0344.html

Contributing threads:

- `PEP 344: Exception Chaining and Embedded Tracebacks
<http://mail.python.org/pipermail/python-dev/2005- May/053821.html>`__
- `PEP 344: Implicit Chaining Semantics
<http://mail.python.org/pipermail/python-dev/2005-May/053944.html>`__
[Python-Dev] PEP 344: Implicit Chaining Semantics
- `PEP 344: Explicit vs. Implicit Chaining
<http://mail.python.org/pipermail/python-dev/2005-May/053945.html>`__
[Python-Dev] PEP 344: Explicit vs. Implicit Chaining
- `Tidier Exceptions
<http://mail.python.org/pipermail/python-dev/2005-May/053671.html>`__

[TAM]

------------------------------------
Adding content to exception messages
------------------------------------

Nicolas Fleury suggested that there should be a standard method of adding
information to an existing exception (to  re-raise it).  Nick Coghlan
suggested that this would be reasonably simple to do with PEP 344, if all
exceptions  were also new-style classes, but Nicolas indicated that this
wouldn't work in some cases.

Contributing threads:

- `Adding content to exception messages
<http://mail.python.org/pipermail/python-dev/2005-May/053903.html>`__
[Python-Dev] Adding content to exception messages

[TAM]
    
===============
Skipped Threads
===============

- `Loading compiled modules under MSYS/MingGW?
<http://mail.python.org/pipermail/python-dev/2005-May/053769.html>`__
- `RFC: rewrite fileinput module to use itertools.
<http://mail.python.org/pipermail/python-dev/2005-May/053820.html>`__
- `Multiple interpreters not compatible with current thread module
<http://mail.python.org/pipermail/python-dev/2005-May/053840.html>`__
- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-May/053875.html>`__
- `Request for dev permissions
<http://mail.python.org/pipermail/python-dev/2005-May/053877.html>`__
- `python-dev Summary for 2005-05-01 through 2005-05-15 [draft]
<http://mail.python.org/pipermail/python-dev/2005-May/053916.html>`__
- `AST manipulation and source code generation
<http://mail.python.org/pipermail/python-dev/2005-May/053991.html>`__
- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-May/054000.html>`__
- `AST branch patches (was Re: PEP 342/343 status?)
<http://mail.python.org/pipermail/python-dev/2005-May/054009.html>`__
- `[Python-checkins] python/dist/src/Lib/test test_site.py, 1.6, 1.7
<http://mail.python.org/pipermail/python-dev/2005-May/054011.html>`__
- `Split MIME headers into multiple lines near a space
<http://mail.python.org/pipermail/python-dev/2005-May/054015.html>`__


From gmccaughan at synaptics-uk.com  Wed Jun 22 13:27:32 2005
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Wed, 22 Jun 2005 12:27:32 +0100
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506211442360.22846@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<ca471dc2050621031618063e94@mail.gmail.com>
	<Pine.LNX.4.63.0506211442360.22846@leviathan.kdart.com>
Message-ID: <200506221227.33222.gmccaughan@synaptics-uk.com>

[GvR:]
> > Huh? C unsigned ints don't flag overflow either -- they perform
> > perfect arithmetic mod 2**32.

[Keith Dart:]
> I was talking about signed ints. Sorry about the confusion. Other
> scripting languages (e.g. perl) do not error on overflow.

C signed ints also don't flag overflow, nor do they -- as you
point out -- in various other languages.

> > (c) The right place to do the overflow checks is in the API wrappers,
> > not in the integer types.
> 
> That would be the "traditional" method.
> 
> I was trying to keep it an object-oriented API. What should "know" the 
> overflow condition is the type object itself. It raises OverFlowError any 
> time this occurs, for any operation, implicitly. I prefer to catch errors 
> earlier, rather than later.

Why "should"?

Sure, catch errors earlier. But *are* the things you'd catch
earlier by having an unsigned-32-bit-integer type actually
errors? Is it, e.g., an "error" to move the low 16 bits into
the high part by writing
    x = (y<<16) & 0xFFFF0000
instead of
    x = (y&0xFFFF) << 16
or to add 1 mod 2^32 by writing
    x = (y+1) & 0xFFFFFFFF
instead of
    if y == 0xFFFFFFFF: x = 0
    else: x = y+1
? Because it sure doesn't seem that way to me. Why is it better,
or more "object-oriented", to have the checking done by a fixed-size
integer type?

> > (b) I don't know what you call a "normal" integer any more; to me,
> > unified long/int is as normal as they come. Trust me, that's the case
> > for most users. Worrying about 32 bits becomes less and less normal.
> 
> By "normal" integer I mean the mathematical definition.

Then you aren't (to me) making sense. You were distinguishing
this from a unified int/long. So far as I can see, a unified int/long
type *does* implement (modulo implementation limits and bugs)
the "mathematical definition". What am I missing?

>                                                         Most Python users 
> don't have to worry about 32 bits now, that is a good thing when you are 
> dealing only with Python. However, if one has to interface to other 
> systems that have definite types with limits, then one must "hack around" 
> this feature.

Why is checking the range of a parameter with a restricted range
a "hack"?

Suppose some "other system" has a function in its interface that
expects a non-zero integer argument, or one with its low bit set.
Do we need a non-zero-integer type and an odd-integer type?

>               I was just thinking how nice it would be if Python had, in 
> addition to unified ("real", "normal") integers it also had built-in
> types that could be more easily mapped to external types (the typical
> set of signed, unsigned, short, long, etc.). Yes, you can check it at
> conversion time, but that would mean extra Python bytecode. It seems you
> think this is a special case, but I think Python may be used as a "glue
> language" fairly often, and some of us would benefit from having those
> extra types as built-ins.

Well, which extra types? One for each of 8, 16, 32, 64 bit and for
each of signed, unsigned? Maybe also "non-negative signed" of each
size? That's 12 new builtin types, so perhaps you'd be proposing a
subset; what subset?

And how are they going to be used?

  - If the conversion to one of these new limited types
    occurs immediately before calling whatever function
    it is that uses it, then what you're really doing is
    a single range-check. Why disguise it as a conversion?

  - If the conversion occurs earlier, then you're restricting
    the ways in which you can calculate the parameter values
    in question. What's the extra value in that?

I expect I'm missing something important. Could you provide some
actual examples of how code using this new feature would look?

-- 
g


From fredrik.johansson at gmail.com  Wed Jun 22 13:41:39 2005
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Wed, 22 Jun 2005 13:41:39 +0200
Subject: [Python-Dev] Decimal floats as default (was: discussion about
	PEP239 and 240)
Message-ID: <3d0cebfb05062204414026741b@mail.gmail.com>

Hi all,

raymond.hettinger at verizon.net  Fri Jun 17 10:36:01 2005 wrote:

> The future direction of the decimal module likely entails literals in
> the form of 123.45d with binary floats continuing to have the form
> 123.45.  This conflicts with the rational literal proposal of having
> 123.45 interpreted as 123 + 45/100.

Decimal literals are a wonderful idea, especially if it means that
decimals and floats can be made to interact with each other directly.
But why not go one step further, making 123.45 decimal and 123.45b
binary?  In fact, I think a good case can be made for replacing the
default float type with a decimal type.

Decimal floats make life easier for humans accustomed to base 10, so
they should be easy to use. This is particularly relevant given
Python's relatively large user base of "non-programmers", but applies
to many domains. Easy-to-use, correct rounding is essential in many
applications that need to process human-readable data (round() would
certainly be more meaningful if it operated on decimals). Not to
mention that arbitrary precision arithmetic just makes the language
more powerful.

Rationals are inappropriate except in highly specialized applications
because of the non-constant size and processing time, but decimals
would only slow down programs by a (usually small) constant factor. I
suspect most Python programs do not demand the performance hardware
floats deliver, nor require the limited precision or particular
behaviour of IEEE 754 binary floats (the need for machine-precision
integers might be greater -- I've written "& 0xffffffffL" many times).

Changing to decimal would not significantly affect users who really
need good numeric performance either. The C interface would convert
Python floats to C doubles as usual, and numarray would function
accordingly. Additionally, "hardware" could be a special value for the
precision in the decimal (future float) context. In that case, decimal
floats could be phased in without breaking compatibility, by leaving
hardware as the default precision.

123.45d is better than Decimal("123.45"), but appending "d" to specify
a quantity with high precision is as illogical as appending "L" to an
integer value to bypass the machine word size limit. I think the step
from hardware floats to arbitrary-precision decimals would be as
natural as going from short to unlimited-size integers.

I've thought of the further implications for complex numbers and the
math library, but I'll stop writing here to listen to feedback in case
there is some obvious technical flaw or people just don't like the
idea :-) Sorry if this has been discussed and/or rejected before (this
is my first post to python-dev, though I've occasionally read the list
since I started using Python extensively about two years ago).

Fredrik Johansson

From ncoghlan at gmail.com  Wed Jun 22 14:32:25 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jun 2005 22:32:25 +1000
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers
	and	Integers
In-Reply-To: <200506221227.33222.gmccaughan@synaptics-uk.com>
References: <2m1x6z7qu6.fsf@starship.python.net>	<ca471dc2050621031618063e94@mail.gmail.com>	<Pine.LNX.4.63.0506211442360.22846@leviathan.kdart.com>
	<200506221227.33222.gmccaughan@synaptics-uk.com>
Message-ID: <42B95A59.3070700@gmail.com>

Gareth McCaughan wrote:
> [Keith Dart:]
>>By "normal" integer I mean the mathematical definition.
> 
> Then you aren't (to me) making sense. You were distinguishing
> this from a unified int/long. So far as I can see, a unified int/long
> type *does* implement (modulo implementation limits and bugs)
> the "mathematical definition". What am I missing?

Hmm, a 'mod_int' type might be an interesting concept (i.e. a type 
that performs integer arithmetic, only each operation is carried out 
modulo some integer).

Then particular bit sizes would be simple ints, modulo the appropriate 
power of two.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From ncoghlan at gmail.com  Wed Jun 22 14:36:54 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jun 2005 22:36:54 +1000
Subject: [Python-Dev] Explicitly declaring expected exceptions for a
	block
In-Reply-To: <1259444531.20050622100949@targeted.org>
References: <1355231971.20050621112518@targeted.org>
	<85f6a31f05062022564619f4f6@mail.gmail.com>
	<42B8295B.8030107@gmail.com>
	<1259444531.20050622100949@targeted.org>
Message-ID: <42B95B66.6090502@gmail.com>

Dmitry Dvoinikov wrote:
> Now, back to original question then, do you think it'd be
> beneficial to have some sort of exception ignoring or expecting
> statement ?

Not really - as I said earlier, I usually have something non-trivial 
in the except or else clause, so I'm not simply ignoring the exceptions.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From ncoghlan at gmail.com  Wed Jun 22 14:45:07 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jun 2005 22:45:07 +1000
Subject: [Python-Dev] python-dev Summary for 2005-05-16 through
 2005-05-31 [draft]
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E801B0F788@its-xchg4.massey.ac.nz>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E801B0F788@its-xchg4.massey.ac.nz>
Message-ID: <42B95D53.9050608@gmail.com>

Tony Meyer wrote:
> .. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html

Now that PEP 346 is on python.org, it would be best to reference that, 
rather than the PEP 3XX URL on my ISP web space (that URL is now just 
a redirect to PEP 346 on python.org).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From eppstein at ics.uci.edu  Tue Jun 21 22:51:06 2005
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue, 21 Jun 2005 13:51:06 -0700
Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops
In-Reply-To: <000401c573e2$16cd8ce0$bb07a044@oemcomputer>
References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer>
Message-ID: <7592058.1119361866@dhcp31-171.ics.uci.edu>

On 6/18/05 4:45 AM -0400 Raymond Hettinger <raymond.hettinger at verizon.net> 
wrote:
> The above recommendations should get the PEP ready for judgement day.

I thought judgement day already happened for this PEP in the "Parade of 
PEPs".  No?

> Also, I recommend tightening the PEP's motivation.  There are only two
> advantages, encoding and readability.  The former is only a minor gain
> because all it saves is a function call, an O(1) savings in an O(n)
> context.  The latter is where the real benefits lay.

The readability part is really my only motivation for this.  In a nutshell, 
I would like a syntax that I could use with little or no explanation with 
people who already understand some concepts of imperative programming but 
have never seen Python before (e.g. when I use Python-like syntax for the 
pseudocode in my algorithms lectures).  The current for x in range(...) 
syntax is not quite there.

In practice, I have been using
    for x in 0, 1, 2, ... n-1:
which does not work as actual programming language syntax but seems to 
communicate my point better than the available syntaxes.

I have to admit, among your proposed options

>     for i between 2 < i <= 10: ...
>     for i over 2 < i <= 10: ...     # chained comparison style
>     for i over [2:11]: ...          # Slice style
>     for i = 3 to 10:  ...           # Basic style

I don't really care for the repetition of the variable name in the first 
two, the third is no more readable to me than the current range syntax, and 
the last one only looks ok to me because I used to program in Pascal, long 
ago.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/


From fredrik.johansson at gmail.com  Wed Jun 22 16:50:12 2005
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Wed, 22 Jun 2005 16:50:12 +0200
Subject: [Python-Dev] Decimal floats as default (was: discussion about
	PEP239 and 240)
In-Reply-To: <200506220942.52768.mclay@python.net>
References: <3d0cebfb05062204414026741b@mail.gmail.com>
	<200506220942.52768.mclay@python.net>
Message-ID: <3d0cebfb05062207503f3256b2@mail.gmail.com>

On 6/22/05, Michael McLay <mclay at python.net> wrote:
> This idea is dead on arrival. The change would break many applications and
> modules. A successful proposal cannot break backwards compatibility. Adding a
> dpython interpreter to the current code base is one possiblity.

Is there actually much code around that relies on the particular
precision of 32- or 64-bit binary floats for arithmetic, and ceases
working when higher precision is available? Note that functions like
struct.pack would be unaffected. If compatibility is a problem, this
could still be a possibility for Python 3.0.

In either case, compatibility can be ensured by allowing both n-digit
decimal and hardware binary precision for floats, settable via a float
context. Then the backwards compatible binary mode can be default, and
"decimal mode" can be set with one line of code. d-suffixed literals
create floats with decimal precision.

There is the alternative of providing decimal literals by using
separate decimal and binary float base types, but in my eyes this
would be redundant. The primary use of binary floats is performance
and compatibility, and both can be achieved with my proposal without
sacrificing the simplicity and elegance of having a single type to
represent non-integral numbers. It makes more sense to extend the
float type with the power and versatility of the decimal module than
to have a special type side by side with a default type that is less
capable.

Fredrik

From gmccaughan at synaptics-uk.com  Wed Jun 22 17:10:27 2005
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Wed, 22 Jun 2005 16:10:27 +0100
Subject: [Python-Dev]
	=?iso-8859-1?q?Is_PEP_237_final_--_Unifying_Long_Int?=
	=?iso-8859-1?q?egers_and=09Integers?=
In-Reply-To: <42B95A59.3070700@gmail.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<200506221227.33222.gmccaughan@synaptics-uk.com>
	<42B95A59.3070700@gmail.com>
Message-ID: <200506221610.28305.gmccaughan@synaptics-uk.com>

On Wednesday 2005-06-22 13:32, Nick Coghlan wrote:
> Gareth McCaughan wrote:
> > [Keith Dart:]
> >>By "normal" integer I mean the mathematical definition.
> > 
> > Then you aren't (to me) making sense. You were distinguishing
> > this from a unified int/long. So far as I can see, a unified int/long
> > type *does* implement (modulo implementation limits and bugs)
> > the "mathematical definition". What am I missing?
> 
> Hmm, a 'mod_int' type might be an interesting concept (i.e. a type 
> that performs integer arithmetic, only each operation is carried out 
> modulo some integer).
> 
> Then particular bit sizes would be simple ints, modulo the appropriate 
> power of two.

It might indeed, but it would be entirely the opposite of what
(I think) Keith wants, namely something that raises an exception
any time a value goes out of range :-).

-- 
g


From skip at pobox.com  Wed Jun 22 21:22:16 2005
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 22 Jun 2005 14:22:16 -0500
Subject: [Python-Dev] Decimal floats as default (was: discussion about
 PEP239 and 240)
In-Reply-To: <3d0cebfb05062207503f3256b2@mail.gmail.com>
References: <3d0cebfb05062204414026741b@mail.gmail.com>
	<200506220942.52768.mclay@python.net>
	<3d0cebfb05062207503f3256b2@mail.gmail.com>
Message-ID: <17081.47720.970132.531615@montanaro.dyndns.org>


    Fredrik> Is there actually much code around that relies on the
    Fredrik> particular precision of 32- or 64-bit binary floats for
    Fredrik> arithmetic, and ceases working when higher precision is
    Fredrik> available? 

Umm, yeah...  The path you take from one or more string literals
representing real numbers through a series of calculations and ending up in
a hardware double-precision floating point number is probably going to be
different at different precisions.

    >>> x = Decimal("1.0000000000001")
    >>> y = Decimal("1.000000000000024567")
    >>> x
    Decimal("1.0000000000001")
    >>> y
    Decimal("1.000000000000024567")
    >>> float(x) 
    1.0000000000000999
    >>> float(y)
    1.0000000000000246
    >>> x/y
    Decimal("1.000000000000075432999999998")
    >>> float(x)/float(y)
    1.0000000000000753
    >>> float(x/y)
    1.0000000000000755

Performance matters too:

    % timeit.py -s 'from decimal import Decimal ; x = Decimal("1.0000000000001") ; y = Decimal("1.000000000000024567")' 'x/y'
    1000 loops, best of 3: 1.39e+03 usec per loop
    % timeit.py -s 'from decimal import Decimal ; x = float(Decimal("1.0000000000001")) ; y = float(Decimal("1.000000000000024567"))' 'x/y'
    1000000 loops, best of 3: 0.583 usec per loop

I imagine a lot of people would be very unhappy if their fp calculations
suddenly began taking 2000x longer, even if their algorithms didn't break.
(For all I know, Raymond might have a C version of Decimal waiting for an
unsuspecting straight man to complain about Decimal's performance and give
him a chance to announce it.)

If nothing else, extension module code that executes

    f = PyFloat_AsDouble(o);

or

    if (PyFloat_Check(o)) {
       ...
    }

would either have to change or those functions would have to be rewritten to
accept Decimal objects and convert them to doubles (probably silently,
because otherwise there would be so many warnings).

For examples of packages that might make large use of these functions, take
a look at Numeric, SciPy, ScientificPython, MayaVi, and any other package
that does lots of floating point arithmetic.

Like Michael wrote, I think this idea is DOA.

Skip

From fredrik.johansson at gmail.com  Wed Jun 22 22:19:39 2005
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Wed, 22 Jun 2005 22:19:39 +0200
Subject: [Python-Dev] Decimal floats as default (was: discussion about
	PEP239 and 240)
In-Reply-To: <17081.47720.970132.531615@montanaro.dyndns.org>
References: <3d0cebfb05062204414026741b@mail.gmail.com>
	<200506220942.52768.mclay@python.net>
	<3d0cebfb05062207503f3256b2@mail.gmail.com>
	<17081.47720.970132.531615@montanaro.dyndns.org>
Message-ID: <3d0cebfb05062213196393d9d0@mail.gmail.com>

On 6/22/05, Skip Montanaro <skip at pobox.com> wrote:
> If nothing else, extension module code that executes
> 
>     f = PyFloat_AsDouble(o);
> 
> or
> 
>     if (PyFloat_Check(o)) {
>        ...
>     }
> 
> would either have to change or those functions would have to be rewritten to
> accept Decimal objects and convert them to doubles (probably silently,
> because otherwise there would be so many warnings).
> 

Silent conversion was the idea.

> Like Michael wrote, I think this idea is DOA.

Granted, then.

However, keeping binary as default does not kill the other idea in my
proposal, which is to extend the float type to cover decimals instead
of having a separate decimal type. I consider this a more important
issue (contradicting the thread title :-) than whether "d" should be
needed to specify decimal precision.

Fredrik

From steven.bethard at gmail.com  Wed Jun 22 23:07:07 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 22 Jun 2005 15:07:07 -0600
Subject: [Python-Dev] python-dev Summary for 2005-05-01 through 2005-05-16
	[draft]
Message-ID: <d11dcfba05062214073ed0b54a@mail.gmail.com>

Here's the May 01-15 draft.  Sorry for the delay.  Please check the
Unicode summary at the end especially closely; I'm not entirely sure I
got that one all right.  Thanks!

As always, please let us know if you have any corrections!

Steve

======================
Summary Announcements
======================

----------------------------------------------
PEP 340 Episode 2: Revenge of the With (Block)
----------------------------------------------

This fornight's Python-Dev was dominated again by another nearly 400
messages on the topic of anonymous block statements. The discussion
was a little more focused than the last thanks mainly to Guido's
introduction of `PEP 340`_. Discussion of this PEP resulted in a
series of other PEPs, including

* `PEP 342`_: Enhanced Iterators, which broke out into a separate
  PEP the parts of `PEP 340`_ that allowed code to pass values into
  iterators using ``continue EXPR`` and yield-expressions.

* `PEP 343`_: Anonymous Block Redux, a dramatically simplified
  version of `PEP 340`_, which removed the looping nature of the
  anonymous blocks and the injection-of-exceptions semantics for
  generators.

* `PEP 3XX`_: User Defined ("with") Statements, which proposed
  non-looping anonymous blocks accompanied by finalization semantics
  for iterators and generators in for loops.

Various details of each of these proposals are discussed below in the
sections:

1. `Enhanced Iterators`_

2. `Separate APIs for Iterators and Anonymous Blocks`_

3. `Looping Anonymous Blocks`_

4. `Loop Finalization`_

At the time of this writing, it looked like the discussion was coming
very close to a final agreement; `PEP 343`_ and `PEP 3XX`_ both agreed
upon the same semantics for the block-statement, the keyword had been
narrowed down to either ``do`` or ``with``, and Guido had agreed to
add back in to `PEP 343`_ some form of exception-injection semantics
for generators.


.. _PEP 340: http://www.python.org/peps/pep-0340.html

.. _PEP 342: http://www.python.org/peps/pep-0342.html

.. _PEP 343: http://www.python.org/peps/pep-0343.html

.. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html

[SJB]


=========
Summaries
=========

------------------
Enhanced Iterators
------------------

`PEP 340`_ incorporated a variety of orthogonal features into a single
proposal. To make the PEP somewhat less monolithic, the method for
passing values into an iterator was broken off into `PEP 342`_. This
method includes:

* updating the iterator protocol to use .__next__() instead of .next()

* introducing a new builtin next()

* allowing continue-statements to pass values into iterators

* allowing generators to receive values with a yield-expression

Though these features had seemed mostly uncontroversial, Guido seemed
inclined to wait for a little more motivation from the co-routiney
people before accepting the proposal.

Contributing threads:

- `Breaking off Enhanced Iterators PEP from PEP 340
<http://mail.python.org/pipermail/python-dev/2005-May/053463.html>`__

[SJB]


------------------------------------------------
Separate APIs for Iterators and Anonymous Blocks
------------------------------------------------

`PEP 340`_ had originally proposed to treat the anonymous block
protocol as an extension of the iterator protocol. Several problems
with this approach were raised, including:

* for-loops could accidentally be used with objects requiring blocks,
  meaning that resources would not get cleaned up properly

* blocks could be used instead of for-loops, violating TOOWTDI

As a result, both `PEP 343`_ and `PEP 3XX`_ propose decorators for
generator functions that will wrap the generator object appropriately
to match the anonymous block protocol. Generator objects without the
proposed decorators would not be usable in anonymous block statements.

Contributing threads:

- `PEP 340 -- loose ends
<http://mail.python.org/pipermail/python-dev/2005-May/053206.html>`__
- `PEP 340 -- concept clarification
<http://mail.python.org/pipermail/python-dev/2005-May/053280.html>`__

[SJB]


------------------------
Looping Anonymous Blocks
------------------------

A few issues arose as a result of `PEP 340`_'s formulation of
anonymous blocks as a variation on a loop.

Because the anonymous blocks of `PEP 340`_ were defined in terms of
while-loops, there was some discussion as to whether they should have
an ``else`` clause like Python ``for`` and ``while`` loops do. There
didn't seem to be one obvious interpretation of an ``else`` block
though, so Guido rejected the ``else`` block proposal.

The big issue with looping anonymous blocks, however, was in the
handling of ``break`` and ``continue`` statements. Many use cases for
anonymous blocks did not require loops. However, because `PEP 340`_
anonymous blocks were implemented in terms of loops, ``break`` and
``continue`` acted much like they would in a loop. This meant that in
code like::

    for item in items:
        with lock:
            if handle(item):
                break

the ``break`` statement would only break out of the anonymous block
(the ``with`` statement) instead of breaking out of the for-loop. This
pretty much shot-down `PEP 340`_; there were too many cases where an
anonymous block didn't look like a loop, and having it behave like one
would have been a major stumbling block in learning the construct.

As a result, both `PEP 343`_ and `PEP 3XX`_ were proposed as
non-looping versions of `PEP 340`_.


Contributing threads:

- `PEP 340: Else clause for block statements
<http://mail.python.org/pipermail/python-dev/2005-May/053190.html>`__
- `PEP 340 -- loose ends
<http://mail.python.org/pipermail/python-dev/2005-May/053206.html>`__
- `PEP 340 -- concept clarification
<http://mail.python.org/pipermail/python-dev/2005-May/053226.html>`__
- `PEP 340: Breaking out.
<http://mail.python.org/pipermail/python-dev/2005-May/053223.html>`__
- `PEP 340: Non-looping version (aka PEP 310 redux)
<http://mail.python.org/pipermail/python-dev/2005-May/053400.html>`__
- `PEP 340 - Remaining issues
<http://mail.python.org/pipermail/python-dev/2005-May/053406.html>`__
- `PEP 340: Deterministic Finalisation (new PEP draft, either a
competitor or update to PEP 340)
<http://mail.python.org/pipermail/python-dev/2005-May/053503.html>`__
- `Merging PEP 310 and PEP 340-redux?
<http://mail.python.org/pipermail/python-dev/2005-May/053591.html>`__
- `PEP 343 - Abstract Block Redux
<http://mail.python.org/pipermail/python-dev/2005-May/053731.html>`__

[SJB]


-----------------
Loop Finalization
-----------------

Greg Ewing pointed out that a generator with a yield inside a
block-statement would require additional work to guarantee its
finalization. For example, if the generator::

   def all_lines(filenames):
       for name in filenames:
           with open(name) as f:
               for line in f:
                   yield line 

were used in code like::

    for line in all_lines(filenames):
        if some_cond(line):
            break

then unless the for-loop performed some sort of finalization on the
all_lines generator, the last-opened file could remain open
indefinitiely.

As a result, `PEP 3XX`_ proposes that for-loops check for a
__finish__() method on their iterators, and if one exists, call that
method when the for-loop completes. Generators like all_lines above,
that put a yield inside a block-statement, would then acquire a
__finish__() method that would raise a TerminateIteration exception
at the point of the last yield. The TerminateIteration exception would
thus cause the block-statement to complete, guaranteeing that the
generator was properly finalized.

Contributing threads:

- `PEP 340 - For loop cleanup, and feature separation
<http://mail.python.org/pipermail/python-dev/2005-May/053432.html>`__
- `PEP 340: Deterministic Finalisation (new PEP draft, either a
competitor or update to PEP 340)
<http://mail.python.org/pipermail/python-dev/2005-May/053503.html>`__
- `PEP 343 - Abstract Block Redux
<http://mail.python.org/pipermail/python-dev/2005-May/053731.html>`__

[SJB]


----------------------------
Breaking out of Nested Loops
----------------------------

As a result of some of the issues of looping anonymous blocks, a few
threads discussed options for breaking out of nested loops. These
mainly worked by augmenting the ``break`` statement with another
keyword (or keywords) that would indicate which loop to break out of.

One proposal suggested that ``break`` be followed with ``for`` or
``while`` to indicate which loop to break out of. But ``break for``
would only really be useful in a while-loop nested within a for-loop,
and ``break while`` would only really be useful in a for-loop nested
within a while-loop. That is, because loops could only be named by
type, the proposal was only useful when loops of different types were
mixed. This suggestion was thus discarded as not being general enough.

A few other suggestions were briefly discussed: adding labels to
loops, using an integer to indicate which "stack level" to break at,
and pushing breaks onto a "break buffer", but Guido killed the
discussion, saying, `"Stop all discussion of breaking out of multiple
loops. It ain't gonna happen before my retirement."
<http://mail.python.org/pipermail/python-dev/2005-May/053592.html>`__

Contributing threads:

- `PEP 340: Breaking out.
<http://mail.python.org/pipermail/python-dev/2005-May/053223.html>`__
- `PEP 340: Deterministic Finalisation (new PEP draft, either a
competitor or update to PEP 340)
<http://mail.python.org/pipermail/python-dev/2005-May/053503.html>`__

[SJB]


------------------------
The future of exceptions
------------------------

Ka-Ping Yee suggested that instead of passing (type, value, traceback)
tuples in exceptions it would be better to put the traceback in
value.traceback.  Guido had also suggested this (in the `PEP 340`_ murk) but
pointed out that this would not work as long as string exceptions exist
(as there is nowhere to put the traceback).

Guido noted that there are no concrete plans as to when string exceptions
will be deprecated and removed (other than in 3.0 at the latest); he
indicated that it could be sooner, if someone wrote a PEP with a timeline
(e.g. deprecated in 2.5, gone in 2.6).

Brett C. volunteered to write a PEP targetted at Python 3000 covering
exception changes (base inheritance, standard attributes (e.g. .traceback),
reworking the built-in exception inheritance hierarchy, and the future of
bare except statements).  

Contributing threads:

- `Tidier Exceptions
<http://mail.python.org/pipermail/python-dev/2005-May/053671.html>`__

.. _PEP 340: http://www.python.org/peps/pep-0340.html

[TAM]


-----------------------------------
Unifying try/except and try/finally
-----------------------------------

Reinhold Birkenfeld submitted a Pre-PEP to allow both except and finally
clauses in try blocks.  For example, a construction like::

        try:
            <suite 1>
        except Ex1:
            <suite 2>
        <more except: clauses>
        else:
            <suite 3>
        finally:
            <suite 4>

would be exactly the same as the legacy::

        try:
            try:
                <suite 1>
            except Ex1:
                <suite 2>
            <more except: clauses>
            else:
                <suite 3>
        finally:
            <suite 4>

Guido liked this idea (so much that he wanted to accept it immediately),
and recommended that it was checked in as a PEP.  However, Tim Peters
pointed out that this functionality was removed from Python (by Guido) way
back in 0.9.6, seemingly because there was confusion about exactly when
the finally clause would be called (explicit is better than implicit!).
Guido clarified that control would only pass forward, and indicated that
he felt that since this is now available in Java (and C#) fewer people
would be confused.  The main concern about this change was that, while the
cost was low, it seemed to add very little value.

Contributing threads:

- `Pre-PEP: Unifying try-except and try-finally
<http://mail.python.org/pipermail/python-dev/2005-May/053290.html>`__
                                                                 
[TAM]


-----------------
Decorator Library
-----------------

Michele Simionato asked whether a module for commonly used decorators, or
utilities to create decorators, was planned.  Raymond Hettinger indicated
that while this was likely in the long term, he felt that it was better if
these first evolved via wikis, recipes, or mailing lists, so that a module
would only be added once best practices and proven winners had emerged.
In the meantime, there is both a `Decorator Library wiki page`_ and
you can try out `Michele's library`_ [zip].

To assist with decorator creation, Michele would like a facility to copy a
function.  Phillip J. Eby noted that the informally-discussed proposal is
to add a mutable __signature__ to functions to assist with signature
preserving decorators.  Raymond suggested a patch adding a __copy__ method
to functions or a patch for the copy module, and Michele indicated that he
would also like to subclass FunctionType with an user-defined __copy__
method.

Contributing threads:

- `my first post: asking about a "decorator" module
<http://mail.python.org/pipermail/python-dev/2005-May/053316.html>`__
- `The decorator module
<http://mail.python.org/pipermail/python-dev/2005-May/053446.html>`__

 .. _Decorator Library wiki page:
http://www.python.org/moin/PythonDecoratorLibrary
 .. _Michele's library:
http://www.phyast.pitt.edu/~micheles/python/decorator.zip

[TAM]

---------------------
Hooking Py_FatalError
---------------------

Errors that invoke Py_FatalError generally signify that the internal state
of Python is in such a poor state that continuing (including raising an
exception) is impossible or unwise; as a result, Py_FatalError outputs the
error to stderr and calls abort().  m.u.k. would like to have a callback to
hook Py_FatalError to avoid this call to abort().  The general consensus
was that effort would be better directed to fixing the causes of fatal
errors than hooking Py_FatalError.  m.u.k.'s use case was for generating
additional logging information; a `callback system patch`_ (revised by
James William Pye) is available for those interested.

Contributing threads:

- `Need to hook Py_FatalError
<http://mail.python.org/pipermail/python-dev/2005-May/053218.html>`__

 .. _callback system patch: http://python.org/sf/1195571

-------------------
Chaining Exceptions
-------------------

Ka-Ping Yee suggested adding information to exceptions when they are raised
in the handler for another exception.  For example::

    def a():
        try:
            raise AError
        except:
            raise BError

raises an exception which is an instance of BError.  This instance could
have an attribute which is instance of AError, containing information about
the original exception.  Use cases include catching a low-level exception
(e.g. socket.error) and turning it into a high-level exception (e.g.
an HTTPRequestFailed exception) and handling problems in exception handling
code.  Guido liked the idea, and discussion fleshed out a tighter
definition; however it was unclear whether adding this now was feasible -
this would perhaps be best added in Python 3000.

Contributing threads:

- `Chained Exceptions
<http://mail.python.org/pipermail/python-dev/2005-May/053672.html>`__

[TAM]

------------------------
Py_UNICODE Documentation
------------------------

Nicholas Bastin started a series of threads discussing an inconsistency
between the Py_UNICODE docs and the behavior on some RedHat systems.
The docs say that Py_UNICODE should be an alias for wchar_t when wchar_t
is available and has 16 bits, but Nick found that pyconfig.h still
reports PY_UNICODE_TYPE as wchar_t, even when PY_UNICODE_SIZE is 4.

An extensive discussion between Nick, Marc-Andre Lemburg and Martin v.
L?wis suggests that the possible Python-internal representations for
Py_UNICODE are:

* 4-byte wchar_t encoded as UTF-32 (UCS-4)
* 2-byte wchar_t encoded as UTF-16
* unsigned short encoded as UTF-16

Python defaults to 2-byte mode, using wchar_t if available (and has 16
bits) and using unsigned short otherwise. You may end up with the
4-byte mode if TCL was built for UCS-4 (this overrides the defaults)
or if you explicitly request it with --enable-unicode=ucs4. To get
UCS-2 when TCL was built for UCS-4, you must explicitly request
--enable-unicode=ucs2. Of course, this will mean that _tkinter can't
be built anymore.

Also noted by this discussion was that even with --enable-unicode=ucs2,
Python continues to support surrogate pairs in the BMP. So for example,
even with a UCS-2 build, u"\U00012345" encodes as a sequence of two
characters; it does not produce a UnicodeError.

At the time of this posting, it did not appear that there was a
documentation patch available yet.

Contributing threads:

- `Py_UNICODE madness
<http://mail.python.org/pipermail/python-dev/2005-May/053264.html>`__
- `New Py_UNICODE doc
<http://mail.python.org/pipermail/python-dev/2005-May/053311.html>`__
- `Python's Unicode width default (New Py_UNICODE doc)
<http://mail.python.org/pipermail/python-dev/2005-May/053574.html>`__

[SJB]


                                                  
===============
Skipped Threads
===============

- `Keyword for block statements
<http://mail.python.org/pipermail/python-dev/2005-May/053189.html>`__
- `PEP 340 - possible new name for block-statement
<http://mail.python.org/pipermail/python-dev/2005-May/053195.html>`__
- `Generating nested data structures with blocks
<http://mail.python.org/pipermail/python-dev/2005-May/053204.html>`__
- `PEP 340 -- Clayton's keyword?
<http://mail.python.org/pipermail/python-dev/2005-May/053377.html>`__
- `PEP 340: Only for try/finally?
<http://mail.python.org/pipermail/python-dev/2005-May/053258.html>`__
- `2 words keyword for block
<http://mail.python.org/pipermail/python-dev/2005-May/053251.html>`__
- `anonymous blocks
<http://mail.python.org/pipermail/python-dev/2005-May/053297.html>`__
- `"begin" as keyword for pep 340
<http://mail.python.org/pipermail/python-dev/2005-May/053315.html>`__
- `PEP 340: propose to get rid of 'as' keyword
<http://mail.python.org/pipermail/python-dev/2005-May/053320.html>`__
- `PEP 340 keyword: after
<http://mail.python.org/pipermail/python-dev/2005-May/053396.html>`__
- `PEP 340 keyword: Extended while syntax
<http://mail.python.org/pipermail/python-dev/2005-May/053409.html>`__
- `PEP 340 - Remaining issues - keyword
<http://mail.python.org/pipermail/python-dev/2005-May/053428.html>`__
- `PEP 340: Examples as class's.
<http://mail.python.org/pipermail/python-dev/2005-May/053423.html>`__
- `Proposed alternative to __next__ and __exit__
<http://mail.python.org/pipermail/python-dev/2005-May/053514.html>`__
- `"with" use case: exception chaining
<http://mail.python.org/pipermail/python-dev/2005-May/053665.html>`__
- `PEP 343: Resource Composition and Idempotent __exit__
<http://mail.python.org/pipermail/python-dev/2005-May/053767.html>`__
- `[Python-checkins] python/nondist/peps pep-0343.txt, 1.8, 1.9
<http://mail.python.org/pipermail/python-dev/2005-May/053766.html>`__
- `the current behavior of try: ... finally:
<http://mail.python.org/pipermail/python-dev/2005-May/053692.html>`__
- `a patch to inspect and a non-feature request
<http://mail.python.org/pipermail/python-dev/2005-May/053653.html>`__
- `Python 2.4 set objects and cyclic garbage
<http://mail.python.org/pipermail/python-dev/2005-May/053630.html>`__
- `CHANGE BayPIGgies: May *THIRD* Thurs
<http://mail.python.org/pipermail/python-dev/2005-May/053628.html>`__
- `Python continually calling sigprocmask() on FreeBSD 5
<http://mail.python.org/pipermail/python-dev/2005-May/053615.html>`__
- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-May/053213.html>`__
- `problems with memory management
<http://mail.python.org/pipermail/python-dev/2005-May/053408.html>`__
- `Adding DBL_MANTISSA and such to Python
<http://mail.python.org/pipermail/python-dev/2005-May/053372.html>`__
- `python-dev Summary for 2005-04-16 through 2005-04-30 [draft]
<http://mail.python.org/pipermail/python-dev/2005-May/053383.html>`__
- `Python Language track at Europython, still possibilities to submit
talks <http://mail.python.org/pipermail/python-dev/2005-May/053303.html>`__
- `(no subject)
<http://mail.python.org/pipermail/python-dev/2005-May/053196.html>`__
- `Kernel panic writing to /dev/dsp with cmpci driver
<http://mail.python.org/pipermail/python-dev/2005-May/053627.html>`__

From skip at pobox.com  Thu Jun 23 01:01:51 2005
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 22 Jun 2005 18:01:51 -0500
Subject: [Python-Dev] PEP 304 - is anyone really interested?
Message-ID: <17081.60895.24439.92843@montanaro.dyndns.org>


I wrote PEP 304, "Controlling Generation of Bytecode Files":

    http://www.python.org/peps/pep-0304.html

quite awhile ago.  The first version appeared in January 2003 in response to
questions from people about controlling/suppressing bytecode generation in
certain situations.  It sat idle for a long while, though from time-to-time
people would ask about the functionality and I'd respond or update the PEP.
In response to another recent question about this topic:

    http://mail.python.org/pipermail/python-list/2005-June/284775.html

and a wave of recommendations by Raymond Hettinger regarding several other
PEPs, I updated the patch to work with current CVS.  Aside from one response
by Thomas Heller noting that my patch upload failed (and which has been
corrected since), I've seen no response either on python-dev or
comp.lang.python.

I really have no personal use for this functionality.  I control all the
computers on which I use Python and don't use any exotic hardware (which
includes Windows as far with its multi-rooted file system as far as I'm
concerned), don't run from read-only media or think that in-memory file
systems are much of an advantage over OS caching.  The best I will ever do
with it is respond to people's inputs.  I'd hate to see it sit for another
two years.  If someone out there is interested in this functionality and
would benefit more from its incorporation into the core, I'd be happy to
hand it off to you.

So speak up folks, otherwise my recommendation is that it be put out of its
misery.

Skip

From trentm at ActiveState.com  Thu Jun 23 01:42:54 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 22 Jun 2005 16:42:54 -0700
Subject: [Python-Dev] PEP 304 - is anyone really interested?
In-Reply-To: <17081.60895.24439.92843@montanaro.dyndns.org>
References: <17081.60895.24439.92843@montanaro.dyndns.org>
Message-ID: <20050622234254.GB13632@ActiveState.com>

[Skip Montanaro wrote]
> 
> I wrote PEP 304, "Controlling Generation of Bytecode Files":
> 
>     http://www.python.org/peps/pep-0304.html
> 
> ...
> So speak up folks, otherwise my recommendation is that it be put out of its
> misery.

I've had use for it before, but have managed to work around the
problems. I think it is a good feature, but I wouldn't have the time to
shepherd the patch.

Trent

-- 
Trent Mick
TrentM at ActiveState.com

From theller at python.net  Thu Jun 23 11:53:27 2005
From: theller at python.net (Thomas Heller)
Date: Thu, 23 Jun 2005 11:53:27 +0200
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers
	and	Integers
References: <2m1x6z7qu6.fsf@starship.python.net>
	<ca471dc2050621031618063e94@mail.gmail.com>
	<Pine.LNX.4.63.0506211442360.22846@leviathan.kdart.com>
	<200506221227.33222.gmccaughan@synaptics-uk.com>
	<42B95A59.3070700@gmail.com>
Message-ID: <u0jpv1pk.fsf@python.net>

Nick Coghlan <ncoghlan at gmail.com> writes:

> Gareth McCaughan wrote:
>> [Keith Dart:]
>>>By "normal" integer I mean the mathematical definition.
>> 
>> Then you aren't (to me) making sense. You were distinguishing
>> this from a unified int/long. So far as I can see, a unified int/long
>> type *does* implement (modulo implementation limits and bugs)
>> the "mathematical definition". What am I missing?
>
> Hmm, a 'mod_int' type might be an interesting concept (i.e. a type 
> that performs integer arithmetic, only each operation is carried out 
> modulo some integer).
>
> Then particular bit sizes would be simple ints, modulo the appropriate 
> power of two.

ctypes provides mutable platform integers and floats of various sizes
(and much more).  Currently they don't have any methods, only the value
attribute.  Maybe it would be useful to implement the standard numeric
methods on them.

Thomas


From paolo_veronelli at libero.it  Thu Jun 23 19:07:13 2005
From: paolo_veronelli at libero.it (Paolino)
Date: Thu, 23 Jun 2005 19:07:13 +0200
Subject: [Python-Dev] is type a usable feature?
Message-ID: <42BAEC41.7090202@libero.it>

Hello developers,I noticed my application was growing strangely while I 
was using type, then I tried this:

while True:
   type('A',(),{})

and saw memory filling up.Is there a clean solution to that?
I see it as a bug in python engeneering,that is why I wrote to you.
Thanks a lot

Paolino

From tim.peters at gmail.com  Thu Jun 23 18:25:14 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 23 Jun 2005 12:25:14 -0400
Subject: [Python-Dev] is type a usable feature?
In-Reply-To: <42BAEC41.7090202@libero.it>
References: <42BAEC41.7090202@libero.it>
Message-ID: <1f7befae050623092570e2b6a0@mail.gmail.com>

[Paolino <paolo_veronelli at libero.it>]
> Hello developers,I noticed my application was growing strangely while I
> was using type, then I tried this:
> 
> while True:
>   type('A',(),{})
> 
> and saw memory filling up.Is there a clean solution to that?
> I see it as a bug in python engeneering,that is why I wrote to you.

Python bugs should be reported on SourceForge:

    http://sourceforge.net/tracker/?group_id=5470&atid=105470

Please specify the Python version and OS.  I do not see memory growth
running the above under Python 2.3.5 or 2.4.1 on Windows, so I don't
have any evidence of a bug here in the Pythons I usually use.

From paolo_veronelli at libero.it  Thu Jun 23 20:15:30 2005
From: paolo_veronelli at libero.it (Paolino)
Date: Thu, 23 Jun 2005 20:15:30 +0200
Subject: [Python-Dev] is type a usable feature?
In-Reply-To: <1f7befae050623092570e2b6a0@mail.gmail.com>
References: <42BAEC41.7090202@libero.it>
	<1f7befae050623092570e2b6a0@mail.gmail.com>
Message-ID: <42BAFC42.5090505@libero.it>

Tim Peters wrote:
> [Paolino <paolo_veronelli at libero.it>]
> 
>>Hello developers,I noticed my application was growing strangely while I
>>was using type, then I tried this:
>>
>>while True:
>>  type('A',(),{})
>>
>>and saw memory filling up.Is there a clean solution to that?
>>I see it as a bug in python engeneering,that is why I wrote to you.
> 
> 
> Python bugs should be reported on SourceForge:
> 
>     http://sourceforge.net/tracker/?group_id=5470&atid=105470
> 
> Please specify the Python version and OS.  I do not see memory growth
> running the above under Python 2.3.5 or 2.4.1 on Windows, so I don't
> have any evidence of a bug here in the Pythons I usually use.
> 
Sorry, the growth happens using ipython shell.With python original shell
       there is nothing wrong.


From paolo_veronelli at tiscali.it  Thu Jun 23 20:08:34 2005
From: paolo_veronelli at tiscali.it (Paolino)
Date: Thu, 23 Jun 2005 20:08:34 +0200
Subject: [Python-Dev] is type a usable feature?
In-Reply-To: <1f7befae050623092570e2b6a0@mail.gmail.com>
References: <42BAEC41.7090202@libero.it>
	<1f7befae050623092570e2b6a0@mail.gmail.com>
Message-ID: <42BAFAA2.7060002@tiscali.it>

Tim Peters wrote:
> [Paolino <paolo_veronelli at libero.it>]
> 
>>Hello developers,I noticed my application was growing strangely while I
>>was using type, then I tried this:
>>
>>while True:
>>  type('A',(),{})
>>
>>and saw memory filling up.Is there a clean solution to that?
>>I see it as a bug in python engeneering,that is why I wrote to you.
> 
> 
> Python bugs should be reported on SourceForge:
> 
>     http://sourceforge.net/tracker/?group_id=5470&atid=105470
> 
> Please specify the Python version and OS.  I do not see memory growth
> running the above under Python 2.3.5 or 2.4.1 on Windows, so I don't
> have any evidence of a bug here in the Pythons I usually use.
> 
Sorry, the growth happens using ipython shell.With python original shell 
       there is nothing wrong.

From tim.peters at gmail.com  Fri Jun 24 00:26:00 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 23 Jun 2005 18:26:00 -0400
Subject: [Python-Dev] refcounting vs PyModule_AddObject
In-Reply-To: <42B341E3.5000609@v.loewis.de>
References: <2mzmtrbxwo.fsf@starship.python.net> <42B0725D.9040200@v.loewis.de>
	<2m4qbybpf2.fsf@starship.python.net> <42B341E3.5000609@v.loewis.de>
Message-ID: <1f7befae05062315266801564c@mail.gmail.com>

[Michael Hudson]
>> I've been looking at this area partly to try and understand this bug:
>>
>>     [ 1163563 ] Sub threads execute in restricted mode
>>
>> but I'm not sure the whole idea of multiple interpreters isn't
>> inherently doomed :-/

[Martin v. L?wis]
> That's what Tim asserts, saying that people who want to use the
> feature should fix it themselves.

I haven't said they're doomed, although I have said that people who
want to use them are on their own.  I think the latter is simply an
obvious truth, since (a) multiple interpreters have been entirely
unexercised by the Python test suite ("if it's not tested, it's
broken" rules); (b) Python developers generally pay no attention to
them; and (c), as the teensy bit of docs for them imply, they're an
"80% solution", but to a problem that's probably more sensitive than
most to glitches in the other 20%.

I've also said that Mark's thread state PEP explicitly disavowed
responsibility for working nicely with multiple interpreters.  I said
that only because that's what his PEP said <wink>.

From greg.ewing at canterbury.ac.nz  Fri Jun 24 04:11:03 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Jun 2005 14:11:03 +1200
Subject: [Python-Dev] PEP 304 - is anyone really interested?
In-Reply-To: <17081.60895.24439.92843@montanaro.dyndns.org>
References: <17081.60895.24439.92843@montanaro.dyndns.org>
Message-ID: <42BB6BB7.5020109@canterbury.ac.nz>

Skip Montanaro wrote:
> I wrote PEP 304, "Controlling Generation of Bytecode Files":
> 
>     http://www.python.org/peps/pep-0304.html

I would like to see some way of having bytecode files put
into platform/version dependent subdirectories, which
would make it easier e.g. to have Python code shared
over a network between machines of different architectures
or running different Python versions. But the scheme this
PEP proposes (a single session-wide location) is too
coarse-grained for what I have in mind. So I would not
support this particular PEP without sufficiently large
changes that it might as well become a new PEP.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From bob at redivi.com  Fri Jun 24 08:00:03 2005
From: bob at redivi.com (Bob Ippolito)
Date: Fri, 24 Jun 2005 02:00:03 -0400
Subject: [Python-Dev] PEP 304 - is anyone really interested?
In-Reply-To: <42BB6BB7.5020109@canterbury.ac.nz>
References: <17081.60895.24439.92843@montanaro.dyndns.org>
	<42BB6BB7.5020109@canterbury.ac.nz>
Message-ID: <E91DE75E-6705-4A87-BA33-3779AF2FAAC5@redivi.com>

On Jun 23, 2005, at 10:11 PM, Greg Ewing wrote:

> Skip Montanaro wrote:
>
>> I wrote PEP 304, "Controlling Generation of Bytecode Files":
>>
>>     http://www.python.org/peps/pep-0304.html
>>
>
> I would like to see some way of having bytecode files put
> into platform/version dependent subdirectories, which
> would make it easier e.g. to have Python code shared
> over a network between machines of different architectures
> or running different Python versions. But the scheme this
> PEP proposes (a single session-wide location) is too
> coarse-grained for what I have in mind. So I would not
> support this particular PEP without sufficiently large
> changes that it might as well become a new PEP.

You could implement that in your sitecustomize.py, unless you have  
Python interpreters that change platform or version mid-session.

I don't see why it needs to be subdirectories, you could make the  
bytecode cache local or somewhere on the server that you have write  
access to.

-bob


From t-meyer at ihug.co.nz  Fri Jun 24 11:16:44 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Fri, 24 Jun 2005 21:16:44 +1200
Subject: [Python-Dev] python-dev Summary for 2005-06-01 through 2005-06-15
	[draft]
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E801DB02F4@its-xchg4.massey.ac.nz>

You've just read two summaries, but here is another one, as we come back up
to speed.  If at all possible, it would be great if we could send this out
in time to catch people for the bug day (very tight, we know), so if anyone
has a chance to check this straight away, that would be great.

Please send any amendments to Steve (steven.bethard at gmail.com) as that
probably gives us the best chance of getting it out on time.

As always, many thanks for the proofreading!

=====================
Summary Announcements
=====================

---------------------------------
Bug Day: Saturday, June 25th 2005
---------------------------------

AMK is organizing another Python Bug Day this Saturday, June 25th. If you're
looking to help out with Python, this  is a great way to get started!

Contributing Threads:

- `Bug day on the 25th?
<http://mail.python.org/pipermail/python-dev/2005-June/054126.html>`__

[SJB]


----------------------
FishEye for Python CVS
----------------------

Peter Moore has kindly set up `Fish Eye for the Python CVS repository`_.
FishEye is a repository browsing,  searching, analysis and monitoring tool,
with great features like RSS feeds, Synthetic changesets, Pretty ediffs and
SQL like searches. Check it out!

.. _Fish Eye for the Python CVS repository:
http://fisheye.cenqua.com/viewrep/python/

Contributing Threads:

- `FishEye on Python CVS Repository
<http://mail.python.org/pipermail/python-dev/2005-June/054127.html>`__

[SJB]


--------------------------------
PyPy Sprint: July 1st - 7th 2005
--------------------------------

The next `PyPy`_ sprint is scheduled right after EuroPython 2005 in
Gothenborg, Sweden. It will focus mainly on  translating PyPy to lower level
backends, so as to move away from running PyPy on top of the CPython
interpreter.  There will be newcomer-friendly introductions, and other
topics are possible, so if you have any interest in PyPy,  now is the time
to help out!

.. _PyPy: http://codespeak.net/pypy

Contributing Threads:

- `Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005
<http://mail.python.org/pipermail/python-dev/2005- June/054162.html>`__

[SJB]


---------------------------------
Reminder: Google's Summer of Code
---------------------------------

Just a reminder that the friendly folks at Python have set up a `wiki`_ and
a `mailing list`_ for questions about  `Google's Summer of Code`_. For
specific details on particular projects (e.g. what needs done to complete
Python SSL  support) participants may also ask questions to the Python-Dev
list.

.. _wiki: http://wiki.python.org/moin/CodingProjectIdeas
.. _mailing list: http://mail.python.org/mailman/listinfo/summerofcode
.. _Google's Summer of Code: http://code.google.com/summerofcode.html

[SJB]


----------------------
hashlib Review Request
----------------------
    

Gregory P. Smith noted that he has finished up the hashlib work he started
on a few months ago for patches `935454`_  and `1121611`_ (where the final
patch is).  He feels that the patch is ready, and would like anyone
interested to  review it; the patch incorporates both OpenSSL hash support
and SHA256+SHA512 support in a single module.  `The  documentation`_ can be
accessed separately, for convenience.


.. _935454: http://python.org/sf/935454
.. _1121611: http://python.org/sf/1121611
.. _The documentation:
http://electricrain.com/greg/hashlib-py25-doc/module-hashlib.html 

Contributing Threads:

- `hashlib - faster md5/sha, adds sha256/512 support
<http://mail.python.org/pipermail/python-dev/2005- June/054156.html>`__

[TAM]

=========
Summaries
=========

----------------
PEP 343 Progress
----------------

The PEP 343 discussions were mostly concluded. Guido posted the newest
version of the PEP to both Python-Dev and  Python-List and the discussions
that followed were brief and mostly in agreement with the proposal.

The PEP 343 syntax was modified slightly to require parentheses if VAR is a
comma-separated list of variables.  This  made the proposal
forward-compatible to extending the with-block for multiple resources. In
the favored extension,  the with-block would take multiple expressions in a
manner analogous to import statements::

    with EXPR1 [as VAR1], EXPR2 [as VAR2], ...:
        BLOCK

There were also some brief discussions about how with-blocks should behave
in the presence of async exceptions like  the KeyboardInterrupt generated
from a ^C. While it seemed like it would be a nice property for with-blocks
to  guarantee that the __exit__ methods would still be called in the
presence of async exceptions, making such a  guarantee proved to be too
complicated.  Thus the final conclusion, as summarized by Nick Coghlan, was
that "with  statements won't make any more guarantees about this than
try/finally statements do".

Contributing Threads:

- `PEP 343 rewrite complete
<http://mail.python.org/pipermail/python-dev/2005-June/054039.html>`__

- `For review: PEP 343: Anonymous Block Redux and Generator Enhancements
<http://mail.python.org/pipermail/python- dev/2005-June/054082.html>`__

- `PEP 343 - next steps
<http://mail.python.org/pipermail/python-dev/2005-June/054138.html>`__

- `PEP 343 question
<http://mail.python.org/pipermail/python-dev/2005-June/054230.html>`__

[SJB]


--------------
Do-While Loops
--------------

Raymond Hettinger asked for a "dowhile" loop of the form::

    dowhile <cond>:
        <body>

which would run <body> once before testing the <cond>, and then proceed as a
normal while-loop. He was subsequently  referred to `PEP 315`_, which
proposes a slightly different syntax for a similar purpose.

The discussion expanded to not only do-while loops, but also loops with
break conditions at locations other than the  beginning and the end of a
loop. A variety of syntax proposals were suggested, but none seemed
compellingly better  than the current syntax::

    while True:
        ...
        if <cond>:
            break
        ...

which supports putting the condition(s) at any location in the loop.

.. _PEP 315: http://www.python.org/peps/pep-0315.html
        
Contributing Threads:

- `Wishlist: dowhile
<http://mail.python.org/pipermail/python-dev/2005-June/054167.html>`__

[SJB]


------------------------------------
Reference Counting in Module Globals
------------------------------------

Both Michael Hudson and Skip Montanaro noticed that Py_INCREFs appeared to
be unnecessary when adding an object to a  module's globals. Armin Rigo
explained that after a module is initialized, the import mechanism makes a
"hidden"  copy of the module's dict so that the module can be reloaded. This
means that objects added as module globals will  always have an extra
reference count in this hidden dict.

However, Armin Rigo agreed with Michael Hudson that this explanation was no
longer applicable after an interpreter  shutdown. The best conclusion he
could draw in this a situation: "it's all quite obscure".

Contributing Threads:

- `refcounting vs PyModule_AddObject
<http://mail.python.org/pipermail/python-dev/2005-June/054232.html>`__
- `[Python-checkins] python/dist/src/Modules _csv.c, 1.37, 1.38
<http://mail.python.org/pipermail/python-dev/2005- June/054239.html>`__

[SJB]

-----------------------------------------
Reorganising the standard library (again)
-----------------------------------------

The ever-popular topic of reorganising the standard library came up again
this fortnight, courtesy of Reinhold  Birkenfeld.  The questions posed
included hierarchy (flat/nested), third party modules, size (batteries
included or  not), and the standard GUI toolkit.

As usual, there was a great deal of discussion, but not a great deal of
consensus about any of these (other than  that including ElementTree in the
standard library would be good), and given the amount of breakage this would
involve (and that Guido didn't weigh in at all), it seems unlikely that much
will change before Python 3000;  although Josiah Carlson indicated that he
had a patch that would avoid a lot of breakage.
    
Contributing Threads:

- `Thoughts on stdlib evolvement
<http://mail.python.org/pipermail/python-dev/2005-June/054092.html>`__

[TAM]

--------------------------
First Mac-tel, now Py-tel?
--------------------------

Guido mentioned that Intel has a free (as in beer) C `compiler for Linux`_,
and that a friend of his (who is  involved in its production and marketing)
would like to see how it performs with Python.  The compiler wasn't news  to
some of the -dev crowd, though, with Martin v. L?wis pointing out a `bug
report on the compiler`_, as well as a  `patch`_, and a `message indicating
that some people had problems`_ with the resulting interpreter.

Martin pointed out that there were some old (2002 and 2004) results
indicating that the Intel compiler was slightly  faster, but couldn't find
any results for the latest version.  Michael Hoffman gave summaries of more
testing, which  gave a 16% speed increase.  He felt that, while this was
significant, he wasted a lot of time dealing with resulting  problems with
extension modules, and so doesn't use as much any more.
    

.. _compiler for Linux:
http://www.intel.com/software/products/compilers/clin/index.htm
.. _bug report on the compiler: http://python.org/sf/1162001
.. _patch: http://python.org/sf/1162023
.. _message indicating that some people had problems:
http://mail.python.org/pipermail/python-list/2005- March/270672.html

Contributing Threads:

- `Compiling Python with Intel compiler?
<http://mail.python.org/pipermail/python-dev/2005-June/054227.html>`__

[TAM]

------------------
sys.path Behaviour
------------------

Reinhold Birkenfeld noticed that sys.path's first element is '' in
interactive sessions, but the directory  containing the script otherwise,
and wondered if this was intentional.  Guido clarified that he's always
liked it  this way, so that if you use os.path.join to join it with a script
name you don't get a spurious ".\" preprended.

The "absolutizing" of sys.path entries, however, is reasonably new; Bob
Ippolito pointed out that is also  `problematic with regards to path
hooks`_.  He has a `patch to fix it`_, but hasn't had a chance to commit it;
Phillip J. Eby noted that the patch doesn't fix completely fix it, however,
and indicated that fixing site.py with  respect to `PEP 302`_ will be quite
challenging.
 
.. _problematic with regards to path hooks:
http://mail.python.org/pipermail/python-dev/2005-April/052885.html
.. _patch to fix it: http://python.org/sf/1174614
.. _PEP 302: http://python.org/peps/pep-0302.html

Contributing Threads:

- `sys.path in interactive session
<http://mail.python.org/pipermail/python-dev/2005-June/054077.html>`__

[TAM]

----------------
More on old bugs
----------------
    
The discussion about what to do with old bugs continued this fortnight.
Against the concern about prematurely  closing old bugs, there was the
suggestion that given that there are such a huge number of open bug reports,
and  since closed bugs can be reopened, this wasn't such a problem.  It was
suggested that the act of closing a bug might  trigger activity to get it
fixed, if necessary.  The thread died off before a consensus was reached,
unfortunately.


Contributing Threads:

- `Closing old bugs
<http://mail.python.org/pipermail/python-dev/2005-June/054038.html>`__

[TAM]

---------------------------------
Improved pseudo-switch statements
---------------------------------

Skip Montanaro has been playing around with getting the Python compiler to
recognize switch-like statements and  generate O(1) code out of them.  The
rules are that the object being compared ('x') can be any expression, but
must  be precisely the same in each elif clause, the comparison operator
must be "==", and the right-hand-side of the test  must evaluate to a simple
hashable constant.  However, if evaluating 'x' has side-effects, then this
would break  code.

Various people felt that it was unwise to allow 'x' to be any expression;
Anthony Baxter suggested that one could  allow any local object that didn't
define a comparison operator.
 
Contributing Threads:

- `Multiple expression eval in compound if statement?
<http://mail.python.org/pipermail/python-dev/2005- June/054164.html>`__

[TAM]

===============
Skipped Threads
===============

- `Adventures with Decimal
<http://mail.python.org/pipermail/python-dev/2005-June/054046.html>`__

- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-June/054049.html>`__

- `AST manipulation and source code generation
<http://mail.python.org/pipermail/python-dev/2005- June/054052.html>`__

- `Vestigial code in threadmodule?
<http://mail.python.org/pipermail/python-dev/2005-June/054054.html>`__

- `[Python-checkins] python/dist/src/Lib sre_compile.py, 1.57, 1.58
<http://mail.python.org/pipermail/python- dev/2005-June/054069.html>`__

- `Split MIME headers into multiple lines near a space
<http://mail.python.org/pipermail/python-dev/2005- June/054090.html>`__

- `python running in several threads
<http://mail.python.org/pipermail/python-dev/2005-June/054110.html>`__

- `problem installing current cvs
<http://mail.python.org/pipermail/python-dev/2005-June/054116.html>`__

- `b32encode and NUL bytes
<http://mail.python.org/pipermail/python-dev/2005-June/054130.html>`__

- `Example workaround classes for using Unicode with csv module...
<http://mail.python.org/pipermail/python- dev/2005-June/054129.html>`__

- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-June/054128.html>`__

- `[Python-checkins] python/dist/src/Doc/lib libtokenize.tex, 1.5, 1.6
<http://mail.python.org/pipermail/python- dev/2005-June/054131.html>`__

- `Five patch reviews & patch request
<http://mail.python.org/pipermail/python-dev/2005-June/054135.html>`__

- `AIX 4.3, Python 2.4.1 fails in test_exceptions with a core dump
<http://mail.python.org/pipermail/python- dev/2005-June/054197.html>`__

- `PEP 342 - Enhanced Iterators
<http://mail.python.org/pipermail/python-dev/2005-June/054201.html>`__

- `A bug in pyconfig.h under Linux?
<http://mail.python.org/pipermail/python-dev/2005-June/054214.html>`__

- `Dynamic class inheritance && something else
<http://mail.python.org/pipermail/python-dev/2005- June/054218.html>`__

- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-June/054229.html>`__


From rhamph at gmail.com  Sun Jun 26 11:34:07 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 26 Jun 2005 03:34:07 -0600
Subject: [Python-Dev] Adding Python-Native Threads
Message-ID: <aac2c7cb0506260234286befd3@mail.gmail.com>

There are some very serious problems with threads today.  Events are
often proposed as a solution to these problems, but they have their own
issues.  A detailed analysis of the two options and some solutions
(most of which are used in my proposal) can be found in Why Events Are
A Bad Idea [0].  I'm going to skip the event aspects and sum up the
thread problems here:

* Expensive (resident memory, address space, time to create and to
             switch between)
* Finely-grained atomicity (python is coarser than C, but still finer
                            than we need)
* Unpredictable (switching between them is not deterministic)
* Uninterruptible (no way to kill them, instead you must set a flag and
                   make sure they check it regularly)
* Fail silently (if they die with an exception it gets printed to the
                 console but the rest of the program is left uninformed)

To resolve these problems I propose adding lightweight cooperative
threads to Python.  They can be built around a Frame object, suspending
and resuming like generators do.

That much is quite easy.  Avoiding the C stack is a bit harder.
However, it can be done if we introduce a new operator for threaded
calls (which I refer to as non-atomic calls), which I propose be
"func@()".  As a bonus this prevents any operation which doesn't use
the non-atomic call syntax from triggering a thread switch accidentally.

After that you need a way of creating new threads.  I propose we use a
"sibling func@()" statement (note that the "@()" would be parsed as
part of the statement itself, and not as an expression).  It would
create a new thread and run func in it, but more importantly it would
also "prejoin" the new thread to the current function.  This means that
for the current function to return to its parent, all of the threads it
created must first return themselves.  It also means that an exception
in any of them will be propagated to the parent of the creator, after
interrupting all of its siblings (with an Interrupted exception) so
that it is permitted to return.

Now for an example of how this would work in practice, using an echo
server.  Note that I am assuming a "con" module (short for concurrent)
that would contain standard functions to support these python-native
threads.


from con import bootstrap, tcplisten, ipv6addr, ipv4addr

def main():
    main_s = tcplisten@(ipv6addr('::1', 6839), ipv4addr('127.0.0.1', 6839))
    while 1:
        sibling echo@(main_s.accept@())

def echo(s):
    try:
        try:
            while 1:
                s.write@(s.read@())
        except (EOFError, IOError):
            pass
    finally:
        s.close()

if __name__ == '__main__':
    try:
        bootstrap(main)
    except KeyboardInterrupt:
        pass


And here is a diagram of the resulting cactus stack, assuming three
clients, two of which are reading and the other is writing.


bootstrap - main - accept
             |
            echo - read
             |
            echo - read
             |
            echo - write


Some notes:
* An idle@() function would be used for all thread switches.  I/O
  functions such as read@() and write@() would use it internally, and
  idle would internally call a function to do poll() on all file
  descriptors.
* idle is the function that will raise Interrupted
* It should be illegal to attempt a non-atomic call once your thread is
  in an interrupted state.  This ensures you cannot get "interrupted
  twice", by having something further up the call stack fail and
  interrupt you while you are already handling an interruption.
* Being a cooperative system, it does not allow you to use multiple
  CPUs simultaneously.  I don't regard this as a significant problem,
  and in any case there are better ways use multiple CPUs if you need to.


References:
[0] http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/index.html

-- 
Adam Olsen, aka Rhamphoryncus

From ronaldoussoren at mac.com  Sun Jun 26 12:17:53 2005
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 26 Jun 2005 12:17:53 +0200
Subject: [Python-Dev] Adding Python-Native Threads
In-Reply-To: <aac2c7cb0506260234286befd3@mail.gmail.com>
References: <aac2c7cb0506260234286befd3@mail.gmail.com>
Message-ID: <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com>


On 26-jun-2005, at 11:34, Adam Olsen wrote:


>
> To resolve these problems I propose adding lightweight cooperative
> threads to Python.  They can be built around a Frame object,  
> suspending
> and resuming like generators do.
>
> That much is quite easy.  Avoiding the C stack is a bit harder.
> However, it can be done if we introduce a new operator for threaded
> calls (which I refer to as non-atomic calls), which I propose be
> "func@()".  As a bonus this prevents any operation which doesn't use
> the non-atomic call syntax from triggering a thread switch  
> accidentally.

Have a look at stackless python. http://www.stackless.com/

Ronald


From florian.proff.schulze at gmx.net  Sun Jun 26 12:26:07 2005
From: florian.proff.schulze at gmx.net (Florian Schulze)
Date: Sun, 26 Jun 2005 12:26:07 +0200
Subject: [Python-Dev] Adding Python-Native Threads
References: <aac2c7cb0506260234286befd3@mail.gmail.com>
Message-ID: <opssy29tz8ttxc4i@news.gmane.org>

Also look at greenlets, they are in the py lib http://codespeak.net/py

Regards,
Florian Schulze


From rhamph at gmail.com  Sun Jun 26 13:04:17 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 26 Jun 2005 05:04:17 -0600
Subject: [Python-Dev] Adding Python-Native Threads
In-Reply-To: <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com>
References: <aac2c7cb0506260234286befd3@mail.gmail.com>
	<4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com>
Message-ID: <aac2c7cb05062604043b22c3a8@mail.gmail.com>

On 6/26/05, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> Have a look at stackless python. http://www.stackless.com/

On 6/26/05, Florian Schulze <florian.proff.schulze at gmx.net> wrote:
> Also look at greenlets, they are in the py lib http://codespeak.net/py

While internally Stackless and Greenlets may (or may not) share a lot
with my proposed python-native threads, they differ greatly in their
intent and the resulting interface they expose.  Stackless and
Greenlets are both tools for building an architecture that uses
threads, while my python-native threads IS that resulting
architecture.

For example, with Greenlets you would use the .switch() method of a
specific greenlet instance to switch to it, and with my python-native
threads you would use the global idle() function which would decide
for itself which thread to switch to.

-- 
Adam Olsen, aka Rhamphoryncus

From rjohnsto at sfu.ca  Thu Jun 23 21:05:52 2005
From: rjohnsto at sfu.ca (Robert W. Johnstone)
Date: Thu, 23 Jun 2005 12:05:52 -0700 (PDT)
Subject: [Python-Dev] Subclassing in 'C'
Message-ID: <4330.199.60.7.247.1119553552.squirrel@webmail.fas.sfu.ca>

Hello,

I'm trying to develop an extension module in C, but am having some
difficulties.  I thought about posting to the general python mailling
list, but since it has to do with the C API I figured this was the correct
location.

My difficulty lies with subclassing a class provided by another external
module.  I have access to the structure definition, so I can determine the
size fo the tp_basicsize member of the PyTypeObject structure.  What I
can't seem to determine is value for the tp_base element.

This is my current "best guess".

	PyObject* superclass;
	superclass = PyObject_GetAttrString( module, "SuperClass" );
	if ( !superclass ) return;
	type.tp_base = superclass->ob_type;
	Py_DECREF( superclass );

	type.tp_new = PyType_GenericNew;
	if ( PyType_Ready( &type ) < 0 ) return;

Unfortunately, I seem to be inheriting from the type object for
SuperClass, and not from SuperClass itself.

I would greatly appreciate some direction on this issue.

Thank-you,

----------------------------------------
Robert Johnstone
Simon Fraser University
http://www.sfu.ca/~rjohnsto/


From ronaldoussoren at mac.com  Sun Jun 26 15:26:35 2005
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 26 Jun 2005 15:26:35 +0200
Subject: [Python-Dev] Subclassing in 'C'
In-Reply-To: <4330.199.60.7.247.1119553552.squirrel@webmail.fas.sfu.ca>
References: <4330.199.60.7.247.1119553552.squirrel@webmail.fas.sfu.ca>
Message-ID: <939D3972-F16D-4E6E-8356-A1A16EC2FC2B@mac.com>


On 23-jun-2005, at 21:05, Robert W. Johnstone wrote:

> Hello,
>
> I'm trying to develop an extension module in C, but am having some
> difficulties.  I thought about posting to the general python mailling
> list, but since it has to do with the C API I figured this was the  
> correct
> location.

The python list would have been the right location.

>
> My difficulty lies with subclassing a class provided by another  
> external
> module.  I have access to the structure definition, so I can  
> determine the
> size fo the tp_basicsize member of the PyTypeObject structure.  What I
> can't seem to determine is value for the tp_base element.
>
> This is my current "best guess".
>
>     PyObject* superclass;
>     superclass = PyObject_GetAttrString( module, "SuperClass" );
>     if ( !superclass ) return;
>     type.tp_base = superclass->ob_type;

type.tp_base = (PyTypeObject*)superclass;

>     Py_DECREF( superclass );
And don't decref the superclass. You may want to add a typecheck that
checks if the superclass is a type.

Ronald

From gvanrossum at gmail.com  Sun Jun 26 16:56:47 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun, 26 Jun 2005 07:56:47 -0700
Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and
	Integers
In-Reply-To: <Pine.LNX.4.63.0506211442360.22846@leviathan.kdart.com>
References: <2m1x6z7qu6.fsf@starship.python.net>
	<Pine.LNX.4.63.0506190029250.12171@leviathan.kdart.com>
	<20050619010354.72AE.JCARLSON@uci.edu>
	<Pine.LNX.4.63.0506190110420.12171@leviathan.kdart.com>
	<ca471dc205062007066ee2883e@mail.gmail.com>
	<Pine.LNX.4.63.0506202033320.12294@leviathan.kdart.com>
	<ca471dc2050621031618063e94@mail.gmail.com>
	<Pine.LNX.4.63.0506211442360.22846@leviathan.kdart.com>
Message-ID: <ca471dc2050626075637ca3d9b@mail.gmail.com>

[me]
> > (c) The right place to do the overflow checks is in the API wrappers,
> > not in the integer types.

[Keith Dart]
> That would be the "traditional" method.
> 
> I was trying to keep it an object-oriented API. What should "know" the
> overflow condition is the type object itself. It raises OverFlowError any
> time this occurs, for any operation, implicitly. I prefer to catch errors
> earlier, rather than later.

Isn't clear to me at all. I might compute a value using some formula
that exceeds 2**32 in some intermediate result but produces an
in-range value in the end. That should be acceptable as an argument. I
also don't see why one approach is more OO than another; sounds like
you have a case of buzzworditis. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Sun Jun 26 17:03:36 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 11:03:36 -0400
Subject: [Python-Dev] Adding Python-Native Threads
In-Reply-To: <aac2c7cb05062604043b22c3a8@mail.gmail.com>
References: <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com>
	<aac2c7cb0506260234286befd3@mail.gmail.com>
	<4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com>
Message-ID: <5.1.1.6.0.20050626104826.01fa7f88@mail.telecommunity.com>

At 05:04 AM 6/26/2005 -0600, Adam Olsen wrote:
>On 6/26/05, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> > Have a look at stackless python. http://www.stackless.com/
>
>On 6/26/05, Florian Schulze <florian.proff.schulze at gmx.net> wrote:
> > Also look at greenlets, they are in the py lib http://codespeak.net/py
>
>While internally Stackless and Greenlets may (or may not) share a lot
>with my proposed python-native threads, they differ greatly in their
>intent and the resulting interface they expose.  Stackless and
>Greenlets are both tools for building an architecture that uses
>threads, while my python-native threads IS that resulting
>architecture.
>
>For example, with Greenlets you would use the .switch() method of a
>specific greenlet instance to switch to it, and with my python-native
>threads you would use the global idle() function which would decide
>for itself which thread to switch to.

See PEP 342 for a competing proposal, that includes a short (<50 lines) 
co-operative threading example.  It uses the 'yield' keyword instead of 
your '@' proposal, and already has a Sourceforge patch implementing it.

It does not propose a specific built-in idler or I/O handler, but such 
things could be added later.  In the short term, I think it's enough that 
Python be *able* to have such things (without using Stackless or 
Greenlets), and maybe include an example of a run loop.  I don't currently 
plan to push for a "standard" or "official" run loop implementation in 
Python 2.5, however, unless all the relevant stakeholders just happen to 
come to some kind of agreement that makes it happen.

Note that while having a universal run loop is a worthy goal, actually 
implementing one is hard.  The experts in this are definitely in the 
Twisted camp, as they have implemented run loops for numerous platforms to 
support compatibility w/various GUI frameworks' event loops.  To some 
extent, this means that the run loop needs to be an interface with multiple 
implementations, and I think that we would need a standard library 
implementation of PEP 246 in order to support it properly.

Anyway, my point is that even deciding on standards for a universal run 
loop is a fairly big project, one that possibly even deserves its own SIG 
to put together.  By contrast, the implementation of simulated threads and 
non-blocking function calls was a weekend project.


From reinhold-birkenfeld-nospam at wolke7.net  Sun Jun 26 18:57:39 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Sun, 26 Jun 2005 18:57:39 +0200
Subject: [Python-Dev] Some RFE for review
Message-ID: <d9mmd6$gjf$1@sea.gmane.org>

Hi,

while bugs and patches are sometimes tricky to close, RFE can be very easy
to decide whether to implement in the first place. So what about working a
bit on this front? Here are several RFE reviewed, perhaps some can be
closed ("should" is always from submitter's point of view):

1193128:
str.translate(None, delchars) should be allowed and only delete delchars
from the string.
Review: may be a good feature, although string._idmap can be passed as the
first parameter too.

1226256:
The "path" module by Jason Orendorff should be in the standard library.
http://www.jorendorff.com/articles/python/path/
Review: the module is great and seems to have a large user base. On c.l.py
there are frequent praises about it.

1216944:
urllib(2) should gain a dict mapping HTTP status codes to the correspondig
status/error text.
Review: I can't see anything speaking against it.

1214675:
warnings should get a removefilter() method. An alternative would be to
fully document the "filters" attribute to allow direct tinkering with it.
Review: As mwh said in a comment, this isn't Java, so the latter may be
the way to go.

1205239:
Shift operands should be allowed to be negative integers, so e.g.
a << -2  is the same as  a >> 2.
Review: Allowing this would open a source of bugs previously well identifiable.

1152248:
In order to read "records" separated by something other than newline, file objects
should either support an additional parameter (the separator) to (x)readlines(),
or gain an additional method which does this.
Review: The former is a no-go, I think, because what is read won't be lines.
The latter is further complicating the file interface, so I would follow the
principle that not every 3-line function should be builtin.

1110010:
A function "attrmap" should be introduced which is used as follows:
attrmap(x)['att'] == getattr(x, 'att')
The submitter mentions the use case of new-style classes without a __dict__ used
at the right of %-style string interpolation.
Review: I don't know whether this is worth it.

1052098:
An environment variable should be supported to set the default encoding.
Review: If one wants this for a single application, he can still implement it himself.

985094:
getattr should support a callable as the second argument, used as follows:
getattr(obj, func) == func(obj)
Review: Highly unintuitive to me.

That's all for today; sorry if it was too much ;)

Reinhold

-- 
Mail address is perfectly valid!


From pje at telecommunity.com  Sun Jun 26 20:17:15 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 14:17:15 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
In-Reply-To: <d9mmd6$gjf$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>

At 06:57 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote:
>1226256:
>The "path" module by Jason Orendorff should be in the standard library.
>http://www.jorendorff.com/articles/python/path/
>Review: the module is great and seems to have a large user base. On c.l.py
>there are frequent praises about it.

I would note that there are some things in the interface that should be 
cleaned up before it becomes a stdlib module.  It has many ways to do the 
same thing, and many of its property and method names are confusing because 
they either do the same thing as a standard function, but have a different 
name (like the 'parent' property that is os.path.dirname in disguise), or 
they have the same name as a standard function but do something different 
(like the 'listdir()' method that returns full paths rather than just 
filenames).  I'm also not keen on the fact that it makes certain things 
properties whose value can change over time; i.e. ctime/mtime/atime and 
size really shouldn't be properties, but rather methods.  I'm also not sure 
how I feel about all the read/write methods that hide the use of file 
objects; these seem like they should remain file object methods, especially 
since PEP 343 will allow easy closing with something like:

     with closing(some_path.open('w')) as f:
         f.write(data)

Granted, this is more verbose than:

     some_path.write_bytes(data)

But brevity isn't always everything.  If these methods are kept I would 
suggest using different names, like "set_bytes()", "set_text()", and 
"set_lines()", because "write" sounds like something you do on an ongoing 
basis to a stream, while these methods just replace the file's entire contents.

Aside from all these concerns, I'm +1 on adding the module.

Here's my list of suggested changes:

* path.joinpath(*args)   -> path.subpath(*args)
* path.listdir()         -> path.subpaths()
* path.splitall()        -> path.parts()
* path.parent            -> path.dirname (and drop dirname() method)
* path.name              -> path.filename (and drop basename() method)
* path.namebase          -> path.filebase (maybe something more descriptive?)
* path.atime/mtime/ctime -> path.atime(), path.mtime(), path.ctime()
* path.size              -> path.filesize()

* drop getcwd(); it makes no sense on a path instance

* add a samepath() method that compares the absolute, case and 
path-normalized versions of two paths, and a samerealpath() method that 
does the same but with symlinks resolved.

And, assuming these file-content methods are kept:

* path.bytes()         -> path.get_file_bytes()
* path.write_bytes()   -> path.set_file_bytes() and path.append_file_bytes()
* path.text()          -> path.get_file_text()
* path.write_text()    -> path.set_file_text() and path.append_file_text()
* path.lines()         -> path.get_file_lines()
* path.write_lines()   -> path.set_file_lines() and path.append_file_lines()


From oren.tirosh at gmail.com  Sun Jun 26 20:52:56 2005
From: oren.tirosh at gmail.com (Oren Tirosh)
Date: Sun, 26 Jun 2005 21:52:56 +0300
Subject: [Python-Dev] Adding Python-Native Threads
In-Reply-To: <aac2c7cb0506260234286befd3@mail.gmail.com>
References: <aac2c7cb0506260234286befd3@mail.gmail.com>
Message-ID: <7168d65a05062611525e5868b5@mail.gmail.com>

On 6/26/05, Adam Olsen <rhamph at gmail.com> wrote:
...
> To resolve these problems I propose adding lightweight cooperative
> threads to Python.  

Speaking of lightweight cooperative threads - has anyone recently
tried to build Python with the pth option? It doesn't quite work out
of the box. How much maintenance would be required to make it work
again?

Oren

From reinhold-birkenfeld-nospam at wolke7.net  Sun Jun 26 21:00:21 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Sun, 26 Jun 2005 21:00:21 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
References: <d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <d9mtj7$28s$1@sea.gmane.org>

Phillip J. Eby wrote:
> At 06:57 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote:
>>1226256:
>>The "path" module by Jason Orendorff should be in the standard library.
>>http://www.jorendorff.com/articles/python/path/
>>Review: the module is great and seems to have a large user base. On c.l.py
>>there are frequent praises about it.

[...]

> Aside from all these concerns, I'm +1 on adding the module.
> 
> Here's my list of suggested changes:

[...]

I agree with your changes list.

One more issue is open: the one of naming. As "path" is already the name of
a module, what would the new object be called to avoid confusion? pathobj?
objpath? Path?

Reinhold

-- 
Mail address is perfectly valid!


From hoffman at ebi.ac.uk  Sun Jun 26 21:19:29 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Sun, 26 Jun 2005 20:19:29 +0100
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
References: <d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>

On Sun, 26 Jun 2005, Phillip J. Eby wrote:

> * drop getcwd(); it makes no sense on a path instance

Personally I use path.getcwd() as a class method all the time. It
makes as much sense as fromkeys() does on a dict instance, which is
technically possible but non-sensical.

> And, assuming these file-content methods are kept:
>
> * path.bytes()         -> path.get_file_bytes()
> * path.write_bytes()   -> path.set_file_bytes() and path.append_file_bytes()
> * path.text()          -> path.get_file_text()
> * path.write_text()    -> path.set_file_text() and path.append_file_text()
> * path.lines()         -> path.get_file_lines()
> * path.write_lines()   -> path.set_file_lines() and path.append_file_lines()

I don't know how often these are used. I don't use them myself. I am
mainly interested in this module so that I don't have to use os.path
anymore.

Reinhold Birkenfeld wrote:

> One more issue is open: the one of naming. As "path" is already the
> name of a module, what would the new object be called to avoid
> confusion? pathobj?  objpath? Path?

I would argue for Path. It fits with the recent cases of:

from sets import Set
from decimal import Decimal
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From skip at pobox.com  Sun Jun 26 21:31:21 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 26 Jun 2005 14:31:21 -0500
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
References: <d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <17087.649.730816.972325@montanaro.dyndns.org>

    Phillip> It has many ways to do the same thing, and many of its property
    Phillip> and method names are confusing because they either do the same
    Phillip> thing as a standard function, but have a different name (like
    Phillip> the 'parent' property that is os.path.dirname in disguise), or
    Phillip> they have the same name as a standard function but do something
    Phillip> different (like the 'listdir()' method that returns full paths
    Phillip> rather than just filenames).  

To the extent that the path module tries to provide a uniform abstraction
that's not saddled with a particular way of doing things (e.g., the Unix way
or the Windows way), I don't think this is necessarily a bad thing.

Skip

From Jack.Jansen at cwi.nl  Sun Jun 26 23:48:54 2005
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Sun, 26 Jun 2005 23:48:54 +0200
Subject: [Python-Dev] [Python-checkins] python/dist/src/Tools/bgen/bgen
	bgenGenerator.py, 1.17, 1.18 bgenObjectDefinition.py, 1.29,
	1.30 bgenType.py, 1.15, 1.16 bgenVariable.py, 1.6,
	1.7 scantools.py, 1.37, 1.38
In-Reply-To: <E1Dlu8e-0004o2-2U@sc8-pr-cvs1.sourceforge.net>
References: <E1Dlu8e-0004o2-2U@sc8-pr-cvs1.sourceforge.net>
Message-ID: <74A681D5-F0EE-4E08-A6FC-264158C6A5E8@cwi.nl>


On 24-jun-2005, at 21:46, tim_one at users.sourceforge.net wrote:

> Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18095/Tools/ 
> bgen/bgen
>
> Modified Files:
>     bgenGenerator.py bgenObjectDefinition.py bgenType.py
>     bgenVariable.py scantools.py
> Log Message:
> Normalize whitespace to avoid offending Bug Day volunteers.

Argh, sorry... I thought I had all my machines educated to do tab  
expansion for Python, but apparently not...
--
Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma  
Goldman



From walter at livinglogic.de  Mon Jun 27 00:22:46 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?D=F6rwald_Walter?=)
Date: Mon, 27 Jun 2005 00:22:46 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <1BCEF0D1-5E33-4485-80F6-010E8C7EC2AC@livinglogic.de>

Phillip J. Eby wrote:

> [...]
> I'm also not keen on the fact that it makes certain things
> properties whose value can change over time; i.e. ctime/mtime/atime  
> and
> size really shouldn't be properties, but rather methods.

I think ctime, mtime and atime should be (or return)  
datetime.datetime objects instead of integer timestamps.

Bye,
    Walter D?rwald


From tim.peters at gmail.com  Mon Jun 27 00:59:35 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 26 Jun 2005 18:59:35 -0400
Subject: [Python-Dev] [Python-checkins] python/dist/src/Tools/bgen/bgen
	bgenGenerator.py, 1.17, 1.18 bgenObjectDefinition.py, 1.29,
	1.30 bgenType.py, 1.15, 1.16 bgenVariable.py, 1.6,
	1.7 scantools.py, 1.37, 1.38
In-Reply-To: <74A681D5-F0EE-4E08-A6FC-264158C6A5E8@cwi.nl>
References: <E1Dlu8e-0004o2-2U@sc8-pr-cvs1.sourceforge.net>
	<74A681D5-F0EE-4E08-A6FC-264158C6A5E8@cwi.nl>
Message-ID: <1f7befae05062615594fb89b7d@mail.gmail.com>

[tim_one at users.sourceforge.net]
>> Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen
>> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18095/Tools/
>> bgen/bgen
>>
>> Modified Files:
>>     bgenGenerator.py bgenObjectDefinition.py bgenType.py
>>     bgenVariable.py scantools.py
>> Log Message:
>> Normalize whitespace to avoid offending Bug Day volunteers.

[Jack Jansen]
> Argh, sorry... I thought I had all my machines educated to do tab
> expansion for Python, but apparently not...

You probably do.  I run Tools/scripts/reindent.py over the whole
source tree from time to time, and check in whatever it changes. 
Converting tabs to 4-space indents is one of the things it does, but
not the only thing.  Most often these days it finds trailing
whitespace at the end of a line, and snips it off.  Once in a great
while, that catches code that *depends* on trailing whitespace
(typically in a long string literal), which is viciously obscure
(nobody looking at the file can see the dependence, and editors may
remove it).  It also removes empty lines at the end of a .py file,
ensures that the final line ends with a newline, and changes "oddball"
indent levels to 4-space indents (curiously, that one triggers
2nd-most often these days).

In any case, it's not like this is work <wink> -- no need to feel apologetic.

From tim.peters at gmail.com  Mon Jun 27 01:15:47 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 26 Jun 2005 19:15:47 -0400
Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib Cookie.py,
	1.17, 1.18
In-Reply-To: <E1DmeHD-0001Jx-QX@sc8-pr-cvs1.sourceforge.net>
References: <E1DmeHD-0001Jx-QX@sc8-pr-cvs1.sourceforge.net>
Message-ID: <1f7befae050626161574adbe07@mail.gmail.com>

[birkenfeld at users.sourceforge.net]
> Update of /cvsroot/python/python/dist/src/Lib
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4891/Lib
> 
> Modified Files:
>        Cookie.py
> Log Message:
> bug [ 1108948 ] Cookie.py produces invalid code
> 
> 
> 
> Index: Cookie.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/Cookie.py,v
> retrieving revision 1.17
> retrieving revision 1.18
> diff -u -d -r1.17 -r1.18
> --- Cookie.py   20 Oct 2003 14:01:49 -0000      1.17
> +++ Cookie.py   26 Jun 2005 21:02:49 -0000      1.18
> @@ -470,9 +470,9 @@
>     def js_output(self, attrs=None):
>         # Print javascript
>         return """
> -        <SCRIPT LANGUAGE="JavaScript">
> +        <script type="text/javascript">
>         <!-- begin hiding
> -        document.cookie = \"%s\"
> +        document.cookie = \"%s\";
>         // end hiding -->
>         </script>
>         """ % ( self.OutputString(attrs), )

I assume this accounts for the current failure of test_cookie:

test_cookie
test test_cookie produced unexpected output:
**********************************************************************
*** mismatch between line 19 of expected output and line 19 of actual output:
-         <SCRIPT LANGUAGE="JavaScript">
+         <script type="text/javascript">
*** mismatch between line 21 of expected output and line 21 of actual output:
-         document.cookie = "Customer="WILE_E_COYOTE"; Path=/acme; Version=1;"
+         document.cookie = "Customer="WILE_E_COYOTE"; Path=/acme; Version=1;";
?                                                                             +
*** mismatch between line 26 of expected output and line 26 of actual output:
-         <SCRIPT LANGUAGE="JavaScript">
+         <script type="text/javascript">
*** mismatch between line 28 of expected output and line 28 of actual output:
-         document.cookie = "Customer="WILE_E_COYOTE"; Path=/acme;"
+         document.cookie = "Customer="WILE_E_COYOTE"; Path=/acme;";
?                                                                  +
**********************************************************************

From t-meyer at ihug.co.nz  Mon Jun 27 02:08:34 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Mon, 27 Jun 2005 12:08:34 +1200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E80328E26F@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E801DB0303@its-xchg4.massey.ac.nz>

[Reinhold Birkenfeld]
>> One more issue is open: the one of naming. As "path" is already the 
>> name of a module, what would the new object be called to avoid 
>> confusion? pathobj?  objpath? Path?

[Michael Hoffman]
> I would argue for Path.

Granted "path" is actually os.path, but I don't think it's wise to have
stdlib modules whose names are differentiated only by case, especially on
Windows (and other case-insensitive filesystems).

=Tony.Meyer


From pje at telecommunity.com  Mon Jun 27 02:46:21 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 20:46:21 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <d9mtj7$28s$1@sea.gmane.org>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050626204554.020efb88@mail.telecommunity.com>

At 09:00 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote:
>One more issue is open: the one of naming. As "path" is already the name of
>a module, what would the new object be called to avoid confusion? pathobj?
>objpath? Path?

I was thinking os.Path, myself.


From pje at telecommunity.com  Mon Jun 27 02:49:18 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 20:49:18 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>

At 08:19 PM 6/26/2005 +0100, Michael Hoffman wrote:
>On Sun, 26 Jun 2005, Phillip J. Eby wrote:
>
> > * drop getcwd(); it makes no sense on a path instance
>
>Personally I use path.getcwd() as a class method all the time. It
>makes as much sense as fromkeys() does on a dict instance, which is
>technically possible but non-sensical.

It's also duplication with os.path; I'm -1 on creating a new staticmethod 
for it.


>Reinhold Birkenfeld wrote:
>
> > One more issue is open: the one of naming. As "path" is already the
> > name of a module, what would the new object be called to avoid
> > confusion? pathobj?  objpath? Path?
>
>I would argue for Path. It fits with the recent cases of:
>
>from sets import Set
>from decimal import Decimal

I like it too, as a class in the os module.


From pje at telecommunity.com  Mon Jun 27 02:52:19 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 20:52:19 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <17087.649.730816.972325@montanaro.dyndns.org>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050626204937.036fa8f0@mail.telecommunity.com>

At 02:31 PM 6/26/2005 -0500, Skip Montanaro wrote:
>     Phillip> It has many ways to do the same thing, and many of its property
>     Phillip> and method names are confusing because they either do the same
>     Phillip> thing as a standard function, but have a different name (like
>     Phillip> the 'parent' property that is os.path.dirname in disguise), or
>     Phillip> they have the same name as a standard function but do something
>     Phillip> different (like the 'listdir()' method that returns full paths
>     Phillip> rather than just filenames).
>
>To the extent that the path module tries to provide a uniform abstraction
>that's not saddled with a particular way of doing things (e.g., the Unix way
>or the Windows way), I don't think this is necessarily a bad thing.

I'm confused by your statements.  First, I didn't notice the path module 
providing any OS-abstractions that aren't already provided by 
os.path.  Second, using inconsistent and confusing names is pretty much 
always a bad thing.  :)


From pje at telecommunity.com  Mon Jun 27 02:54:59 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 20:54:59 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <1BCEF0D1-5E33-4485-80F6-010E8C7EC2AC@livinglogic.de>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>

At 12:22 AM 6/27/2005 +0200, D?rwald Walter wrote:
>Phillip J. Eby wrote:
>>[...]
>>I'm also not keen on the fact that it makes certain things
>>properties whose value can change over time; i.e. ctime/mtime/atime
>>and
>>size really shouldn't be properties, but rather methods.
>
>I think ctime, mtime and atime should be (or return)
>datetime.datetime objects instead of integer timestamps.

With what timezone?  I don't think that can be done portably and 
unambiguously, so I'm -1 on that.


From pje at telecommunity.com  Mon Jun 27 02:57:14 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 20:57:14 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
 forreview)
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E801DB0303@its-xchg4.massey. ac.nz>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E80328E26F@its-xchg4.massey.ac.nz>
Message-ID: <5.1.1.6.0.20050626205505.03700570@mail.telecommunity.com>

At 12:08 PM 6/27/2005 +1200, Tony Meyer wrote:
>[Reinhold Birkenfeld]
> >> One more issue is open: the one of naming. As "path" is already the
> >> name of a module, what would the new object be called to avoid
> >> confusion? pathobj?  objpath? Path?
>
>[Michael Hoffman]
> > I would argue for Path.
>
>Granted "path" is actually os.path, but I don't think it's wise to have
>stdlib modules whose names are differentiated only by case, especially on
>Windows (and other case-insensitive filesystems).

This is the name of a *class*, not a module.  I.e., we are discussing 
adding a Path class to the 'os' module, that implements the interface of 
the "path" module.

We can't call it "path" (as a top-level module) because the interface will 
not be backward-compatible with current uses and installations of the 
"path" module.


From t-meyer at ihug.co.nz  Mon Jun 27 02:58:50 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Mon, 27 Jun 2005 12:58:50 +1200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E80328E302@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E801B0F794@its-xchg4.massey.ac.nz>

[Reinhold Birkenfeld]
>>>> One more issue is open: the one of naming. As "path" is already
>>>> the name of a module, what would the new object be called to
>>>> avoid confusion? pathobj?  objpath? Path?

[Michael Hoffman]
>>> I would argue for Path.

[Tony Meyer
>> Granted "path" is actually os.path, but I don't think it's 
>> wise to have stdlib modules whose names are differentiated only
>> by case, especially on Windows (and other case-insensitive
>> filesystems).

[Phillip J. Eby]
> This is the name of a *class*, not a module.

Sorry - it sounded like the idea was to put this class in a module by itself
(i.e. the class would be os.Path.Path).

> I.e., we are discussing 
> adding a Path class to the 'os' module, that implements the 
> interface of the "path" module.

In that case, I would argue against Path as the name of the class because
it's confusing to have "os.path" be the path module, and "os.Path" be an
class that provides an interface to that module.

I think differentiating things solely on the case of the name is a bad idea.

=Tony.Meyer


From skip at pobox.com  Mon Jun 27 03:16:18 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 26 Jun 2005 20:16:18 -0500
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <1BCEF0D1-5E33-4485-80F6-010E8C7EC2AC@livinglogic.de>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<1BCEF0D1-5E33-4485-80F6-010E8C7EC2AC@livinglogic.de>
Message-ID: <17087.21346.494459.908283@montanaro.dyndns.org>


    Walter> I think ctime, mtime and atime should be (or return)
    Walter> datetime.datetime objects instead of integer timestamps.

+1

Skip

From skip at pobox.com  Mon Jun 27 03:29:11 2005
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 26 Jun 2005 20:29:11 -0500
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <5.1.1.6.0.20050626204937.036fa8f0@mail.telecommunity.com>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626204937.036fa8f0@mail.telecommunity.com>
Message-ID: <17087.22119.665737.68123@montanaro.dyndns.org>


    Phillip> ... but have a different name (like the 'parent' property that
    Phillip> is os.path.dirname in disguise) ...

    Phillip> ... (like the 'listdir()' method that returns full paths rather
    Phillip> than just filenames).

    Skip> To the extent that the path module tries to provide a uniform
    Skip> abstraction that's not saddled with a particular way of doing
    Skip> things (e.g., the Unix way or the Windows way), I don't think this
    Skip> is necessarily a bad thing.

    Phillip> I'm confused by your statements.  First, I didn't notice the
    Phillip> path module providing any OS-abstractions that aren't already
    Phillip> provided by os.path.  Second, using inconsistent and confusing
    Phillip> names is pretty much always a bad thing.  :)

Sorry, let me be more explicit.  "dirname" is the Unix name for "return the
parent of this path".  In the Windows and Mac OS9 worlds (ignore any
possible Posix compatibility for a moment), my guess would be it's probably
something else.  I suspect listdir gets its "return individual filenames
instead of full paths" from the semantics of the Posix opendir/readdir/
closedir functions.  If it makes more sense to return strings that represent
full paths or new path objects that have been absolute-ified, then the minor
semantic change going from os.path.listdir() to the listdir method of
Jason's path objects isn't a big problem to me.

Skip


From bob at redivi.com  Mon Jun 27 03:26:40 2005
From: bob at redivi.com (Bob Ippolito)
Date: Sun, 26 Jun 2005 21:26:40 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
Message-ID: <03118D28-1405-49E8-BF23-729AD0DF54BD@redivi.com>


On Jun 26, 2005, at 8:54 PM, Phillip J. Eby wrote:

> At 12:22 AM 6/27/2005 +0200, D?rwald Walter wrote:
>
>> Phillip J. Eby wrote:
>>
>>> [...]
>>> I'm also not keen on the fact that it makes certain things
>>> properties whose value can change over time; i.e. ctime/mtime/atime
>>> and
>>> size really shouldn't be properties, but rather methods.
>>>
>>
>> I think ctime, mtime and atime should be (or return)
>> datetime.datetime objects instead of integer timestamps.
>>
>
> With what timezone?  I don't think that can be done portably and
> unambiguously, so I'm -1 on that.

That makes no sense, timestamps aren't any better, and datetime  
objects have no time zone set by default anyway.   
datetime.fromtimestamp(time.time()) gives you the same thing as  
datetime.now().

-bob


From pje at telecommunity.com  Mon Jun 27 05:03:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 23:03:23 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <17087.22119.665737.68123@montanaro.dyndns.org>
References: <5.1.1.6.0.20050626204937.036fa8f0@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626204937.036fa8f0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050626230106.020f6cf8@mail.telecommunity.com>

At 08:29 PM 6/26/2005 -0500, Skip Montanaro wrote:

>     Phillip> ... but have a different name (like the 'parent' property that
>     Phillip> is os.path.dirname in disguise) ...
>
>     Phillip> ... (like the 'listdir()' method that returns full paths rather
>     Phillip> than just filenames).
>
>     Skip> To the extent that the path module tries to provide a uniform
>     Skip> abstraction that's not saddled with a particular way of doing
>     Skip> things (e.g., the Unix way or the Windows way), I don't think this
>     Skip> is necessarily a bad thing.
>
>     Phillip> I'm confused by your statements.  First, I didn't notice the
>     Phillip> path module providing any OS-abstractions that aren't already
>     Phillip> provided by os.path.  Second, using inconsistent and confusing
>     Phillip> names is pretty much always a bad thing.  :)
>
>Sorry, let me be more explicit.  "dirname" is the Unix name for "return the
>parent of this path".  In the Windows and Mac OS9 worlds (ignore any
>possible Posix compatibility for a moment), my guess would be it's probably
>something else.  I suspect listdir gets its "return individual filenames
>instead of full paths" from the semantics of the Posix opendir/readdir/
>closedir functions.  If it makes more sense to return strings that represent
>full paths or new path objects that have been absolute-ified, then the minor
>semantic change going from os.path.listdir() to the listdir method of
>Jason's path objects isn't a big problem to me.

The semantics aren't the issue; it's fine and indeed quite useful to have a 
method that returns path objects.  I'm just saying it shouldn't be called 
listdir(), since that's confusing when compared to what the existing 
listdir() function does.  If you look at my original post, you'll see I 
suggested it be called 'subpaths()' instead, to help reflect that it 
returns paths, rather than filenames.


From pje at telecommunity.com  Mon Jun 27 05:09:25 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Jun 2005 23:09:25 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <03118D28-1405-49E8-BF23-729AD0DF54BD@redivi.com>
References: <5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050626230328.0213b640@mail.telecommunity.com>

At 09:26 PM 6/26/2005 -0400, Bob Ippolito wrote:

>On Jun 26, 2005, at 8:54 PM, Phillip J. Eby wrote:
>
>>At 12:22 AM 6/27/2005 +0200, D?rwald Walter wrote:
>>
>>>Phillip J. Eby wrote:
>>>>I'm also not keen on the fact that it makes certain things
>>>>properties whose value can change over time; i.e. ctime/mtime/atime
>>>>and
>>>>size really shouldn't be properties, but rather methods.
>>>
>>>I think ctime, mtime and atime should be (or return)
>>>datetime.datetime objects instead of integer timestamps.
>>
>>With what timezone?  I don't think that can be done portably and
>>unambiguously, so I'm -1 on that.
>
>That makes no sense, timestamps aren't any better,

Sure they are, if what you want is a timestamp.  In any case, the most 
common use case I've seen for mtime and friends is just comparing against a 
previous value, or the value on another file, so it doesn't actually matter 
most of the time what the type of the value is.


>  and datetime
>objects have no time zone set by default anyway.
>datetime.fromtimestamp(time.time()) gives you the same thing as
>datetime.now().

In which case, it's also easy enough to get a datetime if you really want 
one.  I personally would rather do that than complicate the use cases where 
a datetime isn't really needed.  (i.e. most of the time, at least in my 
experience)


From python at rcn.com  Mon Jun 27 06:25:22 2005
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 27 Jun 2005 00:25:22 -0400
Subject: [Python-Dev] Some RFE for review
In-Reply-To: <d9mmd6$gjf$1@sea.gmane.org>
Message-ID: <000201c57ad0$3c3dda80$39facc97@oemcomputer>

> 1193128:
> str.translate(None, delchars) should be allowed and only delete
delchars
> from the string.

I had agreed to this one and it's on my todo list to implement.


> 1214675:
> warnings should get a removefilter() method. An alternative would be
to
> fully document the "filters" attribute to allow direct tinkering with
it.

I'm concerned that removefilter() may not work well in the presence of
multiple modules that use the warnings module.  It may be difficult to
make sure the one removed wasn't subsequently added another module.
Also, the issue is compounded because the order of filter application is
important.


> 1205239:
> Shift operands should be allowed to be negative integers, so e.g.
> a << -2  is the same as  a >> 2.
> Review: Allowing this would open a source of bugs previously well
> identifiable.

The OP is asking why it is different for negative sequence indicies (why
the added convenience was thought to outweigh the loss of error
detection).



> 1152248:
> In order to read "records" separated by something other than newline,
file
> objects
> should either support an additional parameter (the separator) to
> (x)readlines(),
> or gain an additional method which does this.
> Review: The former is a no-go, I think, because what is read won't be
> lines.

Okay, call it a record then.  The OPs request is not a non-starter.
There is a proven precedent in AWK which allows programmer specifiable
record separators.



> The latter is further complicating the file interface, so I would
follow
> the
> principle that not every 3-line function should be builtin.

This is not a design principle.  UserDict.Mixin shows that most of the
mapping API is easily expressible in terms of a few lines and a few
primitives; however, the mapping API has long been proven as valuable
for its expressiveness.  Likewise, Guido's any() and all() builtins can
be expressed in a single line but were accepted anyway.

A more nuanced version of the "principle" is:  if a proposal can be
easily expressed with a small grouping of existing constructs, then must
meet much higher standards of use frequency and expressiveness in order
to be accepted.



> 1110010:
> A function "attrmap" should be introduced which is used as follows:
> attrmap(x)['att'] == getattr(x, 'att')
> The submitter mentions the use case of new-style classes without a
> __dict__ used
> at the right of %-style string interpolation.
> Review: I don't know whether this is worth it.

While potentially useful, the function is entirely unintuitive (it has
to be studied a bit before being able to see what it is for).  Also, the
OP is short on use cases (none were presented).  IMO, this belongs as a
cookbook recipe.


Raymond Hettinger

From reinhold-birkenfeld-nospam at wolke7.net  Mon Jun 27 07:51:11 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Mon, 27 Jun 2005 07:51:11 +0200
Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib Cookie.py,
	1.17, 1.18
In-Reply-To: <1f7befae050626161574adbe07@mail.gmail.com>
References: <E1DmeHD-0001Jx-QX@sc8-pr-cvs1.sourceforge.net>
	<1f7befae050626161574adbe07@mail.gmail.com>
Message-ID: <d9o3nd$o30$1@sea.gmane.org>

Tim Peters wrote:
> [birkenfeld at users.sourceforge.net]
>> Update of /cvsroot/python/python/dist/src/Lib
>> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4891/Lib
>> 
>> Modified Files:
>>        Cookie.py
>> Log Message:
>> bug [ 1108948 ] Cookie.py produces invalid code

[...]

> I assume this accounts for the current failure of test_cookie:

You're right, I forgot to checkin the change to the output file.
Thanks,
Reinhold

-- 
Mail address is perfectly valid!


From hoffman at ebi.ac.uk  Mon Jun 27 09:20:51 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Mon, 27 Jun 2005 08:20:51 +0100
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
Message-ID: <Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>

On Sun, 26 Jun 2005, Phillip J. Eby wrote:

> At 08:19 PM 6/26/2005 +0100, Michael Hoffman wrote:
>> On Sun, 26 Jun 2005, Phillip J. Eby wrote:
>>
>>> * drop getcwd(); it makes no sense on a path instance
>>
>> Personally I use path.getcwd() as a class method all the time. It
>> makes as much sense as fromkeys() does on a dict instance, which is
>> technically possible but non-sensical.
>
> It's also duplication with os.path; I'm -1 on creating a new staticmethod
> for it.

os.getcwd() returns a string, but path.getcwd() returns a new path
object. Almost everything in path is a duplication of os.path--the
difference is that the path methods start and end with path objects.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From reinhold-birkenfeld-nospam at wolke7.net  Mon Jun 27 09:29:24 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Mon, 27 Jun 2005 09:29:24 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>	<d9mmd6$gjf$1@sea.gmane.org>	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
Message-ID: <d9o9fh$4eh$1@sea.gmane.org>

Michael Hoffman wrote:
> On Sun, 26 Jun 2005, Phillip J. Eby wrote:
> 
>> At 08:19 PM 6/26/2005 +0100, Michael Hoffman wrote:
>>> On Sun, 26 Jun 2005, Phillip J. Eby wrote:
>>>
>>>> * drop getcwd(); it makes no sense on a path instance
>>>
>>> Personally I use path.getcwd() as a class method all the time. It
>>> makes as much sense as fromkeys() does on a dict instance, which is
>>> technically possible but non-sensical.
>>
>> It's also duplication with os.path; I'm -1 on creating a new staticmethod
>> for it.
> 
> os.getcwd() returns a string, but path.getcwd() returns a new path
> object. Almost everything in path is a duplication of os.path--the
> difference is that the path methods start and end with path objects.

+1.

Reinhold

-- 
Mail address is perfectly valid!


From ncoghlan at gmail.com  Mon Jun 27 11:34:34 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Jun 2005 19:34:34 +1000
Subject: [Python-Dev] Some RFE for review
In-Reply-To: <d9mmd6$gjf$1@sea.gmane.org>
References: <d9mmd6$gjf$1@sea.gmane.org>
Message-ID: <42BFC82A.4010503@gmail.com>

Reinhold Birkenfeld wrote:
> 1152248:
> In order to read "records" separated by something other than newline, file objects
> should either support an additional parameter (the separator) to (x)readlines(),
> or gain an additional method which does this.
> Review: The former is a no-go, I think, because what is read won't be lines.
> The latter is further complicating the file interface, so I would follow the
> principle that not every 3-line function should be builtin.

As Douglas Alan's sample implementation (and his second attempt [1]) 
show, getting this right (and reasonably efficient) is actually a 
non-trivial exercise. Leveraging the existing xreadlines 
infrastructure is an idea worth considering.

I think it's worth leaving this one open, and see if someone comes up 
with a patch (obviously, this was my opinion from the start, or I 
wouldn't have raised the RFE in response to Douglas's query!)

Cheers,
Nick.

[1] http://mail.python.org/pipermail/python-list/2005-February/268547.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From p.f.moore at gmail.com  Mon Jun 27 14:22:26 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Jun 2005 13:22:26 +0100
Subject: [Python-Dev] Some RFE for review
In-Reply-To: <42BFC82A.4010503@gmail.com>
References: <d9mmd6$gjf$1@sea.gmane.org> <42BFC82A.4010503@gmail.com>
Message-ID: <79990c6b0506270522409ebee@mail.gmail.com>

On 6/27/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As Douglas Alan's sample implementation (and his second attempt [1])
> show, getting this right (and reasonably efficient) is actually a
> non-trivial exercise. Leveraging the existing xreadlines
> infrastructure is an idea worth considering.
> 
> I think it's worth leaving this one open, and see if someone comes up
> with a patch (obviously, this was my opinion from the start, or I
> wouldn't have raised the RFE in response to Douglas's query!)

As a more general approach, would it be worth considering an addition
to itertools which took an iterator which generated "blocks" of items,
and split them on a subsequence? It's a generalisation of the basic
pattern here, and would be able to encapsulate the fiddly "what if a
separator overlaps a block split" logic without locking it down to
string manipulation...

Or does that count as overgeneralisation?

Paul.

From python at rcn.com  Mon Jun 27 16:32:50 2005
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 27 Jun 2005 10:32:50 -0400
Subject: [Python-Dev] Some RFE for review
In-Reply-To: <79990c6b0506270522409ebee@mail.gmail.com>
Message-ID: <000001c57b25$19252bc0$39facc97@oemcomputer>

[Paul Moore on readline getting a record separator argument]
> As a more general approach, would it be worth considering an addition
> to itertools which took an iterator which generated "blocks" of items,
> and split them on a subsequence? 

Nope.  Assign responsibility to the class that has all of the relevant
knowledge (API for retrieving blocks, type of the retrieved data, how
EOF is detected, etc).


> It's a generalisation of the basic
> pattern here, and would be able to encapsulate the fiddly "what if a
> separator overlaps a block split" logic without locking it down to
> string manipulation...

How do you build, scan, and extract the buffer in a type independent
manner?  Are there any use cases for non-string data buffers, a stream
of integers or somesuch?


Raymond

From pje at telecommunity.com  Mon Jun 27 17:06:19 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 27 Jun 2005 11:06:19 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
References: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>

At 08:20 AM 6/27/2005 +0100, Michael Hoffman wrote:
>os.getcwd() returns a string, but path.getcwd() returns a new path
>object.

In that case, I'd expect it to be 'path.fromcwd()' or 'path.cwd()'; i.e. a 
constructor classmethod by analogy with 'dict.fromkeys()' or 
'datetime.now()'.  'getcwd()' looks like it's getting a property of a path 
instance, and doesn't match stdlib conventions for constructors.

So, +1 as long as it's called cwd() or something better (i.e. clearer 
and/or more consistent with stdlib constructor conventions).


From reinhold-birkenfeld-nospam at wolke7.net  Mon Jun 27 17:10:17 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Mon, 27 Jun 2005 17:10:17 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
References: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>	<d9mmd6$gjf$1@sea.gmane.org>	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
Message-ID: <d9p4fk$kvd$1@sea.gmane.org>

Phillip J. Eby wrote:
> At 08:20 AM 6/27/2005 +0100, Michael Hoffman wrote:
>>os.getcwd() returns a string, but path.getcwd() returns a new path
>>object.
> 
> In that case, I'd expect it to be 'path.fromcwd()' or 'path.cwd()'; i.e. a 
> constructor classmethod by analogy with 'dict.fromkeys()' or 
> 'datetime.now()'.  'getcwd()' looks like it's getting a property of a path 
> instance, and doesn't match stdlib conventions for constructors.
> 
> So, +1 as long as it's called cwd() or something better (i.e. clearer 
> and/or more consistent with stdlib constructor conventions).

You're right. +1 for calling it fromcwd().

With that settled, should I rewrite the module? Should I write a PEP?

Reinhold

-- 
Mail address is perfectly valid!


From mcherm at mcherm.com  Mon Jun 27 17:26:53 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon, 27 Jun 2005 08:26:53 -0700
Subject: [Python-Dev] subprocess.call() and stdin
Message-ID: <20050627082653.qnm2jw78t5ogw8k0@login.werra.lunarpages.com>

Stuart Bishop writes:
> When I invoke subprocess.call(), I often want to ensure that the subprocess'
> stdin is closed. This ensures it will die if the subprocess attempts to read
> from stdin rather than block.
>
> This could be done if the subprocess.call() helper closes the input if
> stdin=subprocess.PIPE, which may be the only sane way to handle this
> argument (I can't think of any use cases for spawning a subprocess with an
> input stream that nothing can write too).

+0.5. I agree that if you pass "stdin=subprocess.PIPE" to subprocess.call()
that the current behavior of having the child process block forever is
totally useless. I have little reason to prefer "assume an empty input"
over "let subprocess.call() raise an exception if stdin==subprocess.PIPE"
-- but if I take your word for it that this is a common need, then that's
one good reason.

> It could also be done by adding a subprocess.CLOSED constant, which if
> passed to Popen causes a new closed file descriptor to be given to the
> subprocess.

-1.

It is easy enough to create a closed FD to read from... why complicate the
API?

-- Michael Chermside


From pje at telecommunity.com  Mon Jun 27 17:35:55 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 27 Jun 2005 11:35:55 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <d9p4fk$kvd$1@sea.gmane.org>
References: <5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050627111913.035e2660@mail.telecommunity.com>

At 05:10 PM 6/27/2005 +0200, Reinhold Birkenfeld wrote:
>Phillip J. Eby wrote:
> > At 08:20 AM 6/27/2005 +0100, Michael Hoffman wrote:
> >>os.getcwd() returns a string, but path.getcwd() returns a new path
> >>object.
> >
> > In that case, I'd expect it to be 'path.fromcwd()' or 'path.cwd()'; i.e. a
> > constructor classmethod by analogy with 'dict.fromkeys()' or
> > 'datetime.now()'.  'getcwd()' looks like it's getting a property of a path
> > instance, and doesn't match stdlib conventions for constructors.
> >
> > So, +1 as long as it's called cwd() or something better (i.e. clearer
> > and/or more consistent with stdlib constructor conventions).
>
>You're right. +1 for calling it fromcwd().

I'm leaning slightly towards .cwd() for symmetry with datetime.now(), but 
not enough to argue about it if nobody has objections to fromcwd().


>With that settled, should I rewrite the module? Should I write a PEP?

I think the only questions remaining open are where to put it and what to 
call the class.  I think we should put it in os.path, such that 'from 
os.path import path' gives you the path class for your platform, and using 
one of the path modules directly (e.g. 'from posixpath import path') gives 
you the specific platform's version.  This is useful because sometimes you 
need to manipulate paths that are foreign to your current OS.  For example, 
the distutils and other packages sometimes use POSIX paths for input and 
then convert them to local OS paths.  Also, POSIX path objects would be 
useful for creating or parsing the "path" portion of many kinds of URLs, 
and I have often used functions from posixpath for that myself.

As for a PEP, I doubt a PEP is really required for something this simple; I 
have never seen anyone say, "no, we shouldn't have this in the stdlib".  I 
think it would be more important to write reference documentation and a 
complete test suite.

By the way, it also occurs to me that for the sake of subclassability, the 
methods should not return 'path(somestr)' when creating new objects, but 
instead use self.__class__(somestr).


From grobinson at goombah.com  Mon Jun 27 17:31:52 2005
From: grobinson at goombah.com (Gary Robinson)
Date: Mon, 27 Jun 2005 11:31:52 -0400
Subject: [Python-Dev] Possible C API problem?
Message-ID: <20050627113152.071275.a25edf12@goombah.com>

Hello,

I was asking about a problem I was having over on the C++-python list, 
and they suggested I report it here as a possible Python problem.

I was getting bus errors with a C module I was linking to, so factored 
it down too a very small example that reproduced the problem. Here it 
is:


#include <Python.h>
static  double         gfSumChiSquare = 123.0; 
            
static PyObject *    
getSumChiSquare(PyObject *self, PyObject *args){
    return Py_BuildValue("d", gfSumChiSquare);
    }
    
static PyMethodDef SimMethods[] = {
    {"getSumChiSquare", getSumChiSquare, METH_NOARGS, "Return 
fSumChiSquare"},
    {NULL, NULL, 0, NULL}        /* Sentinel */
};

PyMODINIT_FUNC
inittestfloat(void)
{
    (void) Py_InitModule("testfloat", SimMethods);
}

That caused a bus error 100% of the time when I simply imported the 
module into Python and called getSumChiSquare(), i.e.:

>>> import testfloat
>>> testfloat.getSumChiSquare()
    

However, the problem seems to go away if I use METH_VARARGS, and parse 
the non-existent args with 
PyArg_ParseTuple:

#include <Python.h>
static  double         gfSumChiSquare = 123.0; 
            
static PyObject *    
getSumChiSquare(PyObject *self, PyObject *args){
    if (!PyArg_ParseTuple(args, "", NULL))
        return NULL;    
    return Py_BuildValue("d", gfSumChiSquare);
    }
    
static PyMethodDef SimMethods[] = {
    {"getSumChiSquare", getSumChiSquare, METH_VARARGS, "Return 
fSumChiSquare"},
    {NULL, NULL, 0, NULL}        /* Sentinel */
};

PyMODINIT_FUNC
inittestfloat(void)
{
    (void) Py_InitModule("testfloat", SimMethods);
}


This approach seems to work reliably -- at least variations I've tried 
haven't caused a bus error. But I haven't been able to discern an 
explanation from the docs as to why this would be better. The docs say 
that both METH_VARARGS and METH_NOARGS expect a PyCFunction. So if I am 
calling the function with no arguments, why can't I use METH_NOARGS and 
skip the call to PyArg_ParseTuple?

Could it be that this is a python bug? Or am I doing something wrong?

Note: this is using Python 2.3 on OS X:

    Python 2.3 (#1, Sep 13 2003, 00:49:11) 

Thanks in advance for any help or insight you can give,

Gary


-- 

Gary Robinson
CTO
Emergent Music, LLC
grobinson at goombah.com
207-942-3463
Company: http://www.goombah.com
Blog:    http://www.garyrobinson.net

From mcherm at mcherm.com  Mon Jun 27 17:33:43 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon, 27 Jun 2005 08:33:43 -0700
Subject: [Python-Dev] Decimal floats as default (was:
	discussion	aboutPEP239 and 240)
Message-ID: <20050627083343.vbcal7el0a1wwcwo@login.werra.lunarpages.com>

Fredrik Johansson writes:
> In either case, compatibility can be ensured by allowing both n-digit
> decimal and hardware binary precision for floats, settable via a float
> context.

Perhaps you can show me a design (or working code) that proves me
wrong, but I don't believe that such a design could be made compatible
with the existing Decimal module... certainly while continuing to
maintain compatibility with the Cowlinshaw specification.

> There is the alternative of providing decimal literals by using
> separate decimal and binary float base types

If, by this, you mean adding a "binary float context" modeled after
the Decimal float context and providing access to the underlying FP
flags and traps and generally enhancing the use of binary FP, then
I think it's a great idea. It's probably impossible to write in a
cross-platform manner (because C supplies support for binary FP but
does not offer access to the flags and traps), but this is one of those
few cases where it's worth using platform-and-compiler specific code.

Of course, someone still has to step forward and offer to code it.

-- Michael Chermside


From oren.tirosh at gmail.com  Mon Jun 27 17:40:55 2005
From: oren.tirosh at gmail.com (Oren Tirosh)
Date: Mon, 27 Jun 2005 18:40:55 +0300
Subject: [Python-Dev] Some RFE for review
In-Reply-To: <42BFC82A.4010503@gmail.com>
References: <d9mmd6$gjf$1@sea.gmane.org> <42BFC82A.4010503@gmail.com>
Message-ID: <7168d65a05062708407f7cda82@mail.gmail.com>

On 6/27/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Reinhold Birkenfeld wrote:
> > 1152248:
> > In order to read "records" separated by something other than newline, file objects
> > should either support an additional parameter (the separator) to (x)readlines(),
> > or gain an additional method which does this.
> > Review: The former is a no-go, I think, because what is read won't be lines.
> > The latter is further complicating the file interface, so I would follow the
> > principle that not every 3-line function should be builtin.
> 
> As Douglas Alan's sample implementation (and his second attempt [1])
> show, getting this right (and reasonably efficient) is actually a
> non-trivial exercise. Leveraging the existing xreadlines
> infrastructure is an idea worth considering.

Do you mean the existing xreadlines infrustructure that no longer
exists since 2.4 ?  :-)

An infrastructure that could be leveraged is the readahead buffer used
by the file object's line iterator.

  Oren

From aahz at pythoncraft.com  Mon Jun 27 18:53:24 2005
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 27 Jun 2005 09:53:24 -0700
Subject: [Python-Dev] Decimal floats as default (was:
	discussion	aboutPEP239 and 240)
In-Reply-To: <20050627083343.vbcal7el0a1wwcwo@login.werra.lunarpages.com>
References: <20050627083343.vbcal7el0a1wwcwo@login.werra.lunarpages.com>
Message-ID: <20050627165324.GC4392@panix.com>

On Mon, Jun 27, 2005, Michael Chermside wrote:
>
> If, by this, you mean adding a "binary float context" modeled after
> the Decimal float context and providing access to the underlying FP
> flags and traps and generally enhancing the use of binary FP, then
> I think it's a great idea. It's probably impossible to write in a
> cross-platform manner (because C supplies support for binary FP but
> does not offer access to the flags and traps), but this is one of those
> few cases where it's worth using platform-and-compiler specific code.
> 
> Of course, someone still has to step forward and offer to code it.

...and document and maintain it.  That's always been the sticky part,
along with the requirement that this degrade gracefully when the
platform-specific code doesn't exist.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From trentm at ActiveState.com  Mon Jun 27 20:28:05 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Mon, 27 Jun 2005 11:28:05 -0700
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
References: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
Message-ID: <20050627182805.GC6412@ActiveState.com>

> >os.getcwd() returns a string, but path.getcwd() returns a new path
> >object.
> 
> In that case, I'd expect it to be 'path.fromcwd()' or 'path.cwd()'; i.e. a 
> constructor classmethod by analogy with 'dict.fromkeys()' or 
> 'datetime.now()'.  'getcwd()' looks like it's getting a property of a path 
> instance, and doesn't match stdlib conventions for constructors.
> 
> So, +1 as long as it's called cwd() or something better (i.e. clearer 
> and/or more consistent with stdlib constructor conventions).

What about have it just be the default empty constructor?

    assert path.Path() == os.getcwd() \
        or path.Path() == os.getcwdu()

Dunno if that causes other weirdnesses with the API, though.

Trent

-- 
Trent Mick
TrentM at ActiveState.com

From walter at livinglogic.de  Mon Jun 27 20:52:21 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 27 Jun 2005 20:52:21 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050626230328.0213b640@mail.telecommunity.com>
References: <5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
	<5.1.1.6.0.20050626230328.0213b640@mail.telecommunity.com>
Message-ID: <EC436580-1B99-4E8F-B32A-40E27712D15B@livinglogic.de>

Phillip J. Eby wrote:

> At 09:26 PM 6/26/2005 -0400, Bob Ippolito wrote:
>
>> On Jun 26, 2005, at 8:54 PM, Phillip J. Eby wrote:
>>
>>> At 12:22 AM 6/27/2005 +0200, D?rwald Walter wrote:
>>>
>>>> Phillip J. Eby wrote:
>>>>
>>>>> I'm also not keen on the fact that it makes certain things
>>>>> properties whose value can change over time; i.e. ctime/mtime/ 
>>>>> atime
>>>>> and
>>>>> size really shouldn't be properties, but rather methods.
>>>>
>>>> I think ctime, mtime and atime should be (or return)
>>>> datetime.datetime objects instead of integer timestamps.
>>>
>>> With what timezone?  I don't think that can be done portably and
>>> unambiguously, so I'm -1 on that.
>>
>> That makes no sense, timestamps aren't any better,
>
> Sure they are, if what you want is a timestamp.  In any case, the  
> most common use case I've seen for mtime and friends is just  
> comparing against a previous value, or the value on another file,  
> so it doesn't actually matter most of the time what the type of the  
> value is.

I find timestamp values to be somewhat opaque. So all things being  
equal, I'd prefer datetime objects.

>>  and datetime
>> objects have no time zone set by default anyway.
>> datetime.fromtimestamp(time.time()) gives you the same thing as
>> datetime.now().
>>
>
> In which case, it's also easy enough to get a datetime if you  
> really want one.  I personally would rather do that than complicate  
> the use cases where a datetime isn't really needed.  (i.e. most of  
> the time, at least in my experience)

We should have one uniform way of representing time in Python. IMHO  
datetime objects are the natural choice.

Bye,
    Walter D?rwald


From fredrik.johansson at gmail.com  Mon Jun 27 20:55:29 2005
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Mon, 27 Jun 2005 20:55:29 +0200
Subject: [Python-Dev] Decimal floats as default (was: discussion
	aboutPEP239 and 240)
In-Reply-To: <20050627083343.vbcal7el0a1wwcwo@login.werra.lunarpages.com>
References: <20050627083343.vbcal7el0a1wwcwo@login.werra.lunarpages.com>
Message-ID: <3d0cebfb05062711555c73164b@mail.gmail.com>

On 6/27/05, Michael Chermside <mcherm at mcherm.com> wrote:
> Fredrik Johansson writes:
> > In either case, compatibility can be ensured by allowing both n-digit
> > decimal and hardware binary precision for floats, settable via a float
> > context.
> 
> Perhaps you can show me a design (or working code) that proves me
> wrong, but I don't believe that such a design could be made compatible
> with the existing Decimal module... certainly while continuing to
> maintain compatibility with the Cowlinshaw specification.

Are you thinking of any particular problem?

> > There is the alternative of providing decimal literals by using
> > separate decimal and binary float base types
> 
> If, by this, you mean adding a "binary float context" modeled after
> the Decimal float context and providing access to the underlying FP
> flags and traps and generally enhancing the use of binary FP, then
> I think it's a great idea.

The context (as I envision it) would not be just a "binary float
context", but a universal float context that lets you choose between
binary and decimal precision at run time.

A context for Python's current float type would be a nice idea by
itself, though.

> It's probably impossible to write in a
> cross-platform manner (because C supplies support for binary FP but
> does not offer access to the flags and traps), but this is one of those
> few cases where it's worth using platform-and-compiler specific code.

I agree.

--Fredrik

From hoffman at ebi.ac.uk  Mon Jun 27 21:24:13 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Mon, 27 Jun 2005 20:24:13 +0100
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
References: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
Message-ID: <Pine.LNX.4.62.0506272013490.19675@qnzvnan.rov.np.hx>

On Mon, 27 Jun 2005, Phillip J. Eby wrote:

> At 08:20 AM 6/27/2005 +0100, Michael Hoffman wrote:
>> os.getcwd() returns a string, but path.getcwd() returns a new path
>> object.
>
> In that case, I'd expect it to be 'path.fromcwd()' or 'path.cwd()'; i.e. a
> constructor classmethod by analogy with 'dict.fromkeys()' or
> 'datetime.now()'.  'getcwd()' looks like it's getting a property of a path
> instance, and doesn't match stdlib conventions for constructors.
>
> So, +1 as long as it's called cwd() or something better (i.e. clearer
> and/or more consistent with stdlib constructor conventions).

+1 on cwd().

-1 on making this the default constructor. Essentially the default
constructor returns a path object that will reflect the CWD at the
time that further instance methods are called.

path.cwd() will return a path object that reflects the path at the
time of construction.

This example may be instructive:

>>> from path import path
>>> import os
>>>
>>> default_path = path()
>>> getcwd_path = path.getcwd()
>>> default_path.abspath()
path('/home/hoffman')
>>> getcwd_path.abspath()
path('/home/hoffman')
>>>
>>> os.chdir("etc")
>>> default_path.abspath()
path('/home/hoffman/etc')
>>> getcwd_path.abspath()
path('/home/hoffman')

Unfortunately only some of the methods work on paths created with the
default constructor:

>>> path().listdir()
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/lib/python2.4/site-packages/path.py", line 297, in listdir
     names = os.listdir(self)
OSError: [Errno 2] No such file or directory: ''

Is there support to have all of the methods work when the path is the
empty string? Among other benefits, this would mean that sys.path
could be turned into useful path objects with a simple list
comprehension.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From pje at telecommunity.com  Mon Jun 27 21:37:48 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 27 Jun 2005 15:37:48 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <EC436580-1B99-4E8F-B32A-40E27712D15B@livinglogic.de>
References: <5.1.1.6.0.20050626230328.0213b640@mail.telecommunity.com>
	<5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<5.1.1.6.0.20050626205241.036f9fa0@mail.telecommunity.com>
	<5.1.1.6.0.20050626230328.0213b640@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050627152946.03642d38@mail.telecommunity.com>

At 08:52 PM 6/27/2005 +0200, Walter D?rwald wrote:
>Phillip J. Eby wrote:
> > At 09:26 PM 6/26/2005 -0400, Bob Ippolito wrote:
> >> On Jun 26, 2005, at 8:54 PM, Phillip J. Eby wrote:
> >>> At 12:22 AM 6/27/2005 +0200, D?rwald Walter wrote:
> >>>> Phillip J. Eby wrote:
> >>>>> I'm also not keen on the fact that it makes certain things
> >>>>> properties whose value can change over time; i.e. ctime/mtime/
> >>>>> atime
> >>>>> and
> >>>>> size really shouldn't be properties, but rather methods.
> >>>>
> >>>> I think ctime, mtime and atime should be (or return)
> >>>> datetime.datetime objects instead of integer timestamps.
> >>>
> >>> With what timezone?  I don't think that can be done portably and
> >>> unambiguously, so I'm -1 on that.
> >>
> >> That makes no sense, timestamps aren't any better,
> >
> > Sure they are, if what you want is a timestamp.  In any case, the
> > most common use case I've seen for mtime and friends is just
> > comparing against a previous value, or the value on another file,
> > so it doesn't actually matter most of the time what the type of the
> > value is.
>
>I find timestamp values to be somewhat opaque. So all things being
>equal, I'd prefer datetime objects.

Their opaqueness is actually one of the reasons I prefer them.  :)


> >>  and datetime
> >> objects have no time zone set by default anyway.
> >> datetime.fromtimestamp(time.time()) gives you the same thing as
> >> datetime.now().
> >>
> >
> > In which case, it's also easy enough to get a datetime if you
> > really want one.  I personally would rather do that than complicate
> > the use cases where a datetime isn't really needed.  (i.e. most of
> > the time, at least in my experience)
>
>We should have one uniform way of representing time in Python.

Um, counting the various datetime variants (date, time, datetime), 
timestamps (float and long), and time tuples, Python has 6 ways now.  The 
type chosen for a given API is largely dependent on the *source* of the 
time value.  And the value supplied by most existing OS-level Python APIs 
is either a long or a float.

However, there's also a practicality-beats-purity issue here; using a 
datetime produces an otherwise-unnecessary dependency on the datetime 
module for users of this functionality, despite use cases for an actual 
datetime value being very infrequent.  Date arithmetic on file timestamps 
that can't be trivially done in terms of seconds is rare, as is display of 
file timestamps.  None of these use cases will be significantly harmed by 
having to use datetime.fromtimestamp(); they will probably be importing 
datetime already.  What I don't want is for simple scripts to need to 
import datetime (even indirectly by way of the path class) just to get easy 
access to stat() values.


From pje at telecommunity.com  Mon Jun 27 21:42:39 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 27 Jun 2005 15:42:39 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <Pine.LNX.4.62.0506272013490.19675@qnzvnan.rov.np.hx>
References: <5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050627153850.0364b940@mail.telecommunity.com>

At 08:24 PM 6/27/2005 +0100, Michael Hoffman wrote:
>On Mon, 27 Jun 2005, Phillip J. Eby wrote:
>
> > At 08:20 AM 6/27/2005 +0100, Michael Hoffman wrote:
> >> os.getcwd() returns a string, but path.getcwd() returns a new path
> >> object.
> >
> > In that case, I'd expect it to be 'path.fromcwd()' or 'path.cwd()'; i.e. a
> > constructor classmethod by analogy with 'dict.fromkeys()' or
> > 'datetime.now()'.  'getcwd()' looks like it's getting a property of a path
> > instance, and doesn't match stdlib conventions for constructors.
> >
> > So, +1 as long as it's called cwd() or something better (i.e. clearer
> > and/or more consistent with stdlib constructor conventions).
>
>+1 on cwd().
>
>-1 on making this the default constructor. Essentially the default
>constructor returns a path object that will reflect the CWD at the
>time that further instance methods are called.

Only if we make the default argument to path() be os.curdir, which isn't a 
bad idea.


>Unfortunately only some of the methods work on paths created with the
>default constructor:
>
> >>> path().listdir()
>Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "/usr/lib/python2.4/site-packages/path.py", line 297, in listdir
>      names = os.listdir(self)
>OSError: [Errno 2] No such file or directory: ''

This wouldn't be a problem if the default constructor arg were os.curdir 
(i.e. '.' for most platforms) instead of an empty string.


>Is there support to have all of the methods work when the path is the
>empty string? Among other benefits, this would mean that sys.path
>could be turned into useful path objects with a simple list
>comprehension.

Ugh.  sys.path entries are not path objects, nor should they be.  PEP 302 
(implemented in Python 2.3 and up) allows sys.path to contain any strings 
you like, as interpreted by objects in sys.path_hooks.  Programs that 
assume only filesystem paths appear in sys.path will break in the presence 
of PEP 302-sanctioned import hooks.


From skip at pobox.com  Mon Jun 27 22:45:57 2005
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 27 Jun 2005 15:45:57 -0500
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <20050627182805.GC6412@ActiveState.com>
References: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<20050627182805.GC6412@ActiveState.com>
Message-ID: <17088.25989.693211.15368@montanaro.dyndns.org>

We're getting enough discussion about various aspects of Jason's path module
that perhaps a PEP is warranted.  All this discussion on python-dev is just
going to get lost.

Skip

From mwh at python.net  Mon Jun 27 22:56:44 2005
From: mwh at python.net (Michael Hudson)
Date: Mon, 27 Jun 2005 21:56:44 +0100
Subject: [Python-Dev] Possible C API problem?
In-Reply-To: <20050627113152.071275.a25edf12@goombah.com> (Gary Robinson's
	message of "Mon, 27 Jun 2005 11:31:52 -0400")
References: <20050627113152.071275.a25edf12@goombah.com>
Message-ID: <2moe9r1ptf.fsf@starship.python.net>

Gary Robinson <grobinson at goombah.com> writes:

> That caused a bus error 100% of the time when I simply imported the 
> module into Python and called getSumChiSquare(), i.e.:
>
>>>> import testfloat
>>>> testfloat.getSumChiSquare()

It doesn't for me (CVS HEAD, OS X Panther).

> Could it be that this is a python bug? Or am I doing something wrong?
>
> Note: this is using Python 2.3 on OS X:
>
>     Python 2.3 (#1, Sep 13 2003, 00:49:11) 
>
> Thanks in advance for any help or insight you can give,

Have you, you know, tried to debug the situation yourself?  If you
have gcc installed, you probably have gdb installed too...

Cheers,
mwh

-- 
  You can lead an idiot to knowledge but you cannot make him 
  think.  You can, however, rectally insert the information, 
  printed on stone tablets, using a sharpened poker.        -- Nicolai
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html

From pje at telecommunity.com  Mon Jun 27 23:25:58 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 27 Jun 2005 17:25:58 -0400
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
 review)
In-Reply-To: <17088.25989.693211.15368@montanaro.dyndns.org>
References: <20050627182805.GC6412@ActiveState.com>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<20050627182805.GC6412@ActiveState.com>
Message-ID: <5.1.1.6.0.20050627171642.0364b630@mail.telecommunity.com>

At 03:45 PM 6/27/2005 -0500, Skip Montanaro wrote:
>We're getting enough discussion about various aspects of Jason's path module
>that perhaps a PEP is warranted.  All this discussion on python-dev is just
>going to get lost.

AFAICT, the only unresolved issue outstanding is a compromise or 
Pronouncement regarding the atime/ctime/mtime members' datatype.  This is 
assuming, of course, that making the "empty path" be os.curdir doesn't 
receive any objections, and that nobody strongly prefers 'path.fromcwd()' 
over 'path.cwd()' as the alternate constructor name.

Apart from these fairly minor issues, there is a very short to-do list, 
small enough to do an implementation patch in an evening or 
two.  Documentation might take a similar amount of time after that; mostly 
it'll be copy-paste from the existing os.path docs, though.

As for the open issues, if we can't reach some sane compromise about 
atime/ctime/mtime, I'd suggest just providing the stat() method and let 
people use stat().st_mtime et al.  Alternately, I'd be okay with creating 
last_modified(), last_accessed(), and created_on() methods that return 
datetime objects, as long as there's also atime()/mtime()/ctime() methods 
that return timestamps.



From Scott.Daniels at Acm.Org  Mon Jun 27 23:36:10 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Mon, 27 Jun 2005 14:36:10 -0700
Subject: [Python-Dev] Possible C API problem?
In-Reply-To: <2moe9r1ptf.fsf@starship.python.net>
References: <20050627113152.071275.a25edf12@goombah.com>
	<2moe9r1ptf.fsf@starship.python.net>
Message-ID: <d9pr2u$pqk$1@sea.gmane.org>

Michael Hudson wrote:
> Gary Robinson <grobinson at goombah.com> writes:
>>... bus error 100% of the time ...:

We've boiled it down pretty far, and I've sent him off to
the mac-python folks (looks gcc-compilerish to me, or maybe
fallout from slight changes in C function call semantics).

--Scott David Daniels
Scott.Daniels at Acm.Org


From ncoghlan at gmail.com  Mon Jun 27 23:59:26 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2005 07:59:26 +1000
Subject: [Python-Dev] Some RFE for review
In-Reply-To: <7168d65a05062708407f7cda82@mail.gmail.com>
References: <d9mmd6$gjf$1@sea.gmane.org> <42BFC82A.4010503@gmail.com>
	<7168d65a05062708407f7cda82@mail.gmail.com>
Message-ID: <42C076BE.1050601@gmail.com>

Oren Tirosh wrote:
> An infrastructure that could be leveraged is the readahead buffer used
> by the file object's line iterator.

That's the infrastructure I meant. I was just being sloppy with my 
terminology ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com

From grobinson at goombah.com  Tue Jun 28 00:36:07 2005
From: grobinson at goombah.com (Gary Robinson)
Date: Mon, 27 Jun 2005 18:36:07 -0400
Subject: [Python-Dev] Possible C API problem?
In-Reply-To: <2moe9r1ptf.fsf@starship.python.net>
References: <20050627113152.071275.a25edf12@goombah.com>
	<2moe9r1ptf.fsf@starship.python.net>
Message-ID: <20050627183607.895396.bbd67c79@goombah.com>

> It doesn't for me (CVS HEAD, OS X Panther).

Note sure what you mean "CVS HEAD", you mean the latest python from 
cvs? 2.4? I'm still using the Apple python, which is straight 2.3.

> Have you, you know, tried to debug the situation yourself?  If you
> have gcc installed, you probably have gdb installed too...

It's been around 7 years since I've used C, I've forgotten virtually 
everything I may have known about gdb, I've never worked with the 
C-python API before... meanwhile there is intense time pressure to get 
the next release of our product (http://www.goombah.com) ready. So it's 
just not practical for me to take that on myself now. I'm hoping to get 
some help from other pythonistas where someone will say -- "yes, it's 
getting a bus error for so-and-so reason, and if you do it this other 
way, you'll be fine..."

Thanks,
Gary


-- 

Gary Robinson
CTO
Emergent Music, LLC
grobinson at goombah.com
207-942-3463
Company: http://www.goombah.com
Blog:    http://www.garyrobinson.net

On Mon, 27 Jun 2005 21:56:44 +0100, Michael Hudson wrote:
> Gary Robinson <grobinson at goombah.com> writes:
> 
>>  That caused a bus error 100% of the time when I simply imported the 
>>  module into Python and called getSumChiSquare(), i.e.:
>> 
>>>>>  import testfloat
>>>>>  testfloat.getSumChiSquare()
> 
> It doesn't for me (CVS HEAD, OS X Panther).
> 
>>  Could it be that this is a python bug? Or am I doing something wrong?
>> 
>>  Note: this is using Python 2.3 on OS X:
>> 
>>      Python 2.3 (#1, Sep 13 2003, 00:49:11) 
>> 
>>  Thanks in advance for any help or insight you can give,
> 
> Have you, you know, tried to debug the situation yourself?  If you
> have gcc installed, you probably have gdb installed too...
> 
> Cheers,
> mwh

From tdelaney at avaya.com  Tue Jun 28 00:48:33 2005
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Tue, 28 Jun 2005 08:48:33 +1000
Subject: [Python-Dev] Possible C API problem?
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F05CAF5@au3010avexu1.global.avaya.com>

Gary Robinson wrote:

> It's been around 7 years since I've used C, I've forgotten virtually
> everything I may have known about gdb, I've never worked with the
> C-python API before... meanwhile there is intense time pressure to get
> the next release of our product (http://www.goombah.com) ready. So
> it's 
> just not practical for me to take that on myself now. I'm hoping to
> get 
> some help from other pythonistas where someone will say -- "yes, it's
> getting a bus error for so-and-so reason, and if you do it this other
> way, you'll be fine..."

Well, it appears you have a workaround (METH_VARARGS) so I'd suggest
using that for now, and raise a bug report at SourceForge
<http://sourceforge.net/tracker/?group_id=5470&atid=105470>.

Tim Delaney

From bob at redivi.com  Tue Jun 28 00:58:26 2005
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 27 Jun 2005 18:58:26 -0400
Subject: [Python-Dev] Possible C API problem?
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F05CAF5@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F05CAF5@au3010avexu1.global.avaya.com>
Message-ID: <7D845FB9-A6E2-4F6C-B5FC-477130AEE9CF@redivi.com>


On Jun 27, 2005, at 6:48 PM, Delaney, Timothy (Tim) wrote:

> Gary Robinson wrote:
>
>
>> It's been around 7 years since I've used C, I've forgotten virtually
>> everything I may have known about gdb, I've never worked with the
>> C-python API before... meanwhile there is intense time pressure to  
>> get
>> the next release of our product (http://www.goombah.com) ready. So
>> it's
>> just not practical for me to take that on myself now. I'm hoping to
>> get
>> some help from other pythonistas where someone will say -- "yes, it's
>> getting a bus error for so-and-so reason, and if you do it this other
>> way, you'll be fine..."
>>
>
> Well, it appears you have a workaround (METH_VARARGS) so I'd suggest
> using that for now, and raise a bug report at SourceForge
> <http://sourceforge.net/tracker/?group_id=5470&atid=105470>.

You should also test on Python 2.4.1 and/or Python 2.3.5.  A bug  
report against Python 2.3.0 isn't very useful if it's a bug that's  
already been resolved.

-bob


From adurdin at gmail.com  Tue Jun 28 02:42:20 2005
From: adurdin at gmail.com (Andrew Durdin)
Date: Tue, 28 Jun 2005 10:42:20 +1000
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050627171642.0364b630@mail.telecommunity.com>
References: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<20050627182805.GC6412@ActiveState.com>
	<17088.25989.693211.15368@montanaro.dyndns.org>
	<5.1.1.6.0.20050627171642.0364b630@mail.telecommunity.com>
Message-ID: <59e9fd3a050627174214f5c11e@mail.gmail.com>

On 6/28/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> 
> AFAICT, the only unresolved issue outstanding is a compromise or
> Pronouncement regarding the atime/ctime/mtime members' datatype.  This is
> assuming, of course, that making the "empty path" be os.curdir doesn't
> receive any objections, and that nobody strongly prefers 'path.fromcwd()'
> over 'path.cwd()' as the alternate constructor name.
> 
> Apart from these fairly minor issues, there is a very short to-do list,
> small enough to do an implementation patch in an evening or
> two.  Documentation might take a similar amount of time after that; mostly
> it'll be copy-paste from the existing os.path docs, though.

While we'ew discussing outstanding issues: In a related discussion of
the path module on c.l.py, Thomas Heller pointed out that the path
module doesn't correctly handle unicode paths:

| I have never used the path module before, although I've heard good
| things about it.  But, it seems to have problems with unicode pathnames,
| at least on windows:
| 
| C:\>mkdir sp?m
| 
| C:\sp?m>py24
| Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
| Type "help", "copyright", "credits" or "license" for more information.
| >>> from path import path
| >>> path.getcwd()
| 
| Traceback (most recent call last):
|   File "<stdin>", line 1, in ?
|   File "C:\TSS5\components\_Pythonlib\path.py", line 97, in getcwd
|     return path(os.getcwd())
| UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position
5: ordinal not in range(128)

He suggests a possible fix in his message:
http://mail.python.org/pipermail/python-list/2005-June/287372.html
http://groups-beta.google.com/group/comp.lang.python/msg/b3795a2a0c52b93f

From abo at minkirri.apana.org.au  Tue Jun 28 02:48:22 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Mon, 27 Jun 2005 17:48:22 -0700
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	for	review)
In-Reply-To: <5.1.1.6.0.20050627171642.0364b630@mail.telecommunity.com>
References: <20050627182805.GC6412@ActiveState.com>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<20050627182805.GC6412@ActiveState.com>
	<5.1.1.6.0.20050627171642.0364b630@mail.telecommunity.com>
Message-ID: <1119919702.9174.500.camel@warna.corp.google.com>

On Mon, 2005-06-27 at 14:25, Phillip J. Eby wrote:
[...]
> As for the open issues, if we can't reach some sane compromise about 
> atime/ctime/mtime, I'd suggest just providing the stat() method and let 
> people use stat().st_mtime et al.  Alternately, I'd be okay with creating 
> last_modified(), last_accessed(), and created_on() methods that return 
> datetime objects, as long as there's also atime()/mtime()/ctime() methods 
> that return timestamps.

+1 for atime/mtime/ctime being timestamps
-1 for redundant duplicates that return DateTimes
+1 for a stat() method (there is lots of other goodies in a stat).

-- 
Donovan Baarda <abo at minkirri.apana.org.au>


From nyamatongwe at gmail.com  Tue Jun 28 03:19:45 2005
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Tue, 28 Jun 2005 11:19:45 +1000
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <59e9fd3a050627174214f5c11e@mail.gmail.com>
References: <5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<d9mmd6$gjf$1@sea.gmane.org>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<20050627182805.GC6412@ActiveState.com>
	<17088.25989.693211.15368@montanaro.dyndns.org>
	<5.1.1.6.0.20050627171642.0364b630@mail.telecommunity.com>
	<59e9fd3a050627174214f5c11e@mail.gmail.com>
Message-ID: <50862ebd05062718193ec611c@mail.gmail.com>

Andrew Durdin:

> While we'ew discussing outstanding issues: In a related discussion of
> the path module on c.l.py, Thomas Heller pointed out that the path
> module doesn't correctly handle unicode paths:
> ...

   Here is a patch that avoids failure when paths can not be
represented in a single 8 bit encoding. It adds a _cwd variable in the
initialisation and then calls this rather than os.getcwd. I sent the
patch to Jason as well.

_base = str
_cwd = os.getcwd
try:
   if os.path.supports_unicode_filenames:
       _base = unicode
       _cwd = os.getcwdu
except AttributeError:
   pass

#...

   def getcwd():
       """ Return the current working directory as a path object. """
       return path(_cwd())

   Neil

From facundobatista at gmail.com  Tue Jun 28 08:33:50 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 28 Jun 2005 03:33:50 -0300
Subject: [Python-Dev] Decimal floats as default (was: discussion
	aboutPEP239 and 240)
In-Reply-To: <3d0cebfb05062711555c73164b@mail.gmail.com>
References: <20050627083343.vbcal7el0a1wwcwo@login.werra.lunarpages.com>
	<3d0cebfb05062711555c73164b@mail.gmail.com>
Message-ID: <e04bdf3105062723335cfedf12@mail.gmail.com>

On 6/27/05, Fredrik Johansson <fredrik.johansson at gmail.com> wrote:

> The context (as I envision it) would not be just a "binary float
> context", but a universal float context that lets you choose between
> binary and decimal precision at run time.

You mean something like this?

>>> from __future__ import new_float_behaviour
>>> 1.1
1.1
>>> import sys
>>> sys.float_context.binary = True
>>> 1.1
1.1000000000000001

I don't see why this couldn't be done. The default will be use decimal
fp, and you can switch to binary fp if you need silicon-speed (unless,
of course, you have decimal-on-silicon).

In decimal mode, the context will be a full one, in binary mode, it's
not mandatory to allow access to fpu flags, it could be just like now.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From gvanrossum at gmail.com  Tue Jun 28 08:43:17 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon, 27 Jun 2005 23:43:17 -0700
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050627111913.035e2660@mail.telecommunity.com>
References: <d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<d9p4fk$kvd$1@sea.gmane.org>
	<5.1.1.6.0.20050627111913.035e2660@mail.telecommunity.com>
Message-ID: <ca471dc20506272343cbe7614@mail.gmail.com>

On 6/27/05, Phillip J. Eby <pje at telecommunity.com> wrote:

> I think the only questions remaining open are where to put it and what to
> call the class.

Whoa! Do we really need a completely different mechanism for doing the
same stuff we can already do? The path module seems mostly useful for
folks coming from Java who are used to the Java Path class. With the
massive duplication of functionality we should also consider what to
recommend for the future: will the old os.path module be deprecated,
or are we going to maintain both alternatives forever? (And what about
all the duplication with the os module itself, like the cwd()
constructor?) Remember TOOWTDI.

> I think we should put it in os.path, such that 'from
> os.path import path' gives you the path class for your platform, and using
> one of the path modules directly (e.g. 'from posixpath import path') gives
> you the specific platform's version.

Aargh! Call it anything except path. Having two things nested inside
each other with the same name is begging for confusion forever. We
have a few of these in the stdlib now (StringIO, UserDict etc.) and
they were MISTAKES.

> This is useful because sometimes you
> need to manipulate paths that are foreign to your current OS.  For example,
> the distutils and other packages sometimes use POSIX paths for input and
> then convert them to local OS paths.  Also, POSIX path objects would be
> useful for creating or parsing the "path" portion of many kinds of URLs,
> and I have often used functions from posixpath for that myself.

Right. That's why posixpath etc. always exists, not only when os.name
== "posix".

> As for a PEP, I doubt a PEP is really required for something this simple; I
> have never seen anyone say, "no, we shouldn't have this in the stdlib".  I
> think it would be more important to write reference documentation and a
> complete test suite.

"No, we shouldn't have this in the stdlib."

At least, not without more motivation than "it gets high praise".

> By the way, it also occurs to me that for the sake of subclassability, the
> methods should not return 'path(somestr)' when creating new objects, but
> instead use self.__class__(somestr).

Clearly it needs a PEP.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ajm at flonidan.dk  Tue Jun 28 10:13:55 2005
From: ajm at flonidan.dk (Anders J. Munch)
Date: Tue, 28 Jun 2005 10:13:55 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
Message-ID: <9B1795C95533CA46A83BA1EAD4B01030031EF6@flonidanmail.flonidan.net>

Walter D?rwald wrote:
> 
> We should have one uniform way of representing time in Python. IMHO  
> datetime objects are the natural choice.

Alas datetime objects do not unambiguously identify a point in time.
datetime objects are not timestamps: They represent the related but
different concept of _local time_, which can be good for presentation,
but shouldn't be allowed anywhere near a persistent store.

- Anders

From gvanrossum at gmail.com  Tue Jun 28 10:20:19 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 28 Jun 2005 01:20:19 -0700
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
In-Reply-To: <9B1795C95533CA46A83BA1EAD4B01030031EF6@flonidanmail.flonidan.net>
References: <9B1795C95533CA46A83BA1EAD4B01030031EF6@flonidanmail.flonidan.net>
Message-ID: <ca471dc205062801202519b2ef@mail.gmail.com>

On 6/28/05, Anders J. Munch <ajm at flonidan.dk> wrote:

> Alas datetime objects do not unambiguously identify a point in time.
> datetime objects are not timestamps: They represent the related but
> different concept of _local time_, which can be good for presentation,
> but shouldn't be allowed anywhere near a persistent store.

You misunderstand the datetime module! You can have a datetime object
whose timezone is UTC; or you can have a convention in your API that
datetime objects without timezone represent UTC.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From just at letterror.com  Tue Jun 28 11:41:36 2005
From: just at letterror.com (Just van Rossum)
Date: Tue, 28 Jun 2005 11:41:36 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <5.1.1.6.0.20050627171642.0364b630@mail.telecommunity.com>
Message-ID: <r01050400-1039-D32C33F4E7B811D9ABAA001124365170@[10.0.0.24]>

Phillip J. Eby wrote:

> At 03:45 PM 6/27/2005 -0500, Skip Montanaro wrote:
> >We're getting enough discussion about various aspects of Jason's
> >path module that perhaps a PEP is warranted.  All this discussion on
> >python-dev is just going to get lost.
> 
> AFAICT, the only unresolved issue outstanding is a compromise or
> Pronouncement regarding the atime/ctime/mtime members' datatype. 
> This is assuming, of course, that making the "empty path" be
> os.curdir doesn't receive any objections, and that nobody strongly
> prefers 'path.fromcwd()' over 'path.cwd()' as the alternate
> constructor name.
> 
> Apart from these fairly minor issues, there is a very short to-do
> list, small enough to do an implementation patch in an evening or
> two.  Documentation might take a similar amount of time after that;
> mostly it'll be copy-paste from the existing os.path docs, though.
> 
> As for the open issues, if we can't reach some sane compromise about
> atime/ctime/mtime, I'd suggest just providing the stat() method and
> let people use stat().st_mtime et al.  Alternately, I'd be okay with
> creating last_modified(), last_accessed(), and created_on() methods
> that return datetime objects, as long as there's also
> atime()/mtime()/ctime() methods that return timestamps.

My issues with the 'path' module (basically recapping what I've said on
the subject in the past):

  - It inherits from str/unicode, so path object have many str methods
    that make no sense for paths.
  - On OSX, it inherits from str instead of unicode, due to
    http://python.org/sf/767645
  - I don't like __div__ overloading for join().

Just

From ajm at flonidan.dk  Tue Jun 28 12:52:55 2005
From: ajm at flonidan.dk (Anders J. Munch)
Date: Tue, 28 Jun 2005 12:52:55 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
Message-ID: <9B1795C95533CA46A83BA1EAD4B01030031EF8@flonidanmail.flonidan.net>

> I wrote:
> 
> > Alas datetime objects do not unambiguously identify a point in time.
> > datetime objects are not timestamps: They represent the related but
> > different concept of _local time_, which can be good for 
> presentation,
> > but shouldn't be allowed anywhere near a persistent store.
> 
GvR replied:
> You misunderstand the datetime module! You can have a datetime object
> whose timezone is UTC; or you can have a convention in your API that
> datetime objects without timezone represent UTC.

I can do a lot of things in my own code, and I'm sure the datetime
module provides tools that I can build on to do so, but that doesn't
help in interfacing with other people's code -- such as the standard
library.

Try saving a pickle of datetime.now() and reading it on a computer set
to a different time zone.  The repr will then show the local time on
the _originating_ computer, and figuring out the absolute time this
corresponds to requires knowledge of time zone and DST settings on the
originating computer.

How about datetime.utcnow(), then?  Just use UTC for datetime
timestamps?  But that goes against the grain of the datetime module:

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime, time
>>> datetime.datetime.fromtimestamp(time.time()) - datetime.datetime.now()
datetime.timedelta(0)
>>> datetime.datetime.fromtimestamp(time.time()) - datetime.datetime.utcnow()
datetime.timedelta(0, 7200)

It seems when I subtract the present time from the present time,
there's a two hour difference.

- Anders

From gvanrossum at gmail.com  Tue Jun 28 13:36:50 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 28 Jun 2005 04:36:50 -0700
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
In-Reply-To: <9B1795C95533CA46A83BA1EAD4B01030031EF8@flonidanmail.flonidan.net>
References: <9B1795C95533CA46A83BA1EAD4B01030031EF8@flonidanmail.flonidan.net>
Message-ID: <ca471dc205062804362d327bb8@mail.gmail.com>

[Anders J. Munch]
> > > Alas datetime objects do not unambiguously identify a point in time.
> > > datetime objects are not timestamps: They represent the related but
> > > different concept of _local time_, which can be good for
> > presentation,
> > > but shouldn't be allowed anywhere near a persistent store.

[GvR]
> > You misunderstand the datetime module! You can have a datetime object
> > whose timezone is UTC; or you can have a convention in your API that
> > datetime objects without timezone represent UTC.

[Anders]
> I can do a lot of things in my own code, and I'm sure the datetime
> module provides tools that I can build on to do so, but that doesn't
> help in interfacing with other people's code -- such as the standard
> library.
> 
> Try saving a pickle of datetime.now() and reading it on a computer set
> to a different time zone.  The repr will then show the local time on
> the _originating_ computer, and figuring out the absolute time this
> corresponds to requires knowledge of time zone and DST settings on the
> originating computer.
> 
> How about datetime.utcnow(), then?  Just use UTC for datetime
> timestamps?  But that goes against the grain of the datetime module:

Against the grain? There's just a bug in your example; stop assigning
intentions to datetime that it doesn't have.

> Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import datetime, time
> >>> datetime.datetime.fromtimestamp(time.time()) - datetime.datetime.now()
> datetime.timedelta(0)
> >>> datetime.datetime.fromtimestamp(time.time()) - datetime.datetime.utcnow()
> datetime.timedelta(0, 7200)
> 
> It seems when I subtract the present time from the present time,
> there's a two hour difference.

No, you're mixing intentions. I can't tell if you're doing this
intentionally to make the design look bad or if you don't understand
the design; I'm going to assume the latter (if the former there's no
point to this discussion at all) and explain what you should have
done:

>>> datetime.datetime.utcfromtimestamp(time.time()) - datetime.datetime.utcnow()
datetime.timedelta(0)
>>>

Your bug is similar to comparing centimeters to inches, or speed to
acceleration, or any number of similar mistakes.

What I give you, however, is that a timezone object representing UTC
should be part of the standard library, so that you wouldn't have an
excuse for using tz-less datetime objects to represent UTC.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ajm at flonidan.dk  Tue Jun 28 17:01:51 2005
From: ajm at flonidan.dk (Anders J. Munch)
Date: Tue, 28 Jun 2005 17:01:51 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
Message-ID: <9B1795C95533CA46A83BA1EAD4B01030031EF9@flonidanmail.flonidan.net>

> From: Guido van Rossum [mailto:gvanrossum at gmail.com]
> 
> >>> datetime.datetime.utcfromtimestamp(time.time()) - datetime.datetime.utcnow()
> datetime.timedelta(0)

I overlooked the utcfromtimestamp method, sorry.

> Your bug is similar to comparing centimeters to inches, or speed to
> acceleration, or any number of similar mistakes.

Quite so, and that is exactly the point.  time.time() unambiguously
identifies a point in time.  A datetime object does not.  At least not
unless a tzinfo object is included, and even then there is a corner
case at the end of DST that cannot possibly be handled.

If ctime/mtime/atime were to return datetime objects, that would
pretty much have to be UTC to not lose information in the DST
transition.  I doubt that's what Walter wanted though, as that would
leave users with the job of converting from UTC datetime to local
datetime; - unless perhaps I've overlooked a convenient UTC->local
conversion method?

- Anders

From gvanrossum at gmail.com  Tue Jun 28 17:09:27 2005
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 28 Jun 2005 08:09:27 -0700
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE
	forreview)
In-Reply-To: <9B1795C95533CA46A83BA1EAD4B01030031EF9@flonidanmail.flonidan.net>
References: <9B1795C95533CA46A83BA1EAD4B01030031EF9@flonidanmail.flonidan.net>
Message-ID: <ca471dc2050628080933248f1d@mail.gmail.com>

[Anders J. Munch]
> If ctime/mtime/atime were to return datetime objects, that would
> pretty much have to be UTC to not lose information in the DST
> transition.  I doubt that's what Walter wanted though, as that would
> leave users with the job of converting from UTC datetime to local
> datetime; - unless perhaps I've overlooked a convenient UTC->local
> conversion method?

To be honest, I'm not sure what the point would be of returning
datetime objects for this use case. A time.time()-like value seems
just fine to me. The quest for a single representation of time (as
expressed by Walter's "We should have one uniform way of representing
time in Python") is IMO a mistake; there are too many different use
cases. Note that datetime intentionally doesn't handle things like
leap seconds and alternate calendars. Those things are very
specialized applications and deserve to be handled by
application-specific code.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From t-meyer at ihug.co.nz  Wed Jun 29 03:16:04 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Wed, 29 Jun 2005 13:16:04 +1200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some
	RFEforreview)
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E8033C408B@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E801DB0314@its-xchg4.massey.ac.nz>

Maybe this has already been answered somewhere (although I don't recall
seeing it, and it's not in the sourceforge tracker) but has anyone asked
Jason Orendorff what his opinion about this (including the module in the
stdlib) is?

If someone has, or if he posted it somewhere other than here, could someone
provide a link to it?

=Tony.Meyer


From zilin at cs.nyu.edu  Wed Jun 29 07:55:19 2005
From: zilin at cs.nyu.edu (Zilin Du)
Date: Wed, 29 Jun 2005 01:55:19 -0400 (EDT)
Subject: [Python-Dev] PySWT -- Python binding of SWT using GCJ + SIP
Message-ID: <Pine.GSO.4.61.0506290146500.23431@slinky.cs.nyu.edu>


Hi, all,

I just made an initial Python binding for SWT library from IBM's Eclipse 
project using GCJ + SIP. you can find some information here:

 	http://www.cs.nyu.edu/zilin/pyswt/pmwiki.php

The basic idea is as follows:

 	1. use GCJ compile SWT source codes

 	2. use gcjh to generate C++ header files for SWT

 	3. I modified SIP's code generator to generate C++ wrapper codes.

It is still in very early stage, but most widgets are usable now.  Hope 
I can get more time work on this.

Weclome any feedbacks, and feel free to forward this annocement to 
any place you want@

thanks!

Zilin



From hoffman at ebi.ac.uk  Wed Jun 29 09:11:11 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Wed, 29 Jun 2005 08:11:11 +0100
Subject: [Python-Dev] Adding the 'path' module (was Re: Some
	RFEforreview)
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E801DB0314@its-xchg4.massey.ac.nz>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E8033C408B@its-xchg4.massey.ac.nz>
	<ECBA357DDED63B4995F5C1F5CBE5B1E801DB0314@its-xchg4.massey.ac.nz>
Message-ID: <Pine.LNX.4.62.0506290808001.22373@qnzvnan.rov.np.hx>

On Wed, 29 Jun 2005, Tony Meyer wrote:

> Maybe this has already been answered somewhere (although I don't
> recall seeing it, and it's not in the sourceforge tracker) but has
> anyone asked Jason Orendorff what his opinion about this (including
> the module in the stdlib) is?

I e-mailed Jason earlier this week before submitting the RFE. He said
that "I'd like to see path (or something like it) in the standard
library." He also said he didn't have time to write a PEP at the
moment, but that I should feel free to do so.

As for me, I'm happy for Reinhold to do it if he wants. <wink>
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From gerrit at nl.linux.org  Wed Jun 29 12:15:45 2005
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Wed, 29 Jun 2005 12:15:45 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for
	review)
In-Reply-To: <ca471dc20506272343cbe7614@mail.gmail.com>
References: <d9mmd6$gjf$1@sea.gmane.org>
	<5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com>
	<Pine.LNX.4.62.0506262014050.18093@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050626204633.02102cc8@mail.telecommunity.com>
	<Pine.LNX.4.62.0506270814540.19016@qnzvnan.rov.np.hx>
	<5.1.1.6.0.20050627110223.021064b8@mail.telecommunity.com>
	<d9p4fk$kvd$1@sea.gmane.org>
	<5.1.1.6.0.20050627111913.035e2660@mail.telecommunity.com>
	<ca471dc20506272343cbe7614@mail.gmail.com>
Message-ID: <20050629101545.GA13259@topjaklont.student.utwente.nl>

Guido van Rossum wrote:
> > By the way, it also occurs to me that for the sake of subclassability, the
> > methods should not return 'path(somestr)' when creating new objects, but
> > instead use self.__class__(somestr).
> 
> Clearly it needs a PEP.

I haven't read the rest of the thread
Once, there was a path-related pre-PEP.
http://topjaklont.student.utwente.nl/creaties/path/pep-xxxx.html
It was never finished, nor am I working on it, but it's public domain.
Just wanted to remind potential PEP-authors.

regards,
Gerrit.

-- 
Weather in Twenthe, Netherlands 29/06 10:55:
	18.0?C Few clouds mostly cloudy wind 1.3 m/s None (57 m above NAP)
-- 
In the councils of government, we must guard against the acquisition of
unwarranted influence, whether sought or unsought, by the
military-industrial complex. The potential for the disastrous rise of
misplaced power exists and will persist.
    -Dwight David Eisenhower, January 17, 1961

From lode_leroy at hotmail.com  Tue Jun 28 11:39:03 2005
From: lode_leroy at hotmail.com (lode leroy)
Date: Tue, 28 Jun 2005 11:39:03 +0200
Subject: [Python-Dev] using pyhon from the MSYS shell
Message-ID: <BAY22-F5D1E1FC04146C8B11A68EFFE10@phx.gbl>

I was trying to compile a python plugin (for gimp) using the MSYS shell and 
the MINGW compiler.

I now have put this in 'sitecustomize'
$ cat /c/Python24/lib/site-packages/sitecustomize.py
import sys
import os
import re
if os.environ.get("MSYSTEM") == "MINGW32":
        os.sep='/'
        os.pathsep='/'
        sys.prefix = re.sub('\\\\','/',sys.prefix)
        sys.exec_prefix = re.sub('\\\\','/',sys.exec_prefix)


It would probably be better to have python detect that it's running from 
inside the msys shell,
and output '/'es instead of '\'es.

maybe someone could extend os.path to do this in the standard distribution:
implement an msyspath.py, which calls ntpath for each function, and does a 
replace at the end of the evaluation. Unfortunately, I'm just starting to 
look into python, so I do not know what the cleanest implementation of this 
would be...



From esrever_otua at pythonhacker.is-a-geek.net  Wed Jun 29 04:10:10 2005
From: esrever_otua at pythonhacker.is-a-geek.net (Darryl Dixon)
Date: Wed, 29 Jun 2005 14:10:10 +1200
Subject: [Python-Dev] getch() in msvcrt does not accept extended characters.
Message-ID: <1120011011.9518.22.camel@unixadmindazfc2.chh.co.nz>

Hi all,

    Please CC: me on any replies as I am not subscribed to this list.  I
am the lead maintainer for the ntlmaps project
(http://ntlmaps.sourceforge.net).  Recently a bugreport logged against
ntlmaps has brought an issue to my attention (see:
https://sourceforge.net/tracker/index.php?func=detail&aid=1224877&group_id=69259&atid=523935).  The getpass() function in module getpass does not accept extended characters on Windows.  On Windows, the getpass() source uses the msvcrt module to capture one character at a time from stdin via the _getch() function defined in conio.h.  Microsoft support capturing extended characters via _getch (see: http://support.microsoft.com/default.aspx?scid=kb;en-us;57888), but it requires making a second call to getch() if one of the 'magic' returns is encountered in the first call (0x00 or 0xE0).

The relevant chunk of code in Python that would probably need to be
changed to support this appears to be in msvcrtmodule.c:

static PyObject *
msvcrt_getch(PyObject *self, PyObject *args)
{
	int ch;
	char s[1];

	if (!PyArg_ParseTuple(args, ":getch"))
		return NULL;

	Py_BEGIN_ALLOW_THREADS
	ch = _getch();
	Py_END_ALLOW_THREADS
	s[0] = ch;
	return PyString_FromStringAndSize(s, 1);
}

It would seem that it could be made to support extended characters by changing it to:

static PyObject *
msvcrt_getch(PyObject *self, PyObject *args)
{
	int ch;
	char s[1];

	if (!PyArg_ParseTuple(args, ":getch"))
		return NULL;

	Py_BEGIN_ALLOW_THREADS
	ch = _getch();
+        if (ch == 0x00 || ch == 0XE0)
+            ch = _getch();
	Py_END_ALLOW_THREADS
	s[0] = ch;
	return PyString_FromStringAndSize(s, 1);
}

I haven't yet read the code for PyString_FromStringAndSize(), but presumably it can coerce unicode/extended results out of the int (or could be made to)?

What are the chances of modifying msvcrt_getch() to support extended chars?

The unix_getpass() seems to already work OK with extended characters, so this would ~seem~ to be a logical extension to the Windows version.

Thoughts?



-- 
Darryl Dixon <esrever_otua at pythonhacker.is-a-geek.net>


From fredrik at pythonware.com  Wed Jun 29 15:11:33 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 29 Jun 2005 15:11:33 +0200
Subject: [Python-Dev] getch() in msvcrt does not accept extended
	characters.
References: <1120011011.9518.22.camel@unixadmindazfc2.chh.co.nz>
Message-ID: <d9u68e$vrv$1@sea.gmane.org>

Darryl Dixon wrote:

> Microsoft support capturing extended characters via _getch, but it requires making a
> second call to getch() if one of the 'magic' returns is encountered in the first call (0x00
> or 0xE0).

so why not do that in your python code?

> The relevant chunk of code in Python that would probably need to be
> changed to support this appears to be in msvcrtmodule.c:

if you change msvcrt, you'll break all the programs that uses getch() in
the prescribed way...

</F> 




From aahz at pythoncraft.com  Wed Jun 29 15:32:15 2005
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 29 Jun 2005 06:32:15 -0700
Subject: [Python-Dev] using pyhon from the MSYS shell
In-Reply-To: <BAY22-F5D1E1FC04146C8B11A68EFFE10@phx.gbl>
References: <BAY22-F5D1E1FC04146C8B11A68EFFE10@phx.gbl>
Message-ID: <20050629133215.GB8241@panix.com>

On Tue, Jun 28, 2005, lode leroy wrote:
>
> I was trying to compile a python plugin (for gimp) using the MSYS
> shell and the MINGW compiler.

python-dev is the wrong place for this question; please start with
comp.lang.python (or find another suitable place).
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From skink at evhr.net  Wed Jun 29 15:38:08 2005
From: skink at evhr.net (Fabien)
Date: Wed, 29 Jun 2005 15:38:08 +0200
Subject: [Python-Dev] New python developer
Message-ID: <20050629133808.C90CC7EC04@postix.sdv.fr>

Hello,

I'm using Python for about 2 years, and I would like to go further in
python developement. So that's why I've subscribed to python-dev. And
since I'm not very good in C, I think I will try to help and to submit
patchs in pure python.

-- 
Fabien SCHWOB (and sorry for my english but it's not my mother tongue)


From facundobatista at gmail.com  Wed Jun 29 15:43:18 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 29 Jun 2005 10:43:18 -0300
Subject: [Python-Dev] New python developer
In-Reply-To: <20050629133808.C90CC7EC04@postix.sdv.fr>
References: <20050629133808.C90CC7EC04@postix.sdv.fr>
Message-ID: <e04bdf3105062906434fad2e31@mail.gmail.com>

On 6/29/05, Fabien <skink at evhr.net> wrote:

> I'm using Python for about 2 years, and I would like to go further in
> python developement. So that's why I've subscribed to python-dev. And
> since I'm not very good in C, I think I will try to help and to submit
> patchs in pure python.

I'm not good in C neither, but there's always work to do (not only
coding, but also reviewing bugs, patchs, or dealing with
documentation).

You're more than welcomed.

If I can help you with anything, just let me know.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From aahz at pythoncraft.com  Wed Jun 29 16:52:11 2005
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 29 Jun 2005 07:52:11 -0700
Subject: [Python-Dev] New python developer
In-Reply-To: <20050629133808.C90CC7EC04@postix.sdv.fr>
References: <20050629133808.C90CC7EC04@postix.sdv.fr>
Message-ID: <20050629145211.GA24731@panix.com>

On Wed, Jun 29, 2005, Fabien wrote:
> 
> I'm using Python for about 2 years, and I would like to go further in
> python developement. So that's why I've subscribed to python-dev. And
> since I'm not very good in C, I think I will try to help and to submit
> patchs in pure python.
> 
> -- 
> Fabien SCHWOB (and sorry for my english but it's not my mother tongue)

Welcome!  Your English is just fine.  If you haven't yet, take a look at
http://www.python.org/dev/
Pay particular attention to "Why Develop Python?" and "Intro to
Development".
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From reinhold-birkenfeld-nospam at wolke7.net  Wed Jun 29 21:35:08 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Wed, 29 Jun 2005 21:35:08 +0200
Subject: [Python-Dev] Adding the 'path' module (was Re: Some
	RFEforreview)
In-Reply-To: <Pine.LNX.4.62.0506290808001.22373@qnzvnan.rov.np.hx>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E8033C408B@its-xchg4.massey.ac.nz>	<ECBA357DDED63B4995F5C1F5CBE5B1E801DB0314@its-xchg4.massey.ac.nz>
	<Pine.LNX.4.62.0506290808001.22373@qnzvnan.rov.np.hx>
Message-ID: <d9usni$a19$1@sea.gmane.org>

Michael Hoffman wrote:
> On Wed, 29 Jun 2005, Tony Meyer wrote:
> 
>> Maybe this has already been answered somewhere (although I don't
>> recall seeing it, and it's not in the sourceforge tracker) but has
>> anyone asked Jason Orendorff what his opinion about this (including
>> the module in the stdlib) is?
> 
> I e-mailed Jason earlier this week before submitting the RFE. He said
> that "I'd like to see path (or something like it) in the standard
> library." He also said he didn't have time to write a PEP at the
> moment, but that I should feel free to do so.
> 
> As for me, I'm happy for Reinhold to do it if he wants. <wink>

Well, as it is your RFE... ;)

I'm not eager to write a PEP right now, though I think I said so in a post.
I just wanted to start a discussion here, so that you can write a better
PEP <wink>.

Reinhold

-- 
Mail address is perfectly valid!


From raymond.hettinger at verizon.net  Thu Jun 30 00:08:52 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 29 Jun 2005 18:08:52 -0400
Subject: [Python-Dev] floatobject.c 2.136
In-Reply-To: <E1DiXJz-0003sQ-AS@sc8-pr-cvs1.sourceforge.net>
Message-ID: <005501c57cf7$233b91c0$8d0ba044@oemcomputer>

I'm getting a compiler warning from your checkin:

C:\py25\Objects\floatobject.c(1430) : warning C4244: 'initializing' :
conversion from 'double ' to 'float ', possible loss of data


From mwh at python.net  Thu Jun 30 02:02:37 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 30 Jun 2005 01:02:37 +0100
Subject: [Python-Dev] floatobject.c 2.136
In-Reply-To: <005501c57cf7$233b91c0$8d0ba044@oemcomputer> (Raymond
	Hettinger's message of "Wed, 29 Jun 2005 18:08:52 -0400")
References: <005501c57cf7$233b91c0$8d0ba044@oemcomputer>
Message-ID: <2mwtocyan6.fsf@starship.python.net>

"Raymond Hettinger" <raymond.hettinger at verizon.net> writes:

> I'm getting a compiler warning from your checkin:

"your"?  Mine?

> C:\py25\Objects\floatobject.c(1430) : warning C4244: 'initializing' :
> conversion from 'double ' to 'float ', possible loss of data

That's this line:

            float y = x;

where x is of double type.  This seems an, um, daft warning to me, but
it's easy to fix, and I've fixed it.

Cheers,
mwh

-- 
  The only problem with Microsoft is they just have no taste.
              -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
                                and quoted by Aahz on comp.lang.python

From kbk at shore.net  Thu Jun 30 06:35:04 2005
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu, 30 Jun 2005 00:35:04 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200506300435.j5U4Z49Y015752@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  344 open ( +6) /  2875 closed ( +9) /  3219 total (+15)
Bugs    :  897 open (-17) /  5094 closed (+34) /  5991 total (+17)
RFE     :  191 open ( +3) /   170 closed ( +0) /   361 total ( +3)

New / Reopened Patches
______________________

fileinput openfile patch, bz2fileinput  (2005-06-22)
       http://python.org/sf/1225466  opened by  Stephan Boettcher

Proposal to implement comment rows in csv module  (2005-06-22)
       http://python.org/sf/1225769  opened by  Iain Haslam

Patch for (Doc) bug #1217513  (2005-06-25)
CLOSED http://python.org/sf/1227442  opened by  Peter van Kampen

Patch for (Doc) bug #1225705   (2005-06-25)
CLOSED http://python.org/sf/1227444  opened by  Peter van Kampen

Patch for (Doc) bug #1175022  (2005-06-25)
CLOSED http://python.org/sf/1227489  opened by  Peter van Kampen

Patch for (Doc) bug #1186072  (2005-06-25)
CLOSED http://python.org/sf/1227495  opened by  Peter van Kampen

Patch for (Doc) bug #1166582  (2005-06-25)
CLOSED http://python.org/sf/1227513  opened by  Peter van Kampen

Patch for (Doc) bug #1212195  (2005-06-26)
       http://python.org/sf/1227545  opened by  Peter van Kampen

Patch for (Doc) bug #1219273  (2005-06-26)
       http://python.org/sf/1227568  opened by  Peter van Kampen

Patch for (Doc) bug #1228904  (2005-06-29)
       http://python.org/sf/1229935  opened by  Peter van Kampen

solaris 10 should not define _XOPEN_SOURCE_EXTENDED  (2005-06-27)
       http://python.org/sf/1227966  opened by  Bill Clarke

code.py use sys.excepthook to display exceptions  (2005-06-27)
       http://python.org/sf/1228112  opened by  Miki Tebeka

optionally allow mutable builtin types  (2005-06-28)
       http://python.org/sf/1229239  opened by  Greg Couch

optionally allow mutable builtin types  (2005-06-28)
CLOSED http://python.org/sf/1229280  opened by  Greg Couch

Zipfile fix create_system information  (2005-06-29)
       http://python.org/sf/1229511  opened by  Hendrik Muhs

Patches Closed
______________

Building on Mac OS X 10.1  (2004-01-02)
       http://python.org/sf/869555  closed by  mwh

httplib mentions getreply instead of getresponse  (2005-05-16)
       http://python.org/sf/1203094  closed by  birkenfeld

note that os.chown can have -1 as an argument  (2005-06-01)
       http://python.org/sf/1213031  closed by  birkenfeld

Patch for (Doc) bug #1217513  (2005-06-25)
       http://python.org/sf/1227442  closed by  birkenfeld

Patch for (Doc) bug #1225705   (2005-06-25)
       http://python.org/sf/1227444  closed by  birkenfeld

Patch for (Doc) bug #1175022  (2005-06-25)
       http://python.org/sf/1227489  closed by  birkenfeld

Patch for (Doc) bug #1186072  (2005-06-25)
       http://python.org/sf/1227495  closed by  birkenfeld

Patch for (Doc) bug #1166582  (2005-06-25)
       http://python.org/sf/1227513  closed by  birkenfeld

cgitb: make more usable for 'binary-only' sw (new patch)  (2005-02-19)
       http://python.org/sf/1144549  closed by  birkenfeld

optionally allow mutable builtin types  (2005-06-28)
       http://python.org/sf/1229280  closed by  gregcouch

New / Reopened Bugs
___________________

crash in gcmodule.c on python reinitialization  (2005-06-22)
       http://python.org/sf/1225584  opened by  Alexander Miseler

os.environ documentation should mention unsetenv  (2005-06-22)
CLOSED http://python.org/sf/1225705  opened by  Brian Wellington

SimpleXMLRPCServer example is broken  (2004-10-06)
       http://python.org/sf/1041501  reopened by  kjohnson

Can't take sequence slice past 32K  (2005-06-23)
CLOSED http://python.org/sf/1226404  reopened by  tim_one

Can't take sequence slice past 32K  (2005-06-23)
CLOSED http://python.org/sf/1226404  opened by  Gene Cash

cPickle and pickle dump bug (inf float)  (2005-06-24)
CLOSED http://python.org/sf/1226955  opened by  Andrea Bolzonella

segfault in os module  (2005-06-24)
       http://python.org/sf/1226969  opened by  jacobo_es

Queue class does not inherit object  (2005-06-24)
CLOSED http://python.org/sf/1227166  opened by  Vincent C?t?-Roy

CGIXMLRPCRequestHandler example uses nonexistant "div()"  (2005-06-24)
CLOSED http://python.org/sf/1227181  opened by  Danny Yoo

Two requests to one file are not done in parallel  (2005-06-25)
       http://python.org/sf/1227480  opened by  Markus Franz

subprocess: inheritance of std descriptors inconsistent  (2005-06-26)
       http://python.org/sf/1227748  opened by  Andr? Malo

shelve/bsddb crash on db close  (2005-06-26)
       http://python.org/sf/1227955  opened by  Scott

cPickle and pickle dump bug (inf float)  (2005-06-27)
CLOSED http://python.org/sf/1228015  opened by  Andrea Bolzonella

usage of type "char*" instead of "const char*"  (2005-06-27)
       http://python.org/sf/1228053  opened by  Mattias Ekholm

update to 7.15 zlib & 7.18.1 ZipFile Objects docs  (2005-06-28)
CLOSED http://python.org/sf/1228830  opened by  Brian vdB

weakref example broken  (2005-06-28)
       http://python.org/sf/1228904  opened by  paolo veronelli

error whit dict working whith class  (2005-06-28)
CLOSED http://python.org/sf/1229264  opened by  ST2015

No struct.pack exception for some out of range integers  (2005-06-29)
       http://python.org/sf/1229380  opened by  Adal Chiriliuc

missing Py_DECREF in PyObject_CallMethod  (2005-06-28)
       http://python.org/sf/1229429  opened by  Gary

httplib error checking.  (2005-06-29)
       http://python.org/sf/1229646  opened by  Chris Johns

Tracing C function calls and keyword arguments  (2005-06-29)
       http://python.org/sf/1229680  opened by  Frank Cieslok

Bus error in extension with gcc 3.3  (2005-06-29)
       http://python.org/sf/1229788  opened by  Gary Robinson

Bugs Closed
___________

Line endings problem with Python 2.4.1 cause imports to fail  (2005-06-21)
       http://python.org/sf/1225059  closed by  doerwalter

os.environ documentation should mention unsetenv  (2005-06-22)
       http://python.org/sf/1225705  closed by  birkenfeld

int/long unification and hex()  (2005-06-20)
       http://python.org/sf/1224347  closed by  rhettinger

rsplit introduces spurious CR  (2005-06-21)
       http://python.org/sf/1224672  closed by  birkenfeld

subprocess.Popen fails with closed stdout  (2005-04-10)
       http://python.org/sf/1180160  closed by  astrand

Can't take sequence slice past 32K  (2005-06-23)
       http://python.org/sf/1226404  closed by  mwh

Can't take sequence slice past 32K  (2005-06-23)
       http://python.org/sf/1226404  closed by  birkenfeld

cPickle and pickle dump bug (inf float)  (2005-06-24)
       http://python.org/sf/1226955  closed by  mwh

Queue class does not inherit object  (2005-06-24)
       http://python.org/sf/1227166  closed by  rhettinger

CGIXMLRPCRequestHandler example uses nonexistant "div()"  (2005-06-24)
       http://python.org/sf/1227181  closed by  dyoo

Info from man python not in docs  (2005-06-08)
       http://python.org/sf/1217152  closed by  birkenfeld

Omission in docs for smtplib.SMTP.sendmail()  (2005-06-09)
       http://python.org/sf/1217513  closed by  birkenfeld

httplib docs mentioning HTTPConnection.getreply  (2005-05-15)
       http://python.org/sf/1202475  closed by  birkenfeld

os.waitpid docs don't specify return value for WNOHANG  (2005-04-26)
       http://python.org/sf/1190563  closed by  birkenfeld

cgi.CGIHTTPRequestHandler remembers QUERY_STRING  (2003-09-07)
       http://python.org/sf/801992  closed by  birkenfeld

property example code error  (2005-04-01)
       http://python.org/sf/1175022  closed by  birkenfeld

tempfile.mktemp() omits pid from name  (2003-12-10)
       http://python.org/sf/857566  closed by  birkenfeld

tempnam doc doesn't include link to tmpfile  (2005-04-19)
       http://python.org/sf/1186072  closed by  birkenfeld

IterableUserDict not in docs  (2005-03-19)
       http://python.org/sf/1166582  closed by  birkenfeld

Python keeps file references after calling close methode  (2005-04-10)
       http://python.org/sf/1180237  closed by  birkenfeld

thisid not intialized in pindent.py script  (2004-11-25)
       http://python.org/sf/1072853  closed by  birkenfeld

raw_input() displays wrong unicode prompt  (2005-01-10)
       http://python.org/sf/1099364  closed by  birkenfeld

Cookie.py produces invalid code  (2005-01-25)
       http://python.org/sf/1108948  closed by  birkenfeld

Cross-site scripting on BaseHTTPServer  (2005-01-11)
       http://python.org/sf/1100201  closed by  birkenfeld

urllib2.py assumes 206 is an error  (2005-04-07)
       http://python.org/sf/1178145  closed by  birkenfeld

poorly named variable in urllib2.py  (2005-04-03)
       http://python.org/sf/1175848  closed by  birkenfeld

self.length shield exception in httplib  (2005-03-03)
       http://python.org/sf/1155638  closed by  birkenfeld

BaseCookie does not call value_decode  (2005-03-25)
       http://python.org/sf/1170279  closed by  birkenfeld

doctest.script_from_examples() result sometimes un-exec-able  (2005-03-29)
       http://python.org/sf/1172785  closed by  birkenfeld

Can't read some http URLs using neither urllib, nor urllib2  (2004-11-01)
       http://python.org/sf/1058059  closed by  birkenfeld

CGIHTTPServer: directories/scripts with spaces in their name  (2004-12-19)
       http://python.org/sf/1088039  closed by  birkenfeld

CGI QUERY_STRING  (2004-05-02)
       http://python.org/sf/946247  closed by  birkenfeld

attempt to access sys.argv when it doesn't exist  (2003-11-10)
       http://python.org/sf/839151  closed by  birkenfeld

cPickle and pickle dump bug (inf float)  (2005-06-27)
       http://python.org/sf/1228015  closed by  mwh

update to 7.15 zlib & 7.18.1 ZipFile Objects docs  (2005-06-28)
       http://python.org/sf/1228830  closed by  rhettinger

error whit dict working whith class  (2005-06-28)
       http://python.org/sf/1229264  closed by  birkenfeld

New / Reopened RFE
__________________

Add Jason Orendorff's path module to stdlib  (2005-06-23)
       http://python.org/sf/1226256  opened by  Michael Hoffman


From fredrik.johansson at gmail.com  Thu Jun 30 11:47:16 2005
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Thu, 30 Jun 2005 11:47:16 +0200
Subject: [Python-Dev] Decimal floats as default (was: discussion
	aboutPEP239 and 240)
In-Reply-To: <e04bdf3105062723335cfedf12@mail.gmail.com>
References: <20050627083343.vbcal7el0a1wwcwo@login.werra.lunarpages.com>
	<3d0cebfb05062711555c73164b@mail.gmail.com>
	<e04bdf3105062723335cfedf12@mail.gmail.com>
Message-ID: <3d0cebfb05063002474a53616d@mail.gmail.com>

Should I write a PEP?

- Fredrik

From python at rcn.com  Thu Jun 30 12:42:14 2005
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 30 Jun 2005 06:42:14 -0400
Subject: [Python-Dev] Decimal floats as default (was:
	discussionaboutPEP239 and 240)
In-Reply-To: <3d0cebfb05063002474a53616d@mail.gmail.com>
Message-ID: <000701c57d60$60e54de0$6baf958d@oemcomputer>

> Should I write a PEP?

No.  Just let it evolve naturally.

The next step is for a builtin C version.  That will likely happen as
soon as one of us has the time and inclination to write it.

If that works out well, then writing decimal literals like 123d may be
plausible.

And, if that works out well too, then there would be some basis for
Py3.0 using decimal as the default float.



Raymond

From barry at python.org  Thu Jun 30 16:45:05 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 30 Jun 2005 10:45:05 -0400
Subject: [Python-Dev] Inconsistent API for sets.Set and build-in set
Message-ID: <1120142705.9638.84.camel@geddy.wooz.org>

I've been looking at the API for sets.Set and built-in set objects in
Python 2.4.1 and I think I have found some minor inconsistencies.

Background: we have an object that is very similar to "sets" and we
originally modeled the API after sets.Set since we started with Python
2.3.  Now I'm trying to update the API so that it's consistent with
Python 2.4's built-in set object and I've noticed the following
discrepancies.

set.Set has both an .update() method and a .union_update() method.  They
appear to be completely identical, accepting either a Set object or an
arbitrary sequence.  This is the case despite the docstring difference
for these two methods and the fact that Set.update() isn't documented on
the texinfo page.

Built-in set has only .update() though but it acts just like the set.Set
methods above.  Note that of all these methods, only Set.update() is
documented to take an iterable argument.

These inconsistencies could prove a bit problematic when porting Py2.3
applications using sets.Set to Py2.4 using built-in set.  I'd like to
fix this for Python 2.4.2, and I think the changes are pretty minimal.

If there are no objections, I propose to do the following (only in
Python 2.4 and 2.5):

      * Add set.union_update() as an alias for set.update().
      * Add to docstrings for all methods that 't' can be any iterable.
      * Update texinfo documentation to add Set.update() and
        set.union_update() and explain that all can take iterables

I consider this a bug in 2.4, not a new feature, because without it, it
makes more work in porting applications.

Thoughts?

I'm willing to Just Fix It, but if someone wants to see a patch first,
I'll be happy to generate it and post it to SF.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20050630/f14d35db/attachment.pgp

From aahz at pythoncraft.com  Thu Jun 30 17:31:31 2005
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 30 Jun 2005 08:31:31 -0700
Subject: [Python-Dev] Inconsistent API for sets.Set and build-in set
In-Reply-To: <1120142705.9638.84.camel@geddy.wooz.org>
References: <1120142705.9638.84.camel@geddy.wooz.org>
Message-ID: <20050630153130.GA15385@panix.com>

On Thu, Jun 30, 2005, Barry Warsaw wrote:
>
> If there are no objections, I propose to do the following (only in
> Python 2.4 and 2.5):
> 
>       * Add set.union_update() as an alias for set.update().
>       * Add to docstrings for all methods that 't' can be any iterable.
>       * Update texinfo documentation to add Set.update() and
>         set.union_update() and explain that all can take iterables
> 
> I consider this a bug in 2.4, not a new feature, because without it, it
> makes more work in porting applications.

+0  (I'm only not +1 because I don't use sets much -- I'm still mired in
Python 2.2 -- but I'm always happy to see inconsistencies resolved)

I'll guess that Raymond will probably want 2.5 to have set.union_update()
get a PendingDeprecationWarning.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

From Michaels at rd.bbc.co.uk  Thu Jun 30 17:22:19 2005
From: Michaels at rd.bbc.co.uk (Michael Sparks)
Date: Thu, 30 Jun 2005 16:22:19 +0100
Subject: [Python-Dev] Adding Python-Native Threads
In-Reply-To: <aac2c7cb05062604043b22c3a8@mail.gmail.com>
References: <aac2c7cb0506260234286befd3@mail.gmail.com>
	<4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com>
	<aac2c7cb05062604043b22c3a8@mail.gmail.com>
Message-ID: <200506301622.19915.Michaels@rd.bbc.co.uk>

On Sunday 26 Jun 2005 12:04, Adam Olsen wrote:
> On 6/26/05, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> > Have a look at stackless python. http://www.stackless.com/
> On 6/26/05, Florian Schulze <florian.proff.schulze at gmx.net> wrote:
> > Also look at greenlets, they are in the py lib http://codespeak.net/py
> While internally Stackless and Greenlets may (or may not) share a lot
> with my proposed python-native threads, they differ greatly in their
> intent and the resulting interface they expose.  

Indeed - Greenlets allows you to build the functionality you propose without 
having to change the language.

> > For example, with Greenlets you would use the .switch() method of a 
> specific greenlet instance to switch to it, and with my python-native
> threads you would use the global idle() function which would decide
> for itself which thread to switch to.

This is easy enough to build using greenlets today. I tried writing an
experimental version of our generator scheduling system using greenlets
rather than generators and found it to work very nicely. I'd suggest that if 
you really want this functionality (I can understand why) that you revisit 
greenlets - they probably do what you want.

Mainly replying to say "-1",

Best Regards,


Michael.
-- 
Michael Sparks, Senior R&D Engineer, Digital Media Group
Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This e-mail may contain personal views which are not the views of the BBC.

From python at rcn.com  Thu Jun 30 19:37:00 2005
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 30 Jun 2005 13:37:00 -0400
Subject: [Python-Dev] Inconsistent API for sets.Set and build-in set
In-Reply-To: <1120142705.9638.84.camel@geddy.wooz.org>
Message-ID: <001601c57d9a$52ae08e0$1721a044@oemcomputer>

> If there are no objections, I propose to do the following (only in
> Python 2.4 and 2.5):
> 
>       * Add set.union_update() as an alias for set.update().

No.  It was intentional to drop the duplicate method with the
hard-to-use name.  There was some thought given to deprecating
sets.union_update() but that would have just caused unnecessary grief.



>       * Add to docstrings for all methods that 't' can be any
iterable.
>       * Update texinfo documentation to add Set.update() and
>         set.union_update() and explain that all can take iterables

Feel free to assign a doc patch to me.



> I consider this a bug in 2.4, not a new feature, because without it,
it
> makes more work in porting applications.

Bah.  It's just one of the handful of search/replace steps:

   Set --> set
   ImmutableSet --> frozenset
   union_update --> update

    


> I'm willing to Just Fix It,

Please don't.  All of the differences between set and Set were
intentional improvements (i.e. the hash algorithm's are different).



Raymond

From nidoizo at yahoo.com  Thu Jun 30 20:03:38 2005
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu, 30 Jun 2005 14:03:38 -0400
Subject: [Python-Dev] List copy and clear (was Re: Inconsistent API for
 sets.Set and build-in set)
In-Reply-To: <1120142705.9638.84.camel@geddy.wooz.org>
References: <1120142705.9638.84.camel@geddy.wooz.org>
Message-ID: <da1bnm$m0k$1@sea.gmane.org>

Barry Warsaw wrote:
> I've been looking at the API for sets.Set and built-in set objects in
> Python 2.4.1 and I think I have found some minor inconsistencies.

This comment reminds me another small inconsistency/annoyance.

Should copy and clear functions be added to lists, to be more consistent 
  with dict and set, and easing generic code?

(Sorry if discussed before, I haven't found anything about that in the 
archives)

Regards,
Nicolas


From tismer at stackless.com  Thu Jun 30 21:05:29 2005
From: tismer at stackless.com (Christian Tismer)
Date: Thu, 30 Jun 2005 21:05:29 +0200
Subject: [Python-Dev] Adding Python-Native Threads
In-Reply-To: <200506301622.19915.Michaels@rd.bbc.co.uk>
References: <aac2c7cb0506260234286befd3@mail.gmail.com>	<4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com>	<aac2c7cb05062604043b22c3a8@mail.gmail.com>
	<200506301622.19915.Michaels@rd.bbc.co.uk>
Message-ID: <42C44279.4070707@stackless.com>

Michael Sparks wrote:

...

> Indeed - Greenlets allows you to build the functionality you propose without 
> having to change the language.
> 
> 
>>>For example, with Greenlets you would use the .switch() method of a 
>>
>>specific greenlet instance to switch to it, and with my python-native
>>threads you would use the global idle() function which would decide
>>for itself which thread to switch to.
> 
> 
> This is easy enough to build using greenlets today. I tried writing an
> experimental version of our generator scheduling system using greenlets
> rather than generators and found it to work very nicely. I'd suggest that if 
> you really want this functionality (I can understand why) that you revisit 
> greenlets - they probably do what you want.

I'd like to add that in most production code I saw so far,
people are much more happy with explicit switching of
(green|task)lets. The wish for free-running stuff like real
threads is a typical symptom in an early project phase.
When it becomes more serious, you will love to have as much
control as possible, and leave auto-switching to the very
rare cases like some background monitoring or other stuff
which is not really involved in you control/data flow.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From raymond.hettinger at verizon.net  Thu Jun 30 23:26:03 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 30 Jun 2005 17:26:03 -0400
Subject: [Python-Dev] List copy and clear (was Re: Inconsistent API for
 sets.Set and build-in set)
In-Reply-To: <da1bnm$m0k$1@sea.gmane.org>
Message-ID: <000a01c57dba$520e8700$1330cb97@oemcomputer>

[Nicolas Fleury]
> This comment reminds me another small inconsistency/annoyance.
> 
> Should copy and clear functions be added to lists, to be more
consistent
>   with dict and set, and easing generic code?

I think not.

Use copy.copy() for generic copying -- it works across a wide range of
objects.  Alternatively, use the constructor as generic way to make
duplicates:

   dup = set(s)
   dup = list(l)
   dup = dict(d)
   dup = tuple(t)     # note, the duplicate is original object here :-)

I would think that that generic clearing is a lark.  First, it only
applies to mutable objects.  Second, it would likely only be useful in
the presence of a generic method for adding to the cleared container (as
opposed to the existing append(), add(), and setitem() methods for
lists, sets, and dictionaries respectively).  So, for lists, stick with
the current idiom:

   mylist[:] = []       # clear



Raymond


From raymond.hettinger at verizon.net  Thu Jun 30 23:41:16 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 30 Jun 2005 17:41:16 -0400
Subject: [Python-Dev] Terminology for PEP 343
Message-ID: <000d01c57dbc$71df2420$1330cb97@oemcomputer>

With 343 accepted, we can now add __enter__() and __exit__() methods to
objects.

What term should describe those objects in the documentation?  

For instance, if the new magic methods are added to decimal.Context(),
do we then describe Context objects as "withable" ;-)

The PEP itself doesn't provide much of a clue.  The terms "template",
"wrapper", and "block" are over-broad and do not provide a metaphor for
setup/teardown, r entry/exit, or initialization/finalization.

Whatever term is selected, it should be well thought-out and added to
the glossary.  The choice of words will likely have a great effect on
learnability and on how people think about the new tool.


Raymond




From fdrake at acm.org  Thu Jun 30 23:43:32 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Jun 2005 17:43:32 -0400
Subject: [Python-Dev] List copy and clear (was Re: Inconsistent API for
	sets.Set and build-in set)
In-Reply-To: <000a01c57dba$520e8700$1330cb97@oemcomputer>
References: <000a01c57dba$520e8700$1330cb97@oemcomputer>
Message-ID: <200506301743.32417.fdrake@acm.org>

On Thursday 30 June 2005 17:26, Raymond Hettinger wrote:
 > the current idiom:
 >
 >    mylist[:] = []       # clear

Unless you happen to prefer the other current idiom:

    del mylist[:]


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From esrever_otua at pythonhacker.is-a-geek.net  Wed Jun 29 23:43:15 2005
From: esrever_otua at pythonhacker.is-a-geek.net (Darryl Dixon)
Date: Thu, 30 Jun 2005 09:43:15 +1200
Subject: [Python-Dev] getch() in msvcrt does not accept extended characters.
Message-ID: <1120081396.14913.13.camel@unixadmindazfc2.chh.co.nz>

Hi,

    I'm sorry, I don't seem to have done a very good job at explaining
the situation.  I'll try again:
'getch()' is a low-level function provided on Windows to capture a
single character of input from a user, /without echoing it to the
screen/.  As far as I can tell there's no other way of doing this with
Python on Windows.  The Python interface to this function is in the C
code in msvcrtmodule.c, which has a (very thin) wrapper around the raw
OS system call.  Microsoft provide a way of accepting both normal ASCII
codes, and extended characters via this system call.  Currently, the
Python wrapper in msvcrtmodule.c only supports the semantics of getting
the bare ASCII codes, and not the extended characters.  I would strongly
suggest that it should support both.

So, I guess in answer to the questions raised below;

1) I can't do it in Python code without getch() (hence the original
email)

2) I would argue that in fact getch() is 'broken' in its current Python
implementation, as it doesn't support what the OS implies /should/ be
supported (and, indeed, if I input an extended character in response to
a 'getch()' call, all I get back currently is an empty string).

Hope this helps,
D


Fredrik wrote:
>Darryl Dixon wrote:
>
>> Microsoft support capturing extended characters via _getch, but it requires making a
>> second call to getch() if one of the 'magic' returns is encountered in the first call (0x00
>> or 0xE0).
>
>so why not do that in your python code?
>
>> The relevant chunk of code in Python that would probably need to be
>> changed to support this appears to be in msvcrtmodule.c:
>
>if you change msvcrt, you'll break all the programs that uses getch() in
>the prescribed way...
>
></F> 
-- 
Darryl Dixon <esrever_otua at pythonhacker.is-a-geek.net>