From guido at python.org  Tue Jul  1 01:12:03 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jun 2008 16:12:03 -0700
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806300931x7635cee5t597c09ff7b06fc6f@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
	<58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
	<5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com>
	<ca471dc20806300853m7d2f269bibaf753c18c6f48b3@mail.gmail.com>
	<5c6f2a5d0806300931x7635cee5t597c09ff7b06fc6f@mail.gmail.com>
Message-ID: <ca471dc20806301612m3019d300tabd552590146bcdc@mail.gmail.com>

Mon, Jun 30, 2008 at 9:31 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Mon, Jun 30, 2008 at 4:53 PM, Guido van Rossum <guido at python.org> wrote:
>> FWIW, I'm fine with making these methods on float -- a class method
>> float.fromhex(...) echoes e.g. dict.fromkeys(...) and
>> datetime.fromordinal(...). The to-hex conversion could be x.hex() --
>> we don't tend to use ".toxyz()" as a naming convention much in Python.
>
> Would it be totally outrageous for the float constructor to accept
> hex strings directly?

int('0x10') raises a ValueError as well. You might propose
float('0x...p...', 16) but since the format is so specifically
different I think that's not completely kosher.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Jul  1 22:16:37 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 1 Jul 2008 16:16:37 -0400
Subject: [Python-Dev] Second betas tomorrow
Message-ID: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Wow, I bet this one crept up on you as quickly as it did me!

We have our second planned beta releases for 2.6 and 3.0 tomorrow.  As  
usual I will start looking at blockers and buildbots tomorrow  
afternoon (UTC-4 time) with a plan to start building things at about  
6pm.  Also, I will of course be in #python-dev on freenode to answer  
any questions, or get second opinions.

PEP 361 claims that these will be the last betas.  Whether that's true  
or not depends on how well the beta2's go.  Please help review code or  
fix bugs.  If you know of things that absolutely must go into beta2,  
be sure there is an open release-blocker bug on the issue.

Thanks,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGqQpnEjvBPtnXfVAQLdagP/VooK8+AoPrb1bR7xAxGqg0vC1HOKw5qZ
8VQArzgldz1OnoG24PuKGdaEw7PbHjCMkD0/CyZWjH8/yWawcxV7hKl6RYHJ3GX9
keroo7wz3/NaptJtA9ldoKA5ekV8WVVC5OElgtjKr+v6HorPQSHzUgJiDHYUS1FW
A8fdHipyZds=
=vwYy
-----END PGP SIGNATURE-----

From guido at python.org  Tue Jul  1 22:25:27 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 1 Jul 2008 13:25:27 -0700
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
Message-ID: <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>

I think we should put this one off. The previous betas were done on June 18,
and IMO the next beta should be about a month afterwards, not 2 weeks.

On Tue, Jul 1, 2008 at 1:16 PM, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Wow, I bet this one crept up on you as quickly as it did me!
>
> We have our second planned beta releases for 2.6 and 3.0 tomorrow.  As
> usual I will start looking at blockers and buildbots tomorrow afternoon
> (UTC-4 time) with a plan to start building things at about 6pm.  Also, I
> will of course be in #python-dev on freenode to answer any questions, or get
> second opinions.
>
> PEP 361 claims that these will be the last betas.  Whether that's true or
> not depends on how well the beta2's go.  Please help review code or fix
> bugs.  If you know of things that absolutely must go into beta2, be sure
> there is an open release-blocker bug on the issue.
>
> Thanks,
> - -Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSGqQpnEjvBPtnXfVAQLdagP/VooK8+AoPrb1bR7xAxGqg0vC1HOKw5qZ
> 8VQArzgldz1OnoG24PuKGdaEw7PbHjCMkD0/CyZWjH8/yWawcxV7hKl6RYHJ3GX9
> keroo7wz3/NaptJtA9ldoKA5ekV8WVVC5OElgtjKr+v6HorPQSHzUgJiDHYUS1FW
> A8fdHipyZds=
> =vwYy
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080701/dda8165a/attachment.htm>

From jnoller at gmail.com  Tue Jul  1 22:28:11 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 1 Jul 2008 16:28:11 -0400
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <g4e3p1$77m$1@ger.gmane.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<g4e3p1$77m$1@ger.gmane.org>
Message-ID: <4222a8490807011328v4df3eb9fl7de44be415b334ff@mail.gmail.com>

On Tue, Jul 1, 2008 at 4:23 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Barry Warsaw schrieb:
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Wow, I bet this one crept up on you as quickly as it did me!
>>
>> We have our second planned beta releases for 2.6 and 3.0 tomorrow.  As
>>  usual I will start looking at blockers and buildbots tomorrow  afternoon
>> (UTC-4 time) with a plan to start building things at about  6pm.  Also, I
>> will of course be in #python-dev on freenode to answer  any questions, or
>> get second opinions.
>>
>> PEP 361 claims that these will be the last betas.  Whether that's true  or
>> not depends on how well the beta2's go.  Please help review code or  fix
>> bugs.  If you know of things that absolutely must go into beta2,  be sure
>> there is an open release-blocker bug on the issue.
>
> May I ask if it really makes sense to release the beta tomorrow? Looking
> at the Misc/NEWS files for 2.6 and 3.0, there are around 3-5 entries
> for each release. I know it's good to follow the release plan, but it
> also may save you, the release manager, work for the third beta (which
> I think will be necessary if beta2 is released tomorrow).
>
> Georg
>

Speaking from my minor perspective - I've been sick and MIA, so there
has not been a lot of movement on the pep 371 issues / multiprocessing
bugs since Beta 1, there's still a fair amount of issues to close out.

-jesse

From barry at python.org  Tue Jul  1 22:40:35 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 1 Jul 2008 16:40:35 -0400
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
Message-ID: <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 1, 2008, at 4:25 PM, Guido van Rossum wrote:

> I think we should put this one off. The previous betas were done on  
> June 18, and IMO the next beta should be about a month afterwards,  
> not 2 weeks.

I will not be able to make releases the weeks of July 21st and 28th.   
The next scheduled beta is August 6th.

There are two options.  I could shift everything forward 2 weeks and  
do the next betas on July 16th.  Or we could wait until August 6th.   
That would mean 6 weeks between betas.  It's fine with me either way.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGqWQ3EjvBPtnXfVAQK5owP/Yd1pwWtwelbstnb6xh/dEtILirAyhfyo
kcfQSSFBX+GgkDIx99cxgmJ7nB+xSNSy1MlkXukDj41O2m+dCqcQaxhyim4yqBYC
r/Zc7IIiPT/nNQ/l97z8w0FqBoS/bmk9pqckBzrJfRRW14LZD8m2E/aU+OZeGi6z
0GZn/zwQbYk=
=yC2a
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Tue Jul  1 22:45:21 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 1 Jul 2008 15:45:21 -0500
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
Message-ID: <1afaf6160807011345n650a645fq661908f75a1f6a03@mail.gmail.com>

On Tue, Jul 1, 2008 at 3:40 PM, Barry Warsaw <barry at python.org> wrote:
>
> There are two options.  I could shift everything forward 2 weeks and do the
> next betas on July 16th.  Or we could wait until August 6th.  That would
> mean 6 weeks between betas.  It's fine with me either way.

I vote for shifting things 2 weeks forward.

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From python at rcn.com  Tue Jul  1 22:51:17 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 1 Jul 2008 13:51:17 -0700
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org><ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
Message-ID: <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>

From: "Barry Warsaw" <barry at python.org>
> There are two options.  I could shift everything forward 2 weeks and  
> do the next betas on July 16th.  Or we could wait until August 6th.   
> That would mean 6 weeks between betas.  It's fine with me either way.

+1 for six weeks to allow the code to be more thoroughly exercised.


Raymond

From guido at python.org  Tue Jul  1 22:54:54 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 1 Jul 2008 13:54:54 -0700
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
Message-ID: <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>

On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> wrote:

> From: "Barry Warsaw" <barry at python.org>
>
>> There are two options.  I could shift everything forward 2 weeks and  do
>> the next betas on July 16th.  Or we could wait until August 6th.   That
>> would mean 6 weeks between betas.  It's fine with me either way.
>>
>
> +1 for six weeks to allow the code to be more thoroughly exercised.
>

In that case I'd rather insert an extra beta -- one in 2 weeks and one in 6
weeks.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080701/641c2c52/attachment.htm>

From barry at python.org  Wed Jul  2 00:55:16 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 1 Jul 2008 18:55:16 -0400
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
Message-ID: <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote:

> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com>  
> wrote:
> From: "Barry Warsaw" <barry at python.org>
>
> There are two options.  I could shift everything forward 2 weeks  
> and  do the next betas on July 16th.  Or we could wait until August  
> 6th.   That would mean 6 weeks between betas.  It's fine with me  
> either way.
>
> +1 for six weeks to allow the code to be more thoroughly exercised.
>
> In that case I'd rather insert an extra beta -- one in 2 weeks and  
> one in 6 weeks.

Okay.  I can't actually do it on July 16th, so the revised schedule  
will be:

15-Jul-2008 beta 2
23-Aug-2008 beta 3
03-Sep-2008 rc1
17-Sep-2008 rc2
01-Oct-2008 final releases

I will update PEP 361 now.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGq11HEjvBPtnXfVAQJdYgP8DFVvCHeDzIDliY0bQuw+DXxMuGAxHWFO
BZR2b4sEGFzMRfbGCJOi7wVubc4imwYDIpFXgzFHpWFMfUdBHGaSpnZJhGDxURqp
0vQQ3/nJLy7lpWfDYBy0Sps6XjANQF5SaqeW8KMVsa3X6Spw0fHTmF4xBIjiUaBy
MvydyLNszY4=
=9/s1
-----END PGP SIGNATURE-----

From guido at python.org  Wed Jul  2 01:04:04 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 1 Jul 2008 16:04:04 -0700
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
Message-ID: <ca471dc20807011604q77c00e59kdf0d8b86312a03e6@mail.gmail.com>

On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote:
> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote:
>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> wrote:
>> From: "Barry Warsaw" <barry at python.org>
>>
>> There are two options.  I could shift everything forward 2 weeks and  do
>> the next betas on July 16th.  Or we could wait until August 6th.   That
>> would mean 6 weeks between betas.  It's fine with me either way.
>>
>> +1 for six weeks to allow the code to be more thoroughly exercised.
>>
>> In that case I'd rather insert an extra beta -- one in 2 weeks and one in
>> 6 weeks.
>
> Okay.  I can't actually do it on July 16th, so the revised schedule will be:
>
> 15-Jul-2008 beta 2
> 23-Aug-2008 beta 3
> 03-Sep-2008 rc1
> 17-Sep-2008 rc2
> 01-Oct-2008 final releases
>
> I will update PEP 361 now.

+1

Thanks for being flexible!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Wed Jul  2 01:13:57 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 1 Jul 2008 19:13:57 -0400
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <ca471dc20807011604q77c00e59kdf0d8b86312a03e6@mail.gmail.com>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
	<ca471dc20807011604q77c00e59kdf0d8b86312a03e6@mail.gmail.com>
Message-ID: <535961AE-BCFE-4563-96E0-B883D97A1188@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 1, 2008, at 7:04 PM, Guido van Rossum wrote:

> On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote:
>> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote:
>>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com>  
>>> wrote:
>>> From: "Barry Warsaw" <barry at python.org>
>>>
>>> There are two options.  I could shift everything forward 2 weeks  
>>> and  do
>>> the next betas on July 16th.  Or we could wait until August 6th.    
>>> That
>>> would mean 6 weeks between betas.  It's fine with me either way.
>>>
>>> +1 for six weeks to allow the code to be more thoroughly exercised.
>>>
>>> In that case I'd rather insert an extra beta -- one in 2 weeks and  
>>> one in
>>> 6 weeks.
>>
>> Okay.  I can't actually do it on July 16th, so the revised schedule  
>> will be:
>>
>> 15-Jul-2008 beta 2
>> 23-Aug-2008 beta 3
>> 03-Sep-2008 rc1
>> 17-Sep-2008 rc2
>> 01-Oct-2008 final releases
>>
>> I will update PEP 361 now.
>
> +1
>
> Thanks for being flexible!

Anything for a great release!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGq6NnEjvBPtnXfVAQKOlQP/RYlj6vxHEmlW/mVNIWqBYy/SmmMA6Qw4
hE3Bhb9QYGC5F0kEKyY5BmBVwETe70ahE1X3AOgmLrnHh5XwvGh8sNrFka/3s9sh
vt6XAZh9IoXekZBIOGO4Gz0EtcURVUvAbCzCSXkHCQyL3qoV1r+mxsXVLRV2S4q0
UifMzkOm6WI=
=wDrk
-----END PGP SIGNATURE-----

From brett at python.org  Wed Jul  2 01:27:03 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 1 Jul 2008 16:27:03 -0700
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
Message-ID: <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>

On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote:
>
>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> wrote:
>> From: "Barry Warsaw" <barry at python.org>
>>
>> There are two options.  I could shift everything forward 2 weeks and  do
>> the next betas on July 16th.  Or we could wait until August 6th.   That
>> would mean 6 weeks between betas.  It's fine with me either way.
>>
>> +1 for six weeks to allow the code to be more thoroughly exercised.
>>
>> In that case I'd rather insert an extra beta -- one in 2 weeks and one in
>> 6 weeks.
>
> Okay.  I can't actually do it on July 16th, so the revised schedule will be:
>
> 15-Jul-2008 beta 2
> 23-Aug-2008 beta 3
> 03-Sep-2008 rc1
> 17-Sep-2008 rc2
> 01-Oct-2008 final releases
>
> I will update PEP 361 now.

Is a Google Calendar kept by anyone that lists stuff like planned
release dates, etc.?

-Brett

From musiccomposition at gmail.com  Wed Jul  2 01:28:59 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 1 Jul 2008 18:28:59 -0500
Subject: [Python-Dev] [Python-3000]  Second betas tomorrow
In-Reply-To: <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
	<bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
Message-ID: <1afaf6160807011628m43738e85x3f03064a6df307ef@mail.gmail.com>

On Tue, Jul 1, 2008 at 6:27 PM, Brett Cannon <brett at python.org> wrote:
> Is a Google Calendar kept by anyone that lists stuff like planned
> release dates, etc.?

It's on my personal one. :)



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From barry at python.org  Wed Jul  2 03:44:16 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 1 Jul 2008 21:44:16 -0400
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
	<bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
Message-ID: <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote:

> On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote:
>>
>>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com>  
>>> wrote:
>>> From: "Barry Warsaw" <barry at python.org>
>>>
>>> There are two options.  I could shift everything forward 2 weeks  
>>> and  do
>>> the next betas on July 16th.  Or we could wait until August 6th.    
>>> That
>>> would mean 6 weeks between betas.  It's fine with me either way.
>>>
>>> +1 for six weeks to allow the code to be more thoroughly exercised.
>>>
>>> In that case I'd rather insert an extra beta -- one in 2 weeks and  
>>> one in
>>> 6 weeks.
>>
>> Okay.  I can't actually do it on July 16th, so the revised schedule  
>> will be:
>>
>> 15-Jul-2008 beta 2
>> 23-Aug-2008 beta 3
>> 03-Sep-2008 rc1
>> 17-Sep-2008 rc2
>> 01-Oct-2008 final releases
>>
>> I will update PEP 361 now.
>
> Is a Google Calendar kept by anyone that lists stuff like planned
> release dates, etc.?

http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGrdcHEjvBPtnXfVAQJYxgQAh/+j8pF21H0k1vp+1znOh57MohU7gVP6
7fMnLSzOoA+9w7+pVvJVzWbr09vg41kO6OzqEAoMUPV2BK8ZHePuHZkLDwhCAAYk
nixu2vRZZEGmT6aC0jejwOCY7vy5giTHelX442drKZcuSdNl4x1kvyohBnm0flIH
6B7HRL3Oo2Q=
=5yqD
-----END PGP SIGNATURE-----

From brett at python.org  Wed Jul  2 04:01:33 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 1 Jul 2008 19:01:33 -0700
Subject: [Python-Dev] [Python-3000] Second betas tomorrow
In-Reply-To: <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
	<bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
	<79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org>
Message-ID: <bbaeab100807011901s5fd992cdn87c548bfdaa0866d@mail.gmail.com>

On Tue, Jul 1, 2008 at 6:44 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote:
>
>> On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote:
>>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote:
>>>
>>>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com>
>>>> wrote:
>>>> From: "Barry Warsaw" <barry at python.org>
>>>>
>>>> There are two options.  I could shift everything forward 2 weeks and  do
>>>> the next betas on July 16th.  Or we could wait until August 6th.   That
>>>> would mean 6 weeks between betas.  It's fine with me either way.
>>>>
>>>> +1 for six weeks to allow the code to be more thoroughly exercised.
>>>>
>>>> In that case I'd rather insert an extra beta -- one in 2 weeks and one
>>>> in
>>>> 6 weeks.
>>>
>>> Okay.  I can't actually do it on July 16th, so the revised schedule will
>>> be:
>>>
>>> 15-Jul-2008 beta 2
>>> 23-Aug-2008 beta 3
>>> 03-Sep-2008 rc1
>>> 17-Sep-2008 rc2
>>> 01-Oct-2008 final releases
>>>
>>> I will update PEP 361 now.
>>
>> Is a Google Calendar kept by anyone that lists stuff like planned
>> release dates, etc.?
>
> http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics

Thanks, Barry!

-Brett

From brett at python.org  Wed Jul  2 04:04:44 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 1 Jul 2008 19:04:44 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports?
Message-ID: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>

I just committed r64651 which is my attempt to add support to
fix_imports so that modules that have been split up in 3.0 can be
properly fixed. 2to3's test suite passes and all, but I am not sure if
I botched it somehow since I did the change slightly blind. Can
someone just do a quick check to make sure I did it properly? Also,
what order should renames be declared to give priority to certain
renames (e.g., urllib should probably be renamed to urllib.requeste
over urllib.error when not used in a ``from ... import`` statement).

-Brett

From musiccomposition at gmail.com  Wed Jul  2 04:38:13 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 1 Jul 2008 21:38:13 -0500
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
Message-ID: <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>

On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote:
> I just committed r64651 which is my attempt to add support to
> fix_imports so that modules that have been split up in 3.0 can be
> properly fixed. 2to3's test suite passes and all, but I am not sure if
> I botched it somehow since I did the change slightly blind. Can
> someone just do a quick check to make sure I did it properly? Also,
> what order should renames be declared to give priority to certain
> renames (e.g., urllib should probably be renamed to urllib.requeste
> over urllib.error when not used in a ``from ... import`` statement).

Well for starters, you know the test for fix_imports is disabled, right?

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From musiccomposition at gmail.com  Wed Jul  2 04:42:33 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 1 Jul 2008 21:42:33 -0500
Subject: [Python-Dev] [Python-3000]  Second betas tomorrow
In-Reply-To: <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
	<bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
	<79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org>
Message-ID: <1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com>

On Tue, Jul 1, 2008 at 8:44 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote:
>>
>> Is a Google Calendar kept by anyone that lists stuff like planned
>> release dates, etc.?
>
> http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics

Can I get the non-iCal version?


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From barry at python.org  Wed Jul  2 05:29:10 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 1 Jul 2008 23:29:10 -0400
Subject: [Python-Dev] [Python-3000]  Second betas tomorrow
In-Reply-To: <1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
	<bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
	<79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org>
	<1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com>
Message-ID: <D8E48735-E192-486B-A3D3-CD50C5F99DBF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Jul 1, 2008, at 10:42 PM, Benjamin Peterson wrote:

> On Tue, Jul 1, 2008 at 8:44 PM, Barry Warsaw <barry at python.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote:
>>>
>>> Is a Google Calendar kept by anyone that lists stuff like planned
>>> release dates, etc.?
>>
>> http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics
>
> Can I get the non-iCal version?

http://www.google.com/calendar/feeds/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic

http://www.google.com/calendar/embed?src=b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com&ctz=America/New_York

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGr2BnEjvBPtnXfVAQJKBQP/bme7XNFS74SSmNNYX6Wz7Dq83VSQ8J6A
hZf6k7tTx6I3qv0Xgc2jD9NnNuLmqG+Rw8Ag5CjBtZXgzAoyszluzddJfz3G0032
zPofZx/ekp22u4XJo9iQyrDKinp+qTlDqlQntsscY5l+KXR5P9ahWeWWM9aQw707
VYkxQ2yAA7g=
=fzdc
-----END PGP SIGNATURE-----

From brett at python.org  Wed Jul  2 05:36:59 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 1 Jul 2008 20:36:59 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
Message-ID: <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>

On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote:
>> I just committed r64651 which is my attempt to add support to
>> fix_imports so that modules that have been split up in 3.0 can be
>> properly fixed. 2to3's test suite passes and all, but I am not sure if
>> I botched it somehow since I did the change slightly blind. Can
>> someone just do a quick check to make sure I did it properly? Also,
>> what order should renames be declared to give priority to certain
>> renames (e.g., urllib should probably be renamed to urllib.requeste
>> over urllib.error when not used in a ``from ... import`` statement).
>
> Well for starters, you know the test for fix_imports is disabled, right?
>

Nope, I forgot and turning it on has it failing running under 2.5.

-Brett

From ismail at namtrac.org  Wed Jul  2 07:25:19 2008
From: ismail at namtrac.org (=?UTF-8?Q?=C4=B0smail_D=C3=B6nmez?=)
Date: Wed, 2 Jul 2008 08:25:19 +0300
Subject: [Python-Dev] py3k branch still using -fno-strict-aliasing
Message-ID: <19e566510807012225v6da0e8b2jac05ef407caee1b4@mail.gmail.com>

Hi,

I remember discussing this before and coming to conclusion that
-fno-strict-aliasing would be removed from py3k CFLAGS. But as of now
its still used.

I tested with gcc 4.3.1 on Linux x86_64 and there is no strict
aliasing warning when this flag is removed. Also make testall passes.
Is there any reason to keep this flag? If not see the attached patch.

Regards,
ismail

-- 
Programmer Excuses number 45: I do object-oriented programming - if
the customer objects, I do more programming.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: strict-aliasing.patch
Type: text/x-diff
Size: 1064 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080702/13911566/attachment.patch>

From paddy3118 at googlemail.com  Wed Jul  2 08:08:22 2008
From: paddy3118 at googlemail.com (Paddy 3118)
Date: Wed, 2 Jul 2008 07:08:22 +0100
Subject: [Python-Dev] [issue3214] Suggest change to glossary explanation:
	"Duck Typing"
Message-ID: <3f7cdd360807012308y6eb6f018l6341cb0ace73e4e@mail.gmail.com>

Hi,
I'd like extra opinions on this issue please:
 http://bugs.python.org/issue3214

It's about changing the definition of Duck typing to remove hasattr and
leave just EAFP in the enablers - more detail is in the issue log.

Thanks, Paddy.

From brett at python.org  Wed Jul  2 08:32:54 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 1 Jul 2008 23:32:54 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>
Message-ID: <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com>

On Tue, Jul 1, 2008 at 8:36 PM, Brett Cannon <brett at python.org> wrote:
> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote:
>>> I just committed r64651 which is my attempt to add support to
>>> fix_imports so that modules that have been split up in 3.0 can be
>>> properly fixed. 2to3's test suite passes and all, but I am not sure if
>>> I botched it somehow since I did the change slightly blind. Can
>>> someone just do a quick check to make sure I did it properly? Also,
>>> what order should renames be declared to give priority to certain
>>> renames (e.g., urllib should probably be renamed to urllib.requeste
>>> over urllib.error when not used in a ``from ... import`` statement).
>>
>> Well for starters, you know the test for fix_imports is disabled, right?
>>
>
> Nope, I forgot and turning it on has it failing running under 2.5.
>

And refactor.py cannot be run directly from 2.5 because of a relative
import and in 2.6 (where runpy has extra smarts) it still doesn't work
thanks to main() not being passed an argument is needs (Issue3131).
Looks like 2to3 needs some TLC.

-Brett

From steve at holdenweb.com  Wed Jul  2 13:23:48 2008
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 02 Jul 2008 07:23:48 -0400
Subject: [Python-Dev] [issue3214] Suggest change to glossary
	explanation: "Duck Typing"
In-Reply-To: <3f7cdd360807012308y6eb6f018l6341cb0ace73e4e@mail.gmail.com>
References: <3f7cdd360807012308y6eb6f018l6341cb0ace73e4e@mail.gmail.com>
Message-ID: <g4fog7$7ba$1@ger.gmane.org>

Paddy 3118 wrote:
> Hi,
> I'd like extra opinions on this issue please:
>  http://bugs.python.org/issue3214
> 
> It's about changing the definition of Duck typing to remove hasattr and
> leave just EAFP in the enablers - more detail is in the issue log.
> 
The change seems to make sense. Use of hasattr() to determine method 
availability, while not strictly "look before you leap" because it 
doesn't test for a specific type, certainly isn't EAFP either.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From duncan.booth at suttoncourtenay.org.uk  Wed Jul  2 13:47:02 2008
From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth)
Date: Wed, 2 Jul 2008 11:47:02 +0000 (UTC)
Subject: [Python-Dev] repeated keyword arguments
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<48659116.10302@canterbury.ac.nz>
	<200806281158.41558.steve@pearwood.info>
Message-ID: <Xns9ACF5B23C38D1duncanrcpcouk@127.0.0.1>

"Steven D'Aprano" <steve at pearwood.info> wrote:

> It would be nice to be able to do this:
> 
> defaults = dict(a=5, b=7)
> f(**defaults, a=8)  # override the value of a in defaults
> 
> but unfortunately that gives a syntax error. Reversing the order would 
> override the wrong value. So as Python exists now, no, it's not 
> terribly useful. But it's not inherently a stupid idea.

There is already an easy way to do that using functools.partial, and it is 
documented and therefore presumably deliberate behaviour "If additional 
keyword arguments are supplied, they extend and override keywords."

>>> from functools import partial
>>> def f(a=1, b=2, c=3):
	print a, b, c

	
>>> g = partial(f, b=99)
>>> g()
1 99 3
>>> g(a=100, b=101)
100 101 3




From ncoghlan at gmail.com  Wed Jul  2 14:31:33 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 02 Jul 2008 22:31:33 +1000
Subject: [Python-Dev] Review needed: proposed fix for 2.6 __hash__
 incompatibility (issue 2235)
Message-ID: <486B7525.6070502@gmail.com>

I've posted a possible fix for the __hash__ backwards incompatibilities 
described in issue 2235 [1].

The patch uses a model similar to that used in Py3k (using None is 
indicate "don't inherit __hash__"), but extends it to allowing Py_None 
to be explicitly stored in the tp_hash slot. The major downside is that 
we suffer the cost of an extra pointer comparison on every call to 
PyObject_Hash, but I wasn't able to come up with another solution that 
preserved backwards compatibility while still allowing 
collections.Hashable to function correctly.

The patch involves a few changes to fairly deep components in 
typeobject.c though, so I'd like at least some kind of sanity check 
before I commit it.

Cheers,
Nick.

[1] http://bugs.python.org/issue2235
-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From asmodai at in-nomine.org  Wed Jul  2 16:13:28 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Wed, 2 Jul 2008 16:13:28 +0200
Subject: [Python-Dev] UCS2/UCS4 default
Message-ID: <20080702141328.GW62693@nexus.in-nomine.org>

Guido (and others of course),

back in 2001 you pointed out that you wanted to move to UCS4 completely as
the ideal situation
(http://mail.python.org/pipermail/i18n-sig/2001-June/001107.html) over the
current default UCS2.

Given 3.0 will use Unicode strings as the default, would it also not make
sense to make the switch at this point as well?

The current situation with UCS2 is particularly bad now that the CJK
ideographs Extension B. has been produced (and C is under ballot and D is
under development).

Personally I use nothing else but UCS4 compiled Python binaries for the past
years.

See also http://www.python.org/dev/peps/pep-0261/ for background for the
2001 options.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Expansion of happiness is the purpose of life...

From guido at python.org  Wed Jul  2 16:13:59 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Jul 2008 07:13:59 -0700
Subject: [Python-Dev] Review needed: proposed fix for 2.6 __hash__
	incompatibility (issue 2235)
In-Reply-To: <486B7525.6070502@gmail.com>
References: <486B7525.6070502@gmail.com>
Message-ID: <ca471dc20807020713y75da0bfsec3550ecd0dfac25@mail.gmail.com>

On Wed, Jul 2, 2008 at 5:31 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I've posted a possible fix for the __hash__ backwards incompatibilities
> described in issue 2235 [1].
>
> The patch uses a model similar to that used in Py3k (using None is indicate
> "don't inherit __hash__"), but extends it to allowing Py_None to be
> explicitly stored in the tp_hash slot. The major downside is that we suffer
> the cost of an extra pointer comparison on every call to PyObject_Hash, but
> I wasn't able to come up with another solution that preserved backwards
> compatibility while still allowing collections.Hashable to function
> correctly.

>From your description it seems storing Py_None in the slot acts as a
magic value meaning "this is defined but not usable". However it used
to be pretty common for various code around to call various slots
directly (after a NULL) check. That would have disastrous results if
the slot value was Py_None. Would it be terribly inconvenient if the
magic value was in fact another function, with a public name), whose
sole purpose was to raise an exception?

> The patch involves a few changes to fairly deep components in typeobject.c
> though, so I'd like at least some kind of sanity check before I commit it.
>
> Cheers,
> Nick.
>
> [1] http://bugs.python.org/issue2235

I can't promise I'll have time to look at this before my EuroPython
keynote, but it's important for me to get it right, so if nobody else
jumps in, remind me Tuesday.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Wed Jul  2 16:36:18 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 03 Jul 2008 00:36:18 +1000
Subject: [Python-Dev] Review needed: proposed fix for 2.6 __hash__
 incompatibility (issue 2235)
In-Reply-To: <ca471dc20807020713y75da0bfsec3550ecd0dfac25@mail.gmail.com>
References: <486B7525.6070502@gmail.com>
	<ca471dc20807020713y75da0bfsec3550ecd0dfac25@mail.gmail.com>
Message-ID: <486B9262.80107@gmail.com>

Guido van Rossum wrote:
> On Wed, Jul 2, 2008 at 5:31 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I've posted a possible fix for the __hash__ backwards incompatibilities
>> described in issue 2235 [1].
>>
>> The patch uses a model similar to that used in Py3k (using None is indicate
>> "don't inherit __hash__"), but extends it to allowing Py_None to be
>> explicitly stored in the tp_hash slot. The major downside is that we suffer
>> the cost of an extra pointer comparison on every call to PyObject_Hash, but
>> I wasn't able to come up with another solution that preserved backwards
>> compatibility while still allowing collections.Hashable to function
>> correctly.
> 
>>From your description it seems storing Py_None in the slot acts as a
> magic value meaning "this is defined but not usable". However it used
> to be pretty common for various code around to call various slots
> directly (after a NULL) check. That would have disastrous results if
> the slot value was Py_None. Would it be terribly inconvenient if the
> magic value was in fact another function, with a public name), whose
> sole purpose was to raise an exception?

Not only not inconvenient, but a significant improvement - as well as 
addressing your concern that I missed some code that calls tp_hash 
directly (a concern that I share, particularly since it could be an 
extension module we don't control that ends up doing it), it also gets 
rid of that extra pointer comparison in PyObject_Hash that was bothering me.

>> The patch involves a few changes to fairly deep components in typeobject.c
>> though, so I'd like at least some kind of sanity check before I commit it.
>>
>> Cheers,
>> Nick.
>>
>> [1] http://bugs.python.org/issue2235
> 
> I can't promise I'll have time to look at this before my EuroPython
> keynote, but it's important for me to get it right, so if nobody else
> jumps in, remind me Tuesday.

I'd now advise waiting until I have a chance to implement your idea 
anyway :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From collinw at gmail.com  Wed Jul  2 18:30:14 2008
From: collinw at gmail.com (Collin Winter)
Date: Wed, 2 Jul 2008 09:30:14 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
Message-ID: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>

On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote:
>> I just committed r64651 which is my attempt to add support to
>> fix_imports so that modules that have been split up in 3.0 can be
>> properly fixed. 2to3's test suite passes and all, but I am not sure if
>> I botched it somehow since I did the change slightly blind. Can
>> someone just do a quick check to make sure I did it properly? Also,
>> what order should renames be declared to give priority to certain
>> renames (e.g., urllib should probably be renamed to urllib.requeste
>> over urllib.error when not used in a ``from ... import`` statement).
>
> Well for starters, you know the test for fix_imports is disabled, right?

Why was this test disabled, rather than fixed? That seems a rather
poor solution to the problem of it taking longer than desired to run.

Collin

From musiccomposition at gmail.com  Wed Jul  2 18:34:00 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 2 Jul 2008 11:34:00 -0500
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>
Message-ID: <1afaf6160807020934h37f6e989i772da74b68ced414@mail.gmail.com>

On Wed, Jul 2, 2008 at 11:30 AM, Collin Winter <collinw at gmail.com> wrote:
> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson
>> Well for starters, you know the test for fix_imports is disabled, right?
>
> Why was this test disabled, rather than fixed? That seems a rather
> poor solution to the problem of it taking longer than desired to run.

I believe Martin was the one who disabled it.

>
> Collin
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From collinw at gmail.com  Wed Jul  2 18:36:30 2008
From: collinw at gmail.com (Collin Winter)
Date: Wed, 2 Jul 2008 09:36:30 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>
	<bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com>
Message-ID: <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com>

On Tue, Jul 1, 2008 at 11:32 PM, Brett Cannon <brett at python.org> wrote:
> On Tue, Jul 1, 2008 at 8:36 PM, Brett Cannon <brett at python.org> wrote:
>> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson
>> <musiccomposition at gmail.com> wrote:
>>> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote:
>>>> I just committed r64651 which is my attempt to add support to
>>>> fix_imports so that modules that have been split up in 3.0 can be
>>>> properly fixed. 2to3's test suite passes and all, but I am not sure if
>>>> I botched it somehow since I did the change slightly blind. Can
>>>> someone just do a quick check to make sure I did it properly? Also,
>>>> what order should renames be declared to give priority to certain
>>>> renames (e.g., urllib should probably be renamed to urllib.requeste
>>>> over urllib.error when not used in a ``from ... import`` statement).
>>>
>>> Well for starters, you know the test for fix_imports is disabled, right?
>>>
>>
>> Nope, I forgot and turning it on has it failing running under 2.5.
>>
>
> And refactor.py cannot be run directly from 2.5 because of a relative
> import and in 2.6 (where runpy has extra smarts) it still doesn't work
> thanks to main() not being passed an argument is needs (Issue3131).

Why are you trying to run refactor.py directly, rather than using 2to3
(http://svn.python.org/view/sandbox/trunk/2to3/2to3) as an entry
point?

> Looks like 2to3 needs some TLC.

Agreed. A lot of the pending bugs seem to be related to the version of
lib2to3 in the stdlib, rather than the stand-alone product. Neal
Norwitz and I have been working to turn parts of 2to3 into a more
general refactoring library; once that's done (or even preferably
before), lib2to3 should be removed from the stdlib. It's causing far
more trouble than it's worth.

Collin

From guido at python.org  Wed Jul  2 19:08:01 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Jul 2008 10:08:01 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080702141328.GW62693@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
Message-ID: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>

I think we should continue to leave this up to the distribution. AFAIK
many Linux distros already use UCS4 for everything anyway.

The alternative (no matter what the configure flag is called) is
UTF-16, not UCS-2 though: there is support for surrogate pairs in
various places, including the \U escape and the UTF-8 codec.

I don't want to rule out UTF-16 as internal representation from the
Python language spec, because JVM- and .NET-based implementations
pretty much have no choice in the matter if they want to be compatible
with the native string type (which is very important for performance
and compatibility with other languages on those platforms).

For that reason I think it's also better that the configure script
continues to default to UTF-16 -- this will give the UTF-16 support
code the necessary exercise. (It is mostly a superset of the UCS-4
support code, so I'm less worried about the latter getting enough
exercise.)

--Guido

On Wed, Jul 2, 2008 at 7:13 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
>
> Guido (and others of course),
>
> back in 2001 you pointed out that you wanted to move to UCS4 completely as
> the ideal situation
> (http://mail.python.org/pipermail/i18n-sig/2001-June/001107.html) over the
> current default UCS2.
>
> Given 3.0 will use Unicode strings as the default, would it also not make
> sense to make the switch at this point as well?
>
> The current situation with UCS2 is particularly bad now that the CJK
> ideographs Extension B. has been produced (and C is under ballot and D is
> under development).
>
> Personally I use nothing else but UCS4 compiled Python binaries for the past
> years.
>
> See also http://www.python.org/dev/peps/pep-0261/ for background for the
> 2001 options.
>
> --
> Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
> ????? ?????? ??? ?? ??????
> http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
> Expansion of happiness is the purpose of life...
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org



--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jul  2 19:13:45 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Jul 2008 10:13:45 -0700
Subject: [Python-Dev] Who wants to work with Klocwork?
Message-ID: <ca471dc20807021013u36df2645k644f5e4ac40fe519@mail.gmail.com>

I've got an offer from Klocwork (a static source code analysis
company, www.klocwork.com) to give some developers free access to
their findings from running their bug-finding software over Python
source code. I don't have the bandwidth to deal with this myself, but
I think it would be valuable if we could get some folks to look at
their findings.

We have a similar relationship with one of Klocwork's competitors. In
my experience, each vendor's tool has a different strength, and it is
likely that each will find some important bugs that the other didn't
flag. So IMO it's useful to do this with each vendor that offers...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From asmodai at in-nomine.org  Wed Jul  2 19:19:57 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Wed, 2 Jul 2008 19:19:57 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
Message-ID: <20080702171957.GY62693@nexus.in-nomine.org>

-On [20080702 19:08], Guido van Rossum (guido at python.org) wrote:
>I think we should continue to leave this up to the distribution. AFAIK
>many Linux distros already use UCS4 for everything anyway.

FreeBSD's ports makes it a configure option.

>For that reason I think it's also better that the configure script
>continues to default to UTF-16 -- this will give the UTF-16 support
>code the necessary exercise. (It is mostly a superset of the UCS-4
>support code, so I'm less worried about the latter getting enough
>exercise.)

I was under the impression that it was still UCS2 and thus limiting things
to the BMP only. So you are saying it's UTF-16 nowadays? For both 2.6 and
3.0?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Nature does nothing uselessly...

From guido at python.org  Wed Jul  2 19:42:13 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Jul 2008 10:42:13 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080702171957.GY62693@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
Message-ID: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>

On Wed, Jul 2, 2008 at 10:19 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> -On [20080702 19:08], Guido van Rossum (guido at python.org) wrote:
>>I think we should continue to leave this up to the distribution. AFAIK
>>many Linux distros already use UCS4 for everything anyway.
>
> FreeBSD's ports makes it a configure option.
>
>>For that reason I think it's also better that the configure script
>>continues to default to UTF-16 -- this will give the UTF-16 support
>>code the necessary exercise. (It is mostly a superset of the UCS-4
>>support code, so I'm less worried about the latter getting enough
>>exercise.)
>
> I was under the impression that it was still UCS2 and thus limiting things
> to the BMP only. So you are saying it's UTF-16 nowadays? For both 2.6 and
> 3.0?

Yes. At least in the sense that \Uxxxxxxxx gets translated to a
surrogate pair, and that the UTF-8 codec supports surrogate pairs in
both directions. It's been like this for a long time. What else would
you expect from UTF-16 support?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jul  2 20:17:16 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Jul 2008 11:17:16 -0700
Subject: [Python-Dev] Who wants to work with Klocwork?
In-Reply-To: <ca471dc20807021013u36df2645k644f5e4ac40fe519@mail.gmail.com>
References: <ca471dc20807021013u36df2645k644f5e4ac40fe519@mail.gmail.com>
Message-ID: <ca471dc20807021117k150f43d1r6fcabbbdc4405a10@mail.gmail.com>

Followup: Neal Norwitz <nnorwitz at gmail.com> will coordinate this, send
mail to him if you're interested, not to me. :-)

On Wed, Jul 2, 2008 at 10:13 AM, Guido van Rossum <guido at python.org> wrote:
> I've got an offer from Klocwork (a static source code analysis
> company, www.klocwork.com) to give some developers free access to
> their findings from running their bug-finding software over Python
> source code. I don't have the bandwidth to deal with this myself, but
> I think it would be valuable if we could get some folks to look at
> their findings.
>
> We have a similar relationship with one of Klocwork's competitors. In
> my experience, each vendor's tool has a different strength, and it is
> likely that each will find some important bugs that the other didn't
> flag. So IMO it's useful to do this with each vendor that offers...
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From asmodai at in-nomine.org  Wed Jul  2 20:22:15 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Wed, 2 Jul 2008 20:22:15 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
Message-ID: <20080702182215.GA62693@nexus.in-nomine.org>

-On [20080702 19:42], Guido van Rossum (guido at python.org) wrote:
>Yes. At least in the sense that \Uxxxxxxxx gets translated to a
>surrogate pair, and that the UTF-8 codec supports surrogate pairs in
>both directions. It's been like this for a long time. What else would
>you expect from UTF-16 support?

Well, unless I misunderstand things, a Python 3 compiled with the default
Unicode option gives this:

>>> len("\N{MUSICAL SYMBOL G CLEF}")
2

Whereas a Python 3 with --with-wide-unicode gives:


>>> len("\N{MUSICAL SYMBOL G CLEF}")
1

This, of course, causes problems with splitting, finding, and so on. So that
means that a Python 3 with only 2 byte Unicode support is not to be
used/recommended for Unicode outside of the BMP.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Tomorrow's battle is won during today's practice...

From guido at python.org  Wed Jul  2 20:27:42 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Jul 2008 11:27:42 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080702182215.GA62693@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
Message-ID: <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>

On Wed, Jul 2, 2008 at 11:22 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> -On [20080702 19:42], Guido van Rossum (guido at python.org) wrote:
>>Yes. At least in the sense that \Uxxxxxxxx gets translated to a
>>surrogate pair, and that the UTF-8 codec supports surrogate pairs in
>>both directions. It's been like this for a long time. What else would
>>you expect from UTF-16 support?
>
> Well, unless I misunderstand things, a Python 3 compiled with the default
> Unicode option gives this:
>
>>>> len("\N{MUSICAL SYMBOL G CLEF}")
> 2
>
> Whereas a Python 3 with --with-wide-unicode gives:
>
>
>>>> len("\N{MUSICAL SYMBOL G CLEF}")
> 1
>
> This, of course, causes problems with splitting, finding, and so on.

Understood.

> So that
> means that a Python 3 with only 2 byte Unicode support is not to be
> used/recommended for Unicode outside of the BMP.

I disagree. Instead, I would say that such code needs to be aware of surrogates.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From asmodai at in-nomine.org  Wed Jul  2 20:35:41 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Wed, 2 Jul 2008 20:35:41 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
Message-ID: <20080702183541.GB62693@nexus.in-nomine.org>

-On [20080702 20:27], Guido van Rossum (guido at python.org) wrote:
>I disagree. Instead, I would say that such code needs to be aware of
>surrogates.

Just to make sure I understood you:

Python's code needs to be made aware of surrogates?

If so, do you want me to log issues for the things encountered?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Learn from the past -- don't wear it like a yoke around your neck...

From guido at python.org  Wed Jul  2 20:47:02 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Jul 2008 11:47:02 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080702183541.GB62693@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
Message-ID: <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>

On Wed, Jul 2, 2008 at 11:35 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> -On [20080702 20:27], Guido van Rossum (guido at python.org) wrote:
>>I disagree. Instead, I would say that such code needs to be aware of
>>surrogates.
>
> Just to make sure I understood you:
>
> Python's code needs to be made aware of surrogates?

No, Python already is aware of surrogates. I meant applications
processing non-BMP text should beware of them.

> If so, do you want me to log issues for the things encountered?

If you find places where the Python core or standard library is doing
Unicode processing that would break when surrogates are present you
should file a bug. However this does not mean that every bit of code
that slices a string at an arbitrary point (and hence risks slicing in
the middle of a surrogate) is incorrect -- it all depends on what is
done next with the slice.

I'd also prefer to receive bug reports about breakages actually
encountered in the wild than purely theoretical issues. And in all
cases a fragment of test code to reproduce the problem would be
appreciated.

> --
> Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
> ????? ?????? ??? ?? ??????
> http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
> Learn from the past -- don't wear it like a yoke around your neck...
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Wed Jul  2 21:02:46 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 2 Jul 2008 12:02:46 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>
Message-ID: <bbaeab100807021202n1743d899ge7425ae92f8e555c@mail.gmail.com>

On Wed, Jul 2, 2008 at 9:30 AM, Collin Winter <collinw at gmail.com> wrote:
> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote:
>>> I just committed r64651 which is my attempt to add support to
>>> fix_imports so that modules that have been split up in 3.0 can be
>>> properly fixed. 2to3's test suite passes and all, but I am not sure if
>>> I botched it somehow since I did the change slightly blind. Can
>>> someone just do a quick check to make sure I did it properly? Also,
>>> what order should renames be declared to give priority to certain
>>> renames (e.g., urllib should probably be renamed to urllib.requeste
>>> over urllib.error when not used in a ``from ... import`` statement).
>>
>> Well for starters, you know the test for fix_imports is disabled, right?
>
> Why was this test disabled, rather than fixed? That seems a rather
> poor solution to the problem of it taking longer than desired to run.

I think it may have been to turn off a failing test just before a
release and it was just never switched back on.

-Brett

From brett at python.org  Wed Jul  2 21:06:01 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 2 Jul 2008 12:06:01 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>
	<bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com>
	<43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com>
Message-ID: <bbaeab100807021206l6b47141fq5a214c73948db361@mail.gmail.com>

On Wed, Jul 2, 2008 at 9:36 AM, Collin Winter <collinw at gmail.com> wrote:
> On Tue, Jul 1, 2008 at 11:32 PM, Brett Cannon <brett at python.org> wrote:
>> On Tue, Jul 1, 2008 at 8:36 PM, Brett Cannon <brett at python.org> wrote:
>>> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson
>>> <musiccomposition at gmail.com> wrote:
>>>> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote:
>>>>> I just committed r64651 which is my attempt to add support to
>>>>> fix_imports so that modules that have been split up in 3.0 can be
>>>>> properly fixed. 2to3's test suite passes and all, but I am not sure if
>>>>> I botched it somehow since I did the change slightly blind. Can
>>>>> someone just do a quick check to make sure I did it properly? Also,
>>>>> what order should renames be declared to give priority to certain
>>>>> renames (e.g., urllib should probably be renamed to urllib.requeste
>>>>> over urllib.error when not used in a ``from ... import`` statement).
>>>>
>>>> Well for starters, you know the test for fix_imports is disabled, right?
>>>>
>>>
>>> Nope, I forgot and turning it on has it failing running under 2.5.
>>>
>>
>> And refactor.py cannot be run directly from 2.5 because of a relative
>> import and in 2.6 (where runpy has extra smarts) it still doesn't work
>> thanks to main() not being passed an argument is needs (Issue3131).
>
> Why are you trying to run refactor.py directly, rather than using 2to3
> (http://svn.python.org/view/sandbox/trunk/2to3/2to3) as an entry
> point?
>

Because I honestly did not see it yesterday in my terminal. I blame it
on Canada Day. =)

>> Looks like 2to3 needs some TLC.
>
> Agreed. A lot of the pending bugs seem to be related to the version of
> lib2to3 in the stdlib, rather than the stand-alone product. Neal
> Norwitz and I have been working to turn parts of 2to3 into a more
> general refactoring library; once that's done (or even preferably
> before), lib2to3 should be removed from the stdlib. It's causing far
> more trouble than it's worth.

Fine by me.

-Brett

From martin at v.loewis.de  Wed Jul  2 21:51:49 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Jul 2008 21:51:49 +0200
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>	
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>
Message-ID: <486BDC55.6070506@v.loewis.de>

> Why was this test disabled, rather than fixed? That seems a rather
> poor solution to the problem of it taking longer than desired to run.

I disabled it because I didn't know how to fix it, and created bug
reports 2968 and 2969 in return. It is policy that tests that break
get disabled, rather than keeping them broken.

Regards,
Martin

From martin at v.loewis.de  Wed Jul  2 22:09:57 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Jul 2008 22:09:57 +0200
Subject: [Python-Dev] Can someone check my lib2to3 change
	for	fix_imports?
In-Reply-To: <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>	<bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>	<bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com>
	<43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com>
Message-ID: <486BE095.2030801@v.loewis.de>

> Agreed. A lot of the pending bugs seem to be related to the version of
> lib2to3 in the stdlib, rather than the stand-alone product. Neal
> Norwitz and I have been working to turn parts of 2to3 into a more
> general refactoring library; once that's done (or even preferably
> before), lib2to3 should be removed from the stdlib. It's causing far
> more trouble than it's worth.

I disagree. I think it is quite useful that distutils is able to
invoke it, and other people also asked for that feature on PyCon.

Why do you think the trouble wouldn't be caused if it wasn't
a standard library package?

Regards,
Martin

From collinw at gmail.com  Wed Jul  2 22:39:58 2008
From: collinw at gmail.com (Collin Winter)
Date: Wed, 2 Jul 2008 13:39:58 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <486BDC55.6070506@v.loewis.de>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com>
	<486BDC55.6070506@v.loewis.de>
Message-ID: <43aa6ff70807021339j56d81522hbbfcb754fa027150@mail.gmail.com>

On Wed, Jul 2, 2008 at 12:51 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Why was this test disabled, rather than fixed? That seems a rather
>> poor solution to the problem of it taking longer than desired to run.
>
> I disabled it because I didn't know how to fix it, and created bug
> reports 2968 and 2969 in return.

So you did. I didn't notice them, sorry.

> It is policy that tests that break
> get disabled, rather than keeping them broken.

From collinw at gmail.com  Wed Jul  2 22:40:42 2008
From: collinw at gmail.com (Collin Winter)
Date: Wed, 2 Jul 2008 13:40:42 -0700
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <486BE095.2030801@v.loewis.de>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>
	<bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>
	<bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com>
	<43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com>
	<486BE095.2030801@v.loewis.de>
Message-ID: <43aa6ff70807021340w76804278rd0b90c31a8175faa@mail.gmail.com>

On Wed, Jul 2, 2008 at 1:09 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Agreed. A lot of the pending bugs seem to be related to the version of
>> lib2to3 in the stdlib, rather than the stand-alone product. Neal
>> Norwitz and I have been working to turn parts of 2to3 into a more
>> general refactoring library; once that's done (or even preferably
>> before), lib2to3 should be removed from the stdlib. It's causing far
>> more trouble than it's worth.
>
> I disagree. I think it is quite useful that distutils is able to
> invoke it, and other people also asked for that feature on PyCon.

But distutils currently *doesn't* invoke it, AFAICT (unless that
support is implemented somewhere other than trunk/Lib/distutils/), and
no-one has stepped up to make that happen in the months since PyCon.
Moreover, as I told those people who asked for this at PyCon, 2to3 is
and will never be perfect, meaning that at best, distutils/2to3
integration would look like "python setup.py run2to3", where distutils
is just a long-hand way of running 2to3 over your code.

This strikes me as a waste of time.

> Why do you think the trouble wouldn't be caused if it wasn't
> a standard library package?

Problems with the current setup:
1) People are currently confused as to where they should be commit fixes.
2) Changes to the sandbox version have to be manually merged into the
stdlib version, which is more overhead than I think it's worth. In
addition, the stdlib version lags the sandbox version.
3) At least one bug report (issue3131) has mentioned problems with the
stdlib 2to3 exhibiting problems that the stand-alone version does not.
This is again extra overhead.
4) The test_imports test was commented out because of stdlib test
policy. I'd rather not have that policy imposed on 2to3.

Collin

From martin at v.loewis.de  Thu Jul  3 00:36:59 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 03 Jul 2008 00:36:59 +0200
Subject: [Python-Dev] Can someone check my lib2to3 change for
	fix_imports?
In-Reply-To: <43aa6ff70807021340w76804278rd0b90c31a8175faa@mail.gmail.com>
References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com>	
	<1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com>	
	<bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com>	
	<bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com>	
	<43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com>	
	<486BE095.2030801@v.loewis.de>
	<43aa6ff70807021340w76804278rd0b90c31a8175faa@mail.gmail.com>
Message-ID: <486C030B.6060405@v.loewis.de>

> But distutils currently *doesn't* invoke it, AFAICT

Sure. In 3k, look at Lib/distutils/command/build.py:build_py_2to3.

That's how I ported Django to Py3k.

> 1) People are currently confused as to where they should be commit fixes.

Sure, but it only happens rarely.

> 2) Changes to the sandbox version have to be manually merged into the
> stdlib version, which is more overhead than I think it's worth. In
> addition, the stdlib version lags the sandbox version.

It's not a real problem, IMO, using msgmerge is fairly straight-forward.

> 3) At least one bug report (issue3131) has mentioned problems with the
> stdlib 2to3 exhibiting problems that the stand-alone version does not.
> This is again extra overhead.

I think the 2to3 packaging issue is otherwise unresolved. Do you want
2to3 to be excluded completely from 2.6 and 3.1 releases? If not,
how do you want them packaged? Will it work if packaged in that way?

> 4) The test_imports test was commented out because of stdlib test
> policy. I'd rather not have that policy imposed on 2to3.

It would be possible to comment out the test only in the copy in the
stdlib version, or to omit testing 2to3 in the stdlib altogether,
if that helps.

Regards,
Martin


From asmodai at in-nomine.org  Thu Jul  3 12:48:13 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 12:48:13 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
Message-ID: <20080703104813.GF62693@nexus.in-nomine.org>

My apologies for hammering on this, but I think it is quite important and
currently Python 3.0 seems confused about UCS-2 versus UTF-16.

-On [20080702 20:47], Guido van Rossum (guido at python.org) wrote:
>No, Python already is aware of surrogates. I meant applications
>processing non-BMP text should beware of them.

Just to make sure people are fully aware of the distinctions:

UCS-2 uses 16 bits to encode Unicode data, does NOT support surrogate pairs
and therefore CANNOT represent data beyond U+FFFF (thus only supporting the
Basic Multilingual Plane, BMP). It is a fixed-length character encoding.

UTF-16 also uses 16 bits to encode Unicode data, but DOES support surrogate
pairs and therefore CAN represent data beyond U+FFFF by using said surrogate
pairs (thus supporting all planes). It is a variable-length character
encoding.

So a string representation in UCS-2 means every character occupies 16 bits.
A string representation in UTF-16 means characters can occupy 16 bits or
32-bits.

If one stays within the BMP than all is well, but when you move beyond the
BMP (U+10000 - U+10FFFF) then Python needs to correctly check the string
for surrogate pairs and deal with them internally.

>If you find places where the Python core or standard library is doing
>Unicode processing that would break when surrogates are present you
>should file a bug. However this does not mean that every bit of code
>that slices a string at an arbitrary point (and hence risks slicing in
>the middle of a surrogate) is incorrect -- it all depends on what is
>done next with the slice.

Basically everything but string forming or string printing seems to be
broken for surrogate pairs, from what I can tell.
Also, I think you are confused about slicing in the middle of a surrogate
pair, from a UTF-16 perspective this is 1 codepoint! And as such Python
needs to treat it as one character/codepoint in a string, dealing with
slicing as appropriate. The way you currently describe it is that UTF-16
strings will be treated as UCS-2 when it comes to slicing and the likes.
>From a UTF-16 point of view such slicing can NEVER occur unless you are bit
or byte slicing instead of character/codepoint slicing.

The documentation for len() says:
Return the length (the number of items) of an object.

I think it can be fairly said that an item in a string is a character or
codepoint. Take for example the following string:

a = '\U00020045\u942a' # Two hanzi/kanji/hanja

>From a Unicode perspective we are looking at two characters/codepoints.
When we use a 4-byte Python 3.0 binary we get (as expected):
>>> len(a)
2

When we use a 2-byte Python 3.0 binary (the default) we get (not as
expected):
>>> len(a)
3

>From a UTF-16 perspective a surrogate pair is one character/codepoint and
as such len() should have reported 2 as well. That the sequence is stored
internally as 0xd840 0xdc45 0x942a and occupies 3 bytes is not interesting.
But it seems as if len() is treating the string as being in UCS-2
(fixed-length), which is the only logical explanation for the number 3,
instead of treating it as UTF-16 (variable-length) and reporting the number
2.

Subsequently doing a: print a[1] to get the 0x942a (?) actually requires
a[2] on the 2-byte Python 3.0. As such the code you write for 2-byte and
4-byte Python 3.0 is *different* when you have to deal with the same Unicode
strings! This cannot be the desired situation, can it?

Two more examples:

>>> a.find('?') # 4-byte
1
>>> a.find('?') # 2-byte
2

>>> import re # 4-byte
>>> m = re.search('?', a)
>>> m.start()
1
>>> import re # 2-byte
>>> m = re.search('?', a)
>>> m.start()
2

This, in my opinion, has nothing to do with the application writers, but
more with Python's internals being confused about UCS-2 and UTF-16. We
accept full 32-bit codepoints with the \U escape in strings, and we may even
store it as UTF-16 internally, but we clearly do not deal with it properly
as UTF-16, but rather as UCS-2, when it comes to using said strings with
core functions and modules.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
For wouldst thou not carve at my Soul with thine sword of Supreme Truth?

From solipsis at pitrou.net  Thu Jul  3 13:58:17 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 3 Jul 2008 11:58:17 +0000 (UTC)
Subject: [Python-Dev] UCS2/UCS4 default
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
Message-ID: <loom.20080703T115255-50@post.gmane.org>


Hi,

> Subsequently doing a: print a[1] to get the 0x942a (?) actually requires
> a[2] on the 2-byte Python 3.0.

How is it annoying *in practice*? In actual code the index, instead of
being a constant, will be retrieved through various means such as .find()
or re.search().start()... as you show yourself later in your message.

What is primordial is that Python shows a consistent behaviour, and it
does, since indices returned by .find() et al. have the same meaning as
indices you can use with the [] operator. AFAIK that's why Guido asked
for real-world rather than theoretical examples.

Regards

Antoine.



From ncoghlan at gmail.com  Thu Jul  3 14:39:29 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 03 Jul 2008 22:39:29 +1000
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703104813.GF62693@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702171957.GY62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
Message-ID: <486CC881.5090902@gmail.com>

Jeroen Ruigrok van der Werven wrote:
> The documentation for len() says:
> Return the length (the number of items) of an object.

So what this tells us is that in a UCS-2 build of Python, the "items" in 
a unicode string are not, strictly speaking, Unicode code points or 
characters. Instead, they are successive 16-bit fragments of a UTF-16 
encoded string (which correspond to characters only if there are no 
surrogate pairs present in the string).

Let's look at the options here:

1. System is NOT memory limited (i.e. most desktops): use a UCS-4 Python 
build, which is what most Linux distributions do (I'm not sure about the 
pydotorg provided Windows or Mac OS X builds).

2. System is memory limited, only BMP Unicode code points are used: use 
a UCS-2 Python build, limit yourself to characters on the BMP (possibly 
enforced by use of an appropriate codec to decode input text).

3. System is memory limited, but needs to support characters beyond the 
BMP: use a UCS-2 Python build, handling any codepoints outside the BMP 
in application code.

The current Python approach handles all three cases relatively 
gracefully and with minimal overhead. Dealing natively with surrogate 
pair issues could easily result in pointless complexity for cases 1 and 
2, while completely disallowing codepoints beyond the BMP in a UCS-2 
build would needlessly rule out option 3.

So here's the challenge:

1. If you are advocating disallowing the use of characters outside the 
BMP in a UCS-2 build, enumerate the advantages of doing so (paying 
particular attention to any advantages which cannot be obtained simply 
by using an appropriate codec that disallows non-BMP characters).

2. If you are advocating making the "items" in a Unicode string code 
points even in a UCS-2 build, enumerate all of the string behaviours 
that would have to change, as well as indicating how to avoid causing a 
reduction in speed for cases 1 and 2 above.

Sure, option 2 might be nice to have, but the purity argument isn't 
going to be anywhere near enough motivation to justify the additional 
code complexity - there need to be practical benefits that aren't better 
met just by sacrificing a bit of memory efficiency and switching to a 
UCS-4 build.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mal at egenix.com  Thu Jul  3 15:00:22 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Jul 2008 15:00:22 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486CC881.5090902@gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702171957.GY62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
Message-ID: <486CCD66.70906@egenix.com>

I think the discussion is going in the wrong direction:

The choice between UCS2 and UCS4 builds is really only meant
to enhance the possibility to interface to native OS or
application APIs, e.g. Windows LIBC and Java use UTF-16, glibc
on Unix uses UCS4.

The problem of slicing Unicode objects is far more complicated
than just breaking a surrogate pair. Unicode if full of combining
code points - if you break such a sequence, the output will be
just as wrong; regardless of UCS2 vs. UCS4.

A long time ago we had a discussion about these problems. I had
suggested a new module (unicodeindex IIRC) which takes care of indexing
Unicode strings based on code points (which support for surrogates),
glyphs (taking combining code points into account) and words (with
support for various breaking/non-breaking separation code points).

Trying to solve such issues at the storage level is the wrong
approach, since the problem is application specific and thus requires
a higher-level set of possible solutions.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 03 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania             3 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From djarb at highenergymagic.org  Thu Jul  3 15:14:47 2008
From: djarb at highenergymagic.org (Daniel Arbuckle)
Date: Thu, 3 Jul 2008 06:14:47 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486CC881.5090902@gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
Message-ID: <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>

On Thu, Jul 3, 2008 at 5:39 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 1. If you are advocating disallowing the use of characters outside the BMP
> in a UCS-2 build, enumerate the advantages of doing so (paying particular
> attention to any advantages which cannot be obtained simply by using an
> appropriate codec that disallows non-BMP characters).

Right now, the same python code has different meaning, depending on a
compile-time option that most users didn't even set for themselves.
Moreover, the errors caused by this semantic difference are not
reported. There's just no way to justify that.

You can't solve this problem by saying 'programmers should choose a
codec that limits them to the BMP when they target 2-byte python,'
because the problem specifically arises when code that works correctly
in a 4-byte python is placed into a 2-byte python, an operation
performed by the users rather than by programmers.

Since 2-byte python is apparently a holdover for memory-limited (and
presumably CPU-limited as well) systems, it doesn't make sense to
impose on it the requirement of correctly dealing with surrogate
pairs. Given that, it seems to me that the best solution would be to
make 4-byte python the default, and also to make 2-byte python raise
an exception when it encounters characters outside the BMP. This way,
a mysterious and unreported semantic error becomes an explicit
syntactic error.

For programmers who want to target a 2-byte format (for win32
compatibility, for example), the correct choice of codec is a superior
solution to forcing a 2-byte internal representation on python.

From asmodai at in-nomine.org  Thu Jul  3 15:21:46 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 15:21:46 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486CCD66.70906@egenix.com>
References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com>
Message-ID: <20080703132146.GI62693@nexus.in-nomine.org>

-On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote:
>Unicode if full of combining code points - if you break such a sequence,
>the output will be just as wrong; regardless of UCS2 vs. UCS4.

In my opinion you are confusing two related, but very separated things here.
Combining characters have nothing to do with breaking up the encoding of a
single codepoint. Sure enough, if you arbitrary slice up codepoints that
consist of combining characters then your result is indeed odd looking.

I never said that nor is that the point I am making.

Guido points out that Python supports surrogate pairs and says that if
Python is dealing wrongly with this in the core than it needs to be fixed.
I am pointing out that given the fact we allow surrogate pairs we deal
rather simplistic with it in the core. In fact, we do not consider them at
all. In essence: though we may accept full 21-bit codepoints in the form of
\U00000000 escape sequences and store them internally as UTF-16 (which I
still need to verify) we subsequently deal with them programmatically as
UCS-2, which is plain silly.

You either commit yourself fully to UTF-16 and surrogate pairs or not. Not
some form in-between, because that will ultimately lead to more confusion
due to the difference in results when dealing with Unicode.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Believe in Angels...

From mhammond at skippinet.com.au  Thu Jul  3 15:42:58 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 3 Jul 2008 23:42:58 +1000
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702171957.GY62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>
	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
Message-ID: <031901c8dd12$b7258b60$2570a220$@com.au>

> For programmers who want to target a 2-byte format (for win32
> compatibility, for example)

As MAL said, this is taking the discussion in the wrong direction.

For people on Windows, win32 isn't a "compatibility" consideration.  I
suspect most users of the other platforms MAL mentioned and all others with
their own native unicode implementations would agree.

Cheers,

Mark


From mal at egenix.com  Thu Jul  3 15:57:41 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Jul 2008 15:57:41 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703132146.GI62693@nexus.in-nomine.org>
References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702171957.GY62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>
	<486CCD66.70906@egenix.com>
	<20080703132146.GI62693@nexus.in-nomine.org>
Message-ID: <486CDAD5.1060506@egenix.com>

On 2008-07-03 15:21, Jeroen Ruigrok van der Werven wrote:
> -On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote:
>> Unicode if full of combining code points - if you break such a sequence,
>> the output will be just as wrong; regardless of UCS2 vs. UCS4.
> 
> In my opinion you are confusing two related, but very separated things here.
> Combining characters have nothing to do with breaking up the encoding of a
> single codepoint. Sure enough, if you arbitrary slice up codepoints that
> consist of combining characters then your result is indeed odd looking.
> 
> I never said that nor is that the point I am making.

Please remember that lone surrogate pair code points are perfectly
valid Unicode code points, nevertheless. Just as a lone combining
code point is valid on its own.

> Guido points out that Python supports surrogate pairs and says that if
> Python is dealing wrongly with this in the core than it needs to be fixed.
> I am pointing out that given the fact we allow surrogate pairs we deal
> rather simplistic with it in the core. In fact, we do not consider them at
> all. In essence: though we may accept full 21-bit codepoints in the form of
> \U00000000 escape sequences and store them internally as UTF-16 (which I
> still need to verify) we subsequently deal with them programmatically as
> UCS-2, which is plain silly.

Python applies conversion from non-BMP code points to surroagtes
for UCS builds in a few places and I agree that we should probably
do that at a few more places.

However, these are mainly conversion issues of encoded Unicode
representations vs. the internal Unicode storage where you want
to avoid exceptions in favor of finding a solution that preserves
data.

To make it clear: UCS2 builds of Python do not support non-BMP
code points out of the box.

A programmer will always have to use a codec to map the internal storage
on these builds to the full Unicode code point range. The following
codecs support surrogates on UCS2 builds:

  * UTF-8
  * UTF-16
  * UTF-32
  * unicode-escape
  * raw-unicode-escape

> You either commit yourself fully to UTF-16 and surrogate pairs or not. Not
> some form in-between, because that will ultimately lead to more confusion
> due to the difference in results when dealing with Unicode.

Programmers will have to be aware of the fact that on UCS2
builds of Python non-BMP code points will have to be treated
differently than on UCS4 builds.

I don't see that as a problem. It is in a way similar to
32-bit vs. 64-bit builds of Python or the fact that floating point
numbers work differently depending on the Python platform or
compiler being used.

BTW: Have you ever run into any problems with UCS2 vs. UCS4
in practice that were not easy to solve ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 03 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania             3 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From guido at python.org  Thu Jul  3 15:58:26 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Jul 2008 06:58:26 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703104813.GF62693@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
Message-ID: <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>

On Thu, Jul 3, 2008 at 3:48 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> My apologies for hammering on this, but I think it is quite important and
> currently Python 3.0 seems confused about UCS-2 versus UTF-16.
[...]

Your seem to be suggesting that len(u"\U00012345") should return 1 on
a system that internally uses UTF-16 and hence represents this string
as a surrogate pair.

This is not going to happen. You may as well complain to the authors
of the Java standard about the corresponding problem there.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From djarb at highenergymagic.org  Thu Jul  3 16:38:47 2008
From: djarb at highenergymagic.org (Daniel Arbuckle)
Date: Thu, 3 Jul 2008 07:38:47 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <031901c8dd12$b7258b60$2570a220$@com.au>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
	<031901c8dd12$b7258b60$2570a220$@com.au>
Message-ID: <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>

On Thu, Jul 3, 2008 at 6:42 AM, Mark Hammond <mhammond at skippinet.com.au> wrote:
> For people on Windows, win32 isn't a "compatibility" consideration.  I
> suspect most users of the other platforms MAL mentioned and all others with
> their own native unicode implementations would agree.

I'm sorry, but you're wrong. Interfacing python to interoperate with
the underlying system is compatibility. Surely your own win32
extensions already address this necessity.

Regardless, as I said before, nothing justifies silently changing the
meaning of a program based on an option that most users don't set for
themselves and are not aware of. When such a change would take place,
it should be reported explicitly as an error.

From asmodai at in-nomine.org  Thu Jul  3 16:46:48 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 16:46:48 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
Message-ID: <20080703144648.GA34192@nexus.in-nomine.org>

-On [20080703 15:58], Guido van Rossum (guido at python.org) wrote:
>Your seem to be suggesting that len(u"\U00012345") should return 1 on
>a system that internally uses UTF-16 and hence represents this string
>as a surrogate pair.

>From a Unicode and UTF-16 point of view that makes the most sense. So yes, I
am suggesting that.

>This is not going to happen. You may as well complain to the authors
>of the Java standard about the corresponding problem there.

Why would I need to complain to them? They already fixed it since 1.5.0.

Java 1.5.0's release notes
(http://java.sun.com/developer/technicalArticles/releases/j2se15/):

Supplementary Character Support

32-bit supplementary character support has been carefully added to the
platform as part of the transition to Unicode 4.0 support. Supplementary
characters are encoded as a special pair of UTF16 values to generate a
different character, or codepoint. A surrogate pair is a combination of a
high UTF16 value and a following low UTF16 value. The high and low values
are from a special range of UTF16 values.

In general, when using a String or sequence of characters, the core API
libraries will transparently handle the new supplementary characters for
you.

See also http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html

The methods that accept an int value support all Unicode characters,
including supplementary characters. For example, Character.isLetter(0x2F81A)
returns true because the code point value represents a letter (a CJK
ideograph).

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Life can only be understood backwards, but it must be lived forwards...

From guido at python.org  Thu Jul  3 17:03:55 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Jul 2008 08:03:55 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703144648.GA34192@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
Message-ID: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>

On Thu, Jul 3, 2008 at 7:46 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> -On [20080703 15:58], Guido van Rossum (guido at python.org) wrote:
>>Your seem to be suggesting that len(u"\U00012345") should return 1 on
>>a system that internally uses UTF-16 and hence represents this string
>>as a surrogate pair.
>
> From a Unicode and UTF-16 point of view that makes the most sense. So yes, I
> am suggesting that.
>
>>This is not going to happen. You may as well complain to the authors
>>of the Java standard about the corresponding problem there.
>
> Why would I need to complain to them? They already fixed it since 1.5.0.
>
> Java 1.5.0's release notes
> (http://java.sun.com/developer/technicalArticles/releases/j2se15/):
>
> Supplementary Character Support
>
> 32-bit supplementary character support has been carefully added to the
> platform as part of the transition to Unicode 4.0 support. Supplementary
> characters are encoded as a special pair of UTF16 values to generate a
> different character, or codepoint. A surrogate pair is a combination of a
> high UTF16 value and a following low UTF16 value. The high and low values
> are from a special range of UTF16 values.
>
> In general, when using a String or sequence of characters, the core API
> libraries will transparently handle the new supplementary characters for
> you.
>
> See also http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html
>
> The methods that accept an int value support all Unicode characters,
> including supplementary characters. For example, Character.isLetter(0x2F81A)
> returns true because the code point value represents a letter (a CJK
> ideograph).

I don't see an answer there to the question of whether the length()
method of a Java String object containing a single surrogate pair
returns 1 or 2; I suspect it returns 2. Python 3 supports things like
chr(0x12345) and ord("\U00012345"). (And so does Python 2, using
unichr and unicode literals.)

The one thing that may be missing from Python is things like
interpretation of surrogates by functions like isalpha() and I'm okay
with adding that (since those have to loop over the entire string
anyway).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amauryfa at gmail.com  Thu Jul  3 17:31:57 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 3 Jul 2008 17:31:57 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
Message-ID: <e27efe130807030831q57b907e4h69fe63e4cf3e51ab@mail.gmail.com>

Hello,

2008/7/3 Guido van Rossum <guido at python.org>:
> I don't see an answer there to the question of whether the length()
> method of a Java String object containing a single surrogate pair
> returns 1 or 2; I suspect it returns 2. Python 3 supports things like
> chr(0x12345) and ord("\U00012345"). (And so does Python 2, using
> unichr and unicode literals.)

python2.6 support for supplementary characters is not ideal:
>>> unichr(0x2f81a)
ValueError: unichr() arg not in range(0x10000) (narrow Python build)
>>> ord(u'\U0002F81A')
TypeError: ord() expected a character, but string of length 2 found.

\Uxxxxxxxx seems the only way to enter these characters.
3.0 is much better and passes the two tests above.

The unicodedata module gives good results in both versions:
>>> unicodedata.name(u'\U0002F81A')
'CJK COMPATIBILITY IDEOGRAPH-2F81A'
[34311 refs]
>>> unicodedata.category(u'\U0002F81A')
'Lo'

With python 3.0, I found only two places that refuse large code points
on narrow builds:
the "%c" format, and Py_BuildValue('C'). They should be fixed.

> The one thing that may be missing from Python is things like
> interpretation of surrogates by functions like isalpha() and I'm okay
> with adding that (since those have to loop over the entire string
> anyway).

In this case, a new .isascii() method would be needed for some uses.

-- 
Amaury Forgeot d'Arc

From p.f.moore at gmail.com  Thu Jul  3 17:32:37 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 3 Jul 2008 16:32:37 +0100
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
Message-ID: <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>

On 03/07/2008, Guido van Rossum <guido at python.org> wrote:
> I don't see an answer there to the question of whether the length()
> method of a Java String object containing a single surrogate pair
> returns 1 or 2; I suspect it returns 2.

It appears you're right:

>type testucs.java
class testucs {
    public static void main(String[] args) {
        StringBuilder s = new StringBuilder("Hello, ");
        s.appendCodePoint(0x2F81A);
        System.out.println(s); // Display the string.
        System.out.println(s.length());
    }
}

>java testucs
Hello, ?
9

>java -version
java version "1.6.0_05"
Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing)

> Python 3 supports things like
> chr(0x12345) and ord("\U00012345"). (And so does Python 2, using
> unichr and unicode literals.)

And Java doesn't appear to - that appendCodePoint() method was
wonderfully hard to find :-)

Paul.

From armin.ronacher at active-4.com  Thu Jul  3 18:30:09 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Thu, 3 Jul 2008 16:30:09 +0000 (UTC)
Subject: [Python-Dev] UCS2/UCS4 default
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
Message-ID: <loom.20080703T162738-186@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:

> The one thing that may be missing from Python is things like
> interpretation of surrogates by functions like isalpha() and I'm okay
> with adding that (since those have to loop over the entire string
> anyway).
That and methods to safely iterate and slice strings by codepoint.  Java
supports that via String.codePointCount / String.codePointAt /
String.codePointBefore / String.offsetByCodepoints.  Maybe not on the
unicode/str object itself but as part of unicodedata that would make sense
for applications that have to deal with unicode on that level.

Regards,
Armin


From steve at holdenweb.com  Thu Jul  3 18:35:29 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 03 Jul 2008 12:35:29 -0400
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>	<20080703144648.GA34192@nexus.in-nomine.org>	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
	<79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>
Message-ID: <g4iv4n$3oq$1@ger.gmane.org>

Paul Moore wrote:
> On 03/07/2008, Guido van Rossum <guido at python.org> wrote:
>> I don't see an answer there to the question of whether the length()
>> method of a Java String object containing a single surrogate pair
>> returns 1 or 2; I suspect it returns 2.
> 
> It appears you're right:
> 
>> type testucs.java
> class testucs {
>     public static void main(String[] args) {
>         StringBuilder s = new StringBuilder("Hello, ");
>         s.appendCodePoint(0x2F81A);
>         System.out.println(s); // Display the string.
>         System.out.println(s.length());
>     }
> }
> 
>> java testucs
> Hello, ?
> 9
> 
>> java -version
> java version "1.6.0_05"
> Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
> Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing)
> 
>> Python 3 supports things like
>> chr(0x12345) and ord("\U00012345"). (And so does Python 2, using
>> unichr and unicode literals.)
> 
> And Java doesn't appear to - that appendCodePoint() method was
> wonderfully hard to find :-)
> 
There's also the issue of indexing the Unicode strings. If we are going 
to insist that len(u) counts surrogate pairs as one character then 
random access to the characters of a string is going to be an extremely 
inefficient operation.

Surely it's desirable under all circumstances that

   len(u) == sum(1 for c in u)

and that

   [c for c in u] == [c[i] for i in range(*len(u))]

How would that play under Jeroen's proposed change?

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From asmodai at in-nomine.org  Thu Jul  3 18:41:32 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 18:41:32 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>
References: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
	<79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>
Message-ID: <20080703164132.GB34192@nexus.in-nomine.org>

-On [20080703 17:32], Paul Moore (p.f.moore at gmail.com) wrote:
>        System.out.println(s.length());

I think you want to use codePointCount() to count the Unicode code points.
length() returns Unicode code units.

As http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html explains:

In the J2SE API documentation, Unicode code point is used for character
values in the range between U+0000 and U+10FFFF, and Unicode code unit is
used for 16-bit char values that are code units of the UTF-16 encoding.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Man is the measure of all things...

From guido at python.org  Thu Jul  3 18:48:38 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Jul 2008 09:48:38 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <g4iv4n$3oq$1@ger.gmane.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
	<79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>
	<g4iv4n$3oq$1@ger.gmane.org>
Message-ID: <ca471dc20807030948x418a1607yf928d2e92fa5e54d@mail.gmail.com>

On Thu, Jul 3, 2008 at 9:35 AM, Steve Holden <steve at holdenweb.com> wrote:
> Paul Moore wrote:
>>
>> On 03/07/2008, Guido van Rossum <guido at python.org> wrote:
>>>
>>> I don't see an answer there to the question of whether the length()
>>> method of a Java String object containing a single surrogate pair
>>> returns 1 or 2; I suspect it returns 2.
>>
>> It appears you're right:
>>
>>> type testucs.java
>>
>> class testucs {
>>    public static void main(String[] args) {
>>        StringBuilder s = new StringBuilder("Hello, ");
>>        s.appendCodePoint(0x2F81A);
>>        System.out.println(s); // Display the string.
>>        System.out.println(s.length());
>>    }
>> }
>>
>>> java testucs
>>
>> Hello, ?
>> 9
>>
>>> java -version
>>
>> java version "1.6.0_05"
>> Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
>> Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing)
>>
>>> Python 3 supports things like
>>> chr(0x12345) and ord("\U00012345"). (And so does Python 2, using
>>> unichr and unicode literals.)
>>
>> And Java doesn't appear to - that appendCodePoint() method was
>> wonderfully hard to find :-)
>>
> There's also the issue of indexing the Unicode strings. If we are going to
> insist that len(u) counts surrogate pairs as one character then random
> access to the characters of a string is going to be an extremely inefficient
> operation.

But my whole point is that len(u) should count surrogate pairs as TWO!

> Surely it's desirable under all circumstances that
>
>  len(u) == sum(1 for c in u)
>
> and that
>
>  [c for c in u] == [c[i] for i in range(*len(u))]
>
> How would that play under Jeroen's proposed change?

I am not considering such a change. At best there will be some helper
function in unicodedata, or perhaps a helper method on the 3.0 str
type to iterate over characters instead of 16-bit values. Whether that
iterator should yield 21-bit integer values or strings containing one
character (i.e. perhaps a surrogate pair) and what it would do with
lone surrogate halves is up to the committee to design this API.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From asmodai at in-nomine.org  Thu Jul  3 18:51:40 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 18:51:40 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
References: <20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
Message-ID: <20080703165140.GD34192@nexus.in-nomine.org>

-On [20080703 17:03], Guido van Rossum (guido at python.org) wrote:
>I don't see an answer there to the question of whether the length()
>method of a Java String object containing a single surrogate pair
>returns 1 or 2; I suspect it returns 2.

As
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/CharSequence.html#length()
states:

int length()

Returns the length of this character sequence. The length is the number of
16-bit chars in the sequence. 

But since Java switched to full UTF-16 support in 1.5.0 they extended their
API since the existing methods have probably come too ingrained.

E.g. codePointCount()
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html#codePointCount(char[],%20int,%20int)

>The one thing that may be missing from Python is things like
>interpretation of surrogates by functions like isalpha() and I'm okay
>with adding that (since those have to loop over the entire string
>anyway).

Those would be welcome already, yes. I'll see if I can help out.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Fallen into ever-mourn, with these wings so torn, after your day my dawn...

From asmodai at in-nomine.org  Thu Jul  3 19:01:30 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 19:01:30 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net>
References: <20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net>
Message-ID: <20080703170130.GE34192@nexus.in-nomine.org>

-On [20080703 18:45], James Y Knight (foom at fuhm.net) wrote:
>I think this is misguided.

Only trying to at least correct the current situation, which I consider a
bit of a mess, personally. (Although it seems others share my view.)

>I'd like to have 3 levels of access available:
>1) "byte"-level. In a new implementation I'd probably choose to make  
>all my strings stored in UTF-8, but UTF-16 is fine too.
>2) codepoint-level.
>3) grapheme-level.

Sounds interesting as well and I can very much see the advantages of such
levels and their methods. Especially in the i18n/l10n work I do.

>You should be able to iterate over the string at any of the levels,  
>ask for the nearest codepoint/grapheme boundary to the left or right  
>of an index at a different level, etc.

[snip]

Actually it seems Java already has a lot of similar methods.

>There are a few more desirable operations, to manipulate strings at  
>the grapheme level (because unlike for UTF-8/UTF-16 codepoints,  
>graphemes don't have the nice property of not containing prefixes  
>which are themselves valid graphemes). So, you want a find (and  
>everything else that implicitly does a find operation, like split,  
>replace, strip, etc) which requires that both endpoints of its match  
>are on a grapheme-boundary. [[Probably the easiest way to implement  
>this would be in the regexp engine.]]

Well, your ideas and seeing Java's stuff actually got me excited to work on
these kind of ideas, next to my datetime revamp.

What would the chances for inclusion in Python be if such a PEP + code would
be presented Guido?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Beware of the fury of the patient man...

From foom at fuhm.net  Thu Jul  3 18:45:39 2008
From: foom at fuhm.net (James Y Knight)
Date: Thu, 3 Jul 2008 12:45:39 -0400
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703144648.GA34192@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
Message-ID: <F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net>

On Jul 3, 2008, at 10:46 AM, Jeroen Ruigrok van der Werven wrote:

> -On [20080703 15:58], Guido van Rossum (guido at python.org) wrote:
>> Your seem to be suggesting that len(u"\U00012345") should return 1 on
>> a system that internally uses UTF-16 and hence represents this string
>> as a surrogate pair.
>
> From a Unicode and UTF-16 point of view that makes the most sense.  
> So yes, I
> am suggesting that.


I think this is misguided.

IMO, basically every programming language gets string handling wrong.  
(maybe with the exception of the unreleased perl6? it had some  
interesting moves in this area, but I haven't really been paying  
attention.) Everyone treats strings as arrays, but they are used quite  
differently. For a string, there is hardly ever a time when a  
programmer needs to index it with an arbitrary offset in number of  
codepoints, and the length-in-codepoints is pretty non-useful as well.  
Constant-time access to arbitrary codepoints in a string is pretty  
much unimportant. What *is* of utmost importantance is constant-time  
access to previously-returned points in the string.

I'd like to have 3 levels of access available:
1) "byte"-level. In a new implementation I'd probably choose to make  
all my strings stored in UTF-8, but UTF-16 is fine too.
2) codepoint-level.
3) grapheme-level.

You should be able to iterate over the string at any of the levels,  
ask for the nearest codepoint/grapheme boundary to the left or right  
of an index at a different level, etc.

Python could probably still be made to work kinda like this. I think a  
language designed as such in the first place could be nicer, with  
opaque index objects into the string rather than integers, and such,  
but...whatever.

Let's assume python is changed to always store strings in UTF-16.

All it would take is adding a few more functions to the str object to  
operate on the higher levels. Wherever I say "pos" I mean an integer  
index into the string, at the UTF-16 level. That may sometimes be  
unaligned with the boundary of the representation you're asking about,  
and behavior in that case needs to be specified as well.

.nextcodepoint(curpos, how_many=1) -> returns an index into the string  
how_many codepoints to the right (or left if negative) of the index  
curpos.

.nextgrapheme(curpos, how_many=1) -> returns an index into the string  
how_many graphemes to the right (or left if negative) of the index  
curpos.

.codepoints(from_pos=0, to_pos=None) -> return an iterator of  
codepoints from 'from_pos' to 'to_pos'. I think codepoints could be  
represented as strings themselves (so usually one character, sometimes  
two character strings).

.graphemes(from_pos=0, to_pos=None) -> return an iterator of graphemes  
from 'from_pos' to 'to_pos'. Also could be represented by strings. The  
returned graphemes should probably be normalized.

There are a few more desirable operations, to manipulate strings at  
the grapheme level (because unlike for UTF-8/UTF-16 codepoints,  
graphemes don't have the nice property of not containing prefixes  
which are themselves valid graphemes). So, you want a find (and  
everything else that implicitly does a find operation, like split,  
replace, strip, etc) which requires that both endpoints of its match  
are on a grapheme-boundary. [[Probably the easiest way to implement  
this would be in the regexp engine.]]


A concrete example of that: u'A\N{COMBINING TILDE}\N{COMBINING MACRON  
BELOW}'.find(u'A\N{COMBINING TILDE}') returns 0. But you want a way to  
ask for only a *actual* "A with tilde", not an "A with tilde and  
macron".



Anyhow, I'm not going to tackle this issue or try to push it further,  
but if someone does tackle it, python could grow to have the best  
unicode available. :)

James


From guido at python.org  Thu Jul  3 19:10:07 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Jul 2008 10:10:07 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703170130.GE34192@nexus.in-nomine.org>
References: <20080702171957.GY62693@nexus.in-nomine.org>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net>
	<20080703170130.GE34192@nexus.in-nomine.org>
Message-ID: <ca471dc20807031010w4af82a4cn60a275a13638deb9@mail.gmail.com>

On Thu, Jul 3, 2008 at 10:01 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> What would the chances for inclusion in Python be if such a PEP + code would
> be presented Guido?

As long as it is clear that the len() function and the basic slicing
and indexing operations on strings continue to work in code units
(i.e. 16-bit quantities) and the APIs for dealing with code points
(i.e. treating surrogate pairs as a single character) are a separate
API, there is a chance. Existing code using the existing APIs should
not change its behavior (even if you consider the existing behavior
broken), with the exception of isalpha() and similar APIs, which can
IMO safely be extended to consider surrogate pairs.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From facundobatista at gmail.com  Thu Jul  3 19:12:41 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 3 Jul 2008 14:12:41 -0300
Subject: [Python-Dev] us.pycon.org down?
Message-ID: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com>

(sorry for the crossposting)

Do you know what happened with "http://us.pycon.org/"?

Thank you!

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From rhamph at gmail.com  Thu Jul  3 19:21:24 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 3 Jul 2008 11:21:24 -0600
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486CDAD5.1060506@egenix.com>
References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com>
	<20080703132146.GI62693@nexus.in-nomine.org>
	<486CDAD5.1060506@egenix.com>
Message-ID: <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com>

On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 2008-07-03 15:21, Jeroen Ruigrok van der Werven wrote:
>>
>> -On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote:
>>>
>>> Unicode if full of combining code points - if you break such a sequence,
>>> the output will be just as wrong; regardless of UCS2 vs. UCS4.
>>
>> In my opinion you are confusing two related, but very separated things
>> here.
>> Combining characters have nothing to do with breaking up the encoding of a
>> single codepoint. Sure enough, if you arbitrary slice up codepoints that
>> consist of combining characters then your result is indeed odd looking.
>>
>> I never said that nor is that the point I am making.
>
> Please remember that lone surrogate pair code points are perfectly
> valid Unicode code points, nevertheless. Just as a lone combining
> code point is valid on its own.

That is a big part of these problems.  For all practical purposes, a
surrogate is like a UTF-8 code unit, and must be handled the same way,
so why the heck do they confuse everybody by saying "oh, it's a code
point too!"?


-- 
Adam Olsen, aka Rhamphoryncus

From martin at v.loewis.de  Thu Jul  3 19:31:14 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 03 Jul 2008 19:31:14 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703104813.GF62693@nexus.in-nomine.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702171957.GY62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
Message-ID: <486D0CE2.6010909@v.loewis.de>

> Basically everything but string forming or string printing seems to be
> broken for surrogate pairs, from what I can tell.

We probably disagree what "it works correctly" means. I think everything
works correctly.

> Also, I think you are confused about slicing in the middle of a surrogate
> pair, from a UTF-16 perspective this is 1 codepoint!

Yes, but it is two code units. Python's UTF-16 implementation operates
on code units, not code points.

> And as such Python
> needs to treat it as one character/codepoint in a string, dealing with
> slicing as appropriate.

It does. However, functions such as len, and all indexing, operate in
code units, not code points.

> The way you currently describe it is that UTF-16
> strings will be treated as UCS-2 when it comes to slicing and the likes.

No. In UCS-2, the surrogate range is reserved (for UTF-16). In Python,
it's not reserved, but interpreted as UTF-16.

> From a UTF-16 point of view such slicing can NEVER occur unless you are bit
> or byte slicing instead of character/codepoint slicing.

It most certainly can. UTF-16 is not a character set, but a character
encoding form (unlike UCS-2, which is a coded character set). Slicing
*can* occur at the code unit level. UTF-16 is also understood as a
character encoding scheme (by means of the BOM), then slicing can
occur even on the byte level.

> I think it can be fairly said that an item in a string is a character or
> codepoint.

Not in Python - it's a code unit.

Regards,
Martin

From goodger at python.org  Thu Jul  3 19:32:41 2008
From: goodger at python.org (David Goodger)
Date: Thu, 3 Jul 2008 13:32:41 -0400
Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down?
In-Reply-To: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com>
References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com>
Message-ID: <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com>

On Thu, Jul 3, 2008 at 13:12, Facundo Batista <facundobatista at gmail.com> wrote:
> (sorry for the crossposting)
>
> Do you know what happened with "http://us.pycon.org/"?

Not sure. The machine is still up (it serves www.pycon.org as well).
Either something is misconfigured, or a process can't start, or
something...

I'll ask Jeff Rush (whose machine it's on) and Doug Napoleone (who
knows more about the server than I, and has admin access) to look into
it.

-- 
David Goodger <http://python.net/~goodger>

From martin at v.loewis.de  Thu Jul  3 19:33:23 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 03 Jul 2008 19:33:23 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486CC881.5090902@gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702171957.GY62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
Message-ID: <486D0D63.3090807@v.loewis.de>

> 1. System is NOT memory limited (i.e. most desktops): use a UCS-4 Python
> build, which is what most Linux distributions do (I'm not sure about the
> pydotorg provided Windows or Mac OS X builds).

The Windows builds must continue to use a two-byte representation, as
otherwise PythonWin will break (and anything else that tries to pass
Unicode strings directly to a Win32 *W function).

Regards,
Martin

From asmodai at in-nomine.org  Thu Jul  3 19:35:45 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 19:35:45 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com>
References: <20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com>
	<20080703132146.GI62693@nexus.in-nomine.org>
	<486CDAD5.1060506@egenix.com>
	<aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com>
Message-ID: <20080703173545.GF34192@nexus.in-nomine.org>

-On [20080703 19:21], Adam Olsen (rhamph at gmail.com) wrote:
>On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Please remember that lone surrogate pair code points are perfectly
>> valid Unicode code points, nevertheless. Just as a lone combining
>> code point is valid on its own.
>
>That is a big part of these problems.  For all practical purposes, a
>surrogate is like a UTF-8 code unit, and must be handled the same way,
>so why the heck do they confuse everybody by saying "oh, it's a code
>point too!"?

Because surrogate code points are not Unicode scalar values, isolated UTF-16
code units in the range 0xd800-0xdfff are ill-formed. (D91 from Unicode
5.0/5.1, section 3.9)

So, no, it is not a code point too.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Als men blijft geloven kan de zwaarste steen niet zinken...

From martin at v.loewis.de  Thu Jul  3 19:36:03 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 03 Jul 2008 19:36:03 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486CDAD5.1060506@egenix.com>
References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702171957.GY62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<486CCD66.70906@egenix.com>	<20080703132146.GI62693@nexus.in-nomine.org>
	<486CDAD5.1060506@egenix.com>
Message-ID: <486D0E03.8020007@v.loewis.de>

> Please remember that lone surrogate pair code points are perfectly
> valid Unicode code points, nevertheless. Just as a lone combining
> code point is valid on its own.

Actually, I think they aren't (not any more than an invalid codepoint,
or an unassigned codepoint). They are reserved for UTF-16 only.

I would have to lookup the exact Unicode terminology, but "valid"
is probably not a predicate that they would use.

Regards,
Martin


From martin at v.loewis.de  Thu Jul  3 19:39:00 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 03 Jul 2008 19:39:00 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703164132.GB34192@nexus.in-nomine.org>
References: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>	<20080703144648.GA34192@nexus.in-nomine.org>	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>	<79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>
	<20080703164132.GB34192@nexus.in-nomine.org>
Message-ID: <486D0EB4.10901@v.loewis.de>

> I think you want to use codePointCount() to count the Unicode code points.
> length() returns Unicode code units.
> 
> As http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html explains:
> 
> In the J2SE API documentation, Unicode code point is used for character
> values in the range between U+0000 and U+10FFFF, and Unicode code unit is
> used for 16-bit char values that are code units of the UTF-16 encoding.

So you would like to contribute a function codePointCount to Python's
standard library? Go ahead.

Regards,
Martin


From janssen at parc.com  Thu Jul  3 19:43:58 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 3 Jul 2008 10:43:58 PDT
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <g4iv4n$3oq$1@ger.gmane.org> 
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com>
	<20080703144648.GA34192@nexus.in-nomine.org>
	<ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com>
	<79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com>
	<g4iv4n$3oq$1@ger.gmane.org>
Message-ID: <08Jul3.104407pdt."58698"@synergy1.parc.xerox.com>

> Surely it's desirable under all circumstances that
> 
>    len(u) == sum(1 for c in u)
> 
> and that
> 
>    [c for c in u] == [c[i] for i in range(*len(u))]
> 
> How would that play under Jeroen's proposed change?

Yes, but I think the argument is about what "c" is -- a character or a
codepoint.  Your point about efficiency is well-taken; I doubt that
random access to a particular character in a string has to be
efficient -- kind of a dying technique these days -- but slices and
regexp performance need efficiency guarantees.

Bill

From tjreedy at udel.edu  Thu Jul  3 19:44:19 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 03 Jul 2008 13:44:19 -0400
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>
	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
Message-ID: <g4j35h$i4p$1@ger.gmane.org>



Daniel Arbuckle wrote:

> Regardless, as I said before, nothing justifies silently changing the
> meaning of a program based on an option that most users don't set for
> themselves and are not aware of.

The premise of this thread seems to be that the majority should suffer 
for the benefit of a few.  That is not Python's philosophy.

Python hides many system differences.  It is gradually hiding more.  For 
instance, float('nan') works uniformly in 2.6 (with little performance 
hit), whereas it was system specific in 2.5  But Python does not promise 
to hide all system differences.

If the possible effects of (unicode) string build choice are not 
properly documented, then I agree that they should be, just as they are 
(or, in some cases, I presume are) the effects of underlying OS, 
processor integer and pointer size, float scheme, garbage collection 
scheme, and perhaps something I forgot.

Suggested documentation changes can be submitted to the tracker as 
specific ascii text targeted at a specific location.  If accepted, the 
doc maintainers will adapt submitted text to 'doc style' and add the 
needed markup.  Current response time is usually under a week, perhaps 
even a day.

Documented effects are not 'silent'.  But I am sure they could be made a 
bit louder.  Perhaps someday someone will volunteer to contribute a 
chapter to Using Python on Possible Semantic Variations that would run 
through the issues listed above so they are gathered together in one 
place as well as scattered throughout the Language and Library Reference 
manuals.

 > When such a change would take place,
> it should be reported explicitly as an error.

No, possible changes should be documented so that they are not silent. 
(But I am curious, by 'would' do you mean 'would with the current data' 
or 'theoretically could with chosen data'?)

Terry Jan Reedy


From guido at python.org  Thu Jul  3 19:55:24 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Jul 2008 10:55:24 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <g4j35h$i4p$1@ger.gmane.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
	<031901c8dd12$b7258b60$2570a220$@com.au>
	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
	<g4j35h$i4p$1@ger.gmane.org>
Message-ID: <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>

On Thu, Jul 3, 2008 at 10:44 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> The premise of this thread seems to be that the majority should suffer for
> the benefit of a few.  That is not Python's philosophy.

Who are the many here? Who are the few? I'd venture that (at least for
the foreseeable future, say, until China will finally have taken over
the role of the US as the de-facto dominant super power :-) the many
are people whose app will never see a Unicode character outside the
BMP, or who do such minimal string processing that their code doesn't
care whether it's handling UTF-16-encoded data.

Python's philosophy is also Practicality Beats Purity.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From doug.napoleone at gmail.com  Thu Jul  3 19:53:46 2008
From: doug.napoleone at gmail.com (doug.napoleone at gmail.com)
Date: Thu, 3 Jul 2008 13:53:46 -0400
Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down?
In-Reply-To: <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com>
References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com>
	<4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com>
Message-ID: <f08f4fbb0807031053p59fdd3f4qf2ea7b1d22234db@mail.gmail.com>

In Montana visiting. Will be back at the hotel in about 4 hours. Looks
like base site include is missing or has wrong permissions.

On 7/3/08, David Goodger <goodger at python.org> wrote:
> On Thu, Jul 3, 2008 at 13:12, Facundo Batista <facundobatista at gmail.com>
> wrote:
>> (sorry for the crossposting)
>>
>> Do you know what happened with "http://us.pycon.org/"?
>
> Not sure. The machine is still up (it serves www.pycon.org as well).
> Either something is misconfigured, or a process can't start, or
> something...
>
> I'll ask Jeff Rush (whose machine it's on) and Doug Napoleone (who
> knows more about the server than I, and has admin access) to look into
> it.
>
> --
> David Goodger <http://python.net/~goodger>
> _______________________________________________
> PyCon-organizers mailing list
> PyCon-organizers at python.org
> http://mail.python.org/mailman/listinfo/pycon-organizers
>

From asmodai at in-nomine.org  Thu Jul  3 20:39:02 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 3 Jul 2008 20:39:02 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486D0CE2.6010909@v.loewis.de>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>
	<20080702171957.GY62693@nexus.in-nomine.org>
	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>
	<20080702182215.GA62693@nexus.in-nomine.org>
	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486D0CE2.6010909@v.loewis.de>
Message-ID: <20080703183902.GG34192@nexus.in-nomine.org>

-On [20080703 19:31], "Martin v. L?wis" (martin at v.loewis.de) wrote:
>Yes, but it is two code units. Python's UTF-16 implementation operates
>on code units, not code points.

Thank you, that is the single most important piece of information I got
about this entire thing because it does change the entire approach.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Knowledge comes, but Wisdom lingers...

From goodger at python.org  Thu Jul  3 20:40:35 2008
From: goodger at python.org (David Goodger)
Date: Thu, 3 Jul 2008 14:40:35 -0400
Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down?
In-Reply-To: <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com>
References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com>
	<4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com>
Message-ID: <4335d2c40807031140o61bed415s2588f1e97f2aa655@mail.gmail.com>

On Thu, Jul 3, 2008 at 13:32, David Goodger <goodger at python.org> wrote:
> On Thu, Jul 3, 2008 at 13:12, Facundo Batista <facundobatista at gmail.com> wrote:
>> (sorry for the crossposting)
>>
>> Do you know what happened with "http://us.pycon.org/"?
>
> Not sure. The machine is still up (it serves www.pycon.org as well).
> Either something is misconfigured, or a process can't start, or
> something...
>
> I'll ask Jeff Rush (whose machine it's on) and Doug Napoleone (who
> knows more about the server than I, and has admin access) to look into
> it.

Jeff fixed it. URL rewriting was off by mistake.

-- 
David Goodger <http://python.net/~goodger>

From rhamph at gmail.com  Thu Jul  3 20:50:57 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 3 Jul 2008 12:50:57 -0600
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703173545.GF34192@nexus.in-nomine.org>
References: <20080702182215.GA62693@nexus.in-nomine.org>
	<20080702183541.GB62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com>
	<20080703132146.GI62693@nexus.in-nomine.org>
	<486CDAD5.1060506@egenix.com>
	<aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com>
	<20080703173545.GF34192@nexus.in-nomine.org>
Message-ID: <aac2c7cb0807031150w5a9fcdb6jee5c9adf892390f1@mail.gmail.com>

On Thu, Jul 3, 2008 at 11:35 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> -On [20080703 19:21], Adam Olsen (rhamph at gmail.com) wrote:
>>On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> Please remember that lone surrogate pair code points are perfectly
>>> valid Unicode code points, nevertheless. Just as a lone combining
>>> code point is valid on its own.
>>
>>That is a big part of these problems.  For all practical purposes, a
>>surrogate is like a UTF-8 code unit, and must be handled the same way,
>>so why the heck do they confuse everybody by saying "oh, it's a code
>>point too!"?
>
> Because surrogate code points are not Unicode scalar values, isolated UTF-16
> code units in the range 0xd800-0xdfff are ill-formed. (D91 from Unicode
> 5.0/5.1, section 3.9)
>
> So, no, it is not a code point too.


UTF-16
D91 UTF-16 encoding form: The Unicode encoding form that assigns each
Unicode scalar
     value in the ranges U+0000..U+D7FF and U+E000..U+FFFF to a single unsigned
     16-bit code unit with the same numeric value as the Unicode
scalar value, and that
     assigns each Unicode scalar value in the range U+10000..U+10FFFF
to a surrogate
     pair, according to Table 3-5.
   ? In UTF-16, the code point sequence <004D, 0430, 4E8C, 10302> is represented
     as <004D 0430 4E8C D800 DF02>, where <D800 DF02> corresponds to
     U+10302.
   ? Because surrogate code points are not Unicode scalar values,
isolated UTF-16
     code units in the range D80016..DFFF16 are ill-formed.

In the context of UTF-8 or UTF-32, a Unicode scalar value is a single
code point of a valid character (more or less) and a code unit is the
base unit (1 and 4 bytes respectively) of which 1 or more combine to
form a code point.  In UTF-16, code point becomes synonymous with code
unit and Unicode scalar value becomes one or more code points.  WTF?

-- 
Adam Olsen, aka Rhamphoryncus

From mal at egenix.com  Thu Jul  3 21:07:04 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Jul 2008 21:07:04 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com>
References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>
	<486CCD66.70906@egenix.com>	<20080703132146.GI62693@nexus.in-nomine.org>	<486CDAD5.1060506@egenix.com>
	<aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com>
Message-ID: <486D2358.1010604@egenix.com>

On 2008-07-03 19:21, Adam Olsen wrote:
> On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 2008-07-03 15:21, Jeroen Ruigrok van der Werven wrote:
>>> -On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote:
>>>> Unicode if full of combining code points - if you break such a sequence,
>>>> the output will be just as wrong; regardless of UCS2 vs. UCS4.
>>> In my opinion you are confusing two related, but very separated things
>>> here.
>>> Combining characters have nothing to do with breaking up the encoding of a
>>> single codepoint. Sure enough, if you arbitrary slice up codepoints that
>>> consist of combining characters then your result is indeed odd looking.
>>>
>>> I never said that nor is that the point I am making.
>> Please remember that lone surrogate pair code points are perfectly
>> valid Unicode code points, nevertheless. Just as a lone combining
>> code point is valid on its own.
> 
> That is a big part of these problems.  For all practical purposes, a
> surrogate is like a UTF-8 code unit, and must be handled the same way,
> so why the heck do they confuse everybody by saying "oh, it's a code
> point too!"?

You have to take that up with the Unicode consortium :-)

It would have been better not to add surrogates to the standard
at all. To be fair, I don't think that anybody seriously assumed
at the time that more than 16 bits would be needed.

In practice, you do need to be able to build Unicode strings
that contain half a surrogate (ie. a single code point) or
a combining code point without its anchor code point, so trying
to be smart about detecting surrogates is going to create more
confusion than do good, e.g.

 >>> x1 = u'\udbc0'
 >>> x2 = u'\udc00'
 >>> x1
u'\udbc0'
 >>> x2
u'\udc00'
 >>> len(x1)
1
 >>> len(x2)
1

Having len(x1+x2) == 1 wouldn't be right and break all sorts
of assumptions you normally make about string concatenation.
Which is why len(x1+x2) gives 2 in both UCS2 and UCS4 builds.

The fact that u'\U00100000' can map to a length 1 Unicode string
in UCS4 builds and a length 2 string in UCS2 builds is merely
due to the fact that the unicode-escape codec (which converts
the escaped string literal to a Unicode object) does know about
surrogates and uses them to avoid exceptions.

Programmers need to be aware of this fact, that's all...
just like they need to aware of differences between
integer and float division, different behavior of classic
and new-style classes, etc. etc.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 03 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania             3 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From mal at egenix.com  Thu Jul  3 21:16:03 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Jul 2008 21:16:03 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <20080703173545.GF34192@nexus.in-nomine.org>
References: <20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>
	<486CCD66.70906@egenix.com>	<20080703132146.GI62693@nexus.in-nomine.org>	<486CDAD5.1060506@egenix.com>	<aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com>
	<20080703173545.GF34192@nexus.in-nomine.org>
Message-ID: <486D2573.3070301@egenix.com>

On 2008-07-03 19:35, Jeroen Ruigrok van der Werven wrote:
> -On [20080703 19:21], Adam Olsen (rhamph at gmail.com) wrote:
>> On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> Please remember that lone surrogate pair code points are perfectly
>>> valid Unicode code points, nevertheless. Just as a lone combining
>>> code point is valid on its own.
>> That is a big part of these problems.  For all practical purposes, a
>> surrogate is like a UTF-8 code unit, and must be handled the same way,
>> so why the heck do they confuse everybody by saying "oh, it's a code
>> point too!"?
> 
> Because surrogate code points are not Unicode scalar values, isolated UTF-16
> code units in the range 0xd800-0xdfff are ill-formed. (D91 from Unicode
> 5.0/5.1, section 3.9)

True. They are not valid UTF-16 code units, but a code unit is
just a storage byte representation of a Unicode tranformation...

"""
Code Unit. The minimal bit combination that can represent a unit of encoded text for processing or interchange. The 
Unicode Standard uses 8-bit code units in the UTF-8 encoding form, 16-bit code units in the UTF-16 encoding form, and 
32-bit code units in the UTF-32 encoding form. (See definition D77 in  Section 3.9, Unicode Encoding Forms.)
"""

That's not the same thing as a code point which is an assignment
of a slot in the Unicode character set...

"""
Code Point. Any value in the Unicode codespace; that is, the range of integers from 0 to 10FFFF16. (See definition D10 
in Section 3.4, Characters and Encoding.)
"""

Reference: http://www.unicode.org/glossary/

Also see Chapter 3.4 (http://www.unicode.org/versions/Unicode5.0.0/ch03.pdf#G2212):

"""
Surrogate code points and noncharacters are considered assigned code points,
but not assigned characters.
"""

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 03 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania             3 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From facundobatista at gmail.com  Thu Jul  3 21:16:07 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 3 Jul 2008 16:16:07 -0300
Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down?
In-Reply-To: <4335d2c40807031140o61bed415s2588f1e97f2aa655@mail.gmail.com>
References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com>
	<4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com>
	<4335d2c40807031140o61bed415s2588f1e97f2aa655@mail.gmail.com>
Message-ID: <e04bdf310807031216w2b5df96bgabefb77d828462fb@mail.gmail.com>

2008/7/3 David Goodger <goodger at python.org>:

> Jeff fixed it. URL rewriting was off by mistake.

Thanks! :)

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From mal at egenix.com  Thu Jul  3 21:24:40 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Jul 2008 21:24:40 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <g4j35h$i4p$1@ger.gmane.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
	<g4j35h$i4p$1@ger.gmane.org>
Message-ID: <486D2778.8090503@egenix.com>

On 2008-07-03 19:44, Terry Reedy wrote:
> The premise of this thread seems to be that the majority should suffer 
> for the benefit of a few.  That is not Python's philosophy.

In reality, most Unixes ship with UCS4 builds of Python. Windows
and Mac OS X ship with UCS2 builds. Still, anyone is free to build
their own favorite version - that's freedom of choice, which is good.

Programmers just need to be made aware of the differences in UCS2
and UCS4 builds and deal with it.

Here's talk I've given many many times over the years which explains
some of the details that a Python programmer needs to know when dealing
with Unicode:

http://www.egenix.com/files/python/PyConUK2007-Developing-Unicode-aware-applications-in-Python.pdf

Perhaps I should add a section on UCS2 vs. UCS4 the next time around ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 03 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania             3 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From jeremy at 54oaks.com  Thu Jul  3 21:35:33 2008
From: jeremy at 54oaks.com (Jeremy Link)
Date: Thu, 3 Jul 2008 12:35:33 -0700
Subject: [Python-Dev] problems compiling ctypes
Message-ID: <010601c8dd43$f71ab890$8101a8c0@bocaron>

I've grabbed the latest libffi that contains support for the ARM processor.
I then enable FFI_CLOSURES in the arm/ffi.c file.

 

When I do this, I get compilation errors that it is missing
ffi_prep_closure.

 

Is ffi.c up to date for supporting the ARM platform?

 

Not sure if there is a simple configuration change in one of the files that
will fix *everything* or if ffi.c just doesn't support ARM yet and so it
needs be developed/revamped.

 

Thanks for any help.

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080703/a6a04fab/attachment.htm>

From steve at holdenweb.com  Thu Jul  3 21:59:08 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 03 Jul 2008 15:59:08 -0400
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <486D2778.8090503@egenix.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>	<g4j35h$i4p$1@ger.gmane.org>
	<486D2778.8090503@egenix.com>
Message-ID: <g4jb2l$f3g$1@ger.gmane.org>

M.-A. Lemburg wrote:
> On 2008-07-03 19:44, Terry Reedy wrote:
>> The premise of this thread seems to be that the majority should suffer 
>> for the benefit of a few.  That is not Python's philosophy.
> 
> In reality, most Unixes ship with UCS4 builds of Python. Windows
> and Mac OS X ship with UCS2 builds. Still, anyone is free to build
> their own favorite version - that's freedom of choice, which is good.
> 
> Programmers just need to be made aware of the differences in UCS2
> and UCS4 builds and deal with it.
> 
> Here's talk I've given many many times over the years which explains
> some of the details that a Python programmer needs to know when dealing
> with Unicode:
> 
> http://www.egenix.com/files/python/PyConUK2007-Developing-Unicode-aware-applications-in-Python.pdf 
> 
> 
> Perhaps I should add a section on UCS2 vs. UCS4 the next time around ;-)
> 
The indications are that would be helpful to many people (including myself).

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From tjreedy at udel.edu  Thu Jul  3 23:01:48 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 03 Jul 2008 17:01:48 -0400
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>	<g4j35h$i4p$1@ger.gmane.org>
	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>
Message-ID: <g4jenq$rks$1@ger.gmane.org>



Guido van Rossum wrote:
> On Thu, Jul 3, 2008 at 10:44 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> The premise of this thread seems to be that the majority should suffer for
>> the benefit of a few.  That is not Python's philosophy.

The premise is the OP's idea that Python should switch to all UCS4 to 
create a more pure ('ideal') situation or the idea that len(s) should 
count codepoints (correct term?) for all builds as a matter of purity 
even though on it would be time-costly on 16-bit builds as a matter of 
practicality.

> Who are the many here?

Those who are happy with 3.0 strings as they are for their systems and 
who would not benefit from the proposed change.  In other words, what 
you say below.

 > Who are the few?

Those who are stuck with 16-bit builds and who would benefit from 
32-bits builds because they need to use non basic plane chars and need 
to use the operations for which a change would make a positive difference.

In my opinion, such people with Windows should at least install Linux + 
UCS4 Python as an alternate install.

> I'd venture that (at least for
> the foreseeable future, say, until China will finally have taken over
> the role of the US as the de-facto dominant super power :-) the many
> are people whose app will never see a Unicode character outside the
> BMP, or who do such minimal string processing that their code doesn't
> care whether it's handling UTF-16-encoded data.

Just what I meant.

> Python's philosophy is also Practicality Beats Purity.

Just what I meant, in the form 'Purity does not beat Practicality'.

Having summarized, perhaps too briefly, why Python's basic unicode 
implementation would not change in the near future, I went on to my main 
point, which is that better docs might be an alternative solution to the 
problems raised.

tjr




From martin at v.loewis.de  Thu Jul  3 23:15:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 03 Jul 2008 23:15:40 +0200
Subject: [Python-Dev] problems compiling ctypes
In-Reply-To: <010601c8dd43$f71ab890$8101a8c0@bocaron>
References: <010601c8dd43$f71ab890$8101a8c0@bocaron>
Message-ID: <486D417C.9060503@v.loewis.de>

> Thanks for any help.

This list (python-dev) is not for getting help, but for providing it.
So if you have patches that you would like to discuss, please go
ahead. As you are seeking help, please use python-list at python.org
(aka news:comp.lang.python) instead.

Regards,
Martin

From rhamph at gmail.com  Fri Jul  4 00:00:56 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 3 Jul 2008 16:00:56 -0600
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <g4jenq$rks$1@ger.gmane.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
	<031901c8dd12$b7258b60$2570a220$@com.au>
	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
	<g4j35h$i4p$1@ger.gmane.org>
	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>
	<g4jenq$rks$1@ger.gmane.org>
Message-ID: <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>

On Thu, Jul 3, 2008 at 3:01 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
> The premise is the OP's idea that Python should switch to all UCS4 to create
> a more pure ('ideal') situation or the idea that len(s) should count
> codepoints (correct term?) for all builds as a matter of purity even though
> on it would be time-costly on 16-bit builds as a matter of practicality.

Wrong term - code units and code points are equivalent in UTF-16 and
UTF-32.  What you're looking for is unicode scalar values.


-- 
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Fri Jul  4 00:21:46 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Jul 2008 15:21:46 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<20080703104813.GF62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
	<031901c8dd12$b7258b60$2570a220$@com.au>
	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
	<g4j35h$i4p$1@ger.gmane.org>
	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>
	<g4jenq$rks$1@ger.gmane.org>
	<aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>
Message-ID: <ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com>

On Thu, Jul 3, 2008 at 3:00 PM, Adam Olsen <rhamph at gmail.com> wrote:
> On Thu, Jul 3, 2008 at 3:01 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>
>> The premise is the OP's idea that Python should switch to all UCS4 to create
>> a more pure ('ideal') situation or the idea that len(s) should count
>> codepoints (correct term?) for all builds as a matter of purity even though
>> on it would be time-costly on 16-bit builds as a matter of practicality.
>
> Wrong term - code units and code points are equivalent in UTF-16 and
> UTF-32.  What you're looking for is unicode scalar values.

I don't think so. I have in my lap the Unicode 5.0 standard, which on
page 102, under UTF-16, states (amongst others):

"""
* In UTF-16, the code point sequence <004D, 0430, 4E8C, 10302> is
represented as <004D 0439 4E8C D800 DF02>, where <D800 DF02>
corresponds to U+10302.

* Because surrogate code points are not Unicode scalar values,
isolated UTF-16 code units in the range D800[16]..DFFF[16] are
ill-formed.
"""

>From this I understand they distinguish carefully between code points
and code units -- D800 is a code unit but not a code point, 10302 is a
code point but not a (UTF-16) code unit.

OTOH outside the context of UTF-8, the surrogates are also referred to
as "reserved code points" (e.g. in Table 2-3 on page 27, "Types of
Code Points").

I think the best thing we can do is to use "code points" to refer to
characters and "code units" to the individual 16-bit values in the
UTF-16 encoding; this seems compatible with usage elsewhere in this
thread by most folks.

Also see http://unicode.org/glossary/:

"""
Code Point. Any value in the Unicode codespace; that is, the range of
integers from 0 to 10FFFF16. (See definition D10 in Section 3.4,
Characters and Encoding.)
.
.
.
Code Unit. The minimal bit combination that can represent a unit of
encoded text for processing or interchange. The Unicode Standard uses
8-bit code units in the UTF-8 encoding form, 16-bit code units in the
UTF-16 encoding form, and 32-bit code units in the UTF-32 encoding
form. (See definition D77 in  Section 3.9, Unicode Encoding Forms.)
"""

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Fri Jul  4 00:31:49 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 04 Jul 2008 00:31:49 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>	<g4j35h$i4p$1@ger.gmane.org>	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>	<g4jenq$rks$1@ger.gmane.org>
	<aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>
Message-ID: <486D5355.4000104@v.loewis.de>

> Wrong term - code units and code points are equivalent in UTF-16 and
> UTF-32.  What you're looking for is unicode scalar values.

How so? Section 2.5, UTF-16 says

"code points in the supplementary planes, in the range
U+10000..U+10FFFF, are represented as pairs of 16-bit code units."

So clearly, code points in Unicode range from U+0000..U+10FFFF,
independent of encoding form.

In UTF-16, code units range from 0..65535.

OTOH, "unicode scalar value" is nearly synonymous to "code point":

D76 Unicode Scalar Value. Any Unicode  code point except high-surrogate
and low-surrogate code points.

So codepoint in Terry's message was the right term.

Regards,
Martin

From rhamph at gmail.com  Fri Jul  4 01:50:51 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 3 Jul 2008 17:50:51 -0600
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<486CC881.5090902@gmail.com>
	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
	<031901c8dd12$b7258b60$2570a220$@com.au>
	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
	<g4j35h$i4p$1@ger.gmane.org>
	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>
	<g4jenq$rks$1@ger.gmane.org>
	<aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>
	<ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com>
Message-ID: <aac2c7cb0807031650u7abe63d2q7371bdc1782b7d8@mail.gmail.com>

On Thu, Jul 3, 2008 at 4:21 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jul 3, 2008 at 3:00 PM, Adam Olsen <rhamph at gmail.com> wrote:
>> On Thu, Jul 3, 2008 at 3:01 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>>
>>> The premise is the OP's idea that Python should switch to all UCS4 to create
>>> a more pure ('ideal') situation or the idea that len(s) should count
>>> codepoints (correct term?) for all builds as a matter of purity even though
>>> on it would be time-costly on 16-bit builds as a matter of practicality.
>>
>> Wrong term - code units and code points are equivalent in UTF-16 and
>> UTF-32.  What you're looking for is unicode scalar values.
>
> I don't think so. I have in my lap the Unicode 5.0 standard, which on
> page 102, under UTF-16, states (amongst others):
>
> """
> * In UTF-16, the code point sequence <004D, 0430, 4E8C, 10302> is
> represented as <004D 0439 4E8C D800 DF02>, where <D800 DF02>
> corresponds to U+10302.

The literal interpretation is that the U+10302 code point should get
expanded into <D800 DF02>.  It doesn't say if <D800 DF02> is a pair of
code units or a pair of code points.


> * Because surrogate code points are not Unicode scalar values,
> isolated UTF-16 code units in the range D800[16]..DFFF[16] are
> ill-formed.
> """

So a lone surrogate code unit is not a valid scalar.  It also implies
surrogate code points exist, rather than ruling them out.


> From this I understand they distinguish carefully between code points
> and code units -- D800 is a code unit but not a code point, 10302 is a
> code point but not a (UTF-16) code unit.

I disagree.  They switch between code point and code unit arbitrarily,
never than saying surrogate code points don't exist.


> OTOH outside the context of UTF-8, the surrogates are also referred to
> as "reserved code points" (e.g. in Table 2-3 on page 27, "Types of
> Code Points").

You mean outside the context of UTF-16?  Regarding them as reserved
and lone surrogates as ill-formed code units would have been simpler,
but alas, is not the case.

Regarding changes in 5.1
(http://www.unicode.org/versions/Unicode5.1.0/), I can find this bit
to give some context:

    Rendering Default Ignorable Code Points

    Update the last paragraph on p. 192 of The Unicode Standard,
Version 5.0, in Section 5.20, Default Ignorable Code Points, to read
as follows:

        Replacement Text
        An implementation should ignore all default ignorable code
points in rendering whenever it does not support those code points,
whether they are assigned or not.

        In previous versions of the Unicode Standard, surrogate code
points, private use code points, and some control characters were also
default ignorable code points. However, to avoid security problems,
such characters always should be displayed with a missing glyph, so
that there is a visible indication of their presence in the text. In
Unicode 5.1 these code points are no longer default ignorable code
points. For more information, see UTR #36, "Unicode Security
Considerations."

Clearly they act as if surrogate code points exist.

Finally, we find this in the glossary:

    Unicode Scalar Value. Any Unicode  code point except
high-surrogate and low-surrogate code points. In other words, the
ranges of integers 0 to D7FF16 and E00016 to 10FFFF16 inclusive. (See
definition D76 in  Section 3.9, Unicode Encoding Forms.)

Clearly, each surrogate is a valid code point, regardless of encoding.
 A surrogate pair simultaneously represents both one code point (the
scalar value) and two code points (the surrogate code points).  To be
unambiguous you must instead use either code units (always 2 for
UTF-16) or scalar values (always 1 in any encoding).

The OP wanted it to always be 1, so the correct unambiguous term is
scalar value.


> I think the best thing we can do is to use "code points" to refer to
> characters and "code units" to the individual 16-bit values in the
> UTF-16 encoding; this seems compatible with usage elsewhere in this
> thread by most folks.
>
> Also see http://unicode.org/glossary/:
>
> """
> Code Point. Any value in the Unicode codespace; that is, the range of
> integers from 0 to 10FFFF16. (See definition D10 in Section 3.4,
> Characters and Encoding.)
> .
> .
> .
> Code Unit. The minimal bit combination that can represent a unit of
> encoded text for processing or interchange. The Unicode Standard uses
> 8-bit code units in the UTF-8 encoding form, 16-bit code units in the
> UTF-16 encoding form, and 32-bit code units in the UTF-32 encoding
> form. (See definition D77 in  Section 3.9, Unicode Encoding Forms.)
> """
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Fri Jul  4 05:26:16 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Jul 2008 20:26:16 -0700
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <aac2c7cb0807031650u7abe63d2q7371bdc1782b7d8@mail.gmail.com>
References: <20080702141328.GW62693@nexus.in-nomine.org>
	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>
	<031901c8dd12$b7258b60$2570a220$@com.au>
	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>
	<g4j35h$i4p$1@ger.gmane.org>
	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>
	<g4jenq$rks$1@ger.gmane.org>
	<aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>
	<ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com>
	<aac2c7cb0807031650u7abe63d2q7371bdc1782b7d8@mail.gmail.com>
Message-ID: <ca471dc20807032026r1dfca0a5h2799180dbbed54ca@mail.gmail.com>

On Thu, Jul 3, 2008 at 4:50 PM, Adam Olsen <rhamph at gmail.com> wrote:
> Clearly, each surrogate is a valid code point, regardless of encoding.
>  A surrogate pair simultaneously represents both one code point (the
> scalar value) and two code points (the surrogate code points).  To be
> unambiguous you must instead use either code units (always 2 for
> UTF-16) or scalar values (always 1 in any encoding).
>
> The OP wanted it to always be 1, so the correct unambiguous term is
> scalar value.

Fine, if you want to be completely unambiguous you apparently you
can't use the word code point but you have to use either scalar values
(always Unicode characters) or code units (always part of an encoding,
and 8, 16 or 32 bits).

Regardless of what the OP might want, len() of a surrogate pair will
return 2 (since it counts code units), and we'll have to provide
another API to count scalar values / characters that sees a surrogate
pair as one.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Fri Jul  4 11:39:55 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 4 Jul 2008 10:39:55 +0100
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <e8a0972d0806281912w4328c55tfc55c5a298c3333c@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
	<58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
	<e8a0972d0806281912w4328c55tfc55c5a298c3333c@mail.gmail.com>
Message-ID: <5c6f2a5d0807040239s22fc6b7cx31f27e916b6ba6a2@mail.gmail.com>

On Sun, Jun 29, 2008 at 3:12 AM, Alex Martelli <aleaxit at gmail.com> wrote:
> On Sat, Jun 28, 2008 at 4:46 PM, Raymond Hettinger <python at rcn.com> wrote:
>> Is everyone agreed on a tohex/fromhex pair using the C99 notation as
>> recommended in 754R?
>
> Dunno about everyone, but I'm +1 on that.
>
>
>> Are you thinking of math module functions or as a method and classmethod on
>> floats?
>
> I'd prefer math modules functions.

I'm halfway through implementing this as a pair of float methods.  Are there
compelling reasons to prefer math module functions over float methods, or
vice versa?

Personally, I'm leaning slightly towards float methods:  for me, these
conversions are important enough to belong in the core language.  But I
don't have strong feelings either way.

Mark

From mal at egenix.com  Fri Jul  4 12:08:14 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Jul 2008 12:08:14 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <g4jb2l$f3g$1@ger.gmane.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com>	<20080702182215.GA62693@nexus.in-nomine.org>	<ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com>	<20080702183541.GB62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>	<g4j35h$i4p$1@ger.gmane.org>	<486D2778.8090503@egenix.com>
	<g4jb2l$f3g$1@ger.gmane.org>
Message-ID: <486DF68E.7090000@egenix.com>

On 2008-07-03 21:59, Steve Holden wrote:
> M.-A. Lemburg wrote:
>> On 2008-07-03 19:44, Terry Reedy wrote:
>>> The premise of this thread seems to be that the majority should 
>>> suffer for the benefit of a few.  That is not Python's philosophy.
>>
>> In reality, most Unixes ship with UCS4 builds of Python. Windows
>> and Mac OS X ship with UCS2 builds. Still, anyone is free to build
>> their own favorite version - that's freedom of choice, which is good.
>>
>> Programmers just need to be made aware of the differences in UCS2
>> and UCS4 builds and deal with it.
>>
>> Here's talk I've given many many times over the years which explains
>> some of the details that a Python programmer needs to know when dealing
>> with Unicode:
>>
>> http://www.egenix.com/files/python/PyConUK2007-Developing-Unicode-aware-applications-in-Python.pdf 
>>
>>
>> Perhaps I should add a section on UCS2 vs. UCS4 the next time around ;-)
>
> The indications are that would be helpful to many people (including 
> myself).

Ok, I'll add one for one of the next conferences.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 04 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania             2 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From guido at python.org  Fri Jul  4 15:49:07 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 4 Jul 2008 06:49:07 -0700
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0807040239s22fc6b7cx31f27e916b6ba6a2@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
	<58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
	<e8a0972d0806281912w4328c55tfc55c5a298c3333c@mail.gmail.com>
	<5c6f2a5d0807040239s22fc6b7cx31f27e916b6ba6a2@mail.gmail.com>
Message-ID: <ca471dc20807040649r467d8554n18aa8e7f41daaa9b@mail.gmail.com>

Float methods are fine.

On Fri, Jul 4, 2008 at 2:39 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Sun, Jun 29, 2008 at 3:12 AM, Alex Martelli <aleaxit at gmail.com> wrote:
>> On Sat, Jun 28, 2008 at 4:46 PM, Raymond Hettinger <python at rcn.com> wrote:
>>> Is everyone agreed on a tohex/fromhex pair using the C99 notation as
>>> recommended in 754R?
>>
>> Dunno about everyone, but I'm +1 on that.
>>
>>
>>> Are you thinking of math module functions or as a method and classmethod on
>>> floats?
>>
>> I'd prefer math modules functions.
>
> I'm halfway through implementing this as a pair of float methods.  Are there
> compelling reasons to prefer math module functions over float methods, or
> vice versa?
>
> Personally, I'm leaning slightly towards float methods:  for me, these
> conversions are important enough to belong in the core language.  But I
> don't have strong feelings either way.
>
> Mark
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mail at timgolden.me.uk  Fri Jul  4 17:24:32 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Fri, 04 Jul 2008 16:24:32 +0100
Subject: [Python-Dev] ctypes assertion failure
Message-ID: <486E40B0.4020608@timgolden.me.uk>

This problem was raised on the comtypes-users list
as it prevents comtypes from being imported on Python 2.6
at the moment.

http://bugs.python.org/issue3258

I'll try to find the time to step through to code to work out
what's going on, but it's inside the innards of ctypes which
I've never looked into before.

Could someone confirm at a glance whether this should be
given a high priority, please? It results in an assertion error
in debug mode, a SystemError in non-debug referring to
a NULL return without an Exception set.

Thanks
TJG

From status at bugs.python.org  Fri Jul  4 18:06:28 2008
From: status at bugs.python.org (Python tracker)
Date: Fri,  4 Jul 2008 18:06:28 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080704160628.7A783780B1@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (06/27/08 - 07/04/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1941 open (+33) / 13165 closed (+34) / 15106 total (+67)

Open issues with patches:   607

Average duration of open issues: 704 days.
Median duration of open issues: 1570 days.

Open Issues Breakdown
   open  1915 (+33)
pending    26 ( +0)

Issues Created Or Reopened (71)
_______________________________

isinstance(anything, MetaclassThatDefinesInstancecheck) raises i 06/30/08
       http://bugs.python.org/issue2325    reopened jyasskin                  
       patch                                                                   

test_multiprocessing hangs on OS X 10.5.3                        07/02/08
       http://bugs.python.org/issue3088    reopened jnoller                   
       patch                                                                   

test_multiprocessing causes test_ctypes to fail                  07/02/08
       http://bugs.python.org/issue3125    reopened jnoller                   
       patch                                                                   

"Quick search" box renders too long on FireFox 3                 06/27/08
       http://bugs.python.org/issue3154    reopened benjamin.peterson         
                                                                               

make text is broken                                              06/27/08
CLOSED http://bugs.python.org/issue3217    created  benjamin.peterson         
                                                                               

2to3 Fix_imports optimization                                    06/27/08
       http://bugs.python.org/issue3218    created  nedds                     
       patch                                                                   

repeated keyword arguments                                       06/27/08
CLOSED http://bugs.python.org/issue3219    created  gangesmaster              
       patch                                                                   

Improve Bytes and Byte Array Methods doc                         06/27/08
CLOSED http://bugs.python.org/issue3220    created  tjreedy                   
                                                                               

SystemError: Parent module 'foo' not loaded on import statement  06/27/08
       http://bugs.python.org/issue3221    created  schmir                    
                                                                               

inf*inf gives inf, but inf**2 gives overflow error               06/27/08
CLOSED http://bugs.python.org/issue3222    created  ms                        
                                                                               

py3k warn on use of frame.f_exc*                                 06/27/08
       http://bugs.python.org/issue3223    created  benjamin.peterson         
       easy                                                                    

Small typo in 2.6 what's new                                     06/28/08
CLOSED http://bugs.python.org/issue3224    created  catlee                    
                                                                               

backport python 3.0 language functionality to python 2.5 by addi 06/28/08
CLOSED http://bugs.python.org/issue3225    created  kaizhu                    
                                                                               

can't install on OSX 10.4                                        06/28/08
       http://bugs.python.org/issue3226    created  benjamin.peterson         
                                                                               

os.environ.clear has no effect on child processes                06/28/08
CLOSED http://bugs.python.org/issue3227    created  joe.p.cool                
                                                                               

mailbox.mbox creates files with execute bit set                  06/28/08
       http://bugs.python.org/issue3228    created  pl                        
                                                                               

Language reference, class definitions: missing text in "Programm 06/28/08
CLOSED http://bugs.python.org/issue3229    created  oefe                      
                                                                               

dictobject.c: inappropriate use of PySet_GET_SIZE?               06/28/08
CLOSED http://bugs.python.org/issue3230    created  oefe                      
                                                                               

re.compile fails with some bytes patterns                        06/28/08
       http://bugs.python.org/issue3231    created  pitrou                    
       patch                                                                   

Wrong str->bytes conversion in Lib/encodings/idna.py             06/29/08
       http://bugs.python.org/issue3232    created  pitrou                    
                                                                               

Timestamp stored in ZIP file not correct ?                       06/29/08
CLOSED http://bugs.python.org/issue3233    created  pythonmeister             
                                                                               

subprocess.py strips last character when raising an AttributeErr 06/29/08
CLOSED http://bugs.python.org/issue3234    created  mmokrejs                  
                                                                               

Improve subprocess module usage                                  06/29/08
CLOSED http://bugs.python.org/issue3235    created  mmokrejs                  
                                                                               

ints contructed from strings don't use the smallint constants    06/29/08
CLOSED http://bugs.python.org/issue3236    created  pitrou                    
                                                                               

idlelib3.0 still using xrange                                    06/29/08
CLOSED http://bugs.python.org/issue3237    created  tjreedy                   
                                                                               

backport python 3.0 language functionality to python 2.6 by addi 06/29/08
       http://bugs.python.org/issue3238    created  kaizhu                    
                                                                               

curses/textpad.py incorrectly and redundantly imports ascii      06/30/08
       http://bugs.python.org/issue3239    reopened facundobatista            
       patch                                                                   

IDLE environment corrupts string.letters                         06/30/08
CLOSED http://bugs.python.org/issue3240    created  rupole                    
                                                                               

warnings module prints garbage                                   06/30/08
CLOSED http://bugs.python.org/issue3241    created  schmir                    
                                                                               

Segfault in PyFile_SoftSpace/PyEval_EvalFrameEx with sys.stdout  06/30/08
CLOSED http://bugs.python.org/issue3242    reopened benjamin.peterson         
       patch                                                                   

Support iterable bodies in httplib                               06/30/08
       http://bugs.python.org/issue3243    created  catlee                    
                                                                               

multipart/form-data encoding                                     06/30/08
       http://bugs.python.org/issue3244    created  catlee                    
                                                                               

Memory leak on OS X                                              06/30/08
CLOSED http://bugs.python.org/issue3245    created  fiddlerwoaroof            
                                                                               

configure: WARNING: sys/socket.h: present but cannot be compiled 06/30/08
       http://bugs.python.org/issue3246    created  rrochele                  
                                                                               

dir of an "_sre.SRE_Match" object not working                    07/01/08
CLOSED http://bugs.python.org/issue3247    created  vizcayno                  
                                                                               

ScrolledText can't be placed in a PanedWindow                    07/01/08
       http://bugs.python.org/issue3248    created  gpolo                     
       patch                                                                   

bug adding datetime.timedelta to datetime.date                   07/01/08
CLOSED http://bugs.python.org/issue3249    created  cjw296                    
                                                                               

datetime.time does not support arithmetic                        07/01/08
       http://bugs.python.org/issue3250    created  cjw296                    
       patch                                                                   

references are case insensitive                                  07/01/08
CLOSED http://bugs.python.org/issue3251    created  tds333                    
                                                                               

str.tobytes() and bytes/bytearray.tostr()                        07/01/08
CLOSED http://bugs.python.org/issue3252    created  mark                      
                                                                               

shutil.move bahave unexpected in fat32                           07/01/08
       http://bugs.python.org/issue3253    created  grissiom                  
                                                                               

Suggestion: change default behavior of __ne__                    07/01/08
CLOSED http://bugs.python.org/issue3254    created  cvp                       
                                                                               

[proposal] alternative for re.sub                                07/02/08
       http://bugs.python.org/issue3255    created  ocean-city                
                                                                               

Multiprocessing docs are not 3.0-ready                           07/02/08
       http://bugs.python.org/issue3256    created  mishok13                  
                                                                               

"#define socklen_t int" in pyconfig.h                            07/02/08
CLOSED http://bugs.python.org/issue3257    created  fgoujeon                  
                                                                               

ctypes assertion failure in trunk                                07/02/08
       http://bugs.python.org/issue3258    created  tim.golden                
                                                                               

fix_imports needs to be using the 'as' keyword                   07/02/08
CLOSED http://bugs.python.org/issue3259    created  brett.cannon              
                                                                               

fix_imports does not handle intra-package renames                07/02/08
       http://bugs.python.org/issue3260    created  brett.cannon              
                                                                               

Lib/test/test_cookielib declares utf-8 encoding, but contains no 07/02/08
CLOSED http://bugs.python.org/issue3261    created  leosoto                   
                                                                               

re.split doesn't split with zero-width regex                     07/02/08
       http://bugs.python.org/issue3262    created  mrabarnett                
       patch                                                                   

Odd code fragment in ABC definitions                             07/02/08
CLOSED http://bugs.python.org/issue3263    created  rhettinger                
                                                                               

Use -lcrypto instead of -lcrypt on Solaris 2.6 when available    07/02/08
CLOSED http://bugs.python.org/issue3264    created  mmokrejs                  
                                                                               

Python-2.5.2/Modules/_ctypes/malloc_closure.c:70: error: `MAP_AN 07/03/08
       http://bugs.python.org/issue3265    created  mmokrejs                  
                                                                               

Python-2.5.2/Modules/mmapmodule.c:915: error: `O_RDWR' undeclare 07/03/08
       http://bugs.python.org/issue3266    created  mmokrejs                  
                                                                               

yield in list comprehensions possibly broken in 3.0              07/03/08
CLOSED http://bugs.python.org/issue3267    created  erickt                    
                                                                               

Cleanup of tp_basicsize inheritance                              07/03/08
       http://bugs.python.org/issue3268    created  Rhamphoryncus             
       patch                                                                   

strptime() makes an error concerning second in arg               07/03/08
CLOSED http://bugs.python.org/issue3269    created  nevgor                    
                                                                               

test_multiprocessing: test_listener_client flakiness             07/03/08
       http://bugs.python.org/issue3270    created  jnoller                   
       patch                                                                   

iter.next() or iter.__next__() ?                                 07/03/08
CLOSED http://bugs.python.org/issue3271    created  vizcayno                  
                                                                               

Multiprocessing hangs when multiprocessing.Pool methods are call 07/03/08
       http://bugs.python.org/issue3272    created  mishok13                  
                                                                               

multiprocessing and meaningful errors                            07/03/08
       http://bugs.python.org/issue3273    created  mishok13                  
                                                                               

Py_CLEAR(tmp) seg faults                                         07/03/08
       http://bugs.python.org/issue3274    created  stutzbach                 
                                                                               

Control flow not optimized                                       07/03/08
CLOSED http://bugs.python.org/issue3275    created  quotemstr                 
                                                                               

httplib.HTTPConnection._send_request should not blindly assume d 07/04/08
       http://bugs.python.org/issue3276    created  ludvig.ericson            
       patch                                                                   

socket's OOB data management is broken on FreeBSD                07/04/08
       http://bugs.python.org/issue3277    created  giampaolo.rodola          
                                                                               

socket's SO_OOBINLINE option does not work on FreeBSD            07/04/08
       http://bugs.python.org/issue3278    created  giampaolo.rodola          
                                                                               

import of site.py fails on startup                               07/04/08
       http://bugs.python.org/issue3279    created  rupole                    
                                                                               

%c format does not accept large numbers on ucs-2 builds          07/04/08
       http://bugs.python.org/issue3280    created  amaury.forgeotdarc        
                                                                               

support r"\"                                                     07/04/08
CLOSED http://bugs.python.org/issue3281    created  lidaobing                 
                                                                               

Undefined unicode characters should be non-printable             07/04/08
CLOSED http://bugs.python.org/issue3282    created  amaury.forgeotdarc        
                                                                               

multiprocessing.connection doesn't import AuthenticationError, w 07/04/08
       http://bugs.python.org/issue3283    created  mishok13                  
       patch                                                                   



Issues Now Closed (61)
______________________

DeprecationWarning in zipfile.py while zipping 113000 files       216 days
       http://bugs.python.org/issue1526    loewis                    
                                                                               

zipfile hangs on certain zip files                                202 days
       http://bugs.python.org/issue1622    loewis                    
       patch                                                                   

Move Demo/classes/Rat.py to Lib/fractions.py and fix it up.         6 days
       http://bugs.python.org/issue1682    marketdickinson           
       patch                                                                   

ZIP files with archive comments longer than 4k not recognized as  179 days
       http://bugs.python.org/issue1746    loewis                    
                                                                               

test_audioop.py converted to unittest                             142 days
       http://bugs.python.org/issue2042    benjamin.peterson         
       patch                                                                   

urlparse() does not handle URLs with port numbers properly        127 days
       http://bugs.python.org/issue2195    facundobatista            
                                                                               

subprocess.Popen.communicate takes bytes, not str                  68 days
       http://bugs.python.org/issue2683    georg.brandl              
       patch                                                                   

pickling of large recursive structures crashes cPickle              4 days
       http://bugs.python.org/issue2702    facundobatista            
       patch                                                                   

asynchat forgets packets when push is called from a thread         54 days
       http://bugs.python.org/issue2808    josiahcarlson             
                                                                               

Copy cgi.parse_qs() to urllib.parse                                50 days
       http://bugs.python.org/issue2829    benjamin.peterson         
                                                                               

tests for sys.getsizeof fail on win64                               9 days
       http://bugs.python.org/issue3147    schuppenies               
                                                                               

3.0b1 doesn't seem to install on macs                               8 days
       http://bugs.python.org/issue3174    benjamin.peterson         
                                                                               

Pydoc should ignore __package__ attributes                          8 days
       http://bugs.python.org/issue3190    ncoghlan                  
                                                                               

round docstring is inaccurate                                       7 days
       http://bugs.python.org/issue3191    georg.brandl              
                                                                               

Documentation for fractions module needs work                       2 days
       http://bugs.python.org/issue3197    marketdickinson           
       patch                                                                   

Can't import sqlite3 in Python 2.6b1                                3 days
       http://bugs.python.org/issue3215    loewis                    
                                                                               

make text is broken                                                 4 days
       http://bugs.python.org/issue3217    georg.brandl              
                                                                               

repeated keyword arguments                                          4 days
       http://bugs.python.org/issue3219    benjamin.peterson         
       patch                                                                   

Improve Bytes and Byte Array Methods doc                            4 days
       http://bugs.python.org/issue3220    georg.brandl              
                                                                               

inf*inf gives inf, but inf**2 gives overflow error                  1 days
       http://bugs.python.org/issue3222    marketdickinson           
                                                                               

Small typo in 2.6 what's new                                        0 days
       http://bugs.python.org/issue3224    benjamin.peterson         
                                                                               

backport python 3.0 language functionality to python 2.5 by addi    0 days
       http://bugs.python.org/issue3225    loewis                    
                                                                               

os.environ.clear has no effect on child processes                   0 days
       http://bugs.python.org/issue3227    benjamin.peterson         
                                                                               

Language reference, class definitions: missing text in "Programm    0 days
       http://bugs.python.org/issue3229    benjamin.peterson         
                                                                               

dictobject.c: inappropriate use of PySet_GET_SIZE?                  0 days
       http://bugs.python.org/issue3230    rhettinger                
                                                                               

Timestamp stored in ZIP file not correct ?                          0 days
       http://bugs.python.org/issue3233    loewis                    
                                                                               

subprocess.py strips last character when raising an AttributeErr    0 days
       http://bugs.python.org/issue3234    benjamin.peterson         
                                                                               

Improve subprocess module usage                                     3 days
       http://bugs.python.org/issue3235    mmokrejs                  
                                                                               

ints contructed from strings don't use the smallint constants       1 days
       http://bugs.python.org/issue3236    loewis                    
                                                                               

idlelib3.0 still using xrange                                       0 days
       http://bugs.python.org/issue3237    benjamin.peterson         
                                                                               

IDLE environment corrupts string.letters                            2 days
       http://bugs.python.org/issue3240    loewis                    
                                                                               

warnings module prints garbage                                      0 days
       http://bugs.python.org/issue3241    brett.cannon              
                                                                               

Segfault in PyFile_SoftSpace/PyEval_EvalFrameEx with sys.stdout     1 days
       http://bugs.python.org/issue3242    amaury.forgeotdarc        
       patch                                                                   

Memory leak on OS X                                                 0 days
       http://bugs.python.org/issue3245    benjamin.peterson         
                                                                               

dir of an "_sre.SRE_Match" object not working                       2 days
       http://bugs.python.org/issue3247    amaury.forgeotdarc        
                                                                               

bug adding datetime.timedelta to datetime.date                      3 days
       http://bugs.python.org/issue3249    cjw296                    
                                                                               

references are case insensitive                                     0 days
       http://bugs.python.org/issue3251    georg.brandl              
                                                                               

str.tobytes() and bytes/bytearray.tostr()                           0 days
       http://bugs.python.org/issue3252    lemburg                   
                                                                               

Suggestion: change default behavior of __ne__                       0 days
       http://bugs.python.org/issue3254    cvp                       
                                                                               

"#define socklen_t int" in pyconfig.h                               0 days
       http://bugs.python.org/issue3257    amaury.forgeotdarc        
                                                                               

fix_imports needs to be using the 'as' keyword                      0 days
       http://bugs.python.org/issue3259    brett.cannon              
                                                                               

Lib/test/test_cookielib declares utf-8 encoding, but contains no    0 days
       http://bugs.python.org/issue3261    brett.cannon              
                                                                               

Odd code fragment in ABC definitions                                0 days
       http://bugs.python.org/issue3263    gvanrossum                
                                                                               

Use -lcrypto instead of -lcrypt on Solaris 2.6 when available       0 days
       http://bugs.python.org/issue3264    mmokrejs                  
                                                                               

yield in list comprehensions possibly broken in 3.0                 0 days
       http://bugs.python.org/issue3267    brett.cannon              
                                                                               

strptime() makes an error concerning second in arg                  0 days
       http://bugs.python.org/issue3269    nevgor                    
                                                                               

iter.next() or iter.__next__() ?                                    0 days
       http://bugs.python.org/issue3271    benjamin.peterson         
                                                                               

Control flow not optimized                                          1 days
       http://bugs.python.org/issue3275    georg.brandl              
                                                                               

support r"\"                                                        0 days
       http://bugs.python.org/issue3281    facundobatista            
                                                                               

Undefined unicode characters should be non-printable                0 days
       http://bugs.python.org/issue3282    georg.brandl              
                                                                               

rlcompleter add "(" to callables feature                         2520 days
       http://bugs.python.org/issue449227  facundobatista            
       patch, easy                                                             

Docs don't define sequence-ness very well                        1979 days
       http://bugs.python.org/issue678464  benjamin.peterson         
                                                                               

asyncore.dispactcher: incorrect connect                          1613 days
       http://bugs.python.org/issue889153  josiahcarlson             
                                                                               

catch invalid chunk length in httplib read routine               1590 days
       http://bugs.python.org/issue900744  pdorrell                  
       patch                                                                   

asyncore fixes and improvements                                  1583 days
       http://bugs.python.org/issue909005  josiahcarlson             
       patch                                                                   

asyncore.file_dispatcher should not take fd as argument          1393 days
       http://bugs.python.org/issue1025525 josiahcarlson             
                                                                               

asyncore should handle ECONNRESET in send                        1331 days
       http://bugs.python.org/issue1063924 josiahcarlson             
                                                                               

Add notes to the manual about `is` and methods                    893 days
       http://bugs.python.org/issue1410739 georg.brandl              
       patch, easy                                                             

2.4.2 file.read caches EOF state                                  715 days
       http://bugs.python.org/issue1523853 georg.brandl              
                                                                               

asyncore/asynchat patches                                         387 days
       http://bugs.python.org/issue1736190 josiahcarlson             
       patch                                                                   

asynchat should call "handle_close"                               379 days
       http://bugs.python.org/issue1740572 josiahcarlson             
                                                                               



Top Issues Most Discussed (10)
______________________________

 35 test_multiprocessing hangs on OS X 10.5.3                          2 days
open    http://bugs.python.org/issue3088   

 24 math test fails on Solaris 10                                     13 days
open    http://bugs.python.org/issue3167   

 18 cmath test fails on Solaris 10                                    13 days
open    http://bugs.python.org/issue3168   

 11 Let bin/oct/hex show floats                                       10 days
open    http://bugs.python.org/issue3008   

 10 Use -lcrypto instead of -lcrypt on Solaris 2.6 when available      0 days
closed  http://bugs.python.org/issue3264   

 10 __eq__ / __hash__ check doesn't take inheritance into account    122 days
open    http://bugs.python.org/issue2235   

  8 Segfault in PyFile_SoftSpace/PyEval_EvalFrameEx with sys.stdout    1 days
closed  http://bugs.python.org/issue3242   

  8 "Quick search" box renders too long on FireFox 3                   7 days
pending http://bugs.python.org/issue3154   

  7 re.IGNORECASE not Unicode-ready                                   53 days
open    http://bugs.python.org/issue2834   

  6 ints contructed from strings don't use the smallint constants      1 days
closed  http://bugs.python.org/issue3236   




From unknown_kev_cat at hotmail.com  Sat Jul  5 01:20:34 2008
From: unknown_kev_cat at hotmail.com (Joe Smith)
Date: Fri, 4 Jul 2008 23:20:34 +0000 (UTC)
Subject: [Python-Dev] UCS2/UCS4 default
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>	<g4j35h$i4p$1@ger.gmane.org>	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>	<g4jenq$rks$1@ger.gmane.org>
	<aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>
	<486D5355.4000104@v.loewis.de>
Message-ID: <loom.20080704T225603-695@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:

> 
> > Wrong term - code units and code points are equivalent in UTF-16 and
> > UTF-32.  What you're looking for is unicode scalar values.
> 
> How so? Section 2.5, UTF-16 says
> 
> "code points in the supplementary planes, in the range
> U+10000..U+10FFFF, are represented as pairs of 16-bit code units."
> 
> So clearly, code points in Unicode range from U+0000..U+10FFFF,
> independent of encoding form.
> 
> In UTF-16, code units range from 0..65535.
> 
> OTOH, "unicode scalar value" is nearly synonymous to "code point":
> 
> D76 Unicode Scalar Value. Any Unicode  code point except high-surrogate
> and low-surrogate code points.
> 
> So codepoint in Terry's message was the right term.
> 

No Terry did definitely mean Unicode scalar values. He was describing the "pure"
but impractical "len()" that would count a surrogate pair as "1", not 2, even in
the 32-bit builds.

For what it is worth:
Code point: a number between 0 and 1114111.
Scalar Value: a code point, except the surrogate code points.
Code unit: The basic unit of the encoding. One code unit is always sufficient to
encode some Unicode Scalar values. However, other Unicode scalar values may
require multiple Code units.

Note that a scalar value is a code point. A code point may or may not be a
scalar value. 

Practical len() returns the number of code units of the internal storage format.
Pure len() allegedly would return the number of Unicode scalar values (obviously
a surrogate pair would be considered a single Unicode scalar value).

Please keep in mind that encodings encode Unicode scalar values. Thus a utf-8
code unit sequence (or UTF-32 code unit) that would give a code point in the
surrogate sections is technically in error. (Although python would do well to
ignore this restriction as there may be valid reasons to have a utf-8 sequence
that is not a valid encoded Unicode text sequence)


From martin at v.loewis.de  Sat Jul  5 07:35:18 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 05 Jul 2008 07:35:18 +0200
Subject: [Python-Dev] UCS2/UCS4 default
In-Reply-To: <loom.20080704T225603-695@post.gmane.org>
References: <20080702141328.GW62693@nexus.in-nomine.org>	<ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com>	<20080703104813.GF62693@nexus.in-nomine.org>	<486CC881.5090902@gmail.com>	<e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com>	<031901c8dd12$b7258b60$2570a220$@com.au>	<e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com>	<g4j35h$i4p$1@ger.gmane.org>	<ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com>	<g4jenq$rks$1@ger.gmane.org>	<aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com>	<486D5355.4000104@v.loewis.de>
	<loom.20080704T225603-695@post.gmane.org>
Message-ID: <486F0816.8070707@v.loewis.de>


>> The premise is the OP's idea that Python should switch to all UCS4 to
>> create a more pure ('ideal') situation or the idea that len(s) should
>> count codepoints (correct term?) for all builds as a matter of purity
>> even though on it would be time-costly on 16-bit builds as a matter
>> of practicality.

> No Terry did definitely mean Unicode scalar values.

True. However, using the word "code point" to refer to "Unicode scalar
values" is also correct. He (rather, the OP) wanted to count code
points (i.e. not count code units).

> Practical len() returns the number of code units of the internal storage format.

No, it returns the number of code units.

> Pure len() allegedly would return the number of Unicode scalar values (obviously
> a surrogate pair would be considered a single Unicode scalar value).

Perhaps-not-so-obviously-but-still-intendended, a pure len counting
surrogate pairs as one would *also* count code points.

> Please keep in mind that encodings encode Unicode scalar values.

A "coded character set" is "a character set in which each character is
assigned a numeric code point". So clearly, a character encoding form
encodeds code points.

> Thus a utf-8
> code unit sequence (or UTF-32 code unit) that would give a code point in the
> surrogate sections is technically in error. 

Sure, but this has nothing to do with Terry's terminology use.

Regards,
Martin

From dickinsm at gmail.com  Sat Jul  5 11:39:34 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 5 Jul 2008 10:39:34 +0100
Subject: [Python-Dev] C99 code in the Python core?
Message-ID: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com>

I have a general question and a specific question.  First the general one:

(1) When is it okay to use C99 code in the Python core?  More particularly,
is it considered acceptable to use widely-implemented library functions that
are specified in C99 but not ANSI C, or widely-implemented features that
are new to C99?

Or is C99 code now acceptable pretty much anywhere?  If so, should
PEP 7 be updated?  It currently says: """Use ANSI/ISO standard C
(the 1989 version of the standard)."""

I think there are some C99 features that still aren't implemented
everywhere, even on major platforms.  (Examples are the inverse hyperbolic
trig functions in math.h.)

And the specific question:

(2) Is it okay to use the '%a' format specifier for sprintf, sscanf and friends.
Are there major platforms where this isn't implemented?  (Using
'%a' would make the issue implementation much simpler.)

Mark

From matthieu.brucher at gmail.com  Sat Jul  5 11:59:13 2008
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Sat, 5 Jul 2008 11:59:13 +0200
Subject: [Python-Dev] C99 code in the Python core?
In-Reply-To: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com>
References: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com>
Message-ID: <e76aa17f0807050259xf2ae911v3100af01606cd450@mail.gmail.com>

2008/7/5 Mark Dickinson <dickinsm at gmail.com>:
> I have a general question and a specific question.  First the general one:
>
> (1) When is it okay to use C99 code in the Python core?  More particularly,
> is it considered acceptable to use widely-implemented library functions that
> are specified in C99 but not ANSI C, or widely-implemented features that
> are new to C99?
>
> Or is C99 code now acceptable pretty much anywhere?  If so, should
> PEP 7 be updated?  It currently says: """Use ANSI/ISO standard C
> (the 1989 version of the standard)."""
>
> I think there are some C99 features that still aren't implemented
> everywhere, even on major platforms.  (Examples are the inverse hyperbolic
> trig functions in math.h.)

Hi,

I don't think that C99 is not supported by Visual Studio and there are
no plan for Microsoft to do so.

Matthieu
-- 
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher

From martin at v.loewis.de  Sat Jul  5 12:46:53 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Jul 2008 12:46:53 +0200
Subject: [Python-Dev] C99 code in the Python core?
In-Reply-To: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com>
References: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com>
Message-ID: <486F511D.50809@v.loewis.de>

> (1) When is it okay to use C99 code in the Python core?  More particularly,
> is it considered acceptable to use widely-implemented library functions that
> are specified in C99 but not ANSI C, or widely-implemented features that
> are new to C99?

[C99 is also ANSI C, IIUC. ANSI has adopted ISO/IEC 9899:1999 as a U.S.
 national standard.]

It's ok to use functions of the C99 standard library if you have a
configure test and a fall-back implementation, or if you know that the
function is available on all systems we care about.

> Or is C99 code now acceptable pretty much anywhere?

No. As others have pointed out, Microsoft still hasn't implemented in
Visual C.

> If so, should
> PEP 7 be updated?  It currently says: """Use ANSI/ISO standard C
> (the 1989 version of the standard)."""

No.

> (2) Is it okay to use the '%a' format specifier for sprintf, sscanf and friends.
> Are there major platforms where this isn't implemented?  (Using
> '%a' would make the issue implementation much simpler.)

It's implemented in VS 2008, see

http://msdn.microsoft.com/en-us/library/hf4y5e3w.aspx

On the other hand, people still might try to run Python on older
versions of Solaris, such as Solaris 2.6 (which was released 1997).
I don't know when Solaris' CRT first started to support this.

I'd add a configure test, and, at run-time, raise an exception
if the C library doesn't support it yet somebody tries to use it.

Regards,
Martin

From greg at krypto.org  Sat Jul  5 21:00:35 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 5 Jul 2008 12:00:35 -0700
Subject: [Python-Dev] [Python-3000]  Second betas tomorrow
In-Reply-To: <D8E48735-E192-486B-A3D3-CD50C5F99DBF@python.org>
References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org>
	<ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com>
	<A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org>
	<9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1>
	<ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com>
	<8A43F3E7-BEE8-4656-833D-867328D16D52@python.org>
	<bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com>
	<79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org>
	<1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com>
	<D8E48735-E192-486B-A3D3-CD50C5F99DBF@python.org>
Message-ID: <52dc1c820807051200v9bb3673g967aa6d596520f6a@mail.gmail.com>

On Tue, Jul 1, 2008 at 8:29 PM, Barry Warsaw <barry at python.org> wrote:

> On Jul 1, 2008, at 10:42 PM, Benjamin Peterson wrote:
>
>> On Tue, Jul 1, 2008 at 8:44 PM, Barry Warsaw <barry at python.org> wrote:
>>
>>> On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote:
>>>
>>>>
>>>> Is a Google Calendar kept by anyone that lists stuff like planned
>>>> release dates, etc.?
>>>>
>>>
>>>
>>> http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics
>>>
>>
>> Can I get the non-iCal version?
>>
>
>
> http://www.google.com/calendar/feeds/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic
>
>
> http://www.google.com/calendar/embed?src=b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com&ctz=America/New_York
>
> - -Barry
>
>
And for anyone who hasn't already figured it out.. you can just add
b6v58qvojllt0i6ql654r1vh00 at group.calendar.google.com<http://www.google.com/calendar/embed?src=b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com&ctz=America/New_York>as
a friend in your existing google calendar to see the release schedule
calendar alongside your own.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080705/ed3611be/attachment.htm>

From solipsis at pitrou.net  Sat Jul  5 21:10:37 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 5 Jul 2008 19:10:37 +0000 (UTC)
Subject: [Python-Dev] bytearray and array.array are not thread-safe
Message-ID: <loom.20080705T185033-556@post.gmane.org>


Hello,

Short story: bytearray and array.array by construction allow user code to
reallocate their internal memory buffer. But a raw pointer to the said buffer
can also be obtained by another thread, and used after releasing the GIL (for
CPU-intensive operations like compression). As a consequence, the interpreter
crashes.

Was it envisioned? I see no warning in the docs for the array.array type
(although it has been there for quite some time).

See http://bugs.python.org/issue3139 (reported by Amaury)

Regards

Antoine.



From martin at v.loewis.de  Sun Jul  6 00:28:43 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Jul 2008 00:28:43 +0200
Subject: [Python-Dev] bytearray and array.array are not thread-safe
In-Reply-To: <loom.20080705T185033-556@post.gmane.org>
References: <loom.20080705T185033-556@post.gmane.org>
Message-ID: <486FF59B.6090609@v.loewis.de>

> Short story: bytearray and array.array by construction allow user code to
> reallocate their internal memory buffer. But a raw pointer to the said buffer
> can also be obtained by another thread, and used after releasing the GIL (for
> CPU-intensive operations like compression). As a consequence, the interpreter
> crashes.
> 
> Was it envisioned? 

I guess this wasn't considered. For t#, there is a comment from
Travis that it really shouldn't release the buffer yet, but it does,
anyway.

I propose that new codes s*, t*, w* are added, and that s#,t#,w# refuses
objects which implement a releasebuffer procedure (alternatively, s# etc
might be removed altogether right away). Users of s* then need to pass
in a Py_Buffer view pointer that gets filled, and need to explicitly
release the buffer. For convenience, it might help if the Py_buffer
structure includes a borrowed PyObject* to the underlying object, along
with a PyBuffer_Release procedure/macro.

Regards,
Martin

From grig.gheorghiu at gmail.com  Sun Jul  6 03:02:31 2008
From: grig.gheorghiu at gmail.com (Grig Gheorghiu)
Date: Sat, 5 Jul 2008 18:02:31 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
Message-ID: <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>

On Thu, Jun 26, 2008 at 8:18 AM,  <glyph at divmod.com> wrote:
> Today on planetpython.org, Doug Hellman announced the June issue of Python
> magazine.  The cover story this month is about Pybots, "the fantastic
> automation system that has been put in place to make sure new releases of
> Python software are as robust and stable as possible".
>
> Last week, there was a "beta" release of Python which, according to the
> community buildbots, cannot run any existing python software.  Normally I'd
> be complaining here about Twisted, but in fact Twisted is doing relatively
> well right now; only 80 failing tests.  Django apparently cannot even be
> imported.
>
> The community buildbots have been in a broken state for months now[1]. I've
> been restraining myself from whinging about this, but now that it's getting
> close to release, it's time to get these into shape, or it's time to get rid
> of them.

Hi all,

Sorry for not replying sooner, I was on vacation when this thread
started and I only got back in town yesterday.

To bring my $0.02 to this discussion: the Pybots 'community buildbots'
turned out largely to be a failure. Why? Because there was never
really a 'community' around it, especially a community of project
leaders who would be interested in the state of their projects' tests.
All the machines donated for the Pybots farm belong to people who just
happen to be interested in given projects, but are not really the
leaders of those projects. The only project who constantly stayed on
top of the buildbot status was Twisted, represented by JP Calderone
(although even there the tests were running on my machine, and not on
a machine contributed by the Twisted folks.)

I still haven't given up, and I hope this thread will spur project
leaders into donating time, or resources, to the Pybots project. It
has been my bitter observation about the Open Source world that people
just LOVE to get stuff for free. As soon as you mention more
involvement from them in the form of time, money, hardware resources,
etc., the same brave proponents of cool things to be done are nowhere
to be found.

To come back to this thread, I don't think it's reasonable to expect
the Python core developers to be that interested in the status of the
community buildbots. It is again up to the project leaders to step up
to the plate, donate machines to Pybots, and stay on top of any
breakages that result from Python core checkins. It seems to me that
the Python core developers have always responded promptly and
favorably to reports of breakages coming from the Pybots farm.

Grig

From josiah.carlson at gmail.com  Sun Jul  6 03:52:26 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Sat, 5 Jul 2008 18:52:26 -0700
Subject: [Python-Dev] Packing and unpacking integers
Message-ID: <e6511dbf0807051852i34226efpf3de82ce1ff2599a@mail.gmail.com>

A few years ago (yes, it's been that long), I proposed adding a new
format code to struct that would pack integers as strings, similar to
the 's' format code.  In particular, struct.pack('>60G', v) would be a
60-byte big-endian unsigned integer as a string.  The feature request
is http://bugs.python.org/issue1023290 .

Shortly thereafter, it was decided that it wouldn't become a struct
format code, but instead would find itself as part of binhex.  Raymond
Hettinger was supposed to write the function a couple years ago for, I
believe, Python 2.4 .  It never happened.  It still hasn't happened
for Python 2.5 or 2.6 .

I believe there is still a need for packing integers as strings and
unpacking strings as integers, more specifically, offering to Python
an interface to _PyLong_FromByteArray() and _PyLong_AsByteArray().  I
would be happy to write the functionality and unittests this coming
week for 2.6 and 3.0 if I get the ok.  If not, I can write it for 2.7
and 3.1 .

 - Josiah

From solipsis at pitrou.net  Sun Jul  6 13:17:23 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 06 Jul 2008 13:17:23 +0200
Subject: [Python-Dev] bytearray and array.array are not thread-safe
In-Reply-To: <486FF59B.6090609@v.loewis.de>
References: <loom.20080705T185033-556@post.gmane.org>
	<486FF59B.6090609@v.loewis.de>
Message-ID: <1215343043.5983.7.camel@fsol>

Le dimanche 06 juillet 2008 ? 00:28 +0200, "Martin v. L?wis" a ?crit :
> I propose that new codes s*, t*, w* are added, and that s#,t#,w# refuses
> objects which implement a releasebuffer procedure (alternatively, s# etc
> might be removed altogether right away). Users of s* then need to pass
> in a Py_Buffer view pointer that gets filled, and need to explicitly
> release the buffer. For convenience, it might help if the Py_buffer
> structure includes a borrowed PyObject* to the underlying object, along
> with a PyBuffer_Release procedure/macro.

Why a borrowed reference rather than a new one? It could be decref'ed as
part as the proposed PyBuffer_Release procedure.

Overall it sounds like a clean resolution of the problem.

Regards

Antoine.



From glyph at divmod.com  Sun Jul  6 17:46:30 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Sun, 06 Jul 2008 15:46:30 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
Message-ID: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>

On 01:02 am, grig.gheorghiu at gmail.com wrote:
>To bring my $0.02 to this discussion: the Pybots 'community buildbots'
>turned out largely to be a failure.

Let's not say it's a failure.  Let's instead say that it hasn't yet 
become a success :-).
>I still haven't given up, and I hope this thread will spur project
>leaders into donating time, or resources, to the Pybots project. It
>has been my bitter observation about the Open Source world that people
>just LOVE to get stuff for free. As soon as you mention more
>involvement from them in the form of time, money, hardware resources,
>etc., the same brave proponents of cool things to be done are nowhere
>to be found.

I think this list is the wrong place to go to reach the people who need 
to be reached.  It's python core developers and other people already 
involved in and aware of core development.  That said I'm not sure what 
the *right* place is; I think your blog is syndicated on the unofficial 
planet python, so maybe that's a good place to start.  Sadly, the right 
thing to do in terms of drumming up support is to get someone interested 
in PR and have them go to each project individually, but that might be 
more effort than setting up the buildbots themselves, at least 
initially...

However, let's say that this were tremendously successful, and lots of 
people start paying attention.  I think pybots.org needs to be updated 
to say exactly what a participant interested in python testing needs to 
do, beyond "here's how you set up a buildbot" (a page that is actually a 
daunting-looking blog post which admits it may be somewhat outdated), 
because setting up a buildbot might not be the only thing that the 
project needs.  It's one thing to tell people that they need to be 
helping out (and I'm sure you're right) but it's much more useful to get 
the message out that "we really need people to do X, Y, and Z".  One 
thing I will definitely commit to is that if you make a "cry for help" 
page, I'll blog about it to drive attention to it, and I'll encourage 
the other, perhaps better-read Python bloggers I know to do so as well.

My personal interest at the moment is to get all of the irrelevant red 
off of the community builders page.  Whether or not you believe in an XP 
"green bar" philosophy, the large number of spurious failures is 
distracting.  Who is it that is capable of making appropriate changes? 
Is there something I could do to help with that?  Note that I'm 
committing to say that I can do *that*, but, at least you could shut me 
up by making it my fault ;-).

(I'd also like to improve the labels of the build slaves.  What exactly 
is "x86 Red Hat 9 trunk" testing?  Trunk of what?  What project?)

It would be good to remove the perception that it's somebody else's 
problem as much as possible.  Right now, all these dead buildbots 
suggest to the various communities, "oh, I guess that guy who runs that 
buildbot needs to fix it".  The dead bots should just be killed off, and 
their projects removed from the list, so that if someone wants to get 
involved and set up a bot for lxml, they're not put off of it by the 
fact that it might be rude to the guy who is currently (allegedly) 
running it.

From martin at v.loewis.de  Sun Jul  6 19:25:34 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 06 Jul 2008 19:25:34 +0200
Subject: [Python-Dev] bytearray and array.array are not thread-safe
In-Reply-To: <1215343043.5983.7.camel@fsol>
References: <loom.20080705T185033-556@post.gmane.org>	<486FF59B.6090609@v.loewis.de>
	<1215343043.5983.7.camel@fsol>
Message-ID: <4871000E.2060800@v.loewis.de>

> Why a borrowed reference rather than a new one? It could be decref'ed as
> part as the proposed PyBuffer_Release procedure.

The question is
a) whether a Py_Buffer remains valid even if the object goes away.
   That seems not to be the case, i.e. the caller of getbuffer needs
   to hold onto the object, anyway.
b) whether it would still be correct to call releasebuffer explicitly.
   Of course, as getbuffer would have to fill the object into the view,
   releasebuffer could also DECREF the included object.
   Alternatively, there could be a pair of functions PyBuffer_Get and
   PyBuffer_Release, which would fill the object into the view itself.

So I withdraw issue b; the real question remains whether it is desired
that a buffer will remain alive as long as there is a view to it.
That is a question for the buffer experts to answer; it may also have
impacts on cyclic garbage collection (as inclusion of a view into an
object will mean that the tp_traverse function must also Py_VISIT the
embedded object).

> Overall it sounds like a clean resolution of the problem.

Unfortunately, it's also a significant change at this point. I
personally won't have time to provide a patch, but I think a patch
is needed before the last beta. IOW, the issue should become a
release blocker.

Regards,
Martin

From stephen at xemacs.org  Sun Jul  6 19:40:14 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 07 Jul 2008 02:40:14 +0900
Subject: [Python-Dev] Community buildbots and Python
	release	quality	metrics
In-Reply-To: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
Message-ID: <87y74enc75.fsf@uwakimon.sk.tsukuba.ac.jp>

glyph at divmod.com writes:
 > On 01:02 am, grig.gheorghiu at gmail.com wrote:
 > >To bring my $0.02 to this discussion: the Pybots 'community buildbots'
 > >turned out largely to be a failure.
 > 
 > Let's not say it's a failure.  Let's instead say that it hasn't yet 
 > become a success :-).

+1

 > >I still haven't given up, and I hope this thread will spur project
 > >leaders into donating time, or resources, to the Pybots project.

 > I think this list is the wrong place to go to reach the people who need 
 > to be reached.  It's python core developers and other people already 
 > involved in and aware of core development.  That said I'm not sure what 
 > the *right* place is; I think your blog is syndicated on the unofficial 
 > planet python, so maybe that's a good place to start.  Sadly, the right 
 > thing to do in terms of drumming up support is to get someone interested 
 > in PR and have them go to each project individually, but that might be 
 > more effort than setting up the buildbots themselves, at least 
 > initially...

Exactly, and that's why nobody should be "bitter" about it.  The
problem is that the while overall the effort and rewards look to be
balanced in favor of the rewards, the startup costs for individuals
are quite high.

I think this *is* the place to start, though.  The project leaders
"should" be, and probably generally are, "here".  They have the
strongest interest in any individual 'bot, while Guido is quite
correct in saying python-dev can't afford to have strong interest in
all the bots.

 > However, let's say that this were tremendously successful, and lots of 
 > people start paying attention.  I think pybots.org needs to be updated 
 > to say exactly what a participant interested in python testing needs to 
 > do, beyond "here's how you set up a buildbot" (a page that is actually a 
 > daunting-looking blog post which admits it may be somewhat outdated), 
 > because setting up a buildbot might not be the only thing that the 
 > project needs.  It's one thing to tell people that they need to be 
 > helping out (and I'm sure you're right) but it's much more useful to get 
 > the message out that "we really need people to do X, Y, and Z".  One 
 > thing I will definitely commit to is that if you make a "cry for help" 
 > page, I'll blog about it to drive attention to it, and I'll encourage 
 > the other, perhaps better-read Python bloggers I know to do so as
 > well.

Two suggestions in this vein: First, I think it's established that
some but not all "red community bots" *are* of interest to Python core
development.  While I'm not aware of the technical details, I estimate
that triaging the community 'bot failures is probably similar to
reviewing bugs in the Python tracker.  Perhaps Martin van Loewis and
others who have offered the 5-for-1 review deal would be willing to
extend the definition of "review" to include initial bug reports based
on a red community bot (ie, you review the community 'bot failure and
decide it is something that should concern Python core development).
Perhaps that's not appropriate, but a similar system could be set up.

Second, something intermediate between the occasional half-hour of
triaging bugs and a full-blown PR campaign at the projects would be
documenting the criteria for reporting a failure on a community 'bot
to the Python tracker as a bug, etc.  This would also serve as a basis
for talking to project lurker who might have the odd half-hour to do
some "red bot" triaging.  (By criteria I mean the kinds of things that
Python core considers necessary breakage in new versions that
downstream must address in their own code, vs. regressions in a x.y.z
patchlevel, etc.  The kind of thing that glyph and Guido were
discussing earlier.)

 > It would be good to remove the perception that it's somebody else's 
 > problem as much as possible.

To the extent that a 'bot is running prerelease project against
prerelease Python, this is probably not very doable.  If Python is
stable and the project version is prerelease, it's the project's bug
until proven otherwise, and vice versa.  If both are stable, again
some expertise is probably needed for triage.

I guess that means that one important task is to classfy the bots in a
two-by-two matrix according to stability of project and Python
respectively.


From martin at v.loewis.de  Sun Jul  6 19:29:34 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Jul 2008 19:29:34 +0200
Subject: [Python-Dev] Community buildbots and Python
	release	quality	metrics
In-Reply-To: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
Message-ID: <487100FE.508@v.loewis.de>

> (I'd also like to improve the labels of the build slaves.  What exactly
> is "x86 Red Hat 9 trunk" testing?  Trunk of what?  What project?)

It seems like you would like to edit the master configuration file.
That can be arranged fairly easily.

Regards,
Martin

From martin at v.loewis.de  Sun Jul  6 19:34:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Jul 2008 19:34:06 +0200
Subject: [Python-Dev] Packing and unpacking integers
In-Reply-To: <e6511dbf0807051852i34226efpf3de82ce1ff2599a@mail.gmail.com>
References: <e6511dbf0807051852i34226efpf3de82ce1ff2599a@mail.gmail.com>
Message-ID: <4871020E.5070802@v.loewis.de>

> I believe there is still a need for packing integers as strings and
> unpacking strings as integers, more specifically, offering to Python
> an interface to _PyLong_FromByteArray() and _PyLong_AsByteArray().  I
> would be happy to write the functionality and unittests this coming
> week for 2.6 and 3.0 if I get the ok.  If not, I can write it for 2.7
> and 3.1 .

I think it needs to be deferred to the next releases, given that the
beta release already happened.

If you have any spare time, please look into some of the real serious,
release-blocking bug reports.

Regards,
Martin

From grig.gheorghiu at gmail.com  Sun Jul  6 23:09:37 2008
From: grig.gheorghiu at gmail.com (Grig Gheorghiu)
Date: Sun, 6 Jul 2008 14:09:37 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
Message-ID: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>

On Sun, Jul 6, 2008 at 8:46 AM,  <glyph at divmod.com> wrote:
>
> However, let's say that this were tremendously successful, and lots of
> people start paying attention.  I think pybots.org needs to be updated to
> say exactly what a participant interested in python testing needs to do,
> beyond "here's how you set up a buildbot" (a page that is actually a
> daunting-looking blog post which admits it may be somewhat outdated),
> because setting up a buildbot might not be the only thing that the project
> needs.  It's one thing to tell people that they need to be helping out (and
> I'm sure you're right) but it's much more useful to get the message out that
> "we really need people to do X, Y, and Z".  One thing I will definitely
> commit to is that if you make a "cry for help" page, I'll blog about it to
> drive attention to it, and I'll encourage the other, perhaps better-read
> Python bloggers I know to do so as well.

I have posted 'cries for help' repeatedly on my blog, with generally
little success. See http://agiletesting.blogspot.com/search?q=pybots .
But I will post again. If you can include here a paragraph of what
your vision of the 'X, Y and Z' above is, that'd help too. I think
I've been pretty clear about the benefits that the Pybots farm can
bring to a given project, so all project leaders on this list should
be aware of them IMO. If not, I'd be happy to rehash them. But the
home page of pybots.org is pretty self-explanatory I think.

>
> My personal interest at the moment is to get all of the irrelevant red off
> of the community builders page.  Whether or not you believe in an XP "green
> bar" philosophy, the large number of spurious failures is distracting.  Who
> is it that is capable of making appropriate changes? Is there something I
> could do to help with that?  Note that I'm committing to say that I can do
> *that*, but, at least you could shut me up by making it my fault ;-).
>

I'll send a message to the pybots mailing list asking people whose
buildbots are turned off if they're still interested in running them.
Negative or no answers will mean we can remove them from the farm.

> (I'd also like to improve the labels of the build slaves.  What exactly is
> "x86 Red Hat 9 trunk" testing?  Trunk of what?  What project?)
>

It's not only a question of changing a static label in this case. A
given buildslave can run the tests for multiple projects, in which
case it becomes tricky to change the label on the fly accordingly. As
an aside, the slave you mention was running on my machine, and I used
it to run the Twisted tests, but I shut it down a while ago because
the buildbot process was taking too many resources. If the Twisted
project can donate a machine, I'd be happy to include it in the Pybots
farm ASAP.

> It would be good to remove the perception that it's somebody else's problem
> as much as possible.  Right now, all these dead buildbots suggest to the
> various communities, "oh, I guess that guy who runs that buildbot needs to
> fix it".  The dead bots should just be killed off, and their projects
> removed from the list, so that if someone wants to get involved and set up a
> bot for lxml, they're not put off of it by the fact that it might be rude to
> the guy who is currently (allegedly) running it.

As I said, I'll see what the current owners have to say, and then I'll
report back to this list.

Thanks for offering your help!

Grig

From victor.stinner at haypocalc.com  Mon Jul  7 01:11:52 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 7 Jul 2008 01:11:52 +0200
Subject: [Python-Dev] Play with fuzzing
Message-ID: <200807070111.52959.victor.stinner@haypocalc.com>

Hi,

I wrote a fuzzing "framework" called Fusil and this week I wrote a fuzzer for 
Python. The idea is quite simple: for a module,
 - list all functions, classes and class methods
 - call a function with random arguments (of random types)
 - instanciate a class with random arguments
 - if the class is created correctly, call methods with random arguments

Example:
--------------------- 8< -----------------------------------
print "Call 39/40: linuxaudiodev.open()"
try:
    linuxaudiodev.open(
        # argument 1/2
        u"\u62C0\uFBD7\uB46A\u55E0\uFB7B\uD392\u7CEE",
        # argument 2/2
        52.682,
    )
except Exception, err:
    print >>stderr, "ERROR: %s" % err
--------------------- 8< -----------------------------------

I tried it on CPython 2.5 and then on CPython trunk (future 2.6). I found some 
bugs, see last bug entries in Python bugtracker. Just an example:

http://bugs.python.org/issue3304 
  -> invalid call to PyMem_Free() in fileio_init()

Most bugs crash with a segmentation fault, abort or a denial of service.

If you would like to try my fuzzer, use:
 (1) svn co http://fusil.hachoir.org/svn/trunk fusil
 (2) cd fusil
 (3) ./run_fusil.sh -p projects/python.py --fast --remove ALL

The option --fast goes faster, --remove does remove session directory even if 
Python generated some files, and "ALL" test all modules.

FUSIL IS NOT SAFE! So run it under a different user using to avoid dangerous 
call to os.unlink().

The module list is hardcoded: it's the list of CPython modules written in C.

More informations about Fusil:
   http://fusil.hachoir.org/trac

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From brett at python.org  Mon Jul  7 01:33:14 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 6 Jul 2008 16:33:14 -0700
Subject: [Python-Dev] Play with fuzzing
In-Reply-To: <200807070111.52959.victor.stinner@haypocalc.com>
References: <200807070111.52959.victor.stinner@haypocalc.com>
Message-ID: <bbaeab100807061633x1c3f794and24234b39916bd4b@mail.gmail.com>

On Sun, Jul 6, 2008 at 4:11 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> I wrote a fuzzing "framework" called Fusil and this week I wrote a fuzzer for
> Python. The idea is quite simple: for a module,
>  - list all functions, classes and class methods
>  - call a function with random arguments (of random types)
>  - instanciate a class with random arguments
>  - if the class is created correctly, call methods with random arguments
>
> Example:
> --------------------- 8< -----------------------------------
> print "Call 39/40: linuxaudiodev.open()"
> try:
>    linuxaudiodev.open(
>        # argument 1/2
>        u"\u62C0\uFBD7\uB46A\u55E0\uFB7B\uD392\u7CEE",
>        # argument 2/2
>        52.682,
>    )
> except Exception, err:
>    print >>stderr, "ERROR: %s" % err
> --------------------- 8< -----------------------------------
>
> I tried it on CPython 2.5 and then on CPython trunk (future 2.6). I found some
> bugs, see last bug entries in Python bugtracker. Just an example:
>
> http://bugs.python.org/issue3304
>  -> invalid call to PyMem_Free() in fileio_init()
>

You can use

http://bugs.python.org/issue?%40search_text=&title=&%40columns=title&id=&%40columns=id&creation=&creator=haypo&activity=2008-07-06&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&%40pagesize=50&%40startwith=0&%40queryname=&%40old-queryname=&%40action=search

to see all of the bugs Victor has filed today (looks like eight).

-Brett

From martin at v.loewis.de  Mon Jul  7 06:37:10 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Jul 2008 06:37:10 +0200
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
	<3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>
Message-ID: <48719D76.7040806@v.loewis.de>

> It's not only a question of changing a static label in this case. A
> given buildslave can run the tests for multiple projects, in which
> case it becomes tricky to change the label on the fly accordingly.

I think you could set up different builders for a single slave in
that case (use a slave lock to make them run sequentially).

> As
> an aside, the slave you mention was running on my machine, and I used
> it to run the Twisted tests, but I shut it down a while ago because
> the buildbot process was taking too many resources. If the Twisted
> project can donate a machine, I'd be happy to include it in the Pybots
> farm ASAP.

Please talk to Trent Nelson. He has a Windows machine that he donated
precisely for that kind of activity.

Regards,
Martin


From martin at v.loewis.de  Mon Jul  7 06:38:48 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Jul 2008 06:38:48 +0200
Subject: [Python-Dev] Play with fuzzing
In-Reply-To: <200807070111.52959.victor.stinner@haypocalc.com>
References: <200807070111.52959.victor.stinner@haypocalc.com>
Message-ID: <48719DD8.4070807@v.loewis.de>

> I wrote a fuzzing "framework" called Fusil and this week I wrote a fuzzer for 
> Python. The idea is quite simple: for a module,
>  - list all functions, classes and class methods
>  - call a function with random arguments (of random types)
>  - instanciate a class with random arguments
>  - if the class is created correctly, call methods with random arguments

I was already wondering how you found out all these things. It's quite
amazing!

Thanks,
Martin

From solipsis at pitrou.net  Mon Jul  7 13:57:57 2008
From: solipsis at pitrou.net (Antoine)
Date: Mon, 7 Jul 2008 13:57:57 +0200 (CEST)
Subject: [Python-Dev] bytearray and array.array are not thread-safe
In-Reply-To: <4871000E.2060800@v.loewis.de>
References: <loom.20080705T185033-556@post.gmane.org>
	<486FF59B.6090609@v.loewis.de> <1215343043.5983.7.camel@fsol>
	<4871000E.2060800@v.loewis.de>
Message-ID: <2463911e698f87e837b4296cb13810ea.squirrel@webmail.nerim.net>


> Unfortunately, it's also a significant change at this point. I
> personally won't have time to provide a patch, but I think a patch
> is needed before the last beta. IOW, the issue should become a
> release blocker.

Agreed. Unfortunately I don't have much time to write a patch either.
Perhaps in one or two weeks, but it would be better if someone beats me to
it.

Regards

Antoine.



From solipsis at pitrou.net  Mon Jul  7 14:39:33 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 7 Jul 2008 12:39:33 +0000 (UTC)
Subject: [Python-Dev] buildbots
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
Message-ID: <loom.20080707T123338-370@post.gmane.org>


Hello,

As someone who could (perhaps) (potentially) provide a buildbot machine, there
are several questions which need answering before I take a decision:

- are more buildbots needed and if so, which kinds of platforms/architectures?
- for which software? Python itself? third-party apps and libraries?
- how resource-consuming is it? CPU? memory? disk space? can it run along other
services fine or does it need the whole machine for itself?
- how time-consuming is it (in terms of human work)? I may spend a bit of time
at the start to set it up but I'd like it to it run quite flawlessly afterward.
I'm really not a sysadmin at heart...

I suppose other interested people could ask themselves the same questions...

Just my 2 cents.

Antoine.



From grig.gheorghiu at gmail.com  Mon Jul  7 20:49:05 2008
From: grig.gheorghiu at gmail.com (Grig Gheorghiu)
Date: Mon, 7 Jul 2008 11:49:05 -0700
Subject: [Python-Dev] buildbots
In-Reply-To: <loom.20080707T123338-370@post.gmane.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<loom.20080707T123338-370@post.gmane.org>
Message-ID: <3f09d5a00807071149r42b7b7ddl4eb5a91d21e662b0@mail.gmail.com>

On Mon, Jul 7, 2008 at 5:39 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>
> - are more buildbots needed and if so, which kinds of platforms/architectures?

I can't really answer that question for the python code buildbot farm,
but for the Pybots community project, the platforms we currently have
are in a table on this page:

http://pybots.org/

If you are able to offer something that's not on the list, that'd be
good. But any help at all is appreciated.

I believe Windows has traditionally been under-represented in all
buildbot farms, and it's likely to stay that way...

> - for which software? Python itself? third-party apps and libraries?

For Pybots, we're testing third-party apps and libraries against
changes made to Python core. If you're interested in a 3rd party
project, and you're willing to stay on top of that project's buildbot
status, and notify both the project leaders and the Python core devs
whenever you notice an ugly breakage -- then you're exactly the kind
of guy we need on the Pybots project :-)


> - how resource-consuming is it? CPU? memory? disk space? can it run along other
> services fine or does it need the whole machine for itself?

In my experience, buildbot runs fine on newer hardware. It does
consume CPU, so if you have a slow machine, it might start impacting
your other processes.


> - how time-consuming is it (in terms of human work)? I may spend a bit of time
> at the start to set it up but I'd like it to it run quite flawlessly afterward.
> I'm really not a sysadmin at heart...

The initial learning curve can be a bit steep, but I'm here to help.
Once you add your buildslave to the buildbot farm, things should run
fairly smoothly. You will get notified via email / RSS about
breakages, and then you'll have to invest the time to see what kind of
breakage it is, and to notify the interested parties.

>
> I suppose other interested people could ask themselves the same questions...
>
> Just my 2 cents.
>
> Antoine.

Thanks for the questions, they really help IMO. I also hope the answers helped.

Grig

From grig.gheorghiu at gmail.com  Mon Jul  7 21:54:23 2008
From: grig.gheorghiu at gmail.com (Grig Gheorghiu)
Date: Mon, 7 Jul 2008 12:54:23 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
	<3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>
Message-ID: <3f09d5a00807071254x5b5ee091ye24781a1dd3afa07@mail.gmail.com>

On Sun, Jul 6, 2008 at 2:09 PM, Grig Gheorghiu <grig.gheorghiu at gmail.com> wrote:

> I'll send a message to the pybots mailing list asking people whose
> buildbots are turned off if they're still interested in running them.
> Negative or no answers will mean we can remove them from the farm.
>

OK, I posted a message to the pybots mailing list and I removed 2
slaves. Out of the 6 remaining, 4 are currently active, and one will
hopefully soon be active starting next week. This leaves just one
unanswered for so far. I also got an email from another person
volunteering a buildslave, so we'll soon have 7 machines.

As I said, if anybody else wants to participate in the Pybots project,
please let me know! I'll also post a blog entry on this soon.

Grig

From priyarp.tech at gmail.com  Mon Jul  7 22:48:45 2008
From: priyarp.tech at gmail.com (Pree Raj)
Date: Mon, 7 Jul 2008 13:48:45 -0700
Subject: [Python-Dev] __module__ not found on ported Python
Message-ID: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com>

Hi,
I am trying to port Python to ThreadX.
I have managed to get the prompt.
However when I try "import sys" or any built in module I get an error

__import__ not found.

initmain() in the Initialization code is commented out at present because of
some errors.
Could it be because of this ?

Also, I would like to know which are the MUST HAVE built in modules to be
included for normal working of my ported version of Python.

Thanks,
Priya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080707/e21c2cce/attachment.htm>

From brett at python.org  Mon Jul  7 23:15:32 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 7 Jul 2008 14:15:32 -0700
Subject: [Python-Dev] __module__ not found on ported Python
In-Reply-To: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com>
References: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com>
Message-ID: <bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com>

On Mon, Jul 7, 2008 at 1:48 PM, Pree Raj <priyarp.tech at gmail.com> wrote:
[SNIP]
> Also, I would like to know which are the MUST HAVE built in modules to be
> included for normal working of my ported version of Python.

You can look at sys.builtin_module_names to see what CPython compiles
in. Otherwise you just have to go based on what error messages say. =)

-Brett

From priyarp.tech at gmail.com  Tue Jul  8 01:39:39 2008
From: priyarp.tech at gmail.com (Pree Raj)
Date: Mon, 7 Jul 2008 16:39:39 -0700
Subject: [Python-Dev] __module__ not found on ported Python
In-Reply-To: <bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com>
References: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com>
	<bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com>
Message-ID: <8bb4faa80807071639l7879595ejb2a49affa9536442@mail.gmail.com>

Thanks Brett.
I have been able to do initmain() now.
However, if I do  "import sys" from the python prompt I still get

ImportError: __import__ not found
I am not sure where the initialization is going wrong for this error to show
up.
Can someone please help.







On Mon, Jul 7, 2008 at 2:15 PM, Brett Cannon <brett at python.org> wrote:

> On Mon, Jul 7, 2008 at 1:48 PM, Pree Raj <priyarp.tech at gmail.com> wrote:
> [SNIP]
> > Also, I would like to know which are the MUST HAVE built in modules to be
> > included for normal working of my ported version of Python.
>
> You can look at sys.builtin_module_names to see what CPython compiles
> in. Otherwise you just have to go based on what error messages say. =)
>
> -Brett
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080707/b325d8b0/attachment.htm>

From martin at v.loewis.de  Tue Jul  8 07:32:29 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Jul 2008 07:32:29 +0200
Subject: [Python-Dev] __module__ not found on ported Python
In-Reply-To: <8bb4faa80807071639l7879595ejb2a49affa9536442@mail.gmail.com>
References: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com>	<bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com>
	<8bb4faa80807071639l7879595ejb2a49affa9536442@mail.gmail.com>
Message-ID: <4872FBED.50402@v.loewis.de>

> ImportError: __import__ not found
> I am not sure where the initialization is going wrong for this error to
> show up.
> Can someone please help.

This isn't really the right list to ask for help, at least without
studying some source code prior to posting. The specific error message
is produced in ceval.c, IMPORT_NAME. Use debugging technologies
to trace through the code to find out what went wrong.

Regards,
Martin

From solipsis at pitrou.net  Tue Jul  8 11:33:02 2008
From: solipsis at pitrou.net (Antoine)
Date: Tue, 8 Jul 2008 11:33:02 +0200 (CEST)
Subject: [Python-Dev] buildbots
In-Reply-To: <3f09d5a00807071149r42b7b7ddl4eb5a91d21e662b0@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<loom.20080707T123338-370@post.gmane.org>
	<3f09d5a00807071149r42b7b7ddl4eb5a91d21e662b0@mail.gmail.com>
Message-ID: <3baff3efe0bbb965dd60c3b8a996e9f3.squirrel@webmail.nerim.net>


Hi and thanks for your answers,

> If you are able to offer something that's not on the list, that'd be
> good. But any help at all is appreciated.
>
> I believe Windows has traditionally been under-represented in all
> buildbot farms, and it's likely to stay that way...

Well what I could provide is a 32-bit x86 Debian stable. Rather common
I fear...

> For Pybots, we're testing third-party apps and libraries against
> changes made to Python core. If you're interested in a 3rd party
> project, and you're willing to stay on top of that project's buildbot
> status, and notify both the project leaders and the Python core devs
> whenever you notice an ugly breakage

Not interested /enough/ I think... by your description it sounds the
job should really be done by a core developer of each of those packages
(even if the machine is donated by someone else).

What I could be interested in is to provide a buildbot for Python itself,
but I don't know if that's needed right now. Especially for such a common
platform as a x86 Debian.

Regards

Antoine.



From jeremy at alum.mit.edu  Tue Jul  8 14:49:02 2008
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 8 Jul 2008 08:49:02 -0400
Subject: [Python-Dev] [Python-checkins] buildbot failure in sparc Debian
	3.0
In-Reply-To: <20080702202345.DAD571E400A@bag.python.org>
References: <20080702202345.DAD571E400A@bag.python.org>
Message-ID: <e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com>

Does anyone have a clue about why this test fails only on this
platform?  The test is question is verifying that URLError gets
raised.  From the traceback, it appears that there is an uncaught
exception (URLError) but it fails in an assertRaises() check for
URLError.  That doesn't make much sense unless the variable URLError
refers to different objects, but both modules use "from urllib.error
import URLError".  And, of course, the test works fine on other
platforms.

Jeremy

On Wed, Jul 2, 2008 at 4:23 PM,  <buildbot at python.org> wrote:
> The Buildbot has detected a new failure of sparc Debian 3.0.
> Full details are available at:
>  http://www.python.org/dev/buildbot/all/sparc%20Debian%203.0/builds/330
>
> Buildbot URL: http://www.python.org/dev/buildbot/all/
>
> Buildslave for this Build: klose-debian-sparc
>
> Build Reason:
> Build Source Stamp: [branch branches/py3k] HEAD
> Blamelist: benjamin.peterson
>
> BUILD FAILED: failed test
>
> Excerpt from the test logfile:
> 1 test failed:
>    test_urllib2
>
> ======================================================================
> ERROR: test_badly_named_methods (test.test_urllib2.OpenerDirectorTests)
> ----------------------------------------------------------------------
>
> Traceback (most recent call last):
>  File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/test/test_urllib2.py", line 408, in test_badly_named_methods
>    self.assertRaises(URLError, o.open, scheme+"://example.com/")
>  File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/unittest.py", line 311, in failUnlessRaises
>    callableObj(*args, **kwargs)
>  File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 356, in open
>    response = self._open(req, data)
>  File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 379, in _open
>    'unknown_open', req)
>  File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 334, in _call_chain
>    result = func(*args)
>  File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 1102, in unknown_open
>    raise URLError('unknown url type: %s' % type)
> urllib.error.URLError: <urlopen error unknown url type: do>
>
> make: *** [buildbottest] Error 1
>
> sincerely,
>  -The Buildbot
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From martin at v.loewis.de  Tue Jul  8 21:15:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Jul 2008 21:15:08 +0200
Subject: [Python-Dev] [Python-checkins] buildbot failure in sparc Debian
 3.0
In-Reply-To: <e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com>
References: <20080702202345.DAD571E400A@bag.python.org>
	<e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com>
Message-ID: <4873BCBC.9040104@v.loewis.de>

Jeremy Hylton wrote:
> Does anyone have a clue about why this test fails only on this
> platform?  The test is question is verifying that URLError gets
> raised.  From the traceback, it appears that there is an uncaught
> exception (URLError) but it fails in an assertRaises() check for
> URLError.  That doesn't make much sense unless the variable URLError
> refers to different objects, but both modules use "from urllib.error
> import URLError".  And, of course, the test works fine on other
> platforms.

It might be a transient error, and a complete cleanup of the tree
might fix it. To do so, build a non-existent branch through the web ui,
then build the original branch again; this will cause a fresh checkout.

If the error then persists, my guess it's some kind of compiler issue,
which can be investigated only with access to the machine.

Regards,
Martin

From glyph at divmod.com  Tue Jul  8 21:30:04 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Tue, 08 Jul 2008 19:30:04 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <487100FE.508@v.loewis.de>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
	<487100FE.508@v.loewis.de>
Message-ID: <20080708193004.25821.1534535725.divmod.xquotient.12490@joule.divmod.com>

On 6 Jul, 05:29 pm, martin at v.loewis.de wrote:
>>(I'd also like to improve the labels of the build slaves.  What 
>>exactly
>>is "x86 Red Hat 9 trunk" testing?  Trunk of what?  What project?)
>
>It seems like you would like to edit the master configuration file.
>That can be arranged fairly easily.

How shall we arrange it, then? :)

Whoever is interested, I've got a recent SSH key on 
https://launchpad.net/~glyph/+sshkeys

From glyph at divmod.com  Tue Jul  8 21:56:56 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Tue, 08 Jul 2008 19:56:56 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
	<3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>
Message-ID: <20080708195656.25821.1158092892.divmod.xquotient.12536@joule.divmod.com>

On 6 Jul, 09:09 pm, grig.gheorghiu at gmail.com wrote:
>On Sun, Jul 6, 2008 at 8:46 AM,  <glyph at divmod.com> wrote:
>>
>>It's one thing to tell people that they need to be helping out (and
>>I'm sure you're right) but it's much more useful to get the message 
>>out that
>>"we really need people to do X, Y, and Z".

>I have posted 'cries for help' repeatedly on my blog, with generally
>little success. See http://agiletesting.blogspot.com/search?q=pybots .
>But I will post again. If you can include here a paragraph of what
>your vision of the 'X, Y and Z' above is, that'd help too

It looks to me like the main thing that Pybots needs is help with 
maintenance.  Getting someone to set up an individual buildbot is one 
thing, but keeping it working is the important bit and it looks like 
people are not doing that.  The project would also be greatly aided by 
having dedicated people diagnose errors, report bugs against Python core 
if they're real and report bugs against Pybots if they're spurious.

It would be good to have this effort be centralized and directed because 
it would otherwise be too easy to file duplicate bug reports, or to 
assume "oh, this has been failing for months, someone must have filed a 
bug already".
>It's not only a question of changing a static label in this case. A
>given buildslave can run the tests for multiple projects, in which
>case it becomes tricky to change the label on the fly accordingly.

I'm a bit confused about how the projects being tested changes on the 
fly... but then, this level of specifics is probably best left to the 
pybots mailing list.  Hopefully sometime soon I'll have the time to add 
yet another subscription.

Thanks for cleaning up the buildbots though!  I can see a lot more tests 
actually running now :).

From grig.gheorghiu at gmail.com  Tue Jul  8 22:47:28 2008
From: grig.gheorghiu at gmail.com (Grig Gheorghiu)
Date: Tue, 8 Jul 2008 13:47:28 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080708195656.25821.1158092892.divmod.xquotient.12536@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com>
	<20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com>
	<3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com>
	<20080708195656.25821.1158092892.divmod.xquotient.12536@joule.divmod.com>
Message-ID: <3f09d5a00807081347x631b7ca3w37effe6e75f6fe6@mail.gmail.com>

On Tue, Jul 8, 2008 at 12:56 PM,  <glyph at divmod.com> wrote:
> It looks to me like the main thing that Pybots needs is help with
> maintenance.  Getting someone to set up an individual buildbot is one thing,
> but keeping it working is the important bit and it looks like people are not
> doing that.  The project would also be greatly aided by having dedicated
> people diagnose errors, report bugs against Python core if they're real and
> report bugs against Pybots if they're spurious.
>
> It would be good to have this effort be centralized and directed because it
> would otherwise be too easy to file duplicate bug reports, or to assume "oh,
> this has been failing for months, someone must have filed a bug already".

I agree with all you're saying here. As usual, the devil is in the
details. Finding those 'dedicated people' and also people who would
act as the central point of contact for bug reports etc. turns out to
be very hard in practice. If you have any ideas, I'd be glad to hear
them.

Grig

From tseaver at palladion.com  Fri Jul 11 05:08:11 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 10 Jul 2008 23:08:11 -0400
Subject: [Python-Dev] [Python-checkins] buildbot failure in sparc Debian
	3.0
In-Reply-To: <4873BCBC.9040104@v.loewis.de>
References: <20080702202345.DAD571E400A@bag.python.org>	<e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com>
	<4873BCBC.9040104@v.loewis.de>
Message-ID: <4876CE9B.8050402@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
> Jeremy Hylton wrote:
>> Does anyone have a clue about why this test fails only on this
>> platform?  The test is question is verifying that URLError gets
>> raised.  From the traceback, it appears that there is an uncaught
>> exception (URLError) but it fails in an assertRaises() check for
>> URLError.  That doesn't make much sense unless the variable URLError
>> refers to different objects, but both modules use "from urllib.error
>> import URLError".  And, of course, the test works fine on other
>> platforms.
> 
> It might be a transient error, and a complete cleanup of the tree
> might fix it. To do so, build a non-existent branch through the web ui,
> then build the original branch again; this will cause a fresh checkout.
> 
> If the error then persists, my guess it's some kind of compiler issue,
> which can be investigated only with access to the machine.

I would also be on the lookout for stale .pyc / .pyo files:  I saw a
similar failure recently while testing third-party code, and suspected
both causes:  cleaning out the .pyc files and carefully removing aliased
imports eventually got the problem to go away (at which point I could no
longer reproduce it at all).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIds6b+gerLs4ltQ4RAmIRAJ4pxs0sWLDrpOAilqV+Mx8vKJzeEQCeLMoX
gsFhfjJ4bxwAxgBji7/Jzvw=
=bMRD
-----END PGP SIGNATURE-----


From dickinsm at gmail.com  Fri Jul 11 10:37:36 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 11 Jul 2008 09:37:36 +0100
Subject: [Python-Dev] patch review request: float.hex and float.fromhex
Message-ID: <5c6f2a5d0807110137l3fffd090mbc2c9c80d1f7b05b@mail.gmail.com>

Does anyone have time to review the patch

http://bugs.python.org/file10876/hex_float5.patch

for issue 3008 (float <-> hexadecimal string conversion):

http://bugs.python.org/issue3008

? It would be really good if this could go in before next week's
beta. Guido's approved the idea in principle, but I still need to:

 - get permission from Barry to check in a new feature
   this late in the release cycle, and

 - persuade some other developer to review the patch.

I'll gladly 'pay' for a patch review by reviewing one or
more of someone else's patches.

Mark

From python at rcn.com  Fri Jul 11 11:06:51 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 11 Jul 2008 12:06:51 +0300
Subject: [Python-Dev] patch review request: float.hex and float.fromhex
References: <5c6f2a5d0807110137l3fffd090mbc2c9c80d1f7b05b@mail.gmail.com>
Message-ID: <3A2DC3264D7A4DCD9D64416A9441B8A1@RaymondLaptop1>

From: "Mark Dickinson" <dickinsm at gmail.com>
> Does anyone have time to review the patch
> 
> http://bugs.python.org/file10876/hex_float5.patch
> 
> for issue 3008 (float <-> hexadecimal string conversion):

I'll look at it today and tomorrow.


Raymond

From python at rcn.com  Fri Jul 11 13:24:39 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 11 Jul 2008 14:24:39 +0300
Subject: [Python-Dev] Running Py2.6 with the -3 option
Message-ID: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>

Some effort needs to be made to clear the standard library of -3 warnings.  Running -3 on production code usually involves 
exercising library code so the useful result is obscured by Python complaining about itself.  Since that use case involves the users 
own tests, I don't think the effort needs to be extended to our own unittest suite.  But the rest of the library could likely 
benefit from a good -3 cleanup.


Raymond 


From kirkshorts at hotmail.co.uk  Fri Jul 11 13:27:44 2008
From: kirkshorts at hotmail.co.uk (Andy Scott)
Date: Fri, 11 Jul 2008 11:27:44 +0000
Subject: [Python-Dev] A proposed solution for Issue 502236: Asyncrhonous
 exceptions between threads
Message-ID: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl>

[OK so a newbie post here so many apologies if I am doing this wrong]
Quick Synopsis:
 
 A child thread in an executing Python program can not safely shutdown the program. The issue URL is: http://bugs.python.org/issue502236
 
So my proposal is:
 
Example:
 
  We have three threads -
        t0 - Main system thread
        t1 - Worker thread
        t2 - Worker thread
 
  t1 encounters an issue that means it wants to shut down the application in as safe a way as possible
 
 
A Solution:
 
 1. Put in place a new function call sys.exitapplication, what this would do is:
     a. Mark a flag in t0's data structure saying a request to shutdown has been made
     b. Raise a new exception, SystemShuttingDown, in t1.
 2. As the main interpreter executes it checks the "shutting down flag" in the per thread data and follows one of two paths:
    If it is t0:
     a. Stops execution of the current code sequence
     b. Iterates over all extant threads setting the "system shutdown" flag in the per thread data structure. Setting this flag is a one time deal - it can not be undone once set. (And to avoid issues with multiple threads setting it - it can only ever be a single fixed value so setting it multiple times results in the same answer)
     c. Enters a timed wait loop where it will allow the other threads time to see the signal. It will iterate this loop a set number of times to avoid being blocked on any given thread.
     d. When all threads have exited, or been forcefully closed, raise the SystemShuttingDown exception
 
    If it is not t0:
     a. Stops execution of the current code sequence
     b. Raises the exception, SystemShuttingDown.
 
There are problems with this approach, as I see it they are (but please see the assumptions I have made):
 
P1. If the thread is in a tight loop will it see the exception? Or more generally: when should the exception be raised?
P2. When should the interpreter check this flag?
 
I think the answer to both of these problems is to:
 
 Check the flag, and hence raise the exception, in the following circumstances:
 
  - When the interpreter executes a back loop. So this should catch the jump back to the top of a "while True:" loop
  - Just before the interpreter makes a call to a hooked in non-Python system function, e.g. file I/O, networking &c.
 
 Checking at these points should be the minimal required, I think, to ensure that a given thread can not ignore the exception. It may be possible, or even required, to perform the check every time a Python function call is made.
 
I think this approach would then allow for the finally handlers to be called.
 
Assumptions:
 
[Here I must admit to a large amount of ignorance of the internals of Python at this time. So if my assumptions are incorrect I would greatly appreciate being told so :-) Preferably as polite as possible and any code pointers while welcome unless they point to some very esoteric and arcane area would be best kept general so I feel more of a spur to go learn the code base]
 
 1. The Python interpreter has per thread information.
 2. The Python interpreter can tell if the system, t0, thread is running.
 3. The Python engine has (or can easily obtain) a list of all threads it created.
 4. It is possible to raise exceptions as the byte code is executing.
 
I am mailing this out as:
 
 A. I have no idea if my thoughts are correct or total un-mitigated rubbish :-)
 B. I believe the introduction of this proposal (if I am correct) will require a PEP being raised, which aiui requires building community support (which is very fair imo) so this is me trying to do so :-)
 
So apologies if this post has been total spam (but no eggs) or too long - give a little whistle and it will all be OK again.
 
Andy
--------------------------------------Brain chemistry is not just for Christmas
_________________________________________________________________
Play and win great prizes with Live Search and Kung Fu Panda
http://clk.atdmt.com/UKM/go/101719966/direct/01/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080711/2e69e93e/attachment.htm>

From musiccomposition at gmail.com  Fri Jul 11 14:19:18 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 11 Jul 2008 07:19:18 -0500
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
Message-ID: <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>

On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote:
> Some effort needs to be made to clear the standard library of -3 warnings.
>  Running -3 on production code usually involves exercising library code so
> the useful result is obscured by Python complaining about itself.  Since
> that use case involves the users own tests, I don't think the effort needs
> to be extended to our own unittest suite.  But the rest of the library could
> likely benefit from a good -3 cleanup.

Yes, indeed. We should make sure, however, that the changes in the 2.6
libraries are the absolute minimum to get the job done. (I'm trying to
pretend like this isn't violating the prohibition on all-inclusive
overhauls in the stdlib.)



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From steve at holdenweb.com  Fri Jul 11 15:02:21 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 11 Jul 2008 09:02:21 -0400
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
Message-ID: <g57ll0$kqn$1@ger.gmane.org>

Benjamin Peterson wrote:
> On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote:
>> Some effort needs to be made to clear the standard library of -3 warnings.
>>  Running -3 on production code usually involves exercising library code so
>> the useful result is obscured by Python complaining about itself.  Since
>> that use case involves the users own tests, I don't think the effort needs
>> to be extended to our own unittest suite.  But the rest of the library could
>> likely benefit from a good -3 cleanup.
> 
> Yes, indeed. We should make sure, however, that the changes in the 2.6
> libraries are the absolute minimum to get the job done. (I'm trying to
> pretend like this isn't violating the prohibition on all-inclusive
> overhauls in the stdlib.)
> 
The prohibition is on *gratuitous* changes, basically along the lines of 
"if it ain't broke, don't fix it". The stdlib is definitely broken if it 
raises warnings of that kind.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From musiccomposition at gmail.com  Fri Jul 11 16:50:21 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 11 Jul 2008 09:50:21 -0500
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <g57ll0$kqn$1@ger.gmane.org>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
Message-ID: <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com>

On Fri, Jul 11, 2008 at 8:02 AM, Steve Holden <steve at holdenweb.com> wrote:
> Benjamin Peterson wrote:
>>
>> Yes, indeed. We should make sure, however, that the changes in the 2.6
>> libraries are the absolute minimum to get the job done. (I'm trying to
>> pretend like this isn't violating the prohibition on all-inclusive
>> overhauls in the stdlib.)
>>
> The prohibition is on *gratuitous* changes, basically along the lines of "if
> it ain't broke, don't fix it". The stdlib is definitely broken if it raises
> warnings of that kind.

Just because it's massive breakage fixage doesn't mean that it's
unlikely to break something else. :)

>
> regards
>  Steve
> --
> Steve Holden        +1 571 484 6266   +1 800 494 3119
> Holden Web LLC              http://www.holdenweb.com/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From steve at holdenweb.com  Fri Jul 11 17:08:41 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 11 Jul 2008 11:08:41 -0400
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>	<g57ll0$kqn$1@ger.gmane.org>
	<1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com>
Message-ID: <48777779.70302@holdenweb.com>

Benjamin Peterson wrote:
> On Fri, Jul 11, 2008 at 8:02 AM, Steve Holden <steve at holdenweb.com> wrote:
>> Benjamin Peterson wrote:
>>> Yes, indeed. We should make sure, however, that the changes in the 2.6
>>> libraries are the absolute minimum to get the job done. (I'm trying to
>>> pretend like this isn't violating the prohibition on all-inclusive
>>> overhauls in the stdlib.)
>>>
>> The prohibition is on *gratuitous* changes, basically along the lines of "if
>> it ain't broke, don't fix it". The stdlib is definitely broken if it raises
>> warnings of that kind.
> 
> Just because it's massive breakage fixage doesn't mean that it's
> unlikely to break something else. :)
> 
I agree but, contrariwise, just because we are likely to break other 
things doesn't mean we shouldn't fix the massive breakage.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Fri Jul 11 17:08:41 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 11 Jul 2008 11:08:41 -0400
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>	<g57ll0$kqn$1@ger.gmane.org>
	<1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com>
Message-ID: <48777779.70302@holdenweb.com>

Benjamin Peterson wrote:
> On Fri, Jul 11, 2008 at 8:02 AM, Steve Holden <steve at holdenweb.com> wrote:
>> Benjamin Peterson wrote:
>>> Yes, indeed. We should make sure, however, that the changes in the 2.6
>>> libraries are the absolute minimum to get the job done. (I'm trying to
>>> pretend like this isn't violating the prohibition on all-inclusive
>>> overhauls in the stdlib.)
>>>
>> The prohibition is on *gratuitous* changes, basically along the lines of "if
>> it ain't broke, don't fix it". The stdlib is definitely broken if it raises
>> warnings of that kind.
> 
> Just because it's massive breakage fixage doesn't mean that it's
> unlikely to break something else. :)
> 
I agree but, contrariwise, just because we are likely to break other 
things doesn't mean we shouldn't fix the massive breakage.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From status at bugs.python.org  Fri Jul 11 18:06:49 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 11 Jul 2008 18:06:49 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080711160649.5EDC0782BC@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (07/04/08 - 07/11/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1967 open (+43) / 13199 closed (+17) / 15166 total (+60)

Open issues with patches:   621

Average duration of open issues: 700 days.
Median duration of open issues: 1604 days.

Open Issues Breakdown
   open  1939 (+42)
pending    28 ( +1)

Issues Created Or Reopened (60)
_______________________________

Delete obsolete 'Unicode' in Py3 str docstrings; related fixes   07/04/08
CLOSED http://bugs.python.org/issue3284    created  tjreedy                   
       easy                                                                    

Fraction.from_any()                                              07/04/08
CLOSED http://bugs.python.org/issue3285    created  rhettinger                
       patch                                                                   

IDLE opens window too low on Windows                             07/04/08
       http://bugs.python.org/issue3286    created  tjreedy                   
                                                                               

Fraction constructor should raise TypeError instead of Attribute 07/04/08
CLOSED http://bugs.python.org/issue3287    created  rhettinger                
       patch                                                                   

float.as_integer_ratio method is not documented                  07/05/08
       http://bugs.python.org/issue3288    created  marketdickinson           
                                                                               

unnecessary call to time and localtime slows time.mktime         07/05/08
CLOSED http://bugs.python.org/issue3289    created  nother_jnelson            
                                                                               

python-config --cflags includes irrelevant flags                 07/05/08
       http://bugs.python.org/issue3290    created  sacha                     
                                                                               

rlcompleter doesn't work anymore                                 07/05/08
CLOSED http://bugs.python.org/issue3291    created  pitrou                    
       patch                                                                   

Position index limit; s.insert(i,x) not same as s[i:i]=[x]       07/05/08
       http://bugs.python.org/issue3292    created  tjreedy                   
                                                                               

incorrect comments for PyObject_ReleaseBuffer                    07/05/08
       http://bugs.python.org/issue3293    created  pitrou                    
                                                                               

SVN repository contains an incorrect symbolic link               07/05/08
CLOSED http://bugs.python.org/issue3294    created  pitrou                    
                                                                               

PyExc_BufferError is declared but nowhere defined                07/05/08
CLOSED http://bugs.python.org/issue3295    created  pitrou                    
       patch                                                                   

print function not executed in python 3.0 tutorial               07/05/08
CLOSED http://bugs.python.org/issue3296    created  segfaulthunter            
                                                                               

Python interpreter uses Unicode surrogate pairs only before the  07/06/08
       http://bugs.python.org/issue3297    created  ezio.melotti              
                                                                               

Multiline string with quotes is not parsed correctly.            07/06/08
CLOSED http://bugs.python.org/issue3298    created  Stavros                   
                                                                               

invalid object destruction in re.finditer()                      07/06/08
       http://bugs.python.org/issue3299    created  haypo                     
       patch                                                                   

urllib.quote and unquote - Unicode issues                        07/06/08
       http://bugs.python.org/issue3300    created  mgiuca                    
       patch                                                                   

DoS when lo is negative in bisect.insort_right() / _left()       07/06/08
CLOSED http://bugs.python.org/issue3301    created  haypo                     
       patch                                                                   

segfault on gettext(None)                                        07/06/08
       http://bugs.python.org/issue3302    created  haypo                     
       patch                                                                   

invalid ref count on locale.strcoll() error                      07/06/08
       http://bugs.python.org/issue3303    created  haypo                     
       patch                                                                   

invalid call to PyMem_Free() in fileio_init()                    07/06/08
       http://bugs.python.org/issue3304    created  haypo                     
       patch, easy                                                             

Use Py_XDECREF() instead of Py_DECREF() in MultibyteCodec and Mu 07/06/08
       http://bugs.python.org/issue3305    created  haypo                     
       patch                                                                   

audioop.findmax() crashs with negative length                    07/06/08
CLOSED http://bugs.python.org/issue3306    created  haypo                     
       patch                                                                   

invalid check of _bsddb creation failure                         07/06/08
       http://bugs.python.org/issue3307    created  haypo                     
       patch                                                                   

MinGW built extensions do not load (specified procedure cannot b 07/07/08
CLOSED http://bugs.python.org/issue3308    created  rogerbinns                
                                                                               

missing lock release in BZ2File_iternext()                       07/07/08
       http://bugs.python.org/issue3309    created  haypo                     
       patch, easy                                                             

Out-of-date example 3.0b1 Tutorial Classes page, 'issubclass'    07/07/08
       http://bugs.python.org/issue3310    created  alexis.layton             
                                                                               

block operation on closed socket/pipe for multiprocessing        07/07/08
       http://bugs.python.org/issue3311    created  haypo                     
       patch                                                                   

bugs in _sqlite module                                           07/07/08
       http://bugs.python.org/issue3312    created  haypo                     
       patch                                                                   

dlopen() error with no error message from dlerror()              07/07/08
       http://bugs.python.org/issue3313    created  haypo                     
       patch                                                                   

urllib.parse doesn't import sys                                  07/07/08
CLOSED http://bugs.python.org/issue3314    created  mgiuca                    
       patch                                                                   

abc.rst little error                                             07/07/08
CLOSED http://bugs.python.org/issue3315    created  mishok13                  
       patch                                                                   

Proposal for fix_urllib                                          07/07/08
       http://bugs.python.org/issue3316    created  nedds                     
       patch                                                                   

duplicate lines in zipfile.py                                    07/07/08
       http://bugs.python.org/issue3317    created  amaury.forgeotdarc        
       patch                                                                   

Documentation: timeit: "lower bound" should read "upper bound"   07/08/08
       http://bugs.python.org/issue3318    created  unutbu                    
                                                                               

pystone.main(10) causes ZeroDivisionError                        07/08/08
       http://bugs.python.org/issue3319    created  mokeefe                   
       patch                                                                   

various doc typos                                                07/08/08
       http://bugs.python.org/issue3320    created  dsm001                    
       patch                                                                   

_multiprocessing.Connection() doesn't check handle               07/08/08
       http://bugs.python.org/issue3321    created  haypo                     
       patch                                                                   

bugs in scanstring_str() and scanstring_unicode() of _json modul 07/08/08
       http://bugs.python.org/issue3322    created  haypo                     
                                                                               

Clarify __slots__ behaviour when inheriting                      07/09/08
       http://bugs.python.org/issue3323    created  strangefeatures           
                                                                               

Broken link in online doc                                        07/09/08
       http://bugs.python.org/issue3324    created  ThomasH                   
                                                                               

use of cPickle in multiprocessing                                07/09/08
       http://bugs.python.org/issue3325    created  mishok13                  
       patch                                                                   

py3k shouldn't use -fno-strict-aliasing anymore                  07/09/08
       http://bugs.python.org/issue3326    created  cartman                   
       patch                                                                   

NULL member in modules_by_index                                  07/09/08
       http://bugs.python.org/issue3327    created  krisvale                  
                                                                               

When PyObject_CallMethod fails, refcount is incorrect            07/09/08
       http://bugs.python.org/issue3328    created  dominic.lavoie            
                                                                               

API for setting the memory allocator used by Python              07/09/08
       http://bugs.python.org/issue3329    created  jlaurila                  
                                                                               

webbrowser module doesn't correctly handle '|' character.        07/09/08
       http://bugs.python.org/issue3330    created  AdrianP                   
                                                                               

Possible inconsistency in behavior of list comprehensions vs. ge 07/10/08
       http://bugs.python.org/issue3331    created  carlj                     
                                                                               

DocTest and dict sort.                                           07/10/08
CLOSED http://bugs.python.org/issue3332    created  jedie                     
                                                                               

Need -3 warning for exec statement becoming a function           07/10/08
CLOSED http://bugs.python.org/issue3333    created  rhettinger                
                                                                               

2to3 looses indentation on import fix                            07/10/08
       http://bugs.python.org/issue3334    created  ctheune                   
                                                                               

subprocess lib - opening same command fails                      07/10/08
       http://bugs.python.org/issue3335    created  gtg944q                   
                                                                               

datetime weekday() function                                      07/10/08
       http://bugs.python.org/issue3336    created  ryanboesch                
                                                                               

Fixer for dbm is failing                                         07/11/08
CLOSED http://bugs.python.org/issue3337    created  brett.cannon              
                                                                               

cPickle segfault with deep recursion                             07/11/08
       http://bugs.python.org/issue3338    created  esrever_otua              
                                                                               

dummy_thread LockType.acquire() always returns None, should be T 07/11/08
       http://bugs.python.org/issue3339    created  toymachine                
       patch                                                                   

optparse print_usage(),.. methods are not documented             07/11/08
       http://bugs.python.org/issue3340    created  techtonik                 
                                                                               

"Suggest a change" link                                          07/11/08
       http://bugs.python.org/issue3341    created  techtonik                 
                                                                               

Tracebacks are not properly indented                             07/11/08
       http://bugs.python.org/issue3342    created  amaury.forgeotdarc        
       patch                                                                   

Py_DisplaySourceLine is not documented                           07/11/08
       http://bugs.python.org/issue3343    created  amaury.forgeotdarc        
                                                                               



Issues Now Closed (36)
______________________

async_chat.__init__() parameters                                  221 days
       http://bugs.python.org/issue1519    josiahcarlson             
                                                                               

Error when printing an exception containing a Unicode string       99 days
       http://bugs.python.org/issue2517    ncoghlan                  
       patch                                                                   

performance problem in socket._fileobject.read                     82 days
       http://bugs.python.org/issue2632    gregory.p.smith           
       patch                                                                   

shutil.copytree glob-style filtering [patch]                       76 days
       http://bugs.python.org/issue2663    georg.brandl              
       patch                                                                   

"Report bug" links                                                 61 days
       http://bugs.python.org/issue2823    techtonik                 
                                                                               

cleanup of freelist management                                     52 days
       http://bugs.python.org/issue2862    gregory.p.smith           
       patch, patch                                                            

By default, HTTPSConnection should send header "Host: somehost"    24 days
       http://bugs.python.org/issue3094    gregory.p.smith           
       patch, easy                                                             

glob.py improvements                                               20 days
       http://bugs.python.org/issue3159    facundobatista            
       patch                                                                   

cmath test fails on Solaris 10                                     14 days
       http://bugs.python.org/issue3168    MrJean1                   
       patch                                                                   

sha modules & Modules/Setup.dist                                   13 days
       http://bugs.python.org/issue3183    gregory.p.smith           
                                                                               

float('infinity') should be valid                                  11 days
       http://bugs.python.org/issue3188    marketdickinson           
       patch                                                                   

Improve subprocess module usage                                     6 days
       http://bugs.python.org/issue3235    georg.brandl              
                                                                               

curses/textpad.py incorrectly and redundantly imports ascii         5 days
       http://bugs.python.org/issue3239    facundobatista            
       patch                                                                   

socket's OOB data management is broken on OS X and FreeBSD          3 days
       http://bugs.python.org/issue3277    gregory.p.smith           
                                                                               

socket's SO_OOBINLINE option does not work                          3 days
       http://bugs.python.org/issue3278    gregory.p.smith           
                                                                               

%c format does not accept large numbers on ucs-2 builds             1 days
       http://bugs.python.org/issue3280    amaury.forgeotdarc        
                                                                               

Delete obsolete 'Unicode' in Py3 str docstrings; related fixes      0 days
       http://bugs.python.org/issue3284    benjamin.peterson         
       easy                                                                    

Fraction.from_any()                                                 6 days
       http://bugs.python.org/issue3285    rhettinger                
       patch                                                                   

Fraction constructor should raise TypeError instead of Attribute    6 days
       http://bugs.python.org/issue3287    rhettinger                
       patch                                                                   

unnecessary call to time and localtime slows time.mktime            0 days
       http://bugs.python.org/issue3289    facundobatista            
                                                                               

rlcompleter doesn't work anymore                                    0 days
       http://bugs.python.org/issue3291    benjamin.peterson         
       patch                                                                   

SVN repository contains an incorrect symbolic link                  0 days
       http://bugs.python.org/issue3294    benjamin.peterson         
                                                                               

PyExc_BufferError is declared but nowhere defined                   0 days
       http://bugs.python.org/issue3295    benjamin.peterson         
       patch                                                                   

print function not executed in python 3.0 tutorial                  0 days
       http://bugs.python.org/issue3296    benjamin.peterson         
                                                                               

Multiline string with quotes is not parsed correctly.               0 days
       http://bugs.python.org/issue3298    Stavros                   
                                                                               

DoS when lo is negative in bisect.insort_right() / _left()          4 days
       http://bugs.python.org/issue3301    rhettinger                
       patch                                                                   

audioop.findmax() crashs with negative length                       1 days
       http://bugs.python.org/issue3306    facundobatista            
       patch                                                                   

MinGW built extensions do not load (specified procedure cannot b    1 days
       http://bugs.python.org/issue3308    loewis                    
                                                                               

urllib.parse doesn't import sys                                     0 days
       http://bugs.python.org/issue3314    facundobatista            
       patch                                                                   

abc.rst little error                                                0 days
       http://bugs.python.org/issue3315    benjamin.peterson         
       patch                                                                   

DocTest and dict sort.                                              0 days
       http://bugs.python.org/issue3332    rhettinger                
                                                                               

Need -3 warning for exec statement becoming a function              1 days
       http://bugs.python.org/issue3333    rhettinger                
                                                                               

Fixer for dbm is failing                                            0 days
       http://bugs.python.org/issue3337    brett.cannon              
                                                                               

asyncore.py and "handle_error"                                   1839 days
       http://bugs.python.org/issue760475  josiahcarlson             
                                                                               

asyncore misses socket closes when poll is used                  1515 days
       http://bugs.python.org/issue953599  josiahcarlson             
                                                                               

asyncore should handle also ECONNABORTED in recv                  390 days
       http://bugs.python.org/issue1736101 josiahcarlson             
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 14 threading module can deadlock after fork                        1643 days
open    http://bugs.python.org/issue874900 

 11 MinGW built extensions do not load (specified procedure cannot     1 days
closed  http://bugs.python.org/issue3308   

 10 urllib.quote and unquote - Unicode issues                          5 days
open    http://bugs.python.org/issue3300   

  9 Crash in PyObject_Malloc                                         356 days
open    http://bugs.python.org/issue1758146

  8 urllib2 header capitalization                                    122 days
open    http://bugs.python.org/issue2275   

  7 bytearrays are not thread safe                                    22 days
open    http://bugs.python.org/issue3139   

  7 test_multiprocessing hangs intermittently on POSIX platforms       9 days
open    http://bugs.python.org/issue3088   

  6 API for setting the memory allocator used by Python                2 days
open    http://bugs.python.org/issue3329   

  6 duplicate lines in zipfile.py                                      4 days
open    http://bugs.python.org/issue3317   

  6 Let bin/oct/hex show floats                                       17 days
open    http://bugs.python.org/issue3008   




From rhamph at gmail.com  Fri Jul 11 21:26:33 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 11 Jul 2008 13:26:33 -0600
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <g57ll0$kqn$1@ger.gmane.org>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
Message-ID: <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>

On Fri, Jul 11, 2008 at 7:02 AM, Steve Holden <steve at holdenweb.com> wrote:
> Benjamin Peterson wrote:
>>
>> On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote:
>>>
>>> Some effort needs to be made to clear the standard library of -3
>>> warnings.
>>>  Running -3 on production code usually involves exercising library code
>>> so
>>> the useful result is obscured by Python complaining about itself.  Since
>>> that use case involves the users own tests, I don't think the effort
>>> needs
>>> to be extended to our own unittest suite.  But the rest of the library
>>> could
>>> likely benefit from a good -3 cleanup.
>>
>> Yes, indeed. We should make sure, however, that the changes in the 2.6
>> libraries are the absolute minimum to get the job done. (I'm trying to
>> pretend like this isn't violating the prohibition on all-inclusive
>> overhauls in the stdlib.)
>>
> The prohibition is on *gratuitous* changes, basically along the lines of "if
> it ain't broke, don't fix it". The stdlib is definitely broken if it raises
> warnings of that kind.

Is the stdlib broken or is it the warnings that are broken?  The code
is just fine in 2.6.  Adding pragmas to disable warnings would be just
fine.  Or we could hardcode some warnings as "already seen".

-- 
Adam Olsen, aka Rhamphoryncus

From brett at python.org  Fri Jul 11 22:16:30 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 11 Jul 2008 13:16:30 -0700
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
Message-ID: <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>

On Fri, Jul 11, 2008 at 12:26 PM, Adam Olsen <rhamph at gmail.com> wrote:
> On Fri, Jul 11, 2008 at 7:02 AM, Steve Holden <steve at holdenweb.com> wrote:
>> Benjamin Peterson wrote:
>>>
>>> On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote:
>>>>
>>>> Some effort needs to be made to clear the standard library of -3
>>>> warnings.
>>>>  Running -3 on production code usually involves exercising library code
>>>> so
>>>> the useful result is obscured by Python complaining about itself.  Since
>>>> that use case involves the users own tests, I don't think the effort
>>>> needs
>>>> to be extended to our own unittest suite.  But the rest of the library
>>>> could
>>>> likely benefit from a good -3 cleanup.
>>>
>>> Yes, indeed. We should make sure, however, that the changes in the 2.6
>>> libraries are the absolute minimum to get the job done. (I'm trying to
>>> pretend like this isn't violating the prohibition on all-inclusive
>>> overhauls in the stdlib.)
>>>
>> The prohibition is on *gratuitous* changes, basically along the lines of "if
>> it ain't broke, don't fix it". The stdlib is definitely broken if it raises
>> warnings of that kind.
>
> Is the stdlib broken or is it the warnings that are broken?

Nothing is broken, per se, but the stdlib emits a ton of warnings
through basic usage for Py3K-related changes. We are telling people to
run their code in 2.6 with -3 and to eliminate all warnings in order
to have 2to3 work to transition to 3.0. Having the stdlib itself emit
warnings is just not reasonable.

>  The code
> is just fine in 2.6.  Adding pragmas to disable warnings would be just
> fine.  Or we could hardcode some warnings as "already seen".
>

No, we should eat our own dog food and transition the code over. If
anything it will help with code maintenance between 2.x and 3.x.

-Brett

From josiah.carlson at gmail.com  Sat Jul 12 04:51:58 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Fri, 11 Jul 2008 19:51:58 -0700
Subject: [Python-Dev] A proposed solution for Issue 502236: Asyncrhonous
	exceptions between threads
In-Reply-To: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl>
References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl>
Message-ID: <e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com>

This doesn't need to be an interpreter thing; it's easy to implement
by the user (I've done it about a dozen times using a single global
flag).  If you want it to be automatic, it's even possible to make it
happen automatically using sys.settrace() and friends (you can even
make it reasonably fast if you use a C callback).

 - Josiah

On Fri, Jul 11, 2008 at 4:27 AM, Andy Scott <kirkshorts at hotmail.co.uk> wrote:
> [OK so a newbie post here so many apologies if I am doing this wrong]
>
> Quick Synopsis:
>
>  A child thread in an executing Python program can not safely shutdown the
> program. The issue URL is: http://bugs.python.org/issue502236
>
> So my proposal is:
>
> Example:
>
>   We have three threads -
>         t0 - Main system thread
>         t1 - Worker thread
>         t2 - Worker thread
>
>   t1 encounters an issue that means it wants to shut down the application in
> as safe a way as possible
>
>
> A Solution:
>
>  1. Put in place a new function call sys.exitapplication, what this would do
> is:
>      a. Mark a flag in t0's data structure saying a request to shutdown has
> been made
>      b. Raise a new exception, SystemShuttingDown, in t1.
>  2. As the main interpreter executes it checks the "shutting down flag" in
> the per thread data and follows one of two paths:
>     If it is t0:
>      a. Stops execution of the current code sequence
>      b. Iterates over all extant threads setting the "system shutdown" flag
> in the per thread data structure. Setting this flag is a one time deal - it
> can not be undone once set. (And to avoid issues with multiple threads
> setting it - it can only ever be a single fixed value so setting it multiple
> times results in the same answer)
>      c. Enters a timed wait loop where it will allow the other threads time
> to see the signal. It will iterate this loop a set number of times to avoid
> being blocked on any given thread.
>      d. When all threads have exited, or been forcefully closed, raise the
> SystemShuttingDown exception
>
>     If it is not t0:
>      a. Stops execution of the current code sequence
>      b. Raises the exception, SystemShuttingDown.
>
> There are problems with this approach, as I see it they are (but please see
> the assumptions I have made):
>
> P1. If the thread is in a tight loop will it see the exception? Or more
> generally: when should the exception be raised?
> P2. When should the interpreter check this flag?
>
> I think the answer to both of these problems is to:
>
>  Check the flag, and hence raise the exception, in the following
> circumstances:
>
>   - When the interpreter executes a back loop. So this should catch the jump
> back to the top of a "while True:" loop
>   - Just before the interpreter makes a call to a hooked in non-Python
> system function, e.g. file I/O, networking &c.
>
>  Checking at these points should be the minimal required, I think, to ensure
> that a given thread can not ignore the exception. It may be possible, or
> even required, to perform the check every time a Python function call is
> made.
>
> I think this approach would then allow for the finally handlers to be
> called.
>
> Assumptions:
>
> [Here I must admit to a large amount of ignorance of the internals of Python
> at this time. So if my assumptions are incorrect I would greatly appreciate
> being told so :-) Preferably as polite as possible and any code pointers
> while welcome unless they point to some very esoteric and arcane area would
> be best kept general so I feel more of a spur to go learn the code base]
>
>  1. The Python interpreter has per thread information.
>  2. The Python interpreter can tell if the system, t0, thread is running.
>  3. The Python engine has (or can easily obtain) a list of all threads it
> created.
>  4. It is possible to raise exceptions as the byte code is executing.
>
> I am mailing this out as:
>
>  A. I have no idea if my thoughts are correct or total un-mitigated rubbish
> :-)
>  B. I believe the introduction of this proposal (if I am correct) will
> require a PEP being raised, which aiui requires building community support
> (which is very fair imo) so this is me trying to do so :-)
>
> So apologies if this post has been total spam (but no eggs) or too long -
> give a little whistle and it will all be OK again.
>
> Andy
> --------------------------------------
> Brain chemistry is not just for Christmas
>
>
> ________________________________
> Get Messenger on your Mobile! Get it now!
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com
>
>

From fumanchu at aminus.org  Sat Jul 12 19:08:31 2008
From: fumanchu at aminus.org (Robert Brewer)
Date: Sat, 12 Jul 2008 10:08:31 -0700
Subject: [Python-Dev] A proposed solution for Issue 502236:
	Asyncrhonousexceptions between threads
In-Reply-To: <e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com>
References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl>
	<e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com>
Message-ID: <F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local>

Josiah Carlson wrote:
> This doesn't need to be an interpreter thing; it's easy to implement
> by the user (I've done it about a dozen times using a single global
> flag).  If you want it to be automatic, it's even possible to make it
> happen automatically using sys.settrace() and friends (you can even
> make it reasonably fast if you use a C callback).

Agreed. If someone wants a small library to help do this, especially in
web servers, the latest version of Cherrpy includes a 'process'
subpackage under a generous license. It does all the things Andy
describes via a Bus object:

> Andy Scott wrote:
> > 1. Put in place a new function call sys.exitapplication, what this
> > would do is:
> >      a. Mark a flag in t0's data structure saying a request to
> >         shutdown has been made

This is bus.exit(), which publishes a 'stop' message to all subscribed
'stop' listeners, and then an 'exit' message to any 'exit' listeners.

> >      b. Raise a new exception, SystemShuttingDown, in t1.

That's up to the listener.

> >  2. As the main interpreter executes it checks the "shutting down
> >     flag" in the per thread data and follows one of two paths:
> >     If it is t0:
> >      a. Stops execution of the current code sequence
> >      b. Iterates over all extant threads ...
> >      c. Enters a timed wait loop where it will allow the other
> >         threads time to see the signal. It will iterate this loop
> >         a set number of times to avoid being blocked on any given
> >         thread.

This is implemented as [t.join() for t in threading.enumerate()] in the
main thread.

> >      d. When all threads have exited, or been forcefully closed,
> >         raise the SystemShuttingDown exception

The bus just lets the main thread exit at this point.

> > P1. If the thread is in a tight loop will it see the exception? Or
> > more generally: when should the exception be raised?

That's dependent enough on what work the thread is doing that a
completely generic approach is generally not sufficient. Therefore, the
process.bus sends a 'stop' message, and leaves the implementation of the
receiver up to the author of that thread's logic. Presumably, one
wouldn't register a listener for the 'stop' message unless one knew how
to actually stop.

> > P2. When should the interpreter check this flag?
> >
> > I think the answer to both of these problems is to check the flag,
> > and hence raise the exception, in the following circumstances:
> >   - When the interpreter executes a back loop. So this should catch
> >     the jump back to the top of a "while True:" loop
> >   - Just before the interpreter makes a call to a hooked in non-
> >     Python system function, e.g. file I/O, networking &c.

This is indeed how most well-written apps do it already.

> > Checking at these points should be the minimal required, I think, to
> > ensure that a given thread can not ignore the exception. It may be
> > possible, or even required, to perform the check every time a Python
> > function call is made.

PLEASE don't make Python function calls slower.

> >  1. The Python interpreter has per thread information.
> >  2. The Python interpreter can tell if the system, t0, thread is
> >     running.
> >  3. The Python engine has (or can easily obtain) a list of all
> >     threads it created.
> >  4. It is possible to raise exceptions as the byte code is
executing.

Replace 'Python interpreter' with 'your application' and those become
relatively simple architectural issues: maintain a list of threads, have
them expose an interface to determine if they're running, and make them
monitor a flag to know when another thread is asking them to stop.


Robert Brewer
fumanchu at aminus.org


From matt.giuca at gmail.com  Sat Jul 12 19:27:16 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Sun, 13 Jul 2008 03:27:16 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
Message-ID: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>

Hi all,

My first post to the list. In fact, first time Python hacker, long-time
Python user though. (Melbourne, Australia).

Some of you may have seen for the past week or so my bug report on Roundup,
http://bugs.python.org/issue3300

I've spent a heap of effort on this patch now so I'd really like to get some
more opinions and have this patch considered for Python 3.0.

Basically, urllib.quote and unquote seem not to have been updated since
Python 2.5, and because of this they implicitly perform Latin-1 encoding and
decoding (with respect to percent-encoded characters). I think they should
default to UTF-8 for a number of reasons, including that's what other
software such as web browsers use.

I've submitted a patch which fixes quote and unquote to use UTF-8 by
default. I also added extra arguments allowing the caller to choose the
encoding (after discussion, there was some consensus that this would be
beneficial). I have now completed updating the documentation, writing
extensive test cases, and testing the rest of the standard library for code
breakage - with the result being there wasn't really any, everything seems
to just work nicely with UTF-8. You can read the sordid details of my
investigation in the tracker.

Firstly, it'd be nice to hear if people think this is desirable behaviour.
Secondly, if it's feasible to get this patch in Python 3.0. (I think if it
were delayed to Python 3.1, the code breakage wouldn't justify it). And
thirdly, if the first two are positive, if anyone would like to review this
patch and check it in.

I have extensively tested it, and am now pretty confident that it won't
cause any grief if it's checked in.

Thanks very much,
Matt Giuca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/d6f74f48/attachment.htm>

From brett at python.org  Sat Jul 12 20:46:48 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 12 Jul 2008 11:46:48 -0700
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
Message-ID: <bbaeab100807121146k22046188v858bc70d138a833c@mail.gmail.com>

On Sat, Jul 12, 2008 at 10:27 AM, Matt Giuca <matt.giuca at gmail.com> wrote:
> Hi all,
>
> My first post to the list. In fact, first time Python hacker, long-time
> Python user though. (Melbourne, Australia).
>

Welcome!

> Some of you may have seen for the past week or so my bug report on Roundup,
> http://bugs.python.org/issue3300
>
> I've spent a heap of effort on this patch now so I'd really like to get some
> more opinions and have this patch considered for Python 3.0.
>

Hopefully we can get to it in the near future. Since we are having two
more betas (one of this is next week) hopefully there is enough time
before hitting a release candidate to have this looked at.

> Basically, urllib.quote and unquote seem not to have been updated since
> Python 2.5, and because of this they implicitly perform Latin-1 encoding and
> decoding (with respect to percent-encoded characters). I think they should
> default to UTF-8 for a number of reasons, including that's what other
> software such as web browsers use.
>
> I've submitted a patch which fixes quote and unquote to use UTF-8 by
> default. I also added extra arguments allowing the caller to choose the
> encoding (after discussion, there was some consensus that this would be
> beneficial). I have now completed updating the documentation, writing
> extensive test cases, and testing the rest of the standard library for code
> breakage - with the result being there wasn't really any, everything seems
> to just work nicely with UTF-8. You can read the sordid details of my
> investigation in the tracker.
>
> Firstly, it'd be nice to hear if people think this is desirable behaviour.

Based on what is said in this email, it sounds reasonable.

> Secondly, if it's feasible to get this patch in Python 3.0. (I think if it
> were delayed to Python 3.1, the code breakage wouldn't justify it).

If what you are saying is true, then it can probably go in as a bug
fix (unless someone else knows something about Latin-1 on the Net that
makes this not true).

> And
> thirdly, if the first two are positive, if anyone would like to review this
> patch and check it in.
>

That I can't say I can necessarily due; have my own bug reports to
work through this weekend. =)

-Brett

From janssen at parc.com  Sat Jul 12 23:07:09 2008
From: janssen at parc.com (Bill Janssen)
Date: Sat, 12 Jul 2008 14:07:09 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
Message-ID: <08Jul12.140711pdt."58698"@synergy1.parc.xerox.com>

> Basically, urllib.quote and unquote seem not to have been updated since
> Python 2.5, and because of this they implicitly perform Latin-1 encoding and
> decoding (with respect to percent-encoded characters). I think they should
> default to UTF-8 for a number of reasons, including that's what other
> software such as web browsers use.

The standard here is RFC 3986, from Jan 2005, which says,

  ``When a new URI scheme defines a component that represents textual
data consisting of characters from the Universal Character Set [UCS],
the data should first be encoded as octets according to the UTF-8
character encoding [STD63]; then only those octets that do not
correspond to characters in the unreserved set should be
percent-encoded.''

The "unreserved set" consists of the following ASCII characters:

  ``Characters that are allowed in a URI but do not have a reserved
purpose are called unreserved.  These include uppercase and lowercase
letters, decimal digits, hyphen, period, underscore, and tilde.

   unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"
''

There are a few other wrinkles; it's worth reading section 2.5
carefully.

I'd say, treat the incoming data as either Unicode (if it's a Unicode
string), or some unknown superset of ASCII (which includes both
Latin-1 and UTF-8) if it's a byte-string (and thus in some unknown
encoding), and apply the appropriate transformation.

Bill


From asmodai at in-nomine.org  Sun Jul 13 00:28:01 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sun, 13 Jul 2008 00:28:01 +0200
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
Message-ID: <20080712222801.GC27106@nexus.in-nomine.org>

-On [20080712 19:27], Matt Giuca (matt.giuca at gmail.com) wrote:
>Basically, urllib.quote and unquote seem not to have been updated since Python
>2.5, and because of this they implicitly perform Latin-1 encoding and decoding
>(with respect to percent-encoded characters). I think they should default to
>UTF-8 for a number of reasons, including that's what other software such as web
>browsers use.

Very nice, I had this somewhere on my todo list to work on. I'm very much
in favour, especially since it synchronizes us with the RFCs (for all I
remember reading about it last time).

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Can your hear the Dolphin's cry..?

From martin at v.loewis.de  Sun Jul 13 01:10:01 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 13 Jul 2008 01:10:01 +0200
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <20080712222801.GC27106@nexus.in-nomine.org>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
Message-ID: <487939C9.3040703@v.loewis.de>

> Very nice, I had this somewhere on my todo list to work on. I'm very much
> in favour, especially since it synchronizes us with the RFCs (for all I
> remember reading about it last time).

I still think that it doesn't. The RFCs haven't changed, and can't
change for compatibility reasons. The encoding of non-ASCII characters
in URLs remains as underspecified as it always was.

Now, with IRIs, the situation is different, but I don't think the patch
claims to implement IRIs (and if so, it perhaps shouldn't change URL
processing in doing so).

Regards,
Martin


From matt.giuca at gmail.com  Sun Jul 13 01:15:18 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Sun, 13 Jul 2008 09:15:18 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <20080712222801.GC27106@nexus.in-nomine.org>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
Message-ID: <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>

Thanks for all the replies, and making me feel welcome :)

>
> If what you are saying is true, then it can probably go in as a bug
> fix (unless someone else knows something about Latin-1 on the Net that
> makes this not true).
>

Well from what I've seen, the only time Latin-1 naturally appears on the net
is when you have a web page in Latin-1 (either explicit or inferred; and
note that a browser like Firefox will infer Latin-1 if it sees only ASCII
characters) with a form in it. Submitting the form, the browser will use
Latin-1 to percent-encode the query string.

So if you write a web app and you don't have any non-ASCII characters or
mention the charset, chances are you'll get Latin-1. But I would argue
you're leaving things to chance and you deserve to get funny behaviour. If
you do any of the following:

   - Use a non-ASCII character, encoded as UTF-8 on the page.
   - Send a Content-Type: xxxx; charset=utf-8.
   - In HTML, set a <meta http-equiv="Content-Type: xxxx; charset=utf-8" />.
   - In the form itself, set <form accept-encoding="utf-8">.

then the browser will encode the form data as UTF-8. And most "proper" web
pages should get themselves explicitly served as UTF-8.

That I can't say I can necessarily due; have my own bug reports to
> work through this weekend. =)


OK well I'm busy for the next few days; after that I can do a patch trade
with someone. (That is if I am allowed to do reviews; not sure since I don't
have developer privileges).


On Sun, Jul 13, 2008 at 5:58 AM, Mark Hammond <mhammond at skippinet.com.au>
wrote:

> > My first post to the list. In fact, first time Python hacker,
> > long-time Python user though. (Melbourne, Australia).
>
> Cool - where exactly?  I'm in Wantirna (although not at this very moment -
> I'm in Lithuania, but home again in a couple of days)


Cool :) Balwyn.


> * Please take Martin with a grain of salt ( \I would say "ignore him", but
> that is too strong ;)


Lol, he is a hard man to please, but he's given some good feedback.


On Sun, Jul 13, 2008 at 7:07 AM, Bill Janssen <janssen at parc.com> wrote:

>
> The standard here is RFC 3986, from Jan 2005, which says,
>
>  ``When a new URI scheme defines a component that represents textual
> data consisting of characters from the Universal Character Set [UCS],
> the data should first be encoded as octets according to the UTF-8
> character encoding [STD63]; then only those octets that do not
> correspond to characters in the unreserved set should be
> percent-encoded.''


Ah yes, I was originally hung up on the idea that "URLs had to be encoded in
UTF-8", till Martin pointed out that it only says "new URI scheme" there.
It's perfectly valid to have non-UTF-8-encoded URIs. However in practice
they're almost always UTF-8. So I think introducing the new encoding
argument and having it default to "utf-8" is quite reasonable.

I'd say, treat the incoming data as either Unicode (if it's a Unicode
> string), or some unknown superset of ASCII (which includes both
> Latin-1 and UTF-8) if it's a byte-string (and thus in some unknown
> encoding), and apply the appropriate transformation.
>

Ah there may be some confusion here. We're only dealing with str->str
transformations (which in Python 3 means Unicode strings). You can't put a
bytes in or get a bytes out of either of these functions. I suggested a
"quote_raw" and "unquote_raw" function which would let you do this.

The issue is with the percent-encoded characters in the URI string, which
must be interpreted as bytes, not code points. How then do you convert these
into a Unicode string? (Python 2 did not have this problem, since you simply
output a byte string without caring about the encoding).

On Sun, Jul 13, 2008 at 9:10 AM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > Very nice, I had this somewhere on my todo list to work on. I'm very much
> > in favour, especially since it synchronizes us with the RFCs (for all I
> > remember reading about it last time).
>
> I still think that it doesn't. The RFCs haven't changed, and can't
> change for compatibility reasons. The encoding of non-ASCII characters
> in URLs remains as underspecified as it always was.


Correct. But my patch brings us in-line with that unspecification. The
unpatched version forces you to use Latin-1. My patch lets you specify the
encoding to use.


> Now, with IRIs, the situation is different, but I don't think the patch
> claims to implement IRIs (and if so, it perhaps shouldn't change URL
> processing in doing so).


True. I don't claim to have implemented IRIs or even know enough about them
to do that. I'll read up on these things in the next few days.

However, this is a URI library, not IRI. From what I've seen, it's
percent-encoded URIs coming in from the browser, not IRIs. We just need to
make sure with this patch that IRIs don't become less-supported than they
were before; don't need to explicitly support them.

Cheers,
Matt Giuca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/6a007f88/attachment-0001.htm>

From nd at perlig.de  Sun Jul 13 01:55:48 2008
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Sun, 13 Jul 2008 01:55:48 +0200
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
Message-ID: <200807130155.48346@news.perlig.de>

* Matt Giuca wrote:

> Well from what I've seen, the only time Latin-1 naturally appears on the
> net is when you have a web page in Latin-1 (either explicit or inferred;
> and note that a browser like Firefox will infer Latin-1 if it sees only
> ASCII characters) with a form in it. Submitting the form, the browser
> will use Latin-1 to percent-encode the query string.

This POV is way too browser-centric...

> So if you write a web app and you don't have any non-ASCII characters or
> mention the charset, chances are you'll get Latin-1. But I would argue
> you're leaving things to chance and you deserve to get funny behaviour.
> If you do any of the following:
>
>    - Use a non-ASCII character, encoded as UTF-8 on the page.
>    - Send a Content-Type: xxxx; charset=utf-8.
>    - In HTML, set a <meta http-equiv="Content-Type: xxxx; charset=utf-8"
> />. - In the form itself, set <form accept-encoding="utf-8">.
>
> then the browser will encode the form data as UTF-8. And most "proper"
> web pages should get themselves explicitly served as UTF-8.

... because

1) URL encoding is not limited to web forms at all

2) The web form encoding depends on the browser settings as well (for 
example, try playing around with the internet explorer settings regarding 
query encoding)

3) The process submitting the form may not be a browser at all

4) The web form may not be under your own control (Search engine forms are a 
common example here, e.g. "put this google form snippet onto your webpage")

5) Different cultures do not choose necessarily between latin-1 and utf-8. 
They deal more with things like, say KOI8-R or Big5.

etc pp

Besides all that and without any offense: "most proper" and "should do" and 
the implication that all web browsers behave the same way are not a good 
location to argue from when talking about implementing a standard ;)

nd
-- 
Wenn nur Ingenieure mit Diplom programmieren w?rden, h?tten wir
wahrscheinlich weniger schlechte Software.
Wir h?tten allerdings auch weniger gute Software.
                                   -- Felix von Leitner in dasr

From matt.giuca at gmail.com  Sun Jul 13 02:24:11 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Sun, 13 Jul 2008 10:24:11 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <200807130155.48346@news.perlig.de>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<200807130155.48346@news.perlig.de>
Message-ID: <e6268af30807121724l470e32agb23a018a5f83c757@mail.gmail.com>

> This POV is way too browser-centric...
>

This is but one example. Note that I found web forms to be the least
clear-cut example of choosing an encoding. Most of the time applications
seem to be using UTF-8, and all the standards I have read are moving towards
specifying UTF-8 (from being unspecified). I've never seen a standard
specify or even recommend Latin-1.

Where web forms are concerned, basically setting the form accept-charset or
the page charset is the *maximum amount* of control you have over the
encoding. As you say, it can be encoded by another page or the user can
override their settings. Then what can you do as the server? Nothing ...
there's no way to predict the encoding. So you just handle the cases you
have control over.

5) Different cultures do not choose necessarily between latin-1 and utf-8.
> They deal more with things like, say KOI8-R or Big5.


Exactly. This is exactly my point - Latin-1 is arbitrary from a standards
point of view. It's just one of the many legacy encodings we'd like to
forget. The UTFs are the only options which support all languages, and UTF-8
is the only ASCII-compatible (and therefore URI-compatible) encoding. So we
should aim to support that as the default.

Besides all that and without any offense: "most proper" and "should do" and
> the implication that all web browsers behave the same way are not a good
> location to argue from when talking about implementing a standard ;)


I agree. However if there *was* a proper standard we wouldn't have to argue!
"Most proper" and "should do" is the most confident we can be when dealing
with this standard, as there is no correct encoding.

Does anyone have a suggestion which will be more compatible with the rest of
the world than allowing the user to select an encoding, and defaulting to
"utf-8"?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/d022d9f6/attachment.htm>

From bignose+hates-spam at benfinney.id.au  Sun Jul 13 11:29:22 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Sun, 13 Jul 2008 19:29:22 +1000
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
Message-ID: <8763radte5.fsf@benfinney.id.au>

"Guido van Rossum" <guido at python.org> writes:

> Same here; let's tread carefully here and not change this with 3.0.
> Starting to deprecate in 3.1 and killing in 3.3 would be soon enough.

I'm very glad this is on the table. Even though I'd really like to see
the names become PEP-8-conformant in the 2.x series, the arguments
against such haste are compelling.

So I'm convinced to be +1 on post-3.0 for changing the unittest API.

> I like using only the assertKeyword variants, removing assert_, fail*,
> and assertEquals.

I'm the opposite. I prefer the 'fail*' variants over the 'assert*'
variants, because "fail" tells me exactly what the function will *do*,
while "assert" leaves it implicit what will actually happen if the
assertion is false.

For this reason, I'd prefer the "fail*" names to be recommended, and
the "assert*" names to be, if not deprecated, then at least
second-class citizens.

-- 
 \         ?I think there is a world market for maybe five computers.? |
  `\                             ?Thomas Watson, chairman of IBM, 1943 |
_o__)                                                                  |
Ben Finney


From andrew-pythondev at puzzling.org  Sun Jul 13 13:46:53 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Sun, 13 Jul 2008 21:46:53 +1000
Subject: [Python-Dev] unittest's redundant assertions: asserts
	vs.	failIf/Unlesses
In-Reply-To: <8763radte5.fsf@benfinney.id.au>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
Message-ID: <20080713114653.GE9654@steerpike.home.puzzling.org>

Ben Finney wrote:
> "Guido van Rossum" <guido at python.org> writes:
[...]
> > I like using only the assertKeyword variants, removing assert_, fail*,
> > and assertEquals.
> 
> I'm the opposite. I prefer the 'fail*' variants over the 'assert*'
> variants, because "fail" tells me exactly what the function will *do*,
> while "assert" leaves it implicit what will actually happen if the
> assertion is false.
> 
> For this reason, I'd prefer the "fail*" names to be recommended, and
> the "assert*" names to be, if not deprecated, then at least
> second-class citizens.

On the other hand, ?assert*? names are positive statements of what the behaviour 
of the system-under-test is.  And conversely, ?fail*? names are a bit like
implementation details of how the test is written.  So I personally have a mild
preference for the assert* names.

My suspicion is that it will be very hard to get consensus on the colour of this
particular bike shed :)

-Andrew.


From solipsis at pitrou.net  Sun Jul 13 14:25:35 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 13 Jul 2008 12:25:35 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?unittest=27s_redundant_assertions=3A_asser?=
	=?utf-8?q?ts=09vs=2E=09failIf/Unlesses?=
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
Message-ID: <loom.20080713T121738-271@post.gmane.org>

Andrew Bennetts <andrew-pythondev <at> puzzling.org> writes:
> 
> On the other hand, ?assert*? names are positive statements of what the
> behaviour of the system-under-test is.  And conversely, ?fail*? names are a
> bit like implementation details of how the test is written.  So I personally
> have a mild preference for the assert* names.

The problem with "fail*" is that you get names like "failIfNotEqual" (or perhaps
even "failUnlessNotEqual") which are double negatives and therefore much more
annoying to read and decipher. I always had the same problem when reading Perl's
"unless" statements. They are, IMO, useless complication.

"assert*" names are, well, assertive :-)

(not to mention "assert" is a widely established name in various languages -
including Python - for checking that things went as expected)



From bignose+hates-spam at benfinney.id.au  Sun Jul 13 14:36:27 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Sun, 13 Jul 2008 22:36:27 +1000
Subject: [Python-Dev] unittest's redundant assertions:
	asserts	vs.	failIf/Unlesses
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
Message-ID: <87prpic65w.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> The problem with "fail*" is that you get names like "failIfNotEqual"

That would better be written (preferring PEP 8 names)
"fail_unless_equal".

> (or perhaps even "failUnlessNotEqual")

idem, "fail_if_equal".

> which are double negatives

Exactly. With "if" and "unless" at our disposal, we can avoid such
double negatives.

> (not to mention "assert" is a widely established name in various
> languages - including Python - for checking that things went as
> expected)

That's another reason to avoid "assert" in the name: these methods
*don't* necessarily use the 'assert' statement. Avoiding the
implication that they do use that is a good thing.

-- 
 \     ?Never do anything against conscience even if the state demands |
  `\                                             it.? ?Albert Einstein |
_o__)                                                                  |
Ben Finney


From bignose+hates-spam at benfinney.id.au  Sun Jul 13 14:38:39 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Sun, 13 Jul 2008 22:38:39 +1000
Subject: [Python-Dev] unittest's redundant assertions: asserts
	vs.	failIf/Unlesses
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
Message-ID: <87lk06c628.fsf@benfinney.id.au>

Andrew Bennetts <andrew-pythondev at puzzling.org> writes:

> On the other hand, ?assert*? names are positive statements of what
> the behaviour of the system-under-test is.

That statement is better made by the name of the test case. The method
used for implementing the test shouldn't need to be part of that
statement.

> And conversely, ?fail*? names are a bit like implementation
> details of how the test is written.

Entirely appropriate, since those names are used exactly at the point
of implementing the test, and the names are not visible outside that
implementation. No?

> My suspicion is that it will be very hard to get consensus on the
> colour of this particular bike shed :)

Perhaps so :-)

-- 
 \         ?Quidquid latine dictum sit, altum viditur.?  (?Whatever is |
  `\                      said in Latin, sounds profound.?) ?anonymous |
_o__)                                                                  |
Ben Finney


From ncoghlan at gmail.com  Sun Jul 13 14:51:30 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 13 Jul 2008 22:51:30 +1000
Subject: [Python-Dev] AMD64-W2k8 buildbot wedged
Message-ID: <4879FA52.4040706@gmail.com>

The compilation step on this buildbot is failing because it can't delete 
or overwrite any of the Python DLLs [1]. Is there any way to fix this 
remotely, or at least to tell why kill_python isn't doing the trick?

(Going back a bit further, it looks like test_multiprocessing is the 
culprit as the original hanging test. The number of 64-bit safeness 
warnings being emitted by the current trunk is also fairly worrying)

Cheers,
Nick.

[1] 
http://www.python.org/dev/buildbot/all/AMD64%20W2k8%20trunk/builds/757/step-compile/0

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From solipsis at pitrou.net  Sun Jul 13 15:07:05 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 13 Jul 2008 13:07:05 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?unittest=27s_redundant_assertions=3A=09ass?=
	=?utf-8?q?erts=09vs=2E=09failIf/Unlesses?=
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
	<87prpic65w.fsf@benfinney.id.au>
Message-ID: <loom.20080713T130302-168@post.gmane.org>

Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
> 
> That would better be written (preferring PEP 8 names)
> "fail_unless_equal".

Which is still a double negative ("fail" and "unless" are both negative words).

> That's another reason to avoid "assert" in the name: these methods
> *don't* necessarily use the 'assert' statement.

But all those constructs (assert, assertEqual, etc.) raise the same exception
type named AssertionError which, if you care for naming consistency, is more
important than whether or not they are implemented in terms of each other. :-)




From steve at holdenweb.com  Sun Jul 13 15:31:35 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 13 Jul 2008 09:31:35 -0400
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <loom.20080713T130302-168@post.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>	<8763radte5.fsf@benfinney.id.au>	<20080713114653.GE9654@steerpike.home.puzzling.org>	<loom.20080713T121738-271@post.gmane.org>	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
Message-ID: <g5d03p$air$1@ger.gmane.org>

Antoine Pitrou wrote:
> Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
>> That would better be written (preferring PEP 8 names)
>> "fail_unless_equal".
> 
> Which is still a double negative ("fail" and "unless" are both negative words).
> 
"Fail" isn't a negative. As Guido said, it's a description of the test 
behavior under particular circumstances. "fail_unless_equal" says quite 
clearly that the test requires equality of the values.

>> That's another reason to avoid "assert" in the name: these methods
>> *don't* necessarily use the 'assert' statement.
> 
> But all those constructs (assert, assertEqual, etc.) raise the same exception
> type named AssertionError which, if you care for naming consistency, is more
> important than whether or not they are implemented in terms of each other. :-)
> 
But the important behavior is the failure of the test, not the specific 
exception that is raised to fail it. Or would you prefer tests that 
raise other exceptions to succeed? The exception type is an 
implementation detail, and a fairly unimportant one as far as the 
purpose of testing goes.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From bignose+hates-spam at benfinney.id.au  Sun Jul 13 15:34:51 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Sun, 13 Jul 2008 23:34:51 +1000
Subject: [Python-Dev] unittest's redundant
	assertions:	asserts	vs.	failIf/Unlesses
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
Message-ID: <87ej5xdi10.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
> > That would better be written (preferring PEP 8 names)
> > "fail_unless_equal".
> 
> Which is still a double negative ("fail" and "unless" are both
> negative words).

Hmm, not to this native-English-speaker's ear. "fail" is a verb
stating what will be done, while "unless" and "if" are the conditions
under which it will be done.

> > That's another reason to avoid "assert" in the name: these methods
> > *don't* necessarily use the 'assert' statement.
> 
> But all those constructs (assert, assertEqual, etc.) raise the same
> exception type named AssertionError

Only by default. They can be overridden to raise any exception type.

The only thing they have in common at that point (when the exception
is raised) is that they have failed the test.

-- 
 \       ?First things first, but not necessarily in that order.? ?The |
  `\                                              Doctor, _Doctor Who_ |
_o__)                                                                  |
Ben Finney


From Scott.Daniels at Acm.Org  Sun Jul 13 16:02:01 2008
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun, 13 Jul 2008 07:02:01 -0700
Subject: [Python-Dev] Update bz2 library?
Message-ID: <g5d18v$dav$1@ger.gmane.org>

I just noticed that the bz2lib version was updated to 1.0.5 in December
of 2007, for security reasons.  I suspect it would be good to be sure
to ship this with 2.6 or 3.0.

--Scott David Daniels
Scott.Daniels at Acm.Org


From solipsis at pitrou.net  Sun Jul 13 16:39:48 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 13 Jul 2008 14:39:48 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?unittest=27s_redundant_assertions=3A_asser?=
	=?utf-8?q?ts_vs=2E=09failIf/Unlesses?=
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>	<8763radte5.fsf@benfinney.id.au>	<20080713114653.GE9654@steerpike.home.puzzling.org>	<loom.20080713T121738-271@post.gmane.org>	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
	<g5d03p$air$1@ger.gmane.org>
Message-ID: <loom.20080713T135317-118@post.gmane.org>


Let's split hairs a little...

Steve Holden <steve <at> holdenweb.com> writes:
> "Fail" isn't a negative. As Guido said, it's a description of the test 
> behavior under particular circumstances.

In most circumstances, "fail" is a negative word defined as the contrary of
something else (that is, as the "failure to pass/succeed/perform/achieve/..."),
while the reverse is not true (few people would define "success" or "passing a
test" as the negative of "failure", except in desperate circumstances). Although
I'm not a native English speaker, I don't think our respective languages and
cultures differ on this point.


> "fail_unless_equal" says quite 
> clearly that the test requires equality of the values.

Actually, saying "that the test requires equality of the values" translates
directly into an "assert equals" (or "enforce equals" if you want a stronger
word) rather than a "fail if not equal". It is a grammatical fact...

In other words, if you express a requirement, you intent to say how the
implementation under test is supposed to behave for it to be considered
successful, not the conditions under which its behaviour constitutes a failure. 

As you said, if an exception is thrown which isn't part of the testing protocol
(e.g. something other than an AssertionError), the test is still said to fail...
if the intent of testing were to test for failure conditions, on the contrary,
the test would be said to be passed (because it wouldn't have met the failure
conditions).

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Sun Jul 13 16:45:58 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 13 Jul 2008 15:45:58 +0100
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <loom.20080713T135317-118@post.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>	<8763radte5.fsf@benfinney.id.au>	<20080713114653.GE9654@steerpike.home.puzzling.org>	<loom.20080713T121738-271@post.gmane.org>	<87prpic65w.fsf@benfinney.id.au>	<loom.20080713T130302-168@post.gmane.org>	<g5d03p$air$1@ger.gmane.org>
	<loom.20080713T135317-118@post.gmane.org>
Message-ID: <487A1526.6030401@voidspace.org.uk>

Antoine Pitrou wrote:
> Let's split hairs a little...
>
> Steve Holden <steve <at> holdenweb.com> writes:
>   
>> "Fail" isn't a negative. As Guido said, it's a description of the test 
>> behavior under particular circumstances.
>>     
>
> In most circumstances, "fail" is a negative word defined as the contrary of
> something else (that is, as the "failure to pass/succeed/perform/achieve/..."),
> while the reverse is not true (few people would define "success" or "passing a
> test" as the negative of "failure", except in desperate circumstances). Although
> I'm not a native English speaker, I don't think our respective languages and
> cultures differ on this point.
>
>
>   
>> "fail_unless_equal" says quite 
>> clearly that the test requires equality of the values.
>>     
>
> Actually, saying "that the test requires equality of the values" translates
> directly into an "assert equals" (or "enforce equals" if you want a stronger
> word) rather than a "fail if not equal". It is a grammatical fact...
>
> In other words, if you express a requirement, you intent to say how the
> implementation under test is supposed to behave for it to be considered
> successful, not the conditions under which its behaviour constitutes a failure. 
>   

Agreed. I tend to think of testing as action followed by assertion - I 
do this and this should have happened. Your tests usually define 
'expected behaviour' rather than defining how your code won't fail...

Michael

> As you said, if an exception is thrown which isn't part of the testing protocol
> (e.g. something other than an AssertionError), the test is still said to fail...
> if the intent of testing were to test for failure conditions, on the contrary,
> the test would be said to be passed (because it wouldn't have met the failure
> conditions).
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From kirkshorts at hotmail.co.uk  Sun Jul 13 17:28:16 2008
From: kirkshorts at hotmail.co.uk (Andy Scott)
Date: Sun, 13 Jul 2008 15:28:16 +0000
Subject: [Python-Dev] A proposed solution for Issue 502236:
 Asyncrhonousexceptions between threads
In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local>
References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl>
	<e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local>
Message-ID: <BLU136-W52FFE760D80D27784A462788920@phx.gbl>

Thanks Guys

So it seems as if I've made some pretty basic newbie mistakes then :-$

 1. Posting to the wrong list
 2. Posting about something that already has a work around

oh well never mind - better luck next time ;-)

Only one thing comes to my mind about the comments made saying that this is no longer an issue:

 All of the solutions presented rely on the use of a third party library to control the threading. It appears as if there have been no basic changes to the thread package that ships with Python to resolve this. While it is possible to set and read global variables &c to do something close - it doesn't strike me as a nice solution as Python offers in so many other places.

However, I am willing to concede that as the issue has been dormant (or appears to be) now for ~2years that it can not be a particularly burning issue. So I'll go back and trawl the open issues for something else to help my favourite language get better :-)


Andy
-----
Brain chemistry is not just for Christmas

[snipped the replies to cut down on re-quote spam - cos I kind of hate that]
<snip>

_________________________________________________________________
Invite your Facebook friends to chat on Messenger
http://clk.atdmt.com/UKM/go/101719649/direct/01/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/86826b85/attachment.htm>

From steve at pearwood.info  Sun Jul 13 17:34:53 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 14 Jul 2008 01:34:53 +1000
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <487A1526.6030401@voidspace.org.uk>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<loom.20080713T135317-118@post.gmane.org>
	<487A1526.6030401@voidspace.org.uk>
Message-ID: <200807140134.54676.steve@pearwood.info>

On Mon, 14 Jul 2008 12:45:58 am Michael Foord wrote:

> I tend to think of testing as action followed by assertion -
> I do this and this should have happened. Your tests usually define
> 'expected behaviour' rather than defining how your code won't fail...

Who is the "your" that you are speaking for there? It isn't me.

I tend to think of tests as *both* expected behaviour and unexpected 
behaviour: sometimes a test is most naturally written as "this should 
happen" and sometimes as "this shouldn't happen". It depends on what 
I'm testing.

When it comes to test-driven development, I will often start by 
thinking "What test can I write to make this code fail?" -- not "what 
test can I write to make this code pass?". Having come up with a 
failure mode, the test is often most naturally written as "fail if" 
or "fail unless", and I resent having to turn the condition around into 
a "assert if not" test just to satisfy others who are never going to 
read my code. I wish people would go find another bike shed to 
interfere with.


-- 
Steven

From steve at holdenweb.com  Sun Jul 13 17:56:45 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 13 Jul 2008 11:56:45 -0400
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <loom.20080713T135317-118@post.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>	<8763radte5.fsf@benfinney.id.au>	<20080713114653.GE9654@steerpike.home.puzzling.org>	<loom.20080713T121738-271@post.gmane.org>	<87prpic65w.fsf@benfinney.id.au>	<loom.20080713T130302-168@post.gmane.org>	<g5d03p$air$1@ger.gmane.org>
	<loom.20080713T135317-118@post.gmane.org>
Message-ID: <g5d8k2$ha$1@ger.gmane.org>

Antoine Pitrou wrote:
> Let's split hairs a little...
> 
> Steve Holden <steve <at> holdenweb.com> writes:
>> "Fail" isn't a negative. As Guido said, it's a description of the test 
>> behavior under particular circumstances.
> 
> In most circumstances, "fail" is a negative word defined as the contrary of
> something else (that is, as the "failure to pass/succeed/perform/achieve/..."),
> while the reverse is not true (few people would define "success" or "passing a
> test" as the negative of "failure", except in desperate circumstances). Although
> I'm not a native English speaker, I don't think our respective languages and
> cultures differ on this point.
> 
"The test will fail" is an assertion (in English, not in Python :). It 
is not a negative. "The test will not fail" is an assertion containing a 
negative. "The test will not not fail" is an assertion containing a 
double negative.

> 
>> "fail_unless_equal" says quite 
>> clearly that the test requires equality of the values.
> 
> Actually, saying "that the test requires equality of the values" translates
> directly into an "assert equals" (or "enforce equals" if you want a stronger
> word) rather than a "fail if not equal". It is a grammatical fact...
> 
Right. For extra points, discuss the differences between 
"fail_unless_equal", "fail_if_not_equal", assert_equals" and 
"assert_unequal".

> In other words, if you express a requirement, you intent to say how the
> implementation under test is supposed to behave for it to be considered
> successful, not the conditions under which its behaviour constitutes a failure. 
> 
> As you said, if an exception is thrown which isn't part of the testing protocol
> (e.g. something other than an AssertionError), the test is still said to fail...
> if the intent of testing were to test for failure conditions, on the contrary,
> the test would be said to be passed (because it wouldn't have met the failure
> conditions).
> 
But Guido said:

 > I like using only the assertKeyword variants, removing assert_, fail*,
 > and assertEquals.

So we are now no longer discussing what color to paint the bike shed, we 
are discussing how to describe the color. Let's stop. This is getting silly.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at pearwood.info  Sun Jul 13 18:20:52 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 14 Jul 2008 02:20:52 +1000
Subject: [Python-Dev]
	=?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?=
	=?iso-8859-1?q?asserts_vs=2E=09failIf/Unlesses?=
In-Reply-To: <loom.20080713T135317-118@post.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<g5d03p$air$1@ger.gmane.org>
	<loom.20080713T135317-118@post.gmane.org>
Message-ID: <200807140220.53574.steve@pearwood.info>

On Mon, 14 Jul 2008 12:39:48 am Antoine Pitrou wrote:
> Let's split hairs a little...
>
> Steve Holden <steve <at> holdenweb.com> writes:
> > "Fail" isn't a negative. As Guido said, it's a description of the
> > test behavior under particular circumstances.
>
> In most circumstances, "fail" is a negative word defined as the
> contrary of something else (that is, as the "failure to
> pass/succeed/perform/achieve/..."), while the reverse is not true
> (few people would define "success" or "passing a test" as the
> negative of "failure", except in desperate circumstances).

"Few people"? Do you have studies to support your claim, or are you just 
projecting your own opinion as if it were an objective fact?

I often consider "success" as the negative of failure: my code didn't 
fail. The bridge didn't fall down. The invasion didn't get bogged down 
in a long and pointless guerrilla war. The medicine didn't have massive 
side-effects. These are all successes.

assert_bridge_not_collapsed()
assert_no_guerrilla_war()
assert_if_no_side-effects()

fail_if_bridge_collapsed()
fail_if_guerrilla_war()
fail_if_side_effects()

Speaking for myself, the last three are far more natural to me.



I'll also point out that in science, success is often considered the 
opposite of failure. You design your experiments to disprove the 
theory, to fail, and if they don't fail, the theory is considered 
successful. A theory is considered "correct" when it has failed to 
fail. Some philosophers of science, like the late Karl Popper, consider 
this falsification to be the defining characteristic of science.

[...]

> In other words, if you express a requirement, you intent to say how
> the implementation under test is supposed to behave for it to be
> considered successful, not the conditions under which its behaviour
> constitutes a failure.

Please don't tell me what my intent is. Unless you are a mind-reader, 
you have no way of telling what my intent is.

When I write tests, my intent is often to specify the conditions that 
constitute a failure. I don't want to negate it to specify a success 
just to satisfy you.



-- 
Steven

From martin at v.loewis.de  Sun Jul 13 18:35:49 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Jul 2008 18:35:49 +0200
Subject: [Python-Dev] AMD64-W2k8 buildbot wedged
In-Reply-To: <4879FA52.4040706@gmail.com>
References: <4879FA52.4040706@gmail.com>
Message-ID: <487A2EE5.9020302@v.loewis.de>

Nick Coghlan wrote:
> The compilation step on this buildbot is failing because it can't delete
> or overwrite any of the Python DLLs [1]. Is there any way to fix this
> remotely, or at least to tell why kill_python isn't doing the trick?

That is in the log:

TerminateProcess failed: 5

where 5 is ERROR_ACCESS_DENIED. Now, *why* the system is refusing to
terminate the process is difficult to say. It's a general Windows
problem: the system doesn't support forced process termination in a
reliable manner.

In any case, Trent seems to have fixed the problem.

> The number of 64-bit safeness
> warnings being emitted by the current trunk is also fairly worrying)

Do you have a specific one in mind? The ones truncating size_t/ssize_t
should only matter when you actually do have data larger than 2GiB.

Regards,
Martin

From martin at v.loewis.de  Sun Jul 13 18:37:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Jul 2008 18:37:06 +0200
Subject: [Python-Dev] Update bz2 library?
In-Reply-To: <g5d18v$dav$1@ger.gmane.org>
References: <g5d18v$dav$1@ger.gmane.org>
Message-ID: <487A2F32.1000707@v.loewis.de>

> I just noticed that the bz2lib version was updated to 1.0.5 in December
> of 2007, for security reasons.  I suspect it would be good to be sure
> to ship this with 2.6 or 3.0.

Python 2.6b1 shipped with bzip2 1.0.5.

Regards,
Martin

From mhammond at skippinet.com.au  Sun Jul 13 20:19:57 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Sun, 13 Jul 2008 21:19:57 +0300
Subject: [Python-Dev] unittest's redundant assertions: asserts
	vs.	failIf/Unlesses
In-Reply-To: <200807140134.54676.steve@pearwood.info>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<loom.20080713T135317-118@post.gmane.org>	<487A1526.6030401@voidspace.org.uk>
	<200807140134.54676.steve@pearwood.info>
Message-ID: <004001c8e515$0fb08640$2f1192c0$@com.au>

> On Mon, 14 Jul 2008 12:45:58 am Michael Foord wrote:
> 
> > I tend to think of testing as action followed by assertion -
> > I do this and this should have happened. Your tests usually define
> > 'expected behaviour' rather than defining how your code won't fail...
> 
> Who is the "your" that you are speaking for there? It isn't me.

Although Michael unfortunately chose to use "Your", it is clear to me that
the context implies he actually meant "My" (as in, his)

> I resent having to turn the condition around into
> a "assert if not" test just to satisfy others who are never going to
> read my code. I wish people would go find another bike shed to
> interfere with.

Oh please, just get over it, and try to stick to the issue at hand rather
than trying to score points or acting under the assumption that everyone is
out to give you cause to "resent" things...

As Steve said, this is getting silly...

Mark



From nd at perlig.de  Sun Jul 13 20:54:52 2008
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Sun, 13 Jul 2008 20:54:52 +0200
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807121724l470e32agb23a018a5f83c757@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<200807130155.48346@news.perlig.de>
	<e6268af30807121724l470e32agb23a018a5f83c757@mail.gmail.com>
Message-ID: <200807132054.52146@news.perlig.de>

* Matt Giuca wrote:

> > This POV is way too browser-centric...
>
> This is but one example. Note that I found web forms to be the least
> clear-cut example of choosing an encoding. Most of the time applications
> seem to be using UTF-8, and all the standards I have read are moving
> towards specifying UTF-8 (from being unspecified). I've never seen a
> standard specify or even recommend Latin-1.

Ahem. The HTTP standard does ;-)

> Where web forms are concerned, basically setting the form accept-charset
> or the page charset is the *maximum amount* of control you have over the
> encoding. As you say, it can be encoded by another page or the user can
> override their settings. Then what can you do as the server? Nothing ...

Guessing works pretty well in most of the cases.

> Exactly. This is exactly my point - Latin-1 is arbitrary from a standards
> point of view. It's just one of the many legacy encodings we'd like to
> forget. The UTFs are the only options which support all languages, and
> UTF-8 is the only ASCII-compatible (and therefore URI-compatible)
> encoding. So we should aim to support that as the default.

Latin-1 is not exactly arbitray. Besides being a charset - it maps 
one-to-one to octet values, hence it's commonly used to encode octets and 
is therefore a better fallback than every other encoding.

> I agree. However if there *was* a proper standard we wouldn't have to
> argue! "Most proper" and "should do" is the most confident we can be when
> dealing with this standard, as there is no correct encoding.

Well, the standard says, there are octets to be encoded. I find that proper 
enough.

> Does anyone have a suggestion which will be more compatible with the rest
> of the world than allowing the user to select an encoding, and defaulting
> to "utf-8"?

Default to latin-1 for decoding and utf-8 for encoding. This might be 
confusing though, so maybe you've asked the wrong question ;)

nd
-- 
Real programmers confuse Christmas and Halloween because
DEC 25 = OCT 31.  -- Unknown

                                      (found in ssl_engine_mutex.c)

From rrr at ronadam.com  Sun Jul 13 20:55:24 2008
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 13 Jul 2008 13:55:24 -0500
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <loom.20080713T130302-168@post.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>	<8763radte5.fsf@benfinney.id.au>	<20080713114653.GE9654@steerpike.home.puzzling.org>	<loom.20080713T121738-271@post.gmane.org>	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
Message-ID: <487A4F9C.6090007@ronadam.com>



Antoine Pitrou wrote:
> Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
>> That would better be written (preferring PEP 8 names)
>> "fail_unless_equal".
> 
> Which is still a double negative ("fail" and "unless" are both negative words).
> 
>> That's another reason to avoid "assert" in the name: these methods
>> *don't* necessarily use the 'assert' statement.
> 
> But all those constructs (assert, assertEqual, etc.) raise the same exception
> type named AssertionError which, if you care for naming consistency, is more
> important than whether or not they are implemented in terms of each other. :-)


Regarding: "all those constructs ... raise the same exception type named 
AssertionError.."


As an experiment a while back (out of frustration), I created some tests 
that used more specific (local unittest module only) exceptions.  The 
clarity of the failed errors and the mental effort to debug the problems 
was much improved for me.


I sub-classed unittest.TestCase, and added two new methods and a few local 
  and unique test-only exceptions.

       * test -> a test function to call
       * ref  -> addition helpful debug info


       assertTestReturns(test, expect, ref="")

           raises on failure:

              Wrong_Result_Returned
              Unexpected_Exception_Raised



       assertTestRaises(test, expect, ref="")

           raises on failure:

              No_Exception_Raised
              Wrong_Exception_Raised


These more specific exceptions (over plain AssertionError), allowed for 
more specific error reports.

A testcase with an error would produce a standard python exception so you 
know instantly that you need to fix the test rather than the code being tested.

Also the more specific exception code can better handle some cases where a 
wrong traceback would be returned.  ie.. the test code traceback rather 
than the code being tested traceback.

Is there any interest in going in this direction with unittests?

Ron














From rrr at ronadam.com  Sun Jul 13 20:55:24 2008
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 13 Jul 2008 13:55:24 -0500
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <loom.20080713T130302-168@post.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>	<8763radte5.fsf@benfinney.id.au>	<20080713114653.GE9654@steerpike.home.puzzling.org>	<loom.20080713T121738-271@post.gmane.org>	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
Message-ID: <487A4F9C.6090007@ronadam.com>



Antoine Pitrou wrote:
> Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
>> That would better be written (preferring PEP 8 names)
>> "fail_unless_equal".
> 
> Which is still a double negative ("fail" and "unless" are both negative words).
> 
>> That's another reason to avoid "assert" in the name: these methods
>> *don't* necessarily use the 'assert' statement.
> 
> But all those constructs (assert, assertEqual, etc.) raise the same exception
> type named AssertionError which, if you care for naming consistency, is more
> important than whether or not they are implemented in terms of each other. :-)


Regarding: "all those constructs ... raise the same exception type named 
AssertionError.."


As an experiment a while back (out of frustration), I created some tests 
that used more specific (local unittest module only) exceptions.  The 
clarity of the failed errors and the mental effort to debug the problems 
was much improved for me.


I sub-classed unittest.TestCase, and added two new methods and a few local 
  and unique test-only exceptions.

       * test -> a test function to call
       * ref  -> addition helpful debug info


       assertTestReturns(test, expect, ref="")

           raises on failure:

              Wrong_Result_Returned
              Unexpected_Exception_Raised



       assertTestRaises(test, expect, ref="")

           raises on failure:

              No_Exception_Raised
              Wrong_Exception_Raised


These more specific exceptions (over plain AssertionError), allowed for 
more specific error reports.

A testcase with an error would produce a standard python exception so you 
know instantly that you need to fix the test rather than the code being tested.

Also the more specific exception code can better handle some cases where a 
wrong traceback would be returned.  ie.. the test code traceback rather 
than the code being tested traceback.

Is there any interest in going in this direction with unittests?

Ron














From josiah.carlson at gmail.com  Sun Jul 13 21:15:29 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Sun, 13 Jul 2008 12:15:29 -0700
Subject: [Python-Dev] A proposed solution for Issue 502236:
	Asyncrhonousexceptions between threads
In-Reply-To: <BLU136-W52FFE760D80D27784A462788920@phx.gbl>
References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl>
	<e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com>
	<F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local>
	<BLU136-W52FFE760D80D27784A462788920@phx.gbl>
Message-ID: <e6511dbf0807131215y29f7ca7an9a72d7ada15528ea@mail.gmail.com>

Andy,

You had an idea, and it was a pretty good idea, but the practical
considerations made it a nonstarter.  That's ok, it happens all the
time, among both new and seasoned Python developers alike.  Search for
"a case for top and bottom values" on Google for a bit of a laugh ;) .

 - Josiah

On Sun, Jul 13, 2008 at 8:28 AM, Andy Scott <kirkshorts at hotmail.co.uk> wrote:
> Thanks Guys
>
> So it seems as if I've made some pretty basic newbie mistakes then :-$
>
>  1. Posting to the wrong list
>  2. Posting about something that already has a work around
>
> oh well never mind - better luck next time ;-)
>
> Only one thing comes to my mind about the comments made saying that this is
> no longer an issue:
>
>  All of the solutions presented rely on the use of a third party library to
> control the threading. It appears as if there have been no basic changes to
> the thread package that ships with Python to resolve this. While it is
> possible to set and read global variables &c to do something close - it
> doesn't strike me as a nice solution as Python offers in so many other
> places.
>
> However, I am willing to concede that as the issue has been dormant (or
> appears to be) now for ~2years that it can not be a particularly burning
> issue. So I'll go back and trawl the open issues for something else to help
> my favourite language get better :-)
>
>
> Andy
> -----
> Brain chemistry is not just for Christmas
>
> [snipped the replies to cut down on re-quote spam - cos I kind of hate that]
> ________________________________
> <snip>
>
> ________________________________
> Get fish-slapping on Messenger! Play Now

From solipsis at pitrou.net  Sun Jul 13 22:15:01 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 13 Jul 2008 20:15:01 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?unittest=27s_redundant_assertions=3A_asser?=
	=?utf-8?q?ts_vs=2E=09failIf/Unlesses?=
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>	<8763radte5.fsf@benfinney.id.au>	<20080713114653.GE9654@steerpike.home.puzzling.org>	<loom.20080713T121738-271@post.gmane.org>	<87prpic65w.fsf@benfinney.id.au>	<loom.20080713T130302-168@post.gmane.org>	<g5d03p$air$1@ger.gmane.org>
	<loom.20080713T135317-118@post.gmane.org>
	<g5d8k2$ha$1@ger.gmane.org>
Message-ID: <loom.20080713T200212-415@post.gmane.org>


Steve Holden <steve <at> holdenweb.com> writes:
> But Guido said:
> 
>  > I like using only the assertKeyword variants, removing assert_, fail*,
>  > and assertEquals.
> 
> So we are now no longer discussing what color to paint the bike shed, we 
> are discussing how to describe the color. Let's stop. This is getting silly.

It's certainly getting silly - the language is named Python after all!
However, the whole thread started by someone opposing Guido's statement above,
which at least explains (if not justifies) the origins of this particular
instance of silliness...

(and I've probably made my point explicitly enough, so I won't insist, even if I
sometimes enjoy arguing on my spare time ;-))

cheers

Antoine.



From janssen at parc.com  Sun Jul 13 22:36:06 2008
From: janssen at parc.com (Bill Janssen)
Date: Sun, 13 Jul 2008 13:36:06 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
Message-ID: <08Jul13.133612pdt."58698"@synergy1.parc.xerox.com>

> Ah there may be some confusion here. We're only dealing with str->str
> transformations (which in Python 3 means Unicode strings). You can't put a
> bytes in or get a bytes out of either of these functions. I suggested a
> "quote_raw" and "unquote_raw" function which would let you do this.

Ah, well, that's a problem.  Clearly the unquote is str->bytes, while
the quote is (bytes OR str)->str.  You can't pass a Unicode string back
as the result of unquote *without* passing in an encoding specifier,
because the character set is application-specific.

Bill

From thebeatles42 at gmail.com  Sun Jul 13 22:58:15 2008
From: thebeatles42 at gmail.com (Tom Mullins)
Date: Sun, 13 Jul 2008 16:58:15 -0400
Subject: [Python-Dev] Exception tracebacks
Message-ID: <cb7a3820807131358h590b353clb4d58b12452943a1@mail.gmail.com>

I do not know where cleanup_traceback (in run.py) is called, or really
anything about the inner workings of python, but there is a problem with
sys.tracebacklimit. If it is set to one or zero, "** IDLE Internal
Exception" is printed in the traceback for any exception, not internal ones.
I think tracebacklimit is applied to the traceback before cleanup_traceback
is called, but should be applied after. I imagine the solution is not that
simple, and you may already be aware of the problem, but thanks anyway.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/193ff0ac/attachment.htm>

From aahz at pythoncraft.com  Sun Jul 13 23:42:09 2008
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 13 Jul 2008 14:42:09 -0700
Subject: [Python-Dev] Exception tracebacks
In-Reply-To: <cb7a3820807131358h590b353clb4d58b12452943a1@mail.gmail.com>
References: <cb7a3820807131358h590b353clb4d58b12452943a1@mail.gmail.com>
Message-ID: <20080713214209.GA29334@panix.com>

On Sun, Jul 13, 2008, Tom Mullins wrote:
>
> I do not know where cleanup_traceback (in run.py) is called, or really
> anything about the inner workings of python, but there is a problem with
> sys.tracebacklimit. If it is set to one or zero, "** IDLE Internal
> Exception" is printed in the traceback for any exception, not internal ones.
> I think tracebacklimit is applied to the traceback before cleanup_traceback
> is called, but should be applied after. I imagine the solution is not that
> simple, and you may already be aware of the problem, but thanks anyway.

If you care about this issue, please file a report at bugs.python.org
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

I support the RKAB

From ncoghlan at gmail.com  Mon Jul 14 00:08:05 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 14 Jul 2008 08:08:05 +1000
Subject: [Python-Dev] AMD64-W2k8 buildbot wedged
In-Reply-To: <487A2EE5.9020302@v.loewis.de>
References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de>
Message-ID: <487A7CC5.5070504@gmail.com>

Martin v. L?wis wrote:
> Nick Coghlan wrote:
>> The number of 64-bit safeness
>> warnings being emitted by the current trunk is also fairly worrying)
> 
> Do you have a specific one in mind? The ones truncating size_t/ssize_t
> should only matter when you actually do have data larger than 2GiB.

Nothing specific, I just don't think I've ever actually looked at the 
output of a 64-bit build before and was a little surprised at the number 
of such warnings. I guess they were mostly in modules which are probably 
going to struggle with handling 2+ GiB chunks of data anyway.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From bignose+hates-spam at benfinney.id.au  Mon Jul 14 00:28:41 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 08:28:41 +1000
Subject: [Python-Dev] PEP 8 conformance of unittest (was: unittest's
	redundant assertions: asserts vs. failIf/Unlesses)
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
Message-ID: <87y745bequ.fsf_-_@benfinney.id.au>

"Guido van Rossum" <guido at python.org> writes:

> Same here; let's tread carefully here and not change this with 3.0.
> Starting to deprecate in 3.1 and killing in 3.3 would be soon enough.
> I like using only the assertKeyword variants, removing assert_, fail*,
> and assertEquals.

Are we to interpret the above ("? using only the assertKeyword
variants, removing assert_, ?") as "we should actively remove names
that are PEP 8 compatible from 'unittest', instead preferring names
that go against PEP 8?

I really hope I'm misinterpreting this and that there's another
explanation. Preferably one that includes "we have a plan to make
'unittest' conform with the existing PEP 8".

-- 
 \        ?Pinky, are you pondering what I'm pondering?? ?Umm, I think |
  `\       so, Brain, but three men in a tub? Ooh, that's unsanitary!? |
_o__)                                           ?_Pinky and The Brain_ |
Ben Finney


From fuzzyman at voidspace.org.uk  Mon Jul 14 00:35:30 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 13 Jul 2008 23:35:30 +0100
Subject: [Python-Dev] PEP 8 conformance of unittest
In-Reply-To: <87y745bequ.fsf_-_@benfinney.id.au>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<87y745bequ.fsf_-_@benfinney.id.au>
Message-ID: <487A8332.1030102@voidspace.org.uk>

Ben Finney wrote:
> "Guido van Rossum" <guido at python.org> writes:
>
>   
>> Same here; let's tread carefully here and not change this with 3.0.
>> Starting to deprecate in 3.1 and killing in 3.3 would be soon enough.
>> I like using only the assertKeyword variants, removing assert_, fail*,
>> and assertEquals.
>>     
>
> Are we to interpret the above ("? using only the assertKeyword
> variants, removing assert_, ?") as "we should actively remove names
> that are PEP 8 compatible from 'unittest', instead preferring names
> that go against PEP 8?
>
> I really hope I'm misinterpreting this and that there's another
> explanation. Preferably one that includes "we have a plan to make
> 'unittest' conform with the existing PEP 8".
>
>   
"we have a plan to make 'unittest' conform with the existing PEP 8" - 
that was the outcome of the discussion. PEP 8'ification to begin in 2.7 
/ 3.1 with the deprecation of non-PEP8 compliant method names.

There was some added functionality discussed, but it is too late to get 
this into 2.6 / 3.0.

I volunteered to take it on, and have a record of the whole discussion. 
Unfortunately my writing commitment is going to keep me occupied until 
August - after which it will be one of my highest priorities.

Michael Foord

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From bignose+hates-spam at benfinney.id.au  Mon Jul 14 00:44:59 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 08:44:59 +1000
Subject: [Python-Dev] PEP 8 conformance of unittest
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<87y745bequ.fsf_-_@benfinney.id.au>
	<487A8332.1030102@voidspace.org.uk>
Message-ID: <87lk05bdzo.fsf@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> "we have a plan to make 'unittest' conform with the existing PEP 8"
> - that was the outcome of the discussion. PEP 8'ification to begin
> in 2.7 / 3.1 with the deprecation of non-PEP8 compliant method
> names.

Thanks, that's exactly the reassurance I was hoping for :-)

> I volunteered to take it on, and have a record of the whole
> discussion. Unfortunately my writing commitment is going to keep me
> occupied until August - after which it will be one of my highest
> priorities.

I've contacted you yesterday regarding this, but to reiterate: This
issue is of interest to me, and I'd like to help with it if I can.
Please contact me privately if I can be of assistance, especially if I
can help during your busy period.

-- 
 \      ?The trouble with the rat race is that even if you win, you're |
  `\                       still a rat.? ?Jane Wagner, via Lily Tomlin |
_o__)                                                                  |
Ben Finney


From fuzzyman at voidspace.org.uk  Mon Jul 14 00:51:44 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 13 Jul 2008 23:51:44 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <20080713093936.GA3623@benfinney.id.au>
References: <20080713093936.GA3623@benfinney.id.au>
Message-ID: <487A8700.8000200@voidspace.org.uk>

Ben Finney wrote:
> Howdy Michael,
>
> I'm interested in the changes you're proposing for Python's 'unittest' 
> module. I am (like, I suspect, many Python coders) maintaining my own 
> set of extensions to the module across many projects, so I'd really 
> like to see many of the improvements you discuss actually in the 
> standard library.
>
> What assistance can I offer to help on this issue?
>
>   
I intend to start working on them in August, after I have finished my 
current writing commitments.

The full list of changes proposed (feel free to start - but ping me or 
the list) and not shot down was something like:

Documenting that the assert method names are to be preferred over the 'FailUnless' names (this stirred up some controversy this weekend so should probably not happen).


Adding the following new asserts:

    assertIn    (member, container, msg=None)
    assertNotIn     (member, container, msg=None)
    assertIs     (first, second, msg=None)
    assertNotIs   (first, second, msg=None)
    assertRaisesWithMessage    (exc_class, message, callable, *args, 
**keywargs)


A simple 'RunTests' function that takes a collection of modules, test 
suites or test cases and runs them with TextTestRunner - passing on 
keyword arguments to the test runner. This makes running a test suite 
easier - once you have collected all your test cases together you can 
just pass them to this function so long as you are happy with the 
default runner (possibly allowing an alternative runner class to be 
provided as a keyword argument).


Make the error messages for "assertEquals" and 
"assertNotEquals" more useful - showing the objects that compare 
incorrectly even if an explicit message is passed in. Additionally, when 
comparing lists and tuples that are the same length show the members 
(and indices?) that were different.

Possibly even providing a diff in the case of comparing strings (we have an implementation of this at work).

    assertLessThan
    assertGreaterThan
    assertLessThanOrEquals
    assertGreaterThanOrEquals

Guido suggested various variants on assertEquals:

    assertListEqual(self, list1, list2, msg=None):
    assertDictEqual(self, d1, d2, msg=None):
    assertMultiLineEqual(self, first, second, msg=Non

In my opinion these can all be provided by improving the messages from assertEquals and don't require new methods.

    assertSameElements(self, expected_seq, actual_seq, msg=None):

I usually achieve this with:

    assertEquals(set(actual), set(expected))

A convenience method might be nice, but I'm worried about the API growing in an unbounded way.


Other suggestions that weren't controversial but I might not get to:

    assertRaisesWithMessage taking a regex to match the error message

    expect methods that collect failures and report at the end of the test (allowing an individual test method to raise several errors without stopping)

    assertIsInstance and assertIsSubclass



Michael Foord

-- 

http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From bignose+hates-spam at benfinney.id.au  Mon Jul 14 01:20:23 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 09:20:23 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
Message-ID: <87hcatbcco.fsf@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> Adding the following new asserts:
> 
>    assertIn    (member, container, msg=None)
>    assertNotIn     (member, container, msg=None)
>    assertIs     (first, second, msg=None)
>    assertNotIs   (first, second, msg=None)
>    assertRaisesWithMessage    (exc_class, message, callable, *args,
> **keywargs)
[?]

>    assertLessThan
>    assertGreaterThan
>    assertLessThanOrEquals
>    assertGreaterThanOrEquals
[?]

>    assertListEqual(self, list1, list2, msg=None):
>    assertDictEqual(self, d1, d2, msg=None):
>    assertMultiLineEqual(self, first, second, msg=Non
[?]

>    assertSameElements(self, expected_seq, actual_seq, msg=None):

All these are new, so there is no existing expectation of these names
from users of the standard library 'unittest' module (i.e. no
backward-compatibility concern since these are new methods).

If we're planning to deprecate the existing non-PEP-8 names in 2.7 and
3.1, why would we introduce new names that are non-PEP-8? Wouldn't it
be better to add these as PEP-8 compatible names in the first
instance?

-- 
 \       ?You've got the brain of a four-year-old boy, and I'll bet he |
  `\                         was glad to get rid of it.? ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From steve at holdenweb.com  Mon Jul 14 01:27:55 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 13 Jul 2008 19:27:55 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487A8700.8000200@voidspace.org.uk>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
Message-ID: <487A8F7B.1090901@holdenweb.com>

Michael Foord wrote:
> Ben Finney wrote:
>> Howdy Michael,
>>
>> I'm interested in the changes you're proposing for Python's 'unittest' 
>> module. I am (like, I suspect, many Python coders) maintaining my own 
>> set of extensions to the module across many projects, so I'd really 
>> like to see many of the improvements you discuss actually in the 
>> standard library.
>>
>> What assistance can I offer to help on this issue?
>>
>>   
> I intend to start working on them in August, after I have finished my 
> current writing commitments.
> 
> The full list of changes proposed (feel free to start - but ping me or 
> the list) and not shot down was something like:
> 
> Documenting that the assert method names are to be preferred over the 
> 'FailUnless' names (this stirred up some controversy this weekend so 
> should probably not happen).
> 
> 
> Adding the following new asserts:
> 
>    assertIn    (member, container, msg=None)
>    assertNotIn     (member, container, msg=None)
>    assertIs     (first, second, msg=None)
>    assertNotIs   (first, second, msg=None)

Please, let's call this one "assertIsNot". I know it's valid Python to say

   if a not is b:

but it's a much less natural way of expressing the condition, and (for 
all I know) might even introduce an extra negation operation. "is not" 
is, I believe, treated as a single operator.

 > [...]

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Mon Jul 14 01:27:55 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 13 Jul 2008 19:27:55 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487A8700.8000200@voidspace.org.uk>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
Message-ID: <487A8F7B.1090901@holdenweb.com>

Michael Foord wrote:
> Ben Finney wrote:
>> Howdy Michael,
>>
>> I'm interested in the changes you're proposing for Python's 'unittest' 
>> module. I am (like, I suspect, many Python coders) maintaining my own 
>> set of extensions to the module across many projects, so I'd really 
>> like to see many of the improvements you discuss actually in the 
>> standard library.
>>
>> What assistance can I offer to help on this issue?
>>
>>   
> I intend to start working on them in August, after I have finished my 
> current writing commitments.
> 
> The full list of changes proposed (feel free to start - but ping me or 
> the list) and not shot down was something like:
> 
> Documenting that the assert method names are to be preferred over the 
> 'FailUnless' names (this stirred up some controversy this weekend so 
> should probably not happen).
> 
> 
> Adding the following new asserts:
> 
>    assertIn    (member, container, msg=None)
>    assertNotIn     (member, container, msg=None)
>    assertIs     (first, second, msg=None)
>    assertNotIs   (first, second, msg=None)

Please, let's call this one "assertIsNot". I know it's valid Python to say

   if a not is b:

but it's a much less natural way of expressing the condition, and (for 
all I know) might even introduce an extra negation operation. "is not" 
is, I believe, treated as a single operator.

 > [...]

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From stephen at xemacs.org  Mon Jul 14 01:51:27 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 14 Jul 2008 08:51:27 +0900
Subject: [Python-Dev] unittest's redundant assertions: asserts
	vs.	failIf/Unlesses
In-Reply-To: <g5d03p$air$1@ger.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
	<g5d03p$air$1@ger.gmane.org>
Message-ID: <87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp>

Steve Holden writes:

 > "Fail" isn't a negative. As Guido said, it's a description of the test 
 > behavior under particular circumstances.

This is not true, however.  "Fail" is a description of a potentailly
very large class of behaviors.  Knowing that the test failed does not
tell you which of the failure behaviors occurred, only that the
success behavior did not occur.

The analogy to the fact that True != not not 10 is telling, I think.


From steve at pearwood.info  Mon Jul 14 01:46:03 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 14 Jul 2008 09:46:03 +1000
Subject: [Python-Dev]
	=?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?=
	=?iso-8859-1?q?asserts_vs=2E=09failIf/Unlesses?=
In-Reply-To: <004001c8e515$0fb08640$2f1192c0$@com.au>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<200807140134.54676.steve@pearwood.info>
	<004001c8e515$0fb08640$2f1192c0$@com.au>
Message-ID: <200807140946.05034.steve@pearwood.info>

On Mon, 14 Jul 2008 04:19:57 am Mark Hammond wrote:

> try to stick to the issue at hand

I'm sorry, did I reply to the wrong message? I thought this was part of 
a thread titled "unittest's redundant assertions: asserts vs.	
failIf/Unlesses". What *is* the issue at hand if not the question of 
positive assertions versus fail if/unless?

> As Steve said, this is getting silly...

It certainly is.

Python may be Guido's language, and if he wants to use his dictatorial 
powers to say that tests must be written as positive assertions because 
that's the way he likes it, that's his prerogative. But let's not 
pretend that this particular bike shed colour has any objectively 
rational reason, or that the change won't force some people to have to 
change the way they think about tests.



-- 
Steven

From steve at holdenweb.com  Mon Jul 14 01:54:16 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 13 Jul 2008 19:54:16 -0400
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <200807140946.05034.steve@pearwood.info>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<200807140134.54676.steve@pearwood.info>	<004001c8e515$0fb08640$2f1192c0$@com.au>
	<200807140946.05034.steve@pearwood.info>
Message-ID: <g5e4ja$8rg$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Mon, 14 Jul 2008 04:19:57 am Mark Hammond wrote:
> 
>> try to stick to the issue at hand
> 
> I'm sorry, did I reply to the wrong message? I thought this was part of 
> a thread titled "unittest's redundant assertions: asserts vs.	
> failIf/Unlesses". What *is* the issue at hand if not the question of 
> positive assertions versus fail if/unless?
> 
>> As Steve said, this is getting silly...
> 
> It certainly is.
> 
> Python may be Guido's language, and if he wants to use his dictatorial 
> powers to say that tests must be written as positive assertions because 
> that's the way he likes it, that's his prerogative. But let's not 
> pretend that this particular bike shed colour has any objectively 
> rational reason, or that the change won't force some people to have to 
> change the way they think about tests.
> 
But sometimes even the wrong decision is better than no decision, and I 
suspect most if us can agree that it's better if everyone thinks about 
tests the same way. Otherwise test code is difficult to understand for 
some of its user population.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From bignose+hates-spam at benfinney.id.au  Mon Jul 14 02:06:48 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 10:06:48 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
	<487A8F7B.1090901@holdenweb.com>
Message-ID: <87d4lhba7b.fsf@benfinney.id.au>

Steve Holden <steve at holdenweb.com> writes:

> Michael Foord wrote:
> > Adding the following new asserts:
> >
> >    assertIn    (member, container, msg=None)
> >    assertNotIn     (member, container, msg=None)
> >    assertIs     (first, second, msg=None)
> >    assertNotIs   (first, second, msg=None)
> 
> Please, let's call this one "assertIsNot". I know it's valid Python
> to say
> 
>   if a not is b:
> 
> but it's a much less natural way of expressing the condition, and
> (for all I know) might even introduce an extra negation operation.
> "is not" is, I believe, treated as a single operator.

Dang. You're exactly right.

The problem is, that makes it quite inconsistent with other "not" uses
(such as "assert_not_equal", "assert_not_in", etc.) I would really
prefer that all these "not" uses be gramatically consistent for
predictability. Is this a case where "assert_is_not" should exist
alongside "assert_not_is"?

I know that part of the goal here is to have "preferably only one
obvious way to do it", but I can see *both* those names as "the
obvious way to do it". Is this an instance where the "preferably"
clause must be exercised in the negative?

-- 
 \                ?Every sentence I utter must be understood not as an |
  `\                      affirmation, but as a question.? ?Niels Bohr |
_o__)                                                                  |
Ben Finney


From mithrandi-python-dev at mithrandi.za.net  Mon Jul 14 01:41:23 2008
From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann)
Date: Mon, 14 Jul 2008 01:41:23 +0200
Subject: [Python-Dev] unittest's redundant assertions: asserts
	vs.	failIf/Unlesses
In-Reply-To: <87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
	<g5d03p$air$1@ger.gmane.org>
	<87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20080713234123.GA11170@mithrandi.net>

* Stephen J. Turnbull <stephen at xemacs.org> [2008-07-14 08:51:27 +0900]:

> The analogy to the fact that True != not not 10 is telling, I think.

What?

>>> True == (not not 10)
True
-- 
mithrandi, i Ainil en-Balandor, a faer Ambar
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080714/94860626/attachment.pgp>

From tim.peters at gmail.com  Mon Jul 14 02:32:12 2008
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 13 Jul 2008 20:32:12 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487A8F7B.1090901@holdenweb.com>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com>
Message-ID: <1f7befae0807131732x54f5ed4em23b42b5d5a23dcad@mail.gmail.com>

[Michael Foord]
>> ...
>> Adding the following new asserts:
>>
>> ...
>>   assertNotIs   (first, second, msg=None)

[Steve Holden]
> Please, let's call this one "assertIsNot".

+1

> I know it's valid Python to say
>
>  if a not is b:

Nope, that's a syntax error.

> but it's a much less natural way of expressing the condition, and (for all I
> know) might even introduce an extra negation operation. "is not" is, I
> believe, treated as a single operator.

"is not" and "not in" are both binary infix operators, not to be
confused with the distinct use of "not" on its own as a unary prefix
operator.  "not is" and "in not" are both gibberish.

>>> 1 is not 2
True
>>> 1 is (not 2)
False
>>> 1 not is 2
SyntaxError: invalid syntax

>>> 1 not in [2]
True
>>> 1 in not [2]
SyntaxError: invalid syntax
>>> 1 in (not [2])
Traceback (most recent call last):
   ...
TypeError: argument of type 'bool' is not iterable

From exarkun at divmod.com  Mon Jul 14 02:58:48 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sun, 13 Jul 2008 20:58:48 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487A8700.8000200@voidspace.org.uk>
Message-ID: <20080714005848.4714.1762448186.divmod.quotient.20097@ohm>

On Sun, 13 Jul 2008 23:51:44 +0100, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>Ben Finney wrote:
>>Howdy Michael,
>>
>>I'm interested in the changes you're proposing for Python's 'unittest' 
>>module. I am (like, I suspect, many Python coders) maintaining my own set 
>>of extensions to the module across many projects, so I'd really like to see 
>>many of the improvements you discuss actually in the standard library.
>>
>>What assistance can I offer to help on this issue?
>>
>>
>I intend to start working on them in August, after I have finished my 
>current writing commitments.
>
>The full list of changes proposed (feel free to start - but ping me or the 
>list) and not shot down was something like:
>
>Documenting that the assert method names are to be preferred over the 
>'FailUnless' names (this stirred up some controversy this weekend so should 
>probably not happen).
>
>
>Adding the following new asserts:
>
>    assertIn    (member, container, msg=None)
>    assertNotIn     (member, container, msg=None)
>    assertIs     (first, second, msg=None)
>    assertNotIs   (first, second, msg=None)
>    assertRaisesWithMessage    (exc_class, message, callable, *args, 
>**keywargs)
>

Several of these are implemented in other libraries (Twisted, at least).
You might save some time by grabbing them and their unit tests, rather
than re-implementing them.  Twisted calls `assertIs? `assertIdentical?,
by the way.

> [snip]
>
>Other suggestions that weren't controversial but I might not get to:
>
>    assertRaisesWithMessage taking a regex to match the error message
>

Actually, I remember that someone raised an object to this as being not
as flexible as some might want - an objection I agree with.  Perhaps that
was overruled, but I didn't want this to slip by as "not controversial".

>    expect methods that collect failures and report at the end of the test 
>(allowing an individual test method to raise several errors without 
>stopping)
>
>    assertIsInstance and assertIsSubclass
>

The former of these is also in Twisted already, if you want to copy it.

Jean-Paul

From steve at holdenweb.com  Mon Jul 14 03:06:10 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 13 Jul 2008 21:06:10 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <1f7befae0807131732x54f5ed4em23b42b5d5a23dcad@mail.gmail.com>
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>
	<487A8F7B.1090901@holdenweb.com>
	<1f7befae0807131732x54f5ed4em23b42b5d5a23dcad@mail.gmail.com>
Message-ID: <g5e8q5$hjl$1@ger.gmane.org>

Tim Peters wrote:
> [Michael Foord]
>>> ...
>>> Adding the following new asserts:
>>>
>>> ...
>>>   assertNotIs   (first, second, msg=None)
> 
> [Steve Holden]
>> Please, let's call this one "assertIsNot".
> 
> +1
> 
>> I know it's valid Python to say
>>
>>  if a not is b:
> 
> Nope, that's a syntax error.
> 
Rats, naturally I was thinking of "if not (a is b):"

>> but it's a much less natural way of expressing the condition, and (for all I
>> know) might even introduce an extra negation operation. "is not" is, I
>> believe, treated as a single operator.
> 
> "is not" and "not in" are both binary infix operators, not to be
> confused with the distinct use of "not" on its own as a unary prefix
> operator.  "not is" and "in not" are both gibberish.
> 
>>>> 1 is not 2
> True
>>>> 1 is (not 2)
> False
>>>> 1 not is 2
> SyntaxError: invalid syntax
> 
>>>> 1 not in [2]
> True
>>>> 1 in not [2]
> SyntaxError: invalid syntax
>>>> 1 in (not [2])
> Traceback (most recent call last):
>    ...
> TypeError: argument of type 'bool' is not iterable

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From greg.ewing at canterbury.ac.nz  Mon Jul 14 03:05:34 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 14 Jul 2008 13:05:34 +1200
Subject: [Python-Dev] unittest's redundant assertions:	asserts	vs.
 failIf/Unlesses
In-Reply-To: <87prpic65w.fsf@benfinney.id.au>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
	<87prpic65w.fsf@benfinney.id.au>
Message-ID: <487AA65E.6080007@canterbury.ac.nz>

Ben Finney wrote:

> That's another reason to avoid "assert" in the name: these methods
> *don't* necessarily use the 'assert' statement. Avoiding the
> implication that they do use that is a good thing.

Perhaps some positive alternative such as "verify" could
be used instead.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Mon Jul 14 03:14:28 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 14 Jul 2008 13:14:28 +1200
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
 failIf/Unlesses
In-Reply-To: <g5d03p$air$1@ger.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org>
Message-ID: <487AA874.2010205@canterbury.ac.nz>

Steve Holden wrote:

> "Fail" isn't a negative.

That depends on what you're trying to find out by reading
the code. If you're trying to find out under what conditions
the test succeeds, then it succeeds if it doesn't fail, so
you have a negative.

Whichever convention is chosen, there will be situations in
which you want to mentally negate it. If you start with a
positive, the mental negation produces a single negative.
If you start with a negative, the mental negation produces
a double negative. Therefore it is less confusing overall
to start with as few negatives as possible.

-- 
Greg

From matt.giuca at gmail.com  Mon Jul 14 05:22:23 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Mon, 14 Jul 2008 13:22:23 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <8249917582531821653@unknownmsgid>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
Message-ID: <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>

On Mon, Jul 14, 2008 at 4:54 AM, Andr? Malo <nd at perlig.de> wrote:

>
> Ahem. The HTTP standard does ;-)
>

Really? Can you include a quotation please? The HTTP standard talks a lot
about ISO-8859-1 (Latin-1) in terms of actually raw encoded bytes, but not
in terms of URI percent-encoding (a different issue) as far as I can tell.


>
> > Where web forms are concerned, basically setting the form accept-charset
> > or the page charset is the *maximum amount* of control you have over the
> > encoding. As you say, it can be encoded by another page or the user can
> > override their settings. Then what can you do as the server? Nothing ...
>
> Guessing works pretty well in most of the cases.
>

Are you suggesting that urllib.unquote guess the encoding? It could do that
but it would make things rather unpredictable. I think if this was an
application (such as a web browser), then guessing is OK. But this is a
library function. Library functions should not make arbitrary decisions;
they should be well-specified.

Latin-1 is not exactly arbitray. Besides being a charset - it maps
> one-to-one to octet values, hence it's commonly used to encode octets and
> is therefore a better fallback than every other encoding.
>

True. So the only advantage I see to the current implementation is that if
you really want to, you can take the Latin-1-decoded URI (from unquote) and
explicitly encode it as Latin-1 and then decode it again as whatever
encoding you want. But that would be a hack, would it not? I'd prefer if the
library didn't require a hack just to get the extremely common use case
(UTF-8).


>
> > I agree. However if there *was* a proper standard we wouldn't have to
> > argue! "Most proper" and "should do" is the most confident we can be when
> > dealing with this standard, as there is no correct encoding.
>
> Well, the standard says, there are octets to be encoded. I find that proper
> enough.


Yes but unfortunately we aren't talking about octets any more in Python 3,
but characters. If we're going to follow the standard and encode octets,
then we should be accepting (for quote) and returning (for unquote) bytes
objects, not strings. But as that's going to break most existing code and be
extremely confusing, I think it's best we try and solve this problem for
Unicode strings.


> > Does anyone have a suggestion which will be more compatible with the rest
> > of the world than allowing the user to select an encoding, and defaulting
> > to "utf-8"?
>
> Default to latin-1 for decoding and utf-8 for encoding. This might be
> confusing though, so maybe you've asked the wrong question ;)
>

:o that would break so so much existing code, not to mention being horribly
inconsistent and confusing. Having said that, that's almost what the current
behaviour is (quote uses Latin-1 for characters < 256, and UTF-8 for
characters above; unquote uses Latin-1).

Again I bring up the http server example. If you go to a directory, create a
file with a name such as '??', and then run this code in Python 3.0 from
that directory:

import http.server
s = http.server.HTTPServer(('',8000),
        http.server.SimpleHTTPRequestHandler)
s.serve_forever()

You'll see the file in the directory listing - its HTML will be <a
href="%E6%BC%A2%E5%AD%97">??</a>. But if you click it, you get a 404 because
the server will look for the file named unquote("%E6%BC%A2%E5%AD%97") =
'????\xad\x97'.

If you apply my patch (patch5) *everything* *just* *works*.


On Mon, Jul 14, 2008 at 6:36 AM, Bill Janssen <janssen at parc.com> wrote:

> > Ah there may be some confusion here. We're only dealing with str->str
> > transformations (which in Python 3 means Unicode strings). You can't put
> a
> > bytes in or get a bytes out of either of these functions. I suggested a
> > "quote_raw" and "unquote_raw" function which would let you do this.
>
> Ah, well, that's a problem.  Clearly the unquote is str->bytes, while
> the quote is (bytes OR str)->str.
>

OK so for quote, you're suggesting that we accept either a bytes or a str
object. That sounds quite reasonable (though neither the unpatched or
patched versions accept a bytes at the moment). I'd simply change the code
in quote (from patch5) to do this:

if isinstance(s, str):
    s = s.encode(encoding, errors)
....
res = map(quoter, s)

Now you get this behaviour by default (which may appear confusing but I'd
argue correct given the different semantics of 'h\xfcllo' and b'h\xfcllo'):

>>> urllib.parse.quote(b'h\xfcllo')
'h%FCllo'           # Directly-encoded octets
>>> urllib.parse.quote('h\xfcllo')
'h%C3%BCllo'     # UTF-8 encoded string, then encoded octets

Clearly the unquote is str->bytes, <snip> You can't pass a Unicode string
> back
> as the result of unquote *without* passing in an encoding specifier,
> because the character set is application-specific.
>

So for unquote you're suggesting that it always return a bytes object UNLESS
an encoding is specified? As in:

>>> urllib.parse.unquote('h%C3%BCllo')
b'h\xc3\xbcllo'

I would object to that on two grounds. Firstly, I wouldn't expect or desire
a bytes object. The vast majority of uses for unquote will be to get a
character string out, not bytes. Secondly, there is a mountain of code
(including about 12 modules in the standard library) which call unquote and
don't give the user the encoding option, so it's best if we pick a default
that is what the majority of users will expect. I argue that that's UTF-8.

I'd prefer having a separate unquote_raw function which is str->bytes, and
the unquote function performs the same role as it always have, which is
str->str. But I agree on quote, I think it can be (bytes OR str)->str.

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080714/e2c64a4a/attachment-0001.htm>

From schmir at gmail.com  Mon Jul 14 07:03:55 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Mon, 14 Jul 2008 07:03:55 +0200
Subject: [Python-Dev] AMD64-W2k8 buildbot wedged
In-Reply-To: <487A2EE5.9020302@v.loewis.de>
References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de>
Message-ID: <932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com>

On Sun, Jul 13, 2008 at 6:35 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Nick Coghlan wrote:
>> The compilation step on this buildbot is failing because it can't delete
>> or overwrite any of the Python DLLs [1]. Is there any way to fix this
>> remotely, or at least to tell why kill_python isn't doing the trick?
>
> That is in the log:
>
> TerminateProcess failed: 5
>
> where 5 is ERROR_ACCESS_DENIED. Now, *why* the system is refusing to
> terminate the process is difficult to say. It's a general Windows
> problem: the system doesn't support forced process termination in a
> reliable manner.
>
> In any case, Trent seems to have fixed the problem.
>
>> The number of 64-bit safeness
>> warnings being emitted by the current trunk is also fairly worrying)
>
> Do you have a specific one in mind? The ones truncating size_t/ssize_t
> should only matter when you actually do have data larger than 2GiB.
>

http://bugs.python.org/issue3026 comes to mind.

And I would rather use a little bit different wording: The ones
truncating size_t/ssize_t do matter, unless  you know in advance that
you will always deal with data lesser than 2GiB.
Regards,
- Ralf

From martin at v.loewis.de  Mon Jul 14 07:16:20 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 14 Jul 2008 07:16:20 +0200
Subject: [Python-Dev] AMD64-W2k8 buildbot wedged
In-Reply-To: <932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com>
References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de>
	<932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com>
Message-ID: <487AE124.1030301@v.loewis.de>

> http://bugs.python.org/issue3026 comes to mind.
> 
> And I would rather use a little bit different wording: The ones
> truncating size_t/ssize_t do matter, unless  you know in advance that
> you will always deal with data lesser than 2GiB.

I thought Nick's comment was in the context of the buildbots hanging
in the multiprocessing tests, which I know has only data smaller than
2GiB.

Regards,
Martin

From stephen at xemacs.org  Mon Jul 14 08:27:40 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 14 Jul 2008 15:27:40 +0900
Subject: [Python-Dev] unittest's redundant assertions:
	asserts	vs.	failIf/Unlesses
In-Reply-To: <20080713234123.GA11170@mithrandi.net>
References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<8763radte5.fsf@benfinney.id.au>
	<20080713114653.GE9654@steerpike.home.puzzling.org>
	<loom.20080713T121738-271@post.gmane.org>
	<87prpic65w.fsf@benfinney.id.au>
	<loom.20080713T130302-168@post.gmane.org>
	<g5d03p$air$1@ger.gmane.org>
	<87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080713234123.GA11170@mithrandi.net>
Message-ID: <87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp>

Tristan Seligmann writes:

 > * Stephen J. Turnbull <stephen at xemacs.org> [2008-07-14 08:51:27 +0900]:
 > 
 > > The analogy to the fact that True != not not 10 is telling, I think.
 > 
 > What?
 > 
 > >>> True == (not not 10)
 > True

FWIW, I meant 10 != not not 10.


From ncoghlan at gmail.com  Mon Jul 14 11:14:20 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 14 Jul 2008 19:14:20 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <87d4lhba7b.fsf@benfinney.id.au>
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>	<487A8F7B.1090901@holdenweb.com>
	<87d4lhba7b.fsf@benfinney.id.au>
Message-ID: <487B18EC.6040104@gmail.com>

Ben Finney wrote:
> Steve Holden <steve at holdenweb.com> writes:
> 
>> Michael Foord wrote:
>>> Adding the following new asserts:
>>>
>>>    assertIn    (member, container, msg=None)
>>>    assertNotIn     (member, container, msg=None)
>>>    assertIs     (first, second, msg=None)
>>>    assertNotIs   (first, second, msg=None)
>> Please, let's call this one "assertIsNot". I know it's valid Python
>> to say
>>
>>   if a not is b:
>>
>> but it's a much less natural way of expressing the condition, and
>> (for all I know) might even introduce an extra negation operation.
>> "is not" is, I believe, treated as a single operator.
> 
> Dang. You're exactly right.
> 
> The problem is, that makes it quite inconsistent with other "not" uses
> (such as "assert_not_equal", "assert_not_in", etc.) I would really
> prefer that all these "not" uses be gramatically consistent for
> predictability. Is this a case where "assert_is_not" should exist
> alongside "assert_not_is"?

If we can flip the word order in the language syntax, we can sure as 
heck flip it in a method name :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Mon Jul 14 11:16:33 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 14 Jul 2008 19:16:33 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487A8700.8000200@voidspace.org.uk>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
Message-ID: <487B1971.105@gmail.com>

Michael Foord wrote:
> Ben Finney wrote:
>> Howdy Michael,
>>
>> I'm interested in the changes you're proposing for Python's 'unittest' 
>> module. I am (like, I suspect, many Python coders) maintaining my own 
>> set of extensions to the module across many projects, so I'd really 
>> like to see many of the improvements you discuss actually in the 
>> standard library.
>>
>> What assistance can I offer to help on this issue?
>>
>>   
> I intend to start working on them in August, after I have finished my 
> current writing commitments.

Would it be worth Ben collating your current notes into a draft PEP 
targeting 2.7/3.1?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From hrvoje.niksic at avl.com  Mon Jul 14 10:50:50 2008
From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=)
Date: Mon, 14 Jul 2008 10:50:50 +0200
Subject: [Python-Dev] xmlrpclib.{True,
	False} (was Re:  Assignment to	None)
In-Reply-To: <4856CCBC.5040003@egenix.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<200806091348.44361.steve@pearwood.info>
	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>
	<484EA327.2050704@vector-seven.com>	<48551508.5060500@vector-seven.com>
	<48551A56.1020704@vector-seven.com>	<g338l9$be$1@ger.gmane.org>
	<485529EF.3080209@vector-seven.com> <g33a2m$46q$1@ger.gmane.org>
	<4856CCBC.5040003@egenix.com>
Message-ID: <1216025450.8943.4.camel@localhost>

On Mon, 2008-06-16 at 22:27 +0200, M.-A. Lemburg wrote:
> On 2008-06-15 16:47, Georg Brandl wrote:
> > Thomas Lee schrieb:
> >> Georg Brandl wrote:
> >>> Remember that it must still be possible to write (in 2.6)
> >>>
> >>> True = 0
> >>> assert not True
> >>
> >> Ah of course. Looks like I should just avoid optimizations of 
> >> Name("True") and Name("False") all together. That's a shame!
> > 
> > We can of course decide to make assignment to True and False
> > illegal in 2.7 :)
> 
> Raising a run-time exception would be fine, but not a SyntaxError at
> compile time - this would effectively make it impossible to keep
> code compatible to Python 2.1.

Maybe it wouldn't.  Something like:

try:
    True, False
except NameError:
    globals()['True'] = 1
    globals()['False']

should still work for compatibility.  It would break *existing* code
that tries to be compatible with old releases, but that is unavoidable
in making something illegal that was previously legal.  In this case,
the price is justified by being able to optimize access to None, True,
and False without having to resort to dirty tricks such as
"none=None" (previously None=None) in functions.  It should also enable
the compiler to optimize "while True:" the way it optimizes "while 1:".



From ncoghlan at gmail.com  Mon Jul 14 11:22:37 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 14 Jul 2008 19:22:37 +1000
Subject: [Python-Dev] AMD64-W2k8 buildbot wedged
In-Reply-To: <487AE124.1030301@v.loewis.de>
References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de>
	<932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com>
	<487AE124.1030301@v.loewis.de>
Message-ID: <487B1ADD.6030109@gmail.com>

Martin v. L?wis wrote:
>> http://bugs.python.org/issue3026 comes to mind.
>>
>> And I would rather use a little bit different wording: The ones
>> truncating size_t/ssize_t do matter, unless  you know in advance that
>> you will always deal with data lesser than 2GiB.
> 
> I thought Nick's comment was in the context of the buildbots hanging
> in the multiprocessing tests, which I know has only data smaller than
> 2GiB.

Ah, sorry about the confusion - my chain of thought was a little more 
convoluted than that. The wedged buildbot caused the compile to fail 
after a checkin of mine, which lead to me looking at that buildbot's 
compile log, which had all sorts of warnings which I had never seen 
before because my development machine is a 32-bit Linux box.

Given that I thought all those warnings had been cleared out when 
Py_ssize_t was first added, I was a little surprised by the quantity of 
them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From bignose+hates-spam at benfinney.id.au  Mon Jul 14 12:05:22 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 20:05:22 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
	<487A8F7B.1090901@holdenweb.com>
	<87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com>
Message-ID: <874p6sbx25.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> Ben Finney wrote:
> > The problem is, that makes it quite inconsistent with other "not"
> > uses (such as "assert_not_equal", "assert_not_in", etc.) I would
> > really prefer that all these "not" uses be gramatically consistent
> > for predictability. Is this a case where "assert_is_not" should
> > exist alongside "assert_not_is"?
> 
> If we can flip the word order in the language syntax, we can sure as
> heck flip it in a method name :)

To be clear, I take it you're in favour of the following names (with
no aliases):

    assert_equal                assert_not_equal
    assert_is                   assert_is_not
    assert_in                   assert_not_in
    assert_almost_equal         assert_not_almost_equal

and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by
the others, in the interest of matching Python's 'is not' grammar.

-- 
 \       ?Instead of having ?answers? on a math test, they should just |
  `\               call them ?impressions?, and if you got a different |
_o__)   ?impression?, so what, can't we all be brothers?? ?Jack Handey |
Ben Finney


From steve at holdenweb.com  Mon Jul 14 14:45:06 2008
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 14 Jul 2008 08:45:06 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <874p6sbx25.fsf@benfinney.id.au>
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>	<487A8F7B.1090901@holdenweb.com>	<87d4lhba7b.fsf@benfinney.id.au>
	<487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au>
Message-ID: <g5fhol$411$1@ger.gmane.org>

Ben Finney wrote:
> Nick Coghlan <ncoghlan at gmail.com> writes:
> 
>> Ben Finney wrote:
>>> The problem is, that makes it quite inconsistent with other "not"
>>> uses (such as "assert_not_equal", "assert_not_in", etc.) I would
>>> really prefer that all these "not" uses be gramatically consistent
>>> for predictability. Is this a case where "assert_is_not" should
>>> exist alongside "assert_not_is"?
>> If we can flip the word order in the language syntax, we can sure as
>> heck flip it in a method name :)
> 
> To be clear, I take it you're in favour of the following names (with
> no aliases):
> 
>     assert_equal                assert_not_equal
>     assert_is                   assert_is_not
>     assert_in                   assert_not_in
>     assert_almost_equal         assert_not_almost_equal
> 
> and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by
> the others, in the interest of matching Python's 'is not' grammar.
> 
Well, I'd have said "in the interest of reading correctly in English", 
though I have to acknowledge this may not be an issue for many Python 
users whose first language not is English. "assert_not_is" is just 
dissonant to my ears.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From bignose+hates-spam at benfinney.id.au  Mon Jul 14 15:17:32 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 23:17:32 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk> <487B1971.105@gmail.com>
Message-ID: <87zloka9lf.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> Would it be worth Ben collating your current notes into a draft PEP
> targeting 2.7/3.1?

I'll do it and we'll find out.

-- 
 \         ?A fine is a tax for doing wrong. A tax is a fine for doing |
  `\                                                 well.? ?anonymous |
_o__)                                                                  |
Ben Finney


From ncoghlan at gmail.com  Mon Jul 14 15:17:59 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 14 Jul 2008 23:17:59 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <g5fhol$411$1@ger.gmane.org>
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>	<487A8F7B.1090901@holdenweb.com>	<87d4lhba7b.fsf@benfinney.id.au>	<487B18EC.6040104@gmail.com>
	<874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org>
Message-ID: <487B5207.2020904@gmail.com>

Steve Holden wrote:
> Ben Finney wrote:
>> To be clear, I take it you're in favour of the following names (with
>> no aliases):
>>
>>     assert_equal                assert_not_equal
>>     assert_is                   assert_is_not
>>     assert_in                   assert_not_in
>>     assert_almost_equal         assert_not_almost_equal
>>
>> and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by
>> the others, in the interest of matching Python's 'is not' grammar.
>>
> Well, I'd have said "in the interest of reading correctly in English", 
> though I have to acknowledge this may not be an issue for many Python 
> users whose first language not is English. "assert_not_is" is just 
> dissonant to my ears.

The two reasons aren't that far apart, given that Python's grammar uses 
"is not" because it makes more sense in English. One thing to remember 
is that the word 'is' is actually implied in all of the contracted 
phrases above other than those already including it explicitly.

"x is equal to y"
"x is not equal to y"
"x is y"
"x is not y"
"x is in y"
"x is not in y"
"x is almost equal to y"
"x is not almost equal to y"

As for which phrasing I personally prefer, unit tests and method names 
are areas where I'm quite happy to paint the bike shed the same colour 
as the house :)

Cheers,
Nick.

P.S. Deciphering that somewhat strained metaphor: I don't have a strong 
preference with regards to the unit test method names. While I tend to 
go with the assert* variants when left to my own devices, I have no 
problem sticking to the fail* variants when updating a test that uses 
them. Camel-case vs underscores in method names isn't something that 
particularly worries me either.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From bignose+hates-spam at benfinney.id.au  Mon Jul 14 15:41:19 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 23:41:19 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
	<487A8F7B.1090901@holdenweb.com>
	<87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com>
	<874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org>
Message-ID: <87r69wa8hs.fsf@benfinney.id.au>

Steve Holden <steve at holdenweb.com> writes:

> Ben Finney wrote:
> > and so on; i.e. that 'assert_is_not' breaks the obvious pattern
> > set by the others, in the interest of matching Python's 'is not'
> > grammar.
> 
> Well, I'd have said "in the interest of reading correctly in English",
> though I have to acknowledge this may not be an issue for many Python
> users whose first language not is English. "assert_not_is" is just
> dissonant to my ears.

I'd count this as another (minor) point in favour of making the
'fail*' methods canonical: the names are consistent *and* gramatically
sensible:

    fail_if_equal               fail_unless_equal
    fail_if_is                  fail_unless_is
    fail_if_in                  fail_unless_in
    fail_if_almost_equal        fail_unless_almost_equal

-- 
 \     ?We are not gonna be great; we are not gonna be amazing; we are |
  `\           gonna be *amazingly* amazing!? ?Zaphod Beeblebrox, _The |
_o__)                Hitch-Hiker's Guide To The Galaxy_, Douglas Adams |
Ben Finney


From bignose+hates-spam at benfinney.id.au  Mon Jul 14 15:43:14 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 23:43:14 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
Message-ID: <87mykka8el.fsf@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> The full list of changes proposed (feel free to start - but ping me or
> the list) and not shot down was something like:
[?]

Thanks. I'm working these into another draft PEP that I hope to have
up in a day or two.

-- 
 \     ?[W]e are still the first generation of users, and for all that |
  `\      we may have invented the net, we still don't really get it.? |
_o__)                                                   ?Douglas Adams |
Ben Finney


From python at rcn.com  Mon Jul 14 15:59:22 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Jul 2008 06:59:22 -0700
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk>
	<87mykka8el.fsf@benfinney.id.au>
Message-ID: <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>

> Michael Foord <fuzzyman at voidspace.org.uk> writes:
>
>> The full list of changes proposed (feel free to start - but ping me or
>> the list) and not shot down was something like:
> [?]
>
> Thanks. I'm working these into another draft PEP that I hope to have
> up in a day or two.


Given all of the language changes in 2.6 and 3.0, I would think
that it is dangerous to make any changes at all to the unittest API.
That module is the one anchor in a sea of change.


Raymond




From fuzzyman at voidspace.org.uk  Mon Jul 14 16:21:47 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 14 Jul 2008 15:21:47 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk>	<87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
Message-ID: <487B60FB.5010602@voidspace.org.uk>

Raymond Hettinger wrote:
>> Michael Foord <fuzzyman at voidspace.org.uk> writes:
>>
>>> The full list of changes proposed (feel free to start - but ping me or
>>> the list) and not shot down was something like:
>> [?]
>>
>> Thanks. I'm working these into another draft PEP that I hope to have
>> up in a day or two.
>
>
> Given all of the language changes in 2.6 and 3.0, I would think
> that it is dangerous to make any changes at all to the unittest API.
> That module is the one anchor in a sea of change.
>
As proposed the changes don't remove or rename anything - so there will 
be no code breakage, just additional test methods.

However, as we're into the beta phase I don't think these changes can 
make 2.6 / 3.0 anyway.

Michael


>
> Raymond
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From janssen at parc.com  Mon Jul 14 19:39:42 2008
From: janssen at parc.com (Bill Janssen)
Date: Mon, 14 Jul 2008 10:39:42 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
	<e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>
Message-ID: <08Jul14.103943pdt."58698"@synergy1.parc.xerox.com>

>> Clearly the unquote is str->bytes, <snip> You can't pass a Unicode string back
>> as the result of unquote *without* passing in an encoding specifier,
>> because the character set is application-specific.
> So for unquote you're suggesting that it always return a bytes object
> UNLESS an encoding is specified? As in:
> >> urllib.parse.unquote('h%C3%BCllo')
> b'h\xc3\xbcllo'

Yes, that's correct.  That's what the RFC says we have to do.

> I would object to that on two grounds. Firstly, I wouldn't expect or
> desire a bytes object. The vast majority of uses for unquote will be
> to get a character string out, not bytes. Secondly, there is a
> mountain of code (including about 12 modules in the standard library)
> which call unquote and don't give the user the encoding option, so
> it's best if we pick a default that is what the majority of users will
> expect. I argue that that's UTF-8.

Unfortunately, despite your expectations or desires, the spec doesn't
allow us that luxury.  It's bytes out, and they may even be in a
non-standard (not registered with IANA) encoding.  There's no way to
safely and correctly turn that sequence of bytes into a string.  If
other modules have been mis-using the interface, they are buggy and
should be fixed.  There's a lot of buggy stdlib code in Python around
the older Web standards.

I think it would be great to have another function, unquote_to_string,
which took an extra "encoding" parameter, and returned a string.  It
would also be OK to add a keyword parameter to "unquote", I think,
which provides an encoding, and causes unquote to return a string.
But the standard behavior has to be to return bytes.

> I'd prefer having a separate unquote_raw function which is
> str->bytes, and the unquote function performs the same role as it
> always have, which is str->str.

Actually, it was originally bytes->bytes, because there was no notion
of Unicode strings when it was added.  It perhaps got misunderstood
during the addition of Unicode support to Python; many people have had
trouble wrapping their heads around all this, myself included.

Bill

From ben+python at benfinney.id.au  Mon Jul 14 15:25:32 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 14 Jul 2008 23:25:32 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`
	module
Message-ID: <87vdz8a983.fsf@benfinney.id.au>

:PEP:           XXX
:Title:         Consolidating names and classes in the `unittest` module
:Version:       0.0
:Last-Modified: 2008-07-14
:Author:        Ben Finney <ben+python at benfinney.id.au>
:Status:        Draft
:Type:          Standards Track
:Content-Type:  test/x-rst
:Created:       2008-07-14
:Post-History:


..  contents::


Abstract
========

This PEP proposes to consolidate the names and classes that constitute
the API of the standard library `unittest` module, with the goal of
removing redundant names, conforming with PEP 8, and eliminating
classic classes.


Motivation
==========

The normal use case for the `unittest` module is to subclass its
classes, overriding and re-using its functios and methods. This draws
constant attention to the fact that the existing implementation fails
several current Python standards:

* It does not use new-style classes, preventing e.g. straightforward
  use of ``super`` for calling the fixture set-up and tear-down
  methods.

* It does not conform to PEP 8, requiring users to write their own
  non-PEP-8 conformant names when overriding methods, and encouraging
  extensions to further depart from PEP 8.

* It has many synonyms in its API, which goes against the Zen of
  Python (specifically, that "there should be one, and preferably only
  one, obvious way to do it").


Specification
=============

Use new-style classes throughout
--------------------------------

The following classes will inherit explicitly from the built-in
`object` type, to make all classes in the module part of the new-style
type hierarchy.

* ``TestResult``
* ``TestCase``
* ``TestSuite``
* ``TestLoader``
* ``_WritelnDecorator``
* ``TextTestRunner``
* ``TestProgram``

Remove obsolete names
---------------------

The following attributes are not documented as part of the API and are
marked as obsolete in the implementation. They will be removed.

* ``_makeLoader``
* ``getTestCaseNames``
* ``makeSuite``
* ``findTestCases``

Remove redundant names
----------------------

The following attribute names exist only as synonyms for other names.
They are to be removed, leaving only one name for each attribute in
the API.

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

* ``assertEqual``
* ``assertEquals``
* ``assertNotEqual``
* ``assertNotEquals``
* ``assertAlmostEqual``
* ``assertAlmostEquals``
* ``assertNotAlmostEqual``
* ``assertNotAlmostEquals``
* ``assertRaises``
* ``assert_``
* ``assertTrue``
* ``assertFalse``

Conform API with PEP 8
----------------------

The following names are to be introduced, each replacing an existing
name, to make all names in the module conform with PEP 8. Each name is
shown with the existing name that it replaces.

Where function parameters are to be renamed also, they are shown.
Where function parameters are not to be renamed, they are elided with
the ellipse ("?") symbol.

Module attributes
~~~~~~~~~~~~~~~~~

``_make_loader(prefix, sort_using, suite_class)``
  Replaces ``_makeLoader (prefix, sortUsing, suiteClass)``

``find_test_cases(module, prefix, sort_using, suite_class)``
  Replaces ``findTestCases(module, prefix, sortUsing, suiteClass)``

``get_test_case_names(test_case_class, prefix, sort_using)``
  Replaces ``getTestCaseNames(testCaseClass, prefix, sortUsing)``

``make_suite(test_case_class, prefix, sort_using, suite_class)``
  Replaces ``makeSuite(testCaseClass, prefix, sortUsing, suiteClass)``

``default_test_loader``
  Replaces ``defaultTestLoader``

``TestResult`` atributes
~~~~~~~~~~~~~~~~~~~~~~~~

``add_error(?)``
  Replaces ``addError(?)``

``add_result(?)``
  Replaces ``addResult(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``should_stop``
  Replaces ``shouldStop``

``start_test(?)``
  Replaces ``startTest(?)``

``stop_test(?)``
  Replaces ``stopTest(?)``

``tests_run``
  Replaces ``testsRun``

``was_successful(?)``
  Replaces ``wasSuccessful(?)``

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, method_name='run_test')``
  Replaces ``__init__(self, methodName='runTest')``

``_test_method_doc``
  Replaces ``_testMethodDoc``

``_test_method_name``
  Replaces ``_testMethodName``

``failure_exception``
  Replaces ``failureException``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``default_test_result(?)``
  Replaces ``defaultTestResult(?)``

``fail_if(?)``
  Replaces ``failIf(?)``

``fail_if_almost_equal(?)``
  Replaces ``failIfAlmostEqual(?)``

``fail_if_equal(?)``
  Replaces ``failIfEqual(?)``

``fail_unless(?)``
  Replaces ``failUnless(?)``

``fail_unless_almost_equal(?)`` 
  Replaces ``failUnlessAlmostEqual(?)``

``fail_unless_equal(?)``
  Replaces ``failUnlessEqual(?)``

``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)``
  Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``FunctionTestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, test_func, set_up, tear_down, description)``
  Replaces ``__init__(self, testFunc, setUp, tearDown, description)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``TestSuite`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~

``add_test(?)``
  Replaces ``addTest(?)``

``add_tests(?)``
  Replaces ``addTests(?)``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``TestLoader`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``sort_test_methods_using``
  Replaces ``sortTestMethodsUsing``

``suite_class``
  Replaces ``suiteClass``

``test_method_prefix``
  Replaces ``testMethodPrefix``

``get_test_case_names(self, test_case_class)``
  Replaces ``getTestCaseNames(self, testCaseClass)``

``load_tests_from_module(?)``
  Replaces ``loadTestsFromModule(?)``

``load_tests_from_name(?)``
  Replaces ``loadTestsFromName(?)``

``load_tests_from_names(?)``
  Replaces ``loadTestsFromNames(?)``

``load_tests_from_test_case(self, test_case_class)``
  Replaces ``loadTestsFromTestCase(self, testCaseClass)``

``_TextTestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``show_all``
  Replaces ``showAll``

``add_error(?)``
  Replaces ``addError(?)``

``add_failure(?)``
  Replaces ``addFailure(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``get_description(?)``
  Replaces ``getDescription(?)``

``print_error_list(?)``
  Replaces ``printErrorList(?)``

``print_errors(?)``
  Replaces ``printErrors(?)``

``start_test(?)``
  Replaces ``startTest(?)``

``TextTestRunner`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``_make_result(?)``
  Replaces ``_makeResult(?)``

``TestProgram`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, module, default_test, argv, test_runner, test_loader)``
  Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)``

``create_tests(?)``
  Replaces ``createTests(?)``

``parse_args(?)``
  Replaces ``parseArgs(?)``

``run_tests(?)``
  Replaces ``runTests(?)``

``usage_exit(?)``
  Replaces ``usageExit(?)``


Rationale
=========

New-style classes
-----------------

As a standard library module, `unittest` should not define any classic
classes.

Redundant names
---------------

The current API, with two or in some cases three different names
referencing exactly the same function, leads to an overbroad and
redundant API that violates PEP 20 ("there should be one, and
preferably only one, obvious way to do it").

PEP 8 names
-----------

Although `unittest` (and its predecessor `PyUnit`) are intended to be
familiar to users of other xUnit interfaces, there is no attempt at
direct API compatibility since the only code that Python's `unittest`
interfaces with is other Python code. The module is in the standard
library and its names should all conform with PEP 8.


Backwards Compatibility
=======================

The names to be obsoleted should be deprecated and removed according
to the schedule for modules in PEP 4.

While deprecated, use of the deprecated attributes should raise a
``DeprecationWarning``, with a message stating which replacement name
should be used.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From musiccomposition at gmail.com  Tue Jul 15 00:19:25 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 14 Jul 2008 17:19:25 -0500
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module
In-Reply-To: <87vdz8a983.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
Message-ID: <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com>

On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Use new-style classes throughout
> --------------------------------
>
> The following classes will inherit explicitly from the built-in
> `object` type, to make all classes in the module part of the new-style
> type hierarchy.
>
> * ``TestResult``
> * ``TestCase``
> * ``TestSuite``
> * ``TestLoader``
> * ``_WritelnDecorator``
> * ``TextTestRunner``
> * ``TestProgram``

They already do. __metaclass__ = type is found in unittest.py.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From ben+python at benfinney.id.au  Tue Jul 15 00:18:54 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 08:18:54 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
	<87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
	<487B60FB.5010602@voidspace.org.uk>
Message-ID: <87iqv89kj5.fsf@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> As proposed the changes don't remove or rename anything - so there
> will be no code breakage, just additional test methods.

Right, so I'm putting up a separate PEP just for the renaming. Should
be arriving on this list soon.

> However, as we're into the beta phase I don't think these changes
> can make 2.6 / 3.0 anyway.

Definitely agreed.

-- 
 \        ?You can be a victor without having victims.? ?Harriet Woods |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From bignose+hates-spam at benfinney.id.au  Tue Jul 15 00:21:27 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 08:21:27 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
	<87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
Message-ID: <87ej5w9kew.fsf@benfinney.id.au>

"Raymond Hettinger" <python at rcn.com> writes:

> Given all of the language changes in 2.6 and 3.0, I would think that
> it is dangerous to make any changes at all to the unittest API. That
> module is the one anchor in a sea of change.

Agreed. I'm not proposing to have the unittest API change at all in
Python 2.6 or 3.0. These changes, even the first deprecations, would
not be suitable for anything earlier than 2.7 and 3.1.

-- 
 \        ?When in doubt tell the truth. It will confound your enemies |
  `\   and astound your friends.? ?Mark Twain, _Following the Equator_ |
_o__)                                                                  |
Ben Finney


From steve at pearwood.info  Tue Jul 15 00:40:00 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 15 Jul 2008 08:40:00 +1000
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <g5e4ja$8rg$1@ger.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<200807140946.05034.steve@pearwood.info>
	<g5e4ja$8rg$1@ger.gmane.org>
Message-ID: <200807150840.01490.steve@pearwood.info>

On Mon, 14 Jul 2008 09:54:16 am Steve Holden wrote:
> > Python may be Guido's language, and if he wants to use his
> > dictatorial powers to say that tests must be written as positive
> > assertions because that's the way he likes it, that's his
> > prerogative. But let's not pretend that this particular bike shed
> > colour has any objectively rational reason, or that the change
> > won't force some people to have to change the way they think about
> > tests.
>
> But sometimes even the wrong decision is better than no decision, and
> I suspect most if us can agree that it's better if everyone thinks
> about tests the same way.

There's a term for what happens when everybody in a community or group 
thinks about a subject the same way.

http://en.wikipedia.org/wiki/Groupthink


-- 
Steven

From nas at arctrix.com  Tue Jul 15 00:43:43 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 14 Jul 2008 16:43:43 -0600
Subject: [Python-Dev] git repositories for trunk and py3k
Message-ID: <20080714224343.GA23048@arctrix.com>

Hi,

In case anyone is interested, I have git repositories for both the
trunk and the py3k branch of the Python source code.  They are
up-to-date and so using them with git-svn would be much faster than
starting from scratch.

If anyone is interested, I will find a place to host them.  They are
each about 150 MB in size.

  Neil

From fuzzyman at voidspace.org.uk  Tue Jul 15 01:08:30 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Jul 2008 00:08:30 +0100
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module
In-Reply-To: <87vdz8a983.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
Message-ID: <487BDC6E.2000606@voidspace.org.uk>

Ben Finney wrote:
> [snip..]
>
> Remove redundant names
> ----------------------
>
> The following attribute names exist only as synonyms for other names.
> They are to be removed, leaving only one name for each attribute in
> the API.
>
> ``TestCase`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~
>
> * ``assertEqual``
> * ``assertEquals``
> * ``assertNotEqual``
> * ``assertNotEquals``
> * ``assertAlmostEqual``
> * ``assertAlmostEquals``
> * ``assertNotAlmostEqual``
> * ``assertNotAlmostEquals``
> * ``assertRaises``
> * ``assert_``
> * ``assertTrue``
> * ``assertFalse``
>
>   
Although you may prefer the 'failIf' and 'failUnless' names, the 
consensus in the *last* discussion was that the 'assert*' names were to 
be preferred.

I protest the removal of the assert names - and in the absence of likely 
consensus (and barring a dictat of course) I suggest this part of the 
proposal be excised.

Michael


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From solipsis at pitrou.net  Tue Jul 15 01:19:09 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 14 Jul 2008 23:19:09 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP=3A_Consolidating_names_and_classes_in_?=
	=?utf-8?q?the_=60unittest=60=09module?=
References: <87vdz8a983.fsf@benfinney.id.au>
Message-ID: <loom.20080714T230912-310@post.gmane.org>

Ben Finney <ben+python <at> benfinney.id.au> writes:
> The following attribute names exist only as synonyms for other names.
> They are to be removed, leaving only one name for each attribute in
> the API.

Just for information, here is the current distribution of the two synonym kinds:
(on py3k)

$ grep "self.assert" Lib/test/test_*.py | wc -l
14972
$ grep "self.fail" Lib/test/test_*.py | wc -l
1807

If no rational argument prevails, at least we have data on the past and current
habits of the python-dev community.

regards

Antoine.



From ben+python at benfinney.id.au  Tue Jul 15 01:18:57 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 09:18:57 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module
References: <87vdz8a983.fsf@benfinney.id.au>
	<1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com>
Message-ID: <87abgk9hr2.fsf@benfinney.id.au>

"Benjamin Peterson" <musiccomposition at gmail.com> writes:

> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Use new-style classes throughout
> > --------------------------------
> >
> > The following classes will inherit explicitly from the built-in
> > `object` type, to make all classes in the module part of the new-style
> > type hierarchy.
> >
> > * ``TestResult``
> > * ``TestCase``
> > * ``TestSuite``
> > * ``TestLoader``
> > * ``_WritelnDecorator``
> > * ``TextTestRunner``
> > * ``TestProgram``
> 
> They already do. __metaclass__ = type is found in unittest.py.

Not in the copy I have. Is that in 3.x only, or in 2.x also?

-- 
 \       ?I love to go down to the schoolyard and watch all the little |
  `\   children jump up and down and run around yelling and screaming. |
_o__)             They don't know I'm only using blanks.? ?Emo Philips |
Ben Finney


From musiccomposition at gmail.com  Tue Jul 15 01:22:08 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 14 Jul 2008 18:22:08 -0500
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module
In-Reply-To: <87abgk9hr2.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com>
	<87abgk9hr2.fsf@benfinney.id.au>
Message-ID: <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com>

On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
>
>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> > Use new-style classes throughout
>> > --------------------------------
>> >
>> > The following classes will inherit explicitly from the built-in
>> > `object` type, to make all classes in the module part of the new-style
>> > type hierarchy.
>> >
>> > * ``TestResult``
>> > * ``TestCase``
>> > * ``TestSuite``
>> > * ``TestLoader``
>> > * ``_WritelnDecorator``
>> > * ``TextTestRunner``
>> > * ``TestProgram``
>>
>> They already do. __metaclass__ = type is found in unittest.py.
>
> Not in the copy I have. Is that in 3.x only, or in 2.x also?

Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import unittest
>>> isinstance(unittest.TestCase, object)
True


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From python at rcn.com  Tue Jul 15 01:26:45 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Jul 2008 16:26:45 -0700
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>
	<87iqv89kj5.fsf@benfinney.id.au>
Message-ID: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1>

From: "Ben Finney" <ben+python at benfinney.id.au>
> Right, so I'm putting up a separate PEP just for the renaming. Should
> be arriving on this list soon.

I would like to work with you or someone else who is interested
on an alternative PEP for a separate, simpler test module
using the py.test syntax.  That is much simpler to learn and use.
Instead of self.assertIsNot and whatnot, you write:
   assert a is not b
No need for tons of word-by-word spellings on things we already
have syntax for.  Almost anyone who has used py.test can attest
its syntax is much more natural, easy to learn, easy to both
read and write, and is much lighter weight.  I think some variant
of py.test could be done that is compatable with unittest
and the did not have the "magic" present in earlier versions of py.test.
I wrote a recipe (somewhat rough and incomplete) that shows how
easily this could be done:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194

Raymond

From musiccomposition at gmail.com  Tue Jul 15 01:32:21 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 14 Jul 2008 18:32:21 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
	<487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
Message-ID: <1afaf6160807141632o424907aev777a8889b924ea6c@mail.gmail.com>

On Mon, Jul 14, 2008 at 6:26 PM, Raymond Hettinger <python at rcn.com> wrote:.
>
> I would like to work with you or someone else who is interested
> on an alternative PEP for a separate, simpler test module
> using the py.test syntax.  That is much simpler to learn and use.
> Instead of self.assertIsNot and whatnot, you write:
>  assert a is not b
> No need for tons of word-by-word spellings on things we already
> have syntax for.  Almost anyone who has used py.test can attest
> its syntax is much more natural, easy to learn, easy to both
> read and write, and is much lighter weight.  I think some variant
> of py.test could be done that is compatable with unittest
> and the did not have the "magic" present in earlier versions of py.test.
> I wrote a recipe (somewhat rough and incomplete) that shows how
> easily this could be done:

Bringing the total amount of test modules in the stdlib to 3. OWTDI indeed.

Anyway, I don't think something like needs to be (re)written. nose[1]
is already an excellent implementation of this that I would like to
see in the stdlib.

[1] http://www.somethingaboutorange.com/mrl/projects/nose/



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From steve at pearwood.info  Tue Jul 15 01:40:43 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 15 Jul 2008 09:40:43 +1000
Subject: [Python-Dev]
	=?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?=
	=?iso-8859-1?q?asserts=09vs=2E=09failIf/Unlesses?=
In-Reply-To: <87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<20080713234123.GA11170@mithrandi.net>
	<87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <200807150940.44919.steve@pearwood.info>

On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote:

> FWIW, I meant 10 != not not 10.

>>> 10 != not not 10
  File "<stdin>", line 1
    10 != not not 10
            ^
SyntaxError: invalid syntax

With respect, I think that the fact that you made an analogy with Python 
code that you hadn't tested, got it wrong, then corrected it, and 
*still* got it wrong, is telling. Its part of the pattern of this 
thread. People have repeatedly and consistently asserted that their 
particular favourite bike-shed colour is not just more attractive than 
any other colour, but supposedly objectively and logically better than 
any other colours. It's that second part that I object to.

When it comes to assert* versus fail* tests, this is one case where I 
don't believe "one obvious way to do it" should apply. A similar, and I 
hope uncontroversial, case is the greater-than and less-than operators. 
It would be frankly silly to declare that Python code should always use 
x < y and never y > x on the basis that there should be "one obvious 
way". Sometimes a particular test is most naturally written as g-t, and 
sometimes as l-t, and sometimes the choice between them is just 
arbitrary.

I believe that assert* and fail* tests are the same: while the choice is 
often arbitrary, sometimes a test is naturally written as an assert and 
sometimes as a fail. My own tests often look like this:

fail_if_spam()
fail_unless_ham()
assert_parrot()
fail_if_spanish_inquisition()

because the specific parrot test is best written as an assertion rather 
than a fail. And frankly, I simply don't believe that this imposes an 
undue mental cost on people who might read my code.

It's certainly true that I could invert the nature of the tests:

assert_not_spam()
assert_ham()
assert_parrot()
assert_not_spanish_inquisition()

just for the sake of consistency (and that would be a good thing 
*why*?), but only at the cost of inverting the code inside the test, 
which may or may not be a simple thing to do.


-- 
Steven

From solipsis at pitrou.net  Tue Jul 15 01:41:43 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 14 Jul 2008 23:41:43 +0000 (UTC)
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>
	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
Message-ID: <loom.20080714T233109-142@post.gmane.org>


Hi,

> I think some variant
> of py.test could be done that is compatable with unittest
> and the did not have the "magic" present in earlier versions of py.test.

It already exists:
http://www.somethingaboutorange.com/mrl/projects/nose/

Regards

Antoine.



From ben+python at benfinney.id.au  Tue Jul 15 01:42:18 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 09:42:18 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`
	module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au>
Message-ID: <8763r89go5.fsf@benfinney.id.au>

Significant updates are to the preamble (Python-Version field), the
sections "Use new-style classes throughout", "Module attributes", and
a new Rationale section "Removal of ``assert*`` names".


:PEP:               XXX
:Title:             Consolidating names and classes in the `unittest` module
:Version:           0.0
:Last-Modified:     2008-07-15
:Author:            Ben Finney <ben+python at benfinney.id.au>
:Status:            Draft
:Type:              Standards Track
:Content-Type:      test/x-rst
:Created:           2008-07-14
:Python-Version:    2.7, 3.1
:Post-History:


..  contents::


Abstract
========

This PEP proposes to consolidate the names and classes that constitute
the API of the standard library `unittest` module, with the goal of
removing redundant names, conforming with PEP 8, and eliminating
classic classes.


Motivation
==========

The normal use case for the `unittest` module is to subclass its
classes, overriding and re-using its functios and methods. This draws
constant attention to the fact that the existing implementation fails
several current Python standards:

* It does not use new-style classes, preventing e.g. straightforward
  use of ``super`` for calling the fixture set-up and tear-down
  methods.

* It does not conform to PEP 8, requiring users to write their own
  non-PEP-8 conformant names when overriding methods, and encouraging
  extensions to further depart from PEP 8.

* It has many synonyms in its API, which goes against the Zen of
  Python (specifically, that "there should be one, and preferably only
  one, obvious way to do it").


Specification
=============

Use new-style classes throughout
--------------------------------

The following classes are currently implemented as classic
("old-style") classes, with no metaclass.

* ``TestResult``
* ``TestCase``
* ``TestSuite``
* ``TestLoader``
* ``_WritelnDecorator``
* ``TextTestRunner``
* ``TestProgram``

The `unittest` module will gain the following attribute, to set the
default metaclass for classes in the module and thus make all classes
in the module part of the new-style type hierarchy::

    __metaclass__ = type

Remove obsolete names
---------------------

The following module attributes are not documented as part of the API
and are marked as obsolete in the implementation. They will be
removed.

* ``_makeLoader``
* ``getTestCaseNames``
* ``makeSuite``
* ``findTestCases``

Remove redundant names
----------------------

The following attribute names exist only as synonyms for other names.
They are to be removed, leaving only one name for each attribute in
the API.

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

* ``assertEqual``
* ``assertEquals``
* ``assertNotEqual``
* ``assertNotEquals``
* ``assertAlmostEqual``
* ``assertAlmostEquals``
* ``assertNotAlmostEqual``
* ``assertNotAlmostEquals``
* ``assertRaises``
* ``assert_``
* ``assertTrue``
* ``assertFalse``

Conform API with PEP 8
----------------------

The following names are to be introduced, each replacing an existing
name, to make all names in the module conform with PEP 8. Each name is
shown with the existing name that it replaces.

Where function parameters are to be renamed also, they are shown.
Where function parameters are not to be renamed, they are elided with
the ellipse ("?") symbol.

Module attributes
~~~~~~~~~~~~~~~~~

``default_test_loader``
  Replaces ``defaultTestLoader``

``TestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``add_error(?)``
  Replaces ``addError(?)``

``add_result(?)``
  Replaces ``addResult(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``should_stop``
  Replaces ``shouldStop``

``start_test(?)``
  Replaces ``startTest(?)``

``stop_test(?)``
  Replaces ``stopTest(?)``

``tests_run``
  Replaces ``testsRun``

``was_successful(?)``
  Replaces ``wasSuccessful(?)``

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, method_name='run_test')``
  Replaces ``__init__(self, methodName='runTest')``

``_test_method_doc``
  Replaces ``_testMethodDoc``

``_test_method_name``
  Replaces ``_testMethodName``

``failure_exception``
  Replaces ``failureException``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``default_test_result(?)``
  Replaces ``defaultTestResult(?)``

``fail_if(?)``
  Replaces ``failIf(?)``

``fail_if_almost_equal(?)``
  Replaces ``failIfAlmostEqual(?)``

``fail_if_equal(?)``
  Replaces ``failIfEqual(?)``

``fail_unless(?)``
  Replaces ``failUnless(?)``

``fail_unless_almost_equal(?)`` 
  Replaces ``failUnlessAlmostEqual(?)``

``fail_unless_equal(?)``
  Replaces ``failUnlessEqual(?)``

``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)``
  Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``FunctionTestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, test_func, set_up, tear_down, description)``
  Replaces ``__init__(self, testFunc, setUp, tearDown, description)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``TestSuite`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~

``add_test(?)``
  Replaces ``addTest(?)``

``add_tests(?)``
  Replaces ``addTests(?)``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``TestLoader`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``sort_test_methods_using``
  Replaces ``sortTestMethodsUsing``

``suite_class``
  Replaces ``suiteClass``

``test_method_prefix``
  Replaces ``testMethodPrefix``

``get_test_case_names(self, test_case_class)``
  Replaces ``getTestCaseNames(self, testCaseClass)``

``load_tests_from_module(?)``
  Replaces ``loadTestsFromModule(?)``

``load_tests_from_name(?)``
  Replaces ``loadTestsFromName(?)``

``load_tests_from_names(?)``
  Replaces ``loadTestsFromNames(?)``

``load_tests_from_test_case(self, test_case_class)``
  Replaces ``loadTestsFromTestCase(self, testCaseClass)``

``_TextTestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``show_all``
  Replaces ``showAll``

``add_error(?)``
  Replaces ``addError(?)``

``add_failure(?)``
  Replaces ``addFailure(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``get_description(?)``
  Replaces ``getDescription(?)``

``print_error_list(?)``
  Replaces ``printErrorList(?)``

``print_errors(?)``
  Replaces ``printErrors(?)``

``start_test(?)``
  Replaces ``startTest(?)``

``TextTestRunner`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``_make_result(?)``
  Replaces ``_makeResult(?)``

``TestProgram`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, module, default_test, argv, test_runner, test_loader)``
  Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)``

``create_tests(?)``
  Replaces ``createTests(?)``

``parse_args(?)``
  Replaces ``parseArgs(?)``

``run_tests(?)``
  Replaces ``runTests(?)``

``usage_exit(?)``
  Replaces ``usageExit(?)``


Rationale
=========

New-style classes
-----------------

As a standard library module, `unittest` should not define any classic
classes.

Redundant names
---------------

The current API, with two or in some cases three different names
referencing exactly the same function, leads to an overbroad and
redundant API that violates PEP 20 ("there should be one, and
preferably only one, obvious way to do it").

Removal of ``assert*`` names
----------------------------

There is no overwhelming consensus on whether to remove the
``assert*`` names or the ``fail*`` names; both are in common use. This
proposal argues the ``fail*`` names are slightly superior, for the
following reasons:

* Explicit is better than implicit: The ``fail*`` names state *what
  the function will do* explicitly: fail the test. With the
  ``assert*`` names, the action to be taken is only implicit.

* Avoid false implication: The test methods do not have any necessary
  connection with the built-in ``assert`` statement. Even the
  exception raised, while it defaults to ``AssertionException``, is
  explicitly customisable via the documented ``failure_exception``
  attribute. Choosing the ``fail*`` names avoids the false association
  with either of these.

PEP 8 names
-----------

Although `unittest` (and its predecessor `PyUnit`) are intended to be
familiar to users of other xUnit interfaces, there is no attempt at
direct API compatibility since the only code that Python's `unittest`
interfaces with is other Python code. The module is in the standard
library and its names should all conform with PEP 8.


Backwards Compatibility
=======================

The names to be obsoleted should be deprecated and removed according
to the schedule for modules in PEP 4.

While deprecated, use of the deprecated attributes should raise a
``DeprecationWarning``, with a message stating which replacement name
should be used.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From fuzzyman at voidspace.org.uk  Tue Jul 15 01:43:12 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Jul 2008 00:43:12 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
Message-ID: <487BE490.8080406@voidspace.org.uk>

Raymond Hettinger wrote:
> From: "Ben Finney" <ben+python at benfinney.id.au>
>> Right, so I'm putting up a separate PEP just for the renaming. Should
>> be arriving on this list soon.
>
> I would like to work with you or someone else who is interested
> on an alternative PEP for a separate, simpler test module
> using the py.test syntax.  That is much simpler to learn and use.
> Instead of self.assertIsNot and whatnot, you write:
>   assert a is not b
> No need for tons of word-by-word spellings on things we already
> have syntax for.  

However, to provide readable output for errors in even simple tests 
(like a == b) py.test does magic with stack frames and code objects - in 
order to discover the objects being compared.

As this relies on what are essentially implementation details of the 
Python interpreter it means that some implementations (specifically 
IronPython which doesn't have Python stack frames and only a minimal 
representation of frame objects) will never be able to run it.

I think it would be a bad idea to move *Python testing* itself over to a 
framework like this.

I personally find unittest pretty readable, the feature most lacking is 
autodiscovery of tests which nose does seem to provide very well 
although I haven't used it yet.

Michael

> Almost anyone who has used py.test can attest
> its syntax is much more natural, easy to learn, easy to both
> read and write, and is much lighter weight.  I think some variant
> of py.test could be done that is compatable with unittest
> and the did not have the "magic" present in earlier versions of py.test.
> I wrote a recipe (somewhat rough and incomplete) that shows how
> easily this could be done:
>
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From fuzzyman at voidspace.org.uk  Tue Jul 15 01:45:18 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Jul 2008 00:45:18 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
Message-ID: <487BE50E.6090200@voidspace.org.uk>

Raymond Hettinger wrote:
> From: "Ben Finney" <ben+python at benfinney.id.au>
>> Right, so I'm putting up a separate PEP just for the renaming. Should
>> be arriving on this list soon.
>
> I would like to work with you or someone else who is interested
> on an alternative PEP for a separate, simpler test module
> using the py.test syntax.  That is much simpler to learn and use.
> Instead of self.assertIsNot and whatnot, you write:
>   assert a is not b
> No need for tons of word-by-word spellings on things we already
> have syntax for.  Almost anyone who has used py.test can attest
> its syntax is much more natural, easy to learn, easy to both
> read and write, and is much lighter weight.  I think some variant
> of py.test could be done that is compatable with unittest
> and the did not have the "magic" present in earlier versions of py.test.

Ah, in my haste I skipped over your comment about "magic", my apologies. 
But in the absence of magic how do you propose to provide a meaningful 
error message from the failure of:

assert a == b

To wrap it in a function like "assert equals(a, b)" seems to gain little 
over unittest.

Michael

> I wrote a recipe (somewhat rough and incomplete) that shows how
> easily this could be done:
>
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From jml at mumak.net  Tue Jul 15 01:58:04 2008
From: jml at mumak.net (Jonathan Lange)
Date: Tue, 15 Jul 2008 09:58:04 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487BE490.8080406@voidspace.org.uk>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
	<487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<487BE490.8080406@voidspace.org.uk>
Message-ID: <d06a5cd30807141658t7b1d120era251efc46f7c6a5d@mail.gmail.com>

On Tue, Jul 15, 2008 at 9:43 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> I personally find unittest pretty readable, the feature most lacking is
> autodiscovery of tests which nose does seem to provide very well although I
> haven't used it yet.

FWIW, Twisted's 'trial' has done this since about 2003, and works with
stdlib unit tests. I'd be happy to submit the discovery code to
Python.

jml

From python at rcn.com  Tue Jul 15 02:13:25 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Jul 2008 17:13:25 -0700
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<487BE490.8080406@voidspace.org.uk>
Message-ID: <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1>

From: "Michael Foord" <fuzzyman at voidspace.org.uk>
> However, to provide readable output for errors in even simple tests 
> (like a == b) py.test does magic with stack frames and code objects - in 
> order to discover the objects being compared.

Don't have to go that route.  Can use plain python assert failures with a stacktrace.
Or can trigger pdb, or let the user specify a mode that calls some more advanced
test runner or test reporter with introspection.  This can be done without making
everything hard.


> I think it would be a bad idea to move *Python testing* itself over to a 
> framework like this.

Don't want to convert the python testing.
Would like to offer a lighter-weight alternative to our users.


> I personally find unittest pretty readable, the feature most lacking is 
> autodiscovery of tests which nose does seem to provide very well 
> although I haven't used it yet.

It takes about one day of using py.test to realize have much
cleaner and more readable its syntax is.  Also, writing the
tests is *much* more pleasant.  It has the same clean, clear
joy as writing regular python code.  By comparison, the code
using unittest.py is javaesque.   I've written tons of test with
unittest.py and and find it to be joyless.

I realize there is a matter of taste involved but if you talk to
any regular users of py.test, they will *all* attest to the
syntax being much more readable, lightweight, and pleasant
to use.  It encourages writing tests.

That being said, I think there are less magical, much simpler
ways to implement it.  I think Holger is working on it as we speak.


Raymond

From musiccomposition at gmail.com  Tue Jul 15 02:23:59 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 14 Jul 2008 19:23:59 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487BE490.8080406@voidspace.org.uk>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
	<487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<487BE490.8080406@voidspace.org.uk>
Message-ID: <1afaf6160807141723j65f515c7wbe0d377f2c813ea3@mail.gmail.com>

On Mon, Jul 14, 2008 at 6:43 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>
> However, to provide readable output for errors in even simple tests (like a
> == b) py.test does magic with stack frames and code objects - in order to
> discover the objects being compared.

Maybe what we need to do then is make the assert statement more
powerful. I like the idea of having it call a builtin called
__assert__ which is called by the assert statement. The AST for the
node could be attached.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From musiccomposition at gmail.com  Tue Jul 15 02:30:42 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 14 Jul 2008 19:30:42 -0500
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
In-Reply-To: <8763r89go5.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
Message-ID: <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com>

On Mon, Jul 14, 2008 at 6:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Specification
> =============
>
> Use new-style classes throughout
> --------------------------------
>
> The following classes are currently implemented as classic
> ("old-style") classes, with no metaclass.
>
> * ``TestResult``
> * ``TestCase``
> * ``TestSuite``
> * ``TestLoader``
> * ``_WritelnDecorator``
> * ``TextTestRunner``
> * ``TestProgram``
>
> The `unittest` module will gain the following attribute, to set the
> default metaclass for classes in the module and thus make all classes
> in the module part of the new-style type hierarchy::
>
>    __metaclass__ = type

It's already done.

Line 94-95 in unittest.py (trunk):
# All classes defined herein are 'new-style' classes, allowing use of 'super()'
__metaclass__ = type



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From brett at python.org  Tue Jul 15 02:32:51 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Jul 2008 17:32:51 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <20080714224343.GA23048@arctrix.com>
References: <20080714224343.GA23048@arctrix.com>
Message-ID: <bbaeab100807141732r38fc8b3bi71061221d7393528@mail.gmail.com>

On Mon, Jul 14, 2008 at 3:43 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> Hi,
>
> In case anyone is interested, I have git repositories for both the
> trunk and the py3k branch of the Python source code.  They are
> up-to-date and so using them with git-svn would be much faster than
> starting from scratch.
>
> If anyone is interested, I will find a place to host them.  They are
> each about 150 MB in size.
>

Would you mind putting the time in, Neil, to have them hosted on a
python.org machine like the bzr branches and have them auto-updated?

-Brett

From barry at python.org  Tue Jul 15 03:31:47 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 14 Jul 2008 21:31:47 -0400
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <20080714224343.GA23048@arctrix.com>
References: <20080714224343.GA23048@arctrix.com>
Message-ID: <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 14, 2008, at 6:43 PM, Neil Schemenauer wrote:

> In case anyone is interested, I have git repositories for both the
> trunk and the py3k branch of the Python source code.  They are
> up-to-date and so using them with git-svn would be much faster than
> starting from scratch.
>
> If anyone is interested, I will find a place to host them.  They are
> each about 150 MB in size.

Neil, we should try to host them on code.python.org.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSHv+A3EjvBPtnXfVAQI0zwP9H0lZMvWxwncqg1BmI+df0WTh7+SOsxO2
RHky3TzqkY7wBXXwHPe5d7duWzflsXjB6ljH0AoR7icMs31h5ZUZhGVU/vSYouqk
KhHqCHjnXlnY0qOySthblboih/Uw9ApR9akEsKAxQrzUATbZF93dS5RJg4QjpMn3
HJ06MUTt5y0=
=bm3R
-----END PGP SIGNATURE-----

From fuzzyman at voidspace.org.uk  Tue Jul 15 03:40:31 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Jul 2008 02:40:31 +0100
Subject: [Python-Dev] PEP: Consolidating names and classes in
 the	`unittest` module
In-Reply-To: <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au>	<1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com>	<87abgk9hr2.fsf@benfinney.id.au>
	<1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com>
Message-ID: <487C000F.3050300@voidspace.org.uk>

Benjamin Peterson wrote:
> On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>   
>> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
>>
>>     
>>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>>       
>>>> Use new-style classes throughout
>>>> --------------------------------
>>>>
>>>> The following classes will inherit explicitly from the built-in
>>>> `object` type, to make all classes in the module part of the new-style
>>>> type hierarchy.
>>>>
>>>> * ``TestResult``
>>>> * ``TestCase``
>>>> * ``TestSuite``
>>>> * ``TestLoader``
>>>> * ``_WritelnDecorator``
>>>> * ``TextTestRunner``
>>>> * ``TestProgram``
>>>>         
>>> They already do. __metaclass__ = type is found in unittest.py.
>>>       
>> Not in the copy I have. Is that in 3.x only, or in 2.x also?
>>     
>
> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>   
>>>> import unittest
>>>> isinstance(unittest.TestCase, object)
>>>>         
> True
>
>
>   
That proves nothing:

Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> class x: pass
...
 >>> isinstance(x, object)
True
 >>> isinstance(x, type)
False
 >>> type(x)
<type 'classobj'>

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From python at rcn.com  Tue Jul 15 04:06:49 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Jul 2008 19:06:49 -0700
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
Message-ID: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>

> ``set_up(?)``
>  Replaces ``setUp(?)``
 . .
> ``tear_down(?)``
>  Replaces ``tearDown(?)``

Am I the only one who finds this sort of excessive pep-8 underscoring to be horrorific?

Nobody I know spells setup and teardown as two words. I dread using the module with these new names. Underscores are not fun to 
type. They create a weird_mental_pause when reading them.

It's not going to be easy to remember where they are used (ie. isinstance is still isinstance but isset is now is_set). Go figure.


> fail_if_almost_equal

Another thought is that test suite code is going to get seriously crunched when the new, longer method names meet the 78/80 column 
pep 8 restrictions.

class TestMisc(unittest.test_case):
    def lost_four_spaces_here_already(self):
        self.fail_if_almost_equal('were already on the 34th column before'
                                  'writing anything substantive',
                                  self.testedobject.tested_method(arg1, arg2 +
                                                                  arg3, arg4)
        # Imagine typing and line wrapping dozens more tests like this

Are there any ideas for some short, pithy, mnemonic names that are self-explantory and not full of underscores; something that 
wouldn't suck to type hundreds of times in a good test module?  IMO, the current names are already too long.


> * Explicit is better than implicit:

Don't forgot the opposing forces:

Beautiful is better than ugly.
Readability counts.

These are especially important for the unittest module.  When I'm making tests, I have to type self.very_long_method_name so many 
times it isn't funny.  It's easy to just stop writing tests when you get tired of it.  Long api names with underscores are a 
disincentive (IMO).


> Use new-style classes throughout

This makes sense for 3.1 but of course we already get that automatically for 3.0 ;-)

For 2.7, it doesn't make as much sense.  Since 2.2 came out, there have been many discussions on changing standard library code to 
use new-style classes.  There was always some concern about subtle changes in semantics and for the most part these requests were 
denied.  I think this reason applies with even more force to the unittest module.  Any risk that we subtlely break someone's 
test-suite is a small disaster.  The 2.6 and 2.7 unittests need to be absolutely stable if they are to serve as a transition tool 
(providing a baseline the 3.0/3.1 is expected to match).

Also, are there any expected benefits from making this change in 2.7?  What will the module do differently?

It seems like a risky change for zero-benefit.


Raymond 


From fuzzyman at voidspace.org.uk  Tue Jul 15 04:14:47 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Jul 2008 03:14:47 +0100
Subject: [Python-Dev] PEP: Consolidating names and classes in
 the	`unittest`module (updated 2008-07-15)
In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
Message-ID: <487C0817.50504@voidspace.org.uk>

Raymond Hettinger wrote:
>> ``set_up(?)``
>>  Replaces ``setUp(?)``
> . .
>> ``tear_down(?)``
>>  Replaces ``tearDown(?)``
>
> Am I the only one who finds this sort of excessive pep-8 underscoring 
> to be horrorific?
>
> Nobody I know spells setup and teardown as two words. I dread using 
> the module with these new names. Underscores are not fun to type. They 
> create a weird_mental_pause when reading them.
>
> It's not going to be easy to remember where they are used (ie. 
> isinstance is still isinstance but isset is now is_set). Go figure.

+1

setUp and tearDown should become setup and teardown.

>
>
>> fail_if_almost_equal
>
> Another thought is that test suite code is going to get seriously 
> crunched when the new, longer method names meet the 78/80 column pep 8 
> restrictions.

Well... "assert_not_equal"  is slightly shorter, "assert_notequal" 
slightly more so. Still one char longer than the original though.

>
> class TestMisc(unittest.test_case):
>    def lost_four_spaces_here_already(self):
>        self.fail_if_almost_equal('were already on the 34th column before'
>                                  'writing anything substantive',
>                                  self.testedobject.tested_method(arg1, 
> arg2 +
>                                                                  arg3, 
> arg4)
>        # Imagine typing and line wrapping dozens more tests like this
>
> Are there any ideas for some short, pithy, mnemonic names that are 
> self-explantory and not full of underscores; something that wouldn't 
> suck to type hundreds of times in a good test module?  IMO, the 
> current names are already too long.
>
>
>> * Explicit is better than implicit:
>
> Don't forgot the opposing forces:
>
> Beautiful is better than ugly.
> Readability counts.
>
> These are especially important for the unittest module.  When I'm 
> making tests, I have to type self.very_long_method_name so many times 
> it isn't funny.  It's easy to just stop writing tests when you get 
> tired of it.  Long api names with underscores are a disincentive (IMO).
>
>
>> Use new-style classes throughout
>
> This makes sense for 3.1 but of course we already get that 
> automatically for 3.0 ;-)
>
> For 2.7, it doesn't make as much sense.  Since 2.2 came out, there 
> have been many discussions on changing standard library code to use 
> new-style classes.  There was always some concern about subtle changes 
> in semantics and for the most part these requests were denied.  I 
> think this reason applies with even more force to the unittest 
> module.  Any risk that we subtlely break someone's test-suite is a 
> small disaster.  The 2.6 and 2.7 unittests need to be absolutely 
> stable if they are to serve as a transition tool (providing a baseline 
> the 3.0/3.1 is expected to match).
>
> Also, are there any expected benefits from making this change in 2.7?  
> What will the module do differently?

It would allow you to use properties in testcases. Not sure if there is 
a usecase for this though.

It looks like Benjamin Peterson is right, in Python 2.5 TestCase already 
appears to be a new style class:

Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import unittest
 >>> type(unittest.TestCase)
<type 'type'>
 >>>

>
> It seems like a risky change for zero-benefit.
>

Looks like that part of the PEP is unnecessary.

Michael

>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From jml at mumak.net  Tue Jul 15 04:18:26 2008
From: jml at mumak.net (Jonathan Lange)
Date: Tue, 15 Jul 2008 12:18:26 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest`module (updated 2008-07-15)
In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
Message-ID: <d06a5cd30807141918w3d574c36lb9c84435e014fe36@mail.gmail.com>

On Tue, Jul 15, 2008 at 12:06 PM, Raymond Hettinger <python at rcn.com> wrote:
>> ``set_up(?)``
>>  Replaces ``setUp(?)``
>
> . .
>>
>> ``tear_down(?)``
>>  Replaces ``tearDown(?)``
>
> Am I the only one who finds this sort of excessive pep-8 underscoring to be
> horrorific?
>
> Nobody I know spells setup and teardown as two words. I dread using the
> module with these new names.
>

Hi,

My name's Jonathan, and I spell "set up" as "set up" and "tear down"
as "tear down".

> It's not going to be easy to remember where they are used (ie. isinstance is
> still isinstance but isset is now is_set). Go figure.
>

Yes, guessability via consistency is the important thing here.

>
>> fail_if_almost_equal
>
> Another thought is that test suite code is going to get seriously crunched
> when the new, longer method names meet the 78/80 column pep 8 restrictions.
>
> class TestMisc(unittest.test_case):
>   def lost_four_spaces_here_already(self):
>       self.fail_if_almost_equal('were already on the 34th column before'
>                                 'writing anything substantive',
>                                 self.testedobject.tested_method(arg1, arg2 +
>                                                                 arg3, arg4)
>       # Imagine typing and line wrapping dozens more tests like this
>
> Are there any ideas for some short, pithy, mnemonic names that are
> self-explantory and not full of underscores; something that wouldn't suck to
> type hundreds of times in a good test module?  IMO, the current names are
> already too long.
>

Well, "assert_" is strictly shorter than "fail_unless_".

>
>> * Explicit is better than implicit:
>
> Don't forgot the opposing forces:
>
> Beautiful is better than ugly.
> Readability counts.
>
> These are especially important for the unittest module.  When I'm making
> tests, I have to type self.very_long_method_name so many times it isn't
> funny.  It's easy to just stop writing tests when you get tired of it.  Long
> api names with underscores are a disincentive (IMO).
>

I find underscores easier to read?I suspect this will vary from person
to person. Typing isn't an issue since I use an auto-complete function
in my editor.

jml

From python at rcn.com  Tue Jul 15 04:40:26 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Jul 2008 19:40:26 -0700
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<487C0817.50504@voidspace.org.uk>
Message-ID: <E1D388236201464C84CAE7FE74A15C01@RaymondLaptop1>

> It looks like Benjamin Peterson is right, in Python 2.5 TestCase already appears to be a new style class:

Yep.  I stand corrected.  It looks like that changed five years ago (rev 28064).  Not sure how that slipped through but it doesn't 
seem to have caused any problems.


Raymond 


From janzert at janzert.com  Tue Jul 15 05:15:59 2008
From: janzert at janzert.com (Janzert)
Date: Mon, 14 Jul 2008 23:15:59 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest`module (updated 2008-07-15)
In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
Message-ID: <g5h4og$dub$1@ger.gmane.org>

Raymond Hettinger wrote:
>> ``set_up(?)``
>>  Replaces ``setUp(?)``
> . .
>> ``tear_down(?)``
>>  Replaces ``tearDown(?)``
> 
> Am I the only one who finds this sort of excessive pep-8 underscoring to 
> be horrorific?
> 
> Nobody I know spells setup and teardown as two words. I dread using the 
> module with these new names. Underscores are not fun to type. They 
> create a weird_mental_pause when reading them.
> 

+1

And Merriam-Webster agrees,

http://www.merriam-webster.com/dictionary/setup
http://www.merriam-webster.com/dictionary/teardown

Janzert



From guido at python.org  Tue Jul 15 05:22:05 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Jul 2008 20:22:05 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
	<487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<487BE490.8080406@voidspace.org.uk>
	<A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1>
Message-ID: <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com>

On Mon, Jul 14, 2008 at 5:13 PM, Raymond Hettinger <python at rcn.com> wrote:
> It takes about one day of using py.test to realize have much
> cleaner and more readable its syntax is.  Also, writing the
> tests is *much* more pleasant.  It has the same clean, clear
> joy as writing regular python code.  By comparison, the code
> using unittest.py is javaesque.   I've written tons of test with
> unittest.py and and find it to be joyless.

I, too, have written tons of tests with unittest.py (and Google's
extensions, which follow the same style), and reviewed even more. I
agree that this is pretty joyless, but I'm not at all sure that the
unittest API is the reason. It seems to me that a main problem with
writing test code is and will always remain due to the need to use
mocks, stubs and other similar techniques (e.g. dependency injection).
Typical test code that I've written or reviewed spends more time
setting up the input conditions for testing than it spends checking
the results. Ten lines of mocking code to one self.assertEqual() call
seems typical.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fuzzyman at voidspace.org.uk  Tue Jul 15 05:34:18 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 15 Jul 2008 04:34:18 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com>
References: <20080713093936.GA3623@benfinney.id.au>	
	<487A8700.8000200@voidspace.org.uk>
	<87mykka8el.fsf@benfinney.id.au>	
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>	
	<487B60FB.5010602@voidspace.org.uk>
	<87iqv89kj5.fsf@benfinney.id.au>	
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>	
	<487BE490.8080406@voidspace.org.uk>	
	<A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1>
	<ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com>
Message-ID: <487C1ABA.4010701@voidspace.org.uk>

Guido van Rossum wrote:
> On Mon, Jul 14, 2008 at 5:13 PM, Raymond Hettinger <python at rcn.com> wrote:
>   
>> It takes about one day of using py.test to realize have much
>> cleaner and more readable its syntax is.  Also, writing the
>> tests is *much* more pleasant.  It has the same clean, clear
>> joy as writing regular python code.  By comparison, the code
>> using unittest.py is javaesque.   I've written tons of test with
>> unittest.py and and find it to be joyless.
>>     
>
> I, too, have written tons of tests with unittest.py (and Google's
> extensions, which follow the same style), and reviewed even more. I
> agree that this is pretty joyless, but I'm not at all sure that the
> unittest API is the reason. It seems to me that a main problem with
> writing test code is and will always remain due to the need to use
> mocks, stubs and other similar techniques (e.g. dependency injection).
> Typical test code that I've written or reviewed spends more time
> setting up the input conditions for testing than it spends checking
> the results. Ten lines of mocking code to one self.assertEqual() call
> seems typical.
>
>   
Maybe Python needs a good mocking module in the standard library. There 
are plenty, but we use a particularly nice one at Resolver Systems [1]. :-)

It auto-creates attributes as mocks, allowing you to assert calls made 
to all of its children along with convenience methods like 
'assert_called_with' and has a companion decorator that patches class / 
module level attributes just for the duration of the test.

As we're changing more of our tests over to use these we're finding it 
reduces the volume and complexity of our test code.

Michael Foord

[1] Based on http://code.google.com/p/mock/ although there is some 
outstanding code to sync back to the project.

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From python at rcn.com  Tue Jul 15 06:37:30 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Jul 2008 21:37:30 -0700
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>	
	<487A8700.8000200@voidspace.org.uk>
	<87mykka8el.fsf@benfinney.id.au>	
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>	
	<487B60FB.5010602@voidspace.org.uk>
	<87iqv89kj5.fsf@benfinney.id.au>	
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>	
	<487BE490.8080406@voidspace.org.uk>	
	<A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1>
	<ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com>
	<487C1ABA.4010701@voidspace.org.uk>
Message-ID: <4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1>

 From: "Michael Foord" <fuzzyman at voidspace.org.uk>
> Maybe Python needs a good mocking module in the standard library. There 
> are plenty, but we use a particularly nice one at Resolver Systems [1]. :-)

-1

This comes up occassionally and gets shot down.
http://bugs.python.org/issue708125

Mock objects mean different things to different people.
Some expect more simulated behavior and others want less.
It's rare to find agreement about general purpose mock objects and frameworks.
Mock libraries create their own complexities and burdens on a programmer's memory.
It's often easier to create a small special case mock object
than to remember how to configure a general purpose one.
And, afaict, there is no fan club for some particular python mock
object library -- it seems to only come up in general discussions
about possibilities for growing the unittest module, and almost
never comes up in the context of solving a real problem that
hasn't already be addressed in some other way.


Raymond

From ctb at msu.edu  Tue Jul 15 06:42:54 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Mon, 14 Jul 2008 21:42:54 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1>
References: <87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
	<487B60FB.5010602@voidspace.org.uk>
	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<487BE490.8080406@voidspace.org.uk>
	<A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1>
	<ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com>
	<487C1ABA.4010701@voidspace.org.uk>
	<4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1>
Message-ID: <20080715044253.GA31517@caltech.edu>

On Mon, Jul 14, 2008 at 09:37:30PM -0700, Raymond Hettinger wrote:
-> From: "Michael Foord" <fuzzyman at voidspace.org.uk>
-> >Maybe Python needs a good mocking module in the standard library. There 
-> >are plenty, but we use a particularly nice one at Resolver Systems [1]. :-)
-> 
-> -1
-> 
-> This comes up occassionally and gets shot down.
-> http://bugs.python.org/issue708125
-> 
-> Mock objects mean different things to different people.
-> Some expect more simulated behavior and others want less.
-> It's rare to find agreement about general purpose mock objects and 
-> frameworks.
-> Mock libraries create their own complexities and burdens on a programmer's 
-> memory.
-> It's often easier to create a small special case mock object
-> than to remember how to configure a general purpose one.
-> And, afaict, there is no fan club for some particular python mock
-> object library -- it seems to only come up in general discussions
-> about possibilities for growing the unittest module, and almost
-> never comes up in the context of solving a real problem that
-> hasn't already be addressed in some other way.

Also see:

http://lists.idyll.org/pipermail/testing-in-python/2007-November/000406.html

& associated thread, for those interested in the variety of mock
libraries...

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From python at rcn.com  Tue Jul 15 07:11:07 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Jul 2008 22:11:07 -0700
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au>	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>	<487B60FB.5010602@voidspace.org.uk><87iqv89kj5.fsf@benfinney.id.au>	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>	<487BE490.8080406@voidspace.org.uk>	<A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1><ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com><487C1ABA.4010701@voidspace.org.uk>
	<4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1>
Message-ID: <A0D30B0412654877A54A28CB58C7DA78@RaymondLaptop1>

> From: "Michael Foord" <fuzzyman at voidspace.org.uk>
>> Maybe Python needs a good mocking module in the standard library. There 
>> are plenty, but we use a particularly nice one at Resolver Systems [1]. :-)
> 
> -1
> 
> This comes up occassionally and gets shot down.
> http://bugs.python.org/issue708125

And:  http://bugs.python.org/issue2156


Raymond

From scott+python-dev at scottdial.com  Tue Jul 15 08:17:21 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 15 Jul 2008 02:17:21 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in
 the	`unittest` module
In-Reply-To: <487C000F.3050300@voidspace.org.uk>
References: <87vdz8a983.fsf@benfinney.id.au>	<1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com>	<87abgk9hr2.fsf@benfinney.id.au>	<1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com>
	<487C000F.3050300@voidspace.org.uk>
Message-ID: <487C40F1.9000309@scottdial.com>

Michael Foord wrote:
> Benjamin Peterson wrote:
>> On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney 
>> <ben+python at benfinney.id.au> wrote:
>>> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
>>>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney 
>>>> <ben+python at benfinney.id.au> wrote:
>>>>> Use new-style classes throughout
>>>>> --------------------------------
>>>>>
>>>>> The following classes will inherit explicitly from the built-in
>>>>> `object` type, to make all classes in the module part of the new-style
>>>>> type hierarchy.
>>>>>
[snip]
>>>>>         
>>>> They already do. __metaclass__ = type is found in unittest.py.
>>>>       
>>> Not in the copy I have. Is that in 3.x only, or in 2.x also?
>>>     
>>
>> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
>> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>  
>>>>> import unittest
>>>>> isinstance(unittest.TestCase, object)
>>>>>         
>> True
>>   
> That proves nothing:
> 
> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> class x: pass
> ...
>  >>> isinstance(x, object)
> True
>  >>> isinstance(x, type)
> False
>  >>> type(x)
> <type 'classobj'>
> 

While your retort is accurate, I think it's unintentionally deceptive, 
because you didn't finish your thought.. Benjamin is actually still correct:

Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit 
(Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import unittest
 >>> isinstance(unittest.TestCase, type)
True
 >>> type(unittest.TestCase)
<type 'type'>

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From steve at holdenweb.com  Tue Jul 15 08:21:01 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 15 Jul 2008 02:21:01 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module
In-Reply-To: <487C000F.3050300@voidspace.org.uk>
References: <87vdz8a983.fsf@benfinney.id.au>	<1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com>	<87abgk9hr2.fsf@benfinney.id.au>	<1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com>
	<487C000F.3050300@voidspace.org.uk>
Message-ID: <g5hfl6$7v3$1@ger.gmane.org>

Michael Foord wrote:
> Benjamin Peterson wrote:
>> On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney 
>> <ben+python at benfinney.id.au> wrote:
>>  
>>> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
>>>
>>>    
>>>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney 
>>>> <ben+python at benfinney.id.au> wrote:
>>>>      
>>>>> Use new-style classes throughout
>>>>> --------------------------------
>>>>>
>>>>> The following classes will inherit explicitly from the built-in
>>>>> `object` type, to make all classes in the module part of the new-style
>>>>> type hierarchy.
>>>>>
>>>>> * ``TestResult``
>>>>> * ``TestCase``
>>>>> * ``TestSuite``
>>>>> * ``TestLoader``
>>>>> * ``_WritelnDecorator``
>>>>> * ``TextTestRunner``
>>>>> * ``TestProgram``
>>>>>         
>>>> They already do. __metaclass__ = type is found in unittest.py.
>>>>       
>>> Not in the copy I have. Is that in 3.x only, or in 2.x also?
>>>     
>>
>> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
>> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>  
>>>>> import unittest
>>>>> isinstance(unittest.TestCase, object)
>>>>>         
>> True
>>
>>
>>   
> That proves nothing:
> 
> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> class x: pass
> ...
>  >>> isinstance(x, object)
> True
>  >>> isinstance(x, type)
> False
>  >>> type(x)
> <type 'classobj'>
> 
Shouldn't isinstance(x, object) be True for any x in recent CPythons 
(and hopefully the other implementations too)? It's certainly so for 
functions and modules, for , as well as the less esoteric types and 
instances of classic classes.

Something that often ties students' heads in knots (and this isn't from 
the introductory course):

 >>> isinstance(type, object)
True
 >>> isinstance(object, type)
True

This is a classic property of general object hierarchies based on 
metaclasses. I remember teaching the SmallTalk-80 equivalent to M.Sc. 
studnets in the early 1980s, though the details are now lost in the 
mists of time. Eyes would always widen, and not everyone would get it.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Tue Jul 15 08:40:22 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 15 Jul 2008 02:40:22 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest`module (updated 2008-07-15)
In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
Message-ID: <g5hgpg$an7$1@ger.gmane.org>

Raymond Hettinger wrote:
>> ``set_up(?)``
>>  Replaces ``setUp(?)``
> . .
>> ``tear_down(?)``
>>  Replaces ``tearDown(?)``
> 
> Am I the only one who finds this sort of excessive pep-8 underscoring to 
> be horrorific?
> 
Definitely not. I thin we are in danger of insisting on a foolish 
consistency. I'd far prefer readability (hence my preference for is_not 
over not_is). There's also the (possibly marginal) point that people 
used to Junit can actually understand Python unit tests right now. 
Surely there's something to be said for consistency across languages, 
especially when the idea was lifted from Java in the first place.

> Nobody I know spells setup and teardown as two words. I dread using the 
> module with these new names. Underscores are not fun to type. They 
> create a weird_mental_pause when reading them.
> 
> It's not going to be easy to remember where they are used (ie. 
> isinstance is still isinstance but isset is now is_set). Go figure.
> 
> 
>> fail_if_almost_equal
> 
> Another thought is that test suite code is going to get seriously 
> crunched when the new, longer method names meet the 78/80 column pep 8 
> restrictions.
> 
> class TestMisc(unittest.test_case):
>    def lost_four_spaces_here_already(self):
>        self.fail_if_almost_equal('were already on the 34th column before'
>                                  'writing anything substantive',
>                                  self.testedobject.tested_method(arg1, 
> arg2 +
>                                                                  arg3, 
> arg4)
>        # Imagine typing and line wrapping dozens more tests like this
> 
And the way Thunderbird has wrapped your example makes it obvious that 
72 is a more satisfactory limit, though I agree transmissibility via 
email shouldn't necessarily be a factor.

> Are there any ideas for some short, pithy, mnemonic names that are 
> self-explantory and not full of underscores; something that wouldn't 
> suck to type hundreds of times in a good test module?  IMO, the current 
> names are already too long.
> 
> 
>> * Explicit is better than implicit:
> 
> Don't forgot the opposing forces:
> 
> Beautiful is better than ugly.
> Readability counts.
> 
> These are especially important for the unittest module.  When I'm making 
> tests, I have to type self.very_long_method_name so many times it isn't 
> funny.  It's easy to just stop writing tests when you get tired of it.  
> Long api names with underscores are a disincentive (IMO).
> 
> 
>> Use new-style classes throughout
> 
> This makes sense for 3.1 but of course we already get that automatically 
> for 3.0 ;-)
> 
> For 2.7, it doesn't make as much sense.  Since 2.2 came out, there have 
> been many discussions on changing standard library code to use new-style 
> classes.  There was always some concern about subtle changes in 
> semantics and for the most part these requests were denied.  I think 
> this reason applies with even more force to the unittest module.  Any 
> risk that we subtlely break someone's test-suite is a small disaster.  
> The 2.6 and 2.7 unittests need to be absolutely stable if they are to 
> serve as a transition tool (providing a baseline the 3.0/3.1 is expected 
> to match).
> 
I'm not quite sure how often it has to be pointed out that since 2.5 all 
unittest classes *are* new-style classes. Benjamin has, I believe, 
already mentioned twice that unittest.py contains

__metaclass__ = type

before any class declarations. While this has no effect on pre-2.2 
implementations, surely it means that we've been using new-style classes 
for some time now. Or am I missing some subtlety? 2.5.1 says:

 >>> isinstance(unittest.TestCase, type)
True

> Also, are there any expected benefits from making this change in 2.7?  
> What will the module do differently?
> 
> It seems like a risky change for zero-benefit.
> 
Better revert it, then :-)

But easier to just drop that sentence from the PEP.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From thomas at thomas-lotze.de  Tue Jul 15 08:55:59 2008
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Tue, 15 Jul 2008 08:55:59 +0200
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>
	<487A8F7B.1090901@holdenweb.com>	<87d4lhba7b.fsf@benfinney.id.au>
	<487B18EC.6040104@gmail.com>	<874p6sbx25.fsf@benfinney.id.au>
	<g5fhol$411$1@ger.gmane.org>	<87r69wa8hs.fsf@benfinney.id.au>
Message-ID: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>

Ben Finney wrote:

> I'd count this as another (minor) point in favour of making the 'fail*'
> methods canonical: the names are consistent *and* gramatically sensible:

-1

I'm surprised nobody (that I've noticed) has brought up the point yet that
test code is a lot easier to read if it makes positive assertions. When
reading failure conditions, one has to constantly invert them in order to
deduce the behaviour that is tested. failUnless and friends aren't better
either IMO since while they do work with positive assertions, the method
names themselves are doubly negative. assert* methods are so much more
straightforward to comprehend.

-- 
Thomas




From steve at holdenweb.com  Tue Jul 15 09:04:58 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 15 Jul 2008 03:04:58 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>	<487A8F7B.1090901@holdenweb.com>	<87d4lhba7b.fsf@benfinney.id.au>	<487B18EC.6040104@gmail.com>	<874p6sbx25.fsf@benfinney.id.au>	<g5fhol$411$1@ger.gmane.org>	<87r69wa8hs.fsf@benfinney.id.au>
	<pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>
Message-ID: <g5hi76$ecr$1@ger.gmane.org>

Thomas Lotze wrote:
> Ben Finney wrote:
> 
>> I'd count this as another (minor) point in favour of making the 'fail*'
>> methods canonical: the names are consistent *and* gramatically sensible:
> 
> -1
> 
> I'm surprised nobody (that I've noticed) has brought up the point yet that
> test code is a lot easier to read if it makes positive assertions. When
> reading failure conditions, one has to constantly invert them in order to
> deduce the behaviour that is tested. failUnless and friends aren't better
> either IMO since while they do work with positive assertions, the method
> names themselves are doubly negative. assert* methods are so much more
> straightforward to comprehend.
> 
I think this is where I came in.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From stephen at xemacs.org  Tue Jul 15 10:32:53 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 15 Jul 2008 17:32:53 +0900
Subject: [Python-Dev] unittest's redundant
	assertions:	asserts	vs.	failIf/Unlesses
In-Reply-To: <200807150940.44919.steve@pearwood.info>
References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<20080713234123.GA11170@mithrandi.net>
	<87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp>
	<200807150940.44919.steve@pearwood.info>
Message-ID: <8763r7sg22.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:
 > On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote:
 > 
 > > FWIW, I meant 10 != not not 10.
 > 
 > >>> 10 != not not 10
 >   File "<stdin>", line 1
 >     10 != not not 10
 >             ^
 > SyntaxError: invalid syntax
 > 
 > With respect, I think that the fact that you made an analogy with Python 
 > code that you hadn't tested, got it wrong, then corrected it, and 
 > *still* got it wrong, is telling.

It's telling only if you ignore the fact that it's the ad hominem fallacy.

 > When it comes to assert* versus fail* tests, this is one case where I 
 > don't believe "one obvious way to do it" should apply.

In the context you were talking about, where you don't need to show
anybody your tests, I don't see any reason why TOOWTDI should apply.
In a debugging context, I don't see any reason why it should, either.

But here we're talking about the standard unit-testing framework,
which is used by Python itself.  So it *is* about communication with
other developers, and it *does* make sense to avoid proliferation of
redundant vocabulary.  Some of us have enough trouble remembering when
parentheses are redundant, as you noticed.

Regression test suites aspire to full coverage.  If you mix assert*
and fail* clauses, you are going to need to invert one or the other to
determine whether there are cases that are missed.

IMO those are strong indications that only one of the pair should be
used in a test suite.

 > I believe that assert* and fail* tests are the same:

In logic, they are.  I just think that assert* is much more natural in
several simple cases, such as the pure regression test where you use
Python X.Y to compute foo(), and add "assertEqual(foo(), pyXYresult)"
to the test suite.  I'd be a lot more sympathetic if you'd describe
a similarly plausible and generic application for fail*.


From stephen at xemacs.org  Tue Jul 15 10:56:57 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 15 Jul 2008 17:56:57 +0900
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`module (updated 2008-07-15)
In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
Message-ID: <874p6rsexy.fsf@uwakimon.sk.tsukuba.ac.jp>

Raymond Hettinger writes:

 > Nobody I know spells setup and teardown as two words.

I set up a house of cards.  When I'm done, I'm done with setup.
Similarly for "tear down" and "teardown".  The two word forms are
verbs, the one word forms are nouns.  I don't think it's worth a
column to make that distinction, though.

 > Another thought is that test suite code is going to get seriously crunched when the new, longer method names meet the 78/80 column 
 > pep 8 restrictions.
 > 
 > class TestMisc(unittest.test_case):
 >     def lost_four_spaces_here_already(self):

Eight spaces, actually.  Make that 13 by the time you get to the "."
after "self" in the next line.

 > Are there any ideas for some short, pithy, mnemonic names that are
 > self-explantory and not full of underscores; something that
 > wouldn't suck to type hundreds of times in a good test module?

"test" or "check" instead of "assert" or "fail_unless" comes to mind
to shorten the prefix.  But the best I can come up with for
"fail_unless_equal" is something like "equalize" which really fails
EIBTI.


From stephen at xemacs.org  Tue Jul 15 11:18:05 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 15 Jul 2008 18:18:05 +0900
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest`	module (updated 2008-07-15)
In-Reply-To: <8763r89go5.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
Message-ID: <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:

 > Removal of ``assert*`` names
 > ----------------------------
 > 
 > There is no overwhelming consensus on whether to remove the
 > ``assert*`` names or the ``fail*`` names;

7 to 1 is overwhelming in my book.  See
Message-ID: <loom.20080714T230912-310 at post.gmane.org>
From: Antoine Pitrou <solipsis at pitrou.net>

While people's preferences are important, I think there is a very good
case to made that keeping this much continuity in the test suite as
possible is more so.

 > * Explicit is better than implicit: The ``fail*`` names state *what
 >   the function will do* explicitly: fail the test. With the
 >   ``assert*`` names, the action to be taken is only implicit.

EIBTI applies with the most force to "local" names, ie, specific to a
particular function, class, or program.  Here we propose to impose a
community-wide convention.  I think we can document it explicitly and
expect near-instant uptake on the appropriate connotations to "assert"
(especially since that connotation is pretty much universal across
languages with an assert facility, anyway).

 > * Avoid false implication: The test methods do not have any necessary
 >   connection with the built-in ``assert`` statement.

Data point: Use of `Assert' as a test method in the XEmacs test suite
has never caused any confusion with either C-level asserts, or with
the Lisp function `assert'.


From ncoghlan at gmail.com  Tue Jul 15 11:40:51 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Jul 2008 19:40:51 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <487BE50E.6090200@voidspace.org.uk>
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<487BE50E.6090200@voidspace.org.uk>
Message-ID: <487C70A3.3090808@gmail.com>

Michael Foord wrote:
> Raymond Hettinger wrote:
>> From: "Ben Finney" <ben+python at benfinney.id.au>
>>> Right, so I'm putting up a separate PEP just for the renaming. Should
>>> be arriving on this list soon.
>>
>> I would like to work with you or someone else who is interested
>> on an alternative PEP for a separate, simpler test module
>> using the py.test syntax.  That is much simpler to learn and use.
>> Instead of self.assertIsNot and whatnot, you write:
>>   assert a is not b
>> No need for tons of word-by-word spellings on things we already
>> have syntax for.  Almost anyone who has used py.test can attest
>> its syntax is much more natural, easy to learn, easy to both
>> read and write, and is much lighter weight.  I think some variant
>> of py.test could be done that is compatable with unittest
>> and the did not have the "magic" present in earlier versions of py.test.
> 
> Ah, in my haste I skipped over your comment about "magic", my apologies. 
> But in the absence of magic how do you propose to provide a meaningful 
> error message from the failure of:
> 
> assert a == b
> 
> To wrap it in a function like "assert equals(a, b)" seems to gain little 
> over unittest.

Aside from the question of providing nice error messages, two questions 
that immediately come to mind for me are:
- how do I run my tests with -O or -OO? (since the compiler will throw 
all the assert statements away before any Python code gets a chance to 
look at them)
- how do I test that code raises an expected exception?
- how do I explicitly fail a test case? (e.g. I'll often do this when I 
want to test an operation with a variety of different inputs - I'll test 
for all of the inputs of interest, collecting the failures in a list, 
then reporting a single error message at the end detailing all of the 
cases that failed)

And I've also never had any problem whatsoever debugging unit tests with 
print statements - one of the effects of the -v switch is to display 
anything which is written to stderr/stdout on the console again.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ben+python at benfinney.id.au  Tue Jul 15 12:04:33 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 20:04:33 +1000
Subject: [Python-Dev] Default metaclass in Python 3.0 modules (was: PEP:
	Consolidating names and classes in the `unittest` module
	(updated 2008-07-15))
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com>
Message-ID: <87skub8nv2.fsf_-_@benfinney.id.au>

"Benjamin Peterson" <musiccomposition at gmail.com> writes:

> On Mon, Jul 14, 2008 at 6:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > The `unittest` module will gain the following attribute, to set the
> > default metaclass for classes in the module and thus make all classes
> > in the module part of the new-style type hierarchy::
> >
> >    __metaclass__ = type
> 
> It's already done.
> 
> Line 94-95 in unittest.py (trunk):
> # All classes defined herein are 'new-style' classes, allowing use of 'super()'
> __metaclass__ = type

Hmm, you're right; I see that in Python 2.5.2 'unittest.py'.

Why is it not there in 3.0's 'unittest.py'? Is this achieved some
other way?

-- 
 \         ?Pinky, are you pondering what I'm pondering?? ?I think so, |
  `\     Brain, but if they called them ?Sad Meals?, kids wouldn't buy |
_o__)                                    them!? ?_Pinky and The Brain_ |
Ben Finney


From eric+python-dev at trueblade.com  Tue Jul 15 12:08:50 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 15 Jul 2008 06:08:50 -0400
Subject: [Python-Dev] Default metaclass in Python 3.0 modules
In-Reply-To: <87skub8nv2.fsf_-_@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com>
	<87skub8nv2.fsf_-_@benfinney.id.au>
Message-ID: <487C7732.4000102@trueblade.com>

Ben Finney wrote:
> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
> 
>> On Mon, Jul 14, 2008 at 6:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>> The `unittest` module will gain the following attribute, to set the
>>> default metaclass for classes in the module and thus make all classes
>>> in the module part of the new-style type hierarchy::
>>>
>>>    __metaclass__ = type
>> It's already done.
>>
>> Line 94-95 in unittest.py (trunk):
>> # All classes defined herein are 'new-style' classes, allowing use of 'super()'
>> __metaclass__ = type
> 
> Hmm, you're right; I see that in Python 2.5.2 'unittest.py'.
> 
> Why is it not there in 3.0's 'unittest.py'? Is this achieved some
> other way?

In 3.0 there are only new-style classes, so nothing needs to be done there.

From bignose+hates-spam at benfinney.id.au  Tue Jul 15 12:17:23 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 20:17:23 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
Message-ID: <87od4z8n9o.fsf@benfinney.id.au>

"Raymond Hettinger" <python at rcn.com> writes:

> > ``set_up(?)``
> >  Replaces ``setUp(?)``
> . .
> > ``tear_down(?)``
> >  Replaces ``tearDown(?)``
> 
> Am I the only one who finds this sort of excessive pep-8
> underscoring to be horrorific?
> 
> Nobody I know spells setup and teardown as two words.

I spell them as two words. The existing unittest framework also spells
them as two words, using camelCase.

> It's not going to be easy to remember where they are used (ie.
> isinstance is still isinstance but isset is now is_set). Go figure.

I don't see the connection of this sentence to the existing discussion.

> Another thought is that test suite code is going to get seriously
> crunched when the new, longer method names meet the 78/80 column pep
> 8 restrictions.
> 
> class TestMisc(unittest.test_case):
>    def lost_four_spaces_here_already(self):
>        self.fail_if_almost_equal('were already on the 34th column before'
>                                  'writing anything substantive',
>                                  self.testedobject.tested_method(arg1, arg2 +
>                                                                  arg3, arg4)
>        # Imagine typing and line wrapping dozens more tests like this

Yikes. Why are you using such huge indents? Break the line where
indents won't be so enormous.

class TestMisc(unittest.test_case):
   def lost_four_spaces_here_already(self):
       self.fail_if_almost_equal(
           'we know this is going to be long, so we indent before'
           'writing anything substantive',
           self.testedobject.tested_method(
               arg1, arg2 + arg3, arg4)

> Are there any ideas for some short, pithy, mnemonic names that are
> self-explantory and not full of underscores; something that wouldn't
> suck to type hundreds of times in a good test module?  IMO, the
> current names are already too long.

I'm very much in favour of having the namessay exactly what they're
testing. They need to be long because there are many of them doing
slightly-different things, so need careful disambiguation.

> Beautiful is better than ugly.
> Readability counts.
> 
> These are especially important for the unittest module.  When I'm
> making tests, I have to type self.very_long_method_name so many times
> it isn't funny.  It's easy to just stop writing tests when you get
> tired of it.  Long api names with underscores are a disincentive
> (IMO).

Yes, this is something that deserves consideration.

-- 
 \            ?Why was I with her? She reminds me of you. In fact, she |
  `\                reminds me more of you than you do!? ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From bignose+hates-spam at benfinney.id.au  Tue Jul 15 12:26:41 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 20:26:41 +1000
Subject: [Python-Dev] Default metaclass in Python 3.0 modules
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com>
	<87skub8nv2.fsf_-_@benfinney.id.au>
	<487C7732.4000102@trueblade.com>
Message-ID: <87k5fn8mu6.fsf@benfinney.id.au>

Eric Smith <eric+python-dev at trueblade.com> writes:

> Ben Finney wrote:
> > "Benjamin Peterson" <musiccomposition at gmail.com> writes:
> >> Line 94-95 in unittest.py (trunk):
> >> # All classes defined herein are 'new-style' classes, allowing use of 'super()'
> >> __metaclass__ = type
> >
> > Hmm, you're right; I see that in Python 2.5.2 'unittest.py'.
> >
> > Why is it not there in 3.0's 'unittest.py'? Is this achieved some
> > other way?
> 
> In 3.0 there are only new-style classes, so nothing needs to be done
> there.

What makes that happen in the case where a class declares no
superclass? Is there an invisible enforced "__metaclass__ = type" for
every module? Where can I read about this change?

-- 
 \       ?The apparent lesson of the Inquisition is that insistence on |
  `\         uniformity of belief is fatal to intellectual, moral, and |
_o__)    spiritual health.? ?_The Uses Of The Past_, Herbert J. Muller |
Ben Finney


From bignose+hates-spam at benfinney.id.au  Tue Jul 15 12:31:46 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 20:31:46 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest`	module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <87fxqb8mlp.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:

> Ben Finney writes:
> 
>  > Removal of ``assert*`` names
>  > ----------------------------
>  > 
>  > There is no overwhelming consensus on whether to remove the
>  > ``assert*`` names or the ``fail*`` names;
> 
> 7 to 1 is overwhelming in my book.  See
> Message-ID: <loom.20080714T230912-310 at post.gmane.org>
> From: Antoine Pitrou <solipsis at pitrou.net>

That measured only usage of unittest *within the Python standard
library*. Is that the only body of unittest-using code we need
consider?

> While people's preferences are important, I think there is a very
> good case to made that keeping this much continuity in the test
> suite as possible is more so.

That's a separate argument, then. One which I don't dismiss, but it
needs to be stated as such.

-- 
 \           ?A poet more than thirty years old is simply an overgrown |
  `\                                         child.? ?Henry L. Mencken |
_o__)                                                                  |
Ben Finney


From p.f.moore at gmail.com  Tue Jul 15 12:41:29 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Jul 2008 11:41:29 +0100
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
Message-ID: <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com>

On 15/07/2008, Barry Warsaw <barry at python.org> wrote:
> > In case anyone is interested, I have git repositories for both the
> > trunk and the py3k branch of the Python source code.  They are
> > up-to-date and so using them with git-svn would be much faster than
> > starting from scratch.
> >
> > If anyone is interested, I will find a place to host them.  They are
> > each about 150 MB in size.
> >
>
> Neil, we should try to host them on code.python.org.

If we're setting up a variety of DVCS systems there, I'd be willing to
set up Mercurial repos (I have my own local one, just trunk at the
moment but py3k would be easy enough to add). I'd need access in order
to set up any sort of sync process, though.

Paul.

From asmodai at in-nomine.org  Tue Jul 15 12:54:25 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Tue, 15 Jul 2008 12:54:25 +0200
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
In-Reply-To: <87fxqb8mlp.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
Message-ID: <20080715105425.GI60130@nexus.in-nomine.org>

-On [20080715 12:35], Ben Finney (bignose+hates-spam at benfinney.id.au) wrote:
>That measured only usage of unittest *within the Python standard
>library*. Is that the only body of unittest-using code we need
>consider?

Some greps on random Python projects give me a 4-10:1 ratio for assert*
versus fail*.

Personally I also find the assert* syntax preferable over fail*.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
If I am telling you the Truth now, do you believe it..?

From p.f.moore at gmail.com  Tue Jul 15 12:58:04 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Jul 2008 11:58:04 +0100
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest`module (updated 2008-07-15)
In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
Message-ID: <79990c6b0807150358x7ec93c8avefefa19a153aaf70@mail.gmail.com>

On 15/07/2008, Raymond Hettinger <python at rcn.com> wrote:
> > ``set_up(?)``
> >  Replaces ``setUp(?)``
> >
> . .
> > ``tear_down(?)``
> >  Replaces ``tearDown(?)``
> >
>
> Am I the only one who finds this sort of excessive pep-8 underscoring to be
> horrorific?

No.

> Nobody I know spells setup and teardown as two words. I dread using the
> module with these new names. Underscores are not fun to type. They create a
> weird_mental_pause when reading them.

I agree. The java-esque setUp and tearDown are (in my view) ugly, but
set_up and tear_down are as bad.

+1 for setup and teardown.

+1 also for thinking a *lot* harder about naming, and not just going
with phrases_joined_up_with_underscores, or
phrasesWithNoSpacesAndFunnyCapitalisation.

There are certainly issues with the simple

    assert expression

approach, but ugliness and unreadability aren't two of them.

Paul.

From ncoghlan at gmail.com  Tue Jul 15 12:58:57 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Jul 2008 20:58:57 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
 the	`unittest`module (updated 2008-07-15)
In-Reply-To: <87od4z8n9o.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au>
Message-ID: <487C82F1.9010506@gmail.com>

Ben Finney wrote:
>>        self.fail_if_almost_equal('were already on the 34th column before'
>>                                  'writing anything substantive',
>>                                  self.testedobject.tested_method(arg1, arg2 +
>>                                                                  arg3, arg4)
>>        # Imagine typing and line wrapping dozens more tests like this
> 
> Yikes. Why are you using such huge indents? Break the line where
> indents won't be so enormous.

While true, that doesn't change the fact that fail_if_almost_equal is an 
undesirably long method name.

> I'm very much in favour of having the namessay exactly what they're
> testing. They need to be long because there are many of them doing
> slightly-different things, so need careful disambiguation.

The "almost_equal" methods for floating point comparison are the main 
culprits for excessive method length, closely followed by the 
"fail_unless" variants.

(21, 'failUnlessAlmostEqual') -> 24 with underscores
(21, 'assertNotAlmostEquals') -> 25 with underscores
(20, 'assertNotAlmostEqual')  -> 24 with underscores
(18, 'assertAlmostEquals')    -> 20 with underscores
(17, 'failIfAlmostEqual')     -> 20 with underscores
(17, 'assertAlmostEqual')     -> 19 with underscores
(16, 'failUnlessRaises')      -> 18 with underscores
(15, 'failUnlessEqual')       -> 17 with underscores
(15, 'assertNotEquals')       -> 17 with underscores
(14, 'assertNotEqual')        -> 16 with underscores
(12, 'assertRaises')          -> 13 with underscores
(12, 'assertEquals')          -> 13 with underscores
(11, 'failIfEqual')           -> 13 with underscores
(11, 'assertFalse')           -> 12 with underscores
(11, 'assertEqual')           -> 12 with underscores
(10, 'failUnless')            -> 11 with underscores
(10, 'assertTrue')            -> 11 with underscores
(7, 'assert_')                ->  7 with underscores
(6, 'failIf')                 ->  7 with underscores
(4, 'fail')                   ->  4 with underscores

One option for rationalising the API would be to merely keep the 
shortest version of each phrase (i.e. use assert* instead of 
fail_unless* for the positive tests and fail_if* instead of assert_not* 
for the negative tests, and always drop the trailing 's' from 'equals'). 
This simple rule would eliminate 12 of the 20 methods (including the 
four longest method names and 7 of the longest 9), leaving the following 
minimal set of methods:

fail_if_almost_equal (20)
assert_almost_equal  (19)
assert_raises        (13)
fail_if_equal        (13)
assert_equal         (12)
assert_              (7)
fail_if              (7)
fail                 (4)

Adding in possible positive and negative forms of the proposed new 
methods (using words more appropriate for a method rather than copying 
the infix operators):

fail_if_identical (17)
fail_if_contains  (16)
assert_identical (16)
assert_contains  (15)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Tue Jul 15 13:02:16 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Jul 2008 21:02:16 +1000
Subject: [Python-Dev] Default metaclass in Python 3.0 modules
In-Reply-To: <87k5fn8mu6.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com>	<87skub8nv2.fsf_-_@benfinney.id.au>	<487C7732.4000102@trueblade.com>
	<87k5fn8mu6.fsf@benfinney.id.au>
Message-ID: <487C83B8.2080403@gmail.com>

Ben Finney wrote:
> Eric Smith <eric+python-dev at trueblade.com> writes:
> 
>> Ben Finney wrote:
>>> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
>>>> Line 94-95 in unittest.py (trunk):
>>>> # All classes defined herein are 'new-style' classes, allowing use of 'super()'
>>>> __metaclass__ = type
>>> Hmm, you're right; I see that in Python 2.5.2 'unittest.py'.
>>>
>>> Why is it not there in 3.0's 'unittest.py'? Is this achieved some
>>> other way?
>> In 3.0 there are only new-style classes, so nothing needs to be done
>> there.
> 
> What makes that happen in the case where a class declares no
> superclass? Is there an invisible enforced "__metaclass__ = type" for
> every module? Where can I read about this change?

The magic is actually in 2.x, not in 3.0. In 2.x, if you don't explicit 
set the metaclass (or inherit explicitly from an object which sets it), 
then the default metaclass is 'classobj'. In 3.0, that magic goes away 
and the default metaclass is just 'type'.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From stephen at xemacs.org  Tue Jul 15 13:52:15 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 15 Jul 2008 20:52:15 +0900
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`	module (updated 2008-07-15)
In-Reply-To: <87fxqb8mlp.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
Message-ID: <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:

 > > Message-ID: <loom.20080714T230912-310 at post.gmane.org>
 > > From: Antoine Pitrou <solipsis at pitrou.net>
 > 
 > That measured only usage of unittest *within the Python standard
 > library*. Is that the only body of unittest-using code we need
 > consider?

Yes, for the purposes of this PEP.  We already know that many people
want various different things.  You want fail* /rather than/ assert*,
but Steven d'Aprono wants /both/, and I prefer assert* /exclusively/.
I don't see why we all shouldn't be satisfied[1], so the content of
unittest should not set policy for our own projects.  So there should
be other modules (perhaps in the stdlib, perhaps not) to satisfy those
preferences not catered to by stdlib's unittest.

Thus this PEP should restrict it's concern to revising unittest to
conform to PEPs and help standardize Python's own testing, without
trying to impose standards on the whole community of Python users.

That's my rationale.  YMMV.

Footnotes: 
[1]  For myself, if the fail*-only proposal wins, I'd switch to that;
fighting the tide on this isn't my idea of fun.  But other assert*
fans may be more faithful to their original preference.


From barry at python.org  Tue Jul 15 14:29:53 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Jul 2008 08:29:53 -0400
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com>
Message-ID: <16F705A1-663A-47C4-873D-5B90E5690AF5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 15, 2008, at 6:41 AM, Paul Moore wrote:

> On 15/07/2008, Barry Warsaw <barry at python.org> wrote:
>>> In case anyone is interested, I have git repositories for both the
>>> trunk and the py3k branch of the Python source code.  They are
>>> up-to-date and so using them with git-svn would be much faster than
>>> starting from scratch.
>>>
>>> If anyone is interested, I will find a place to host them.  They are
>>> each about 150 MB in size.
>>>
>>
>> Neil, we should try to host them on code.python.org.
>
> If we're setting up a variety of DVCS systems there, I'd be willing to
> set up Mercurial repos (I have my own local one, just trunk at the
> moment but py3k would be easy enough to add). I'd need access in order
> to set up any sort of sync process, though.

We need to get the python.org admins involved in the process at this  
point.  I'm pretty sure that Thomas runs the Bazaar sync operations on  
one of his machines and pushes to code.python.org.  I don't know if  
that's an option for you and/or whether we should bring this in- 
house.  I do remember that at the time, there were issues with bzr-svn  
requiring svn 1.5, which hadn't been released yet.  Now that that's  
out, and there is a more stable generic fast-import process, perhaps  
bringing the sync operations in will be easier.  Unfortunately I will  
have no cycles for this in the next several weeks.

When we put up the experimental bzr repos, we generalized the scripts  
we use to set up access control based on ssh.  It would be a good  
thing if we can use the same arrangement to control access to git and  
hg.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSHyYQXEjvBPtnXfVAQKNUwP/bH9XYHqZPLipE/cWvI6MpFSrLHU7ANPd
oa/VBsb0QiO3G1+dWj6Q7czJ2QvV/IE0VmXi37AXy3NSp7of80hRDRdpwNabDvnI
cRKSueL/7qCb6+HG0DKoZ+4oh1g94jAok2naUNzffxdx4qFGlQ72WK3bQH1ZrNlp
k5ynDPw9Xdg=
=oC0s
-----END PGP SIGNATURE-----

From barry at python.org  Tue Jul 15 14:32:37 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Jul 2008 08:32:37 -0400
Subject: [Python-Dev] Reminder: beta 2's schedule for tomorrow
Message-ID: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A reminder: the second betas of Python 2.6 and 3.0 are schedule for  
tomorrow.  I will try to hang out on #python-dev today and will start  
looking at the trackers and buildbots.  Hopefully, we're on track to  
get the releases out!

If there is anything you need a decision on, please follow up to this  
thread.  I'm inundated with email so I can't watch every thread on the  
mailing lists.  Or ping me on #python-dev.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSHyY5XEjvBPtnXfVAQKvYgP+J4GYVDWOraKhErC4wRl0ylEdHcc9LypB
O7yxhBjb+tLu54IImxLkj/Nzff4uUQI6zsA6lg87A4b+sC0/0ltH4+vGkZaq8z7/
xSlP0b0UOaBpOEhTR8ZJK3DJBjSk97IR8Ty3MV1DuM0cczYltorCmQVpodA5ciXj
PRy/LAIalJg=
=DTQM
-----END PGP SIGNATURE-----

From ben+python at benfinney.id.au  Tue Jul 15 15:03:01 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 23:03:01 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
Message-ID: <877ibn8flm.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> fail_if_almost_equal is an undesirably long method name.

I disagree. It says what the method does, as precisely as necessary to
distinguish it from other methods of the class. It is as long as it
needs to be to say that while still being readable and PEP 8
compliant.

> One option for rationalising the API would be to merely keep the
> shortest version of each phrase (i.e. use assert* instead of
> fail_unless* for the positive tests and fail_if* instead of
> assert_not* for the negative tests, and always drop the trailing 's'
> from 'equals').

I think clarity, consistency, and discoverability are more important
criteria than saving a few characters in a method name.

> Adding in possible positive and negative forms of the proposed new
> methods

A major point of this PEP is to *remove* redundant names in the API.

-- 
 \          ?It's dangerous to be right when the government is wrong.? |
  `\                                   ?Francois Marie Arouet Voltaire |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Tue Jul 15 15:03:52 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 23:03:52 +1000
Subject: [Python-Dev] Default metaclass in Python 3.0 modules
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com>
	<87skub8nv2.fsf_-_@benfinney.id.au>
	<487C7732.4000102@trueblade.com>
	<87k5fn8mu6.fsf@benfinney.id.au> <487C83B8.2080403@gmail.com>
Message-ID: <873amb8fk7.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> Ben Finney wrote:
> > What makes that happen in the case where a class declares no
> > superclass? Is there an invisible enforced "__metaclass__ = type"
> > for every module? Where can I read about this change?
> 
> The magic is actually in 2.x, not in 3.0. In 2.x, if you don't
> explicit set the metaclass (or inherit explicitly from an object
> which sets it), then the default metaclass is 'classobj'. In 3.0,
> that magic goes away and the default metaclass is just 'type'.

That helps. Thanks.

-- 
 \          ?When I get real bored, I like to drive downtown and get a |
  `\         great parking spot, then sit in my car and count how many |
_o__)                    people ask me if I'm leaving.? ?Steven Wright |
Ben Finney


From andrew-pythondev at puzzling.org  Tue Jul 15 15:05:33 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Tue, 15 Jul 2008 23:05:33 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
In-Reply-To: <87fxqb8mlp.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
Message-ID: <20080715130533.GB17917@steerpike.home.puzzling.org>

Ben Finney wrote:
> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
> 
> > Ben Finney writes:
> > 
> >  > Removal of ``assert*`` names
> >  > ----------------------------
> >  > 
> >  > There is no overwhelming consensus on whether to remove the
> >  > ``assert*`` names or the ``fail*`` names;
> > 
> > 7 to 1 is overwhelming in my book.  See
> > Message-ID: <loom.20080714T230912-310 at post.gmane.org>
> > From: Antoine Pitrou <solipsis at pitrou.net>
> 
> That measured only usage of unittest *within the Python standard
> library*. Is that the only body of unittest-using code we need
> consider?

Three more data points then:

bzr: 13228 assert* vs. 770 fail*.

Twisted: 6149 assert* vs. 1666 fail*.

paramiko: 431 assert* vs. 4 fail*.

The data seems pretty overwhelmingly in favour of keeping assert*. 

-Andrew.


From dickinsm at gmail.com  Tue Jul 15 15:09:17 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 15 Jul 2008 14:09:17 +0100
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
Message-ID: <5c6f2a5d0807150609o136715f8gda5e599f1330f13c@mail.gmail.com>

On Tue, Jul 15, 2008 at 1:32 PM, Barry Warsaw <barry at python.org> wrote:
> If there is anything you need a decision on, please follow up to this
> thread.  I'm inundated with email so I can't watch every thread on the
> mailing lists.  Or ping me on #python-dev.

Can I request permission to check in the patch attached to issue 3008
(float <-> hex) conversion?  This is the revised version of the bin/oct/hex
for floats patch that Raymond had to withdraw shortly after the first beta.

Link to the issue:

http://bugs.python.org/issue3008

and the long recent thread on python-dev

http://mail.python.org/pipermail/python-dev/2008-June/080558.html

Mark

From solipsis at pitrou.net  Tue Jul 15 15:25:16 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Jul 2008 13:25:16 +0000 (UTC)
Subject: [Python-Dev] Mercurial mirrors
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com>
Message-ID: <loom.20080715T131942-636@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:
> 
> If we're setting up a variety of DVCS systems there, I'd be willing to
> set up Mercurial repos (I have my own local one, just trunk at the
> moment but py3k would be easy enough to add).

Using the convert extension or using hgsvn?
I already have public mirrors using hgsvn (synced every 10 minutes):
http://hg.pitrou.net/public/py3k/py3k/
http://hg.pitrou.net/public/cpython/trunk/

I don't know if I'm the only one using them... I know Ralf Schmitt has his own
mirrors at http://hgpy.de/py/ , but they don't have the full history.

Regards

Antoine.



From ncoghlan at gmail.com  Tue Jul 15 15:29:34 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Jul 2008 23:29:34 +1000
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
Message-ID: <487CA63E.4000207@gmail.com>

Barry Warsaw wrote:
> A reminder: the second betas of Python 2.6 and 3.0 are schedule for 
> tomorrow.  I will try to hang out on #python-dev today and will start 
> looking at the trackers and buildbots.  Hopefully, we're on track to get 
> the releases out!
> 
> If there is anything you need a decision on, please follow up to this 
> thread.  I'm inundated with email so I can't watch every thread on the 
> mailing lists.  Or ping me on #python-dev.

I'll be checking in the fix for issue 2235 shortly (the problem with 
__hash__ not being inherited in 2.x). A pre-review from Guido would have 
been nice (since monkeying with typeobject.c is always a somewhat 
delicate operation), but it looks like he didn't get a chance to get 
back to it after Europython.

I'm also going to forward port the underlying implementation to Py3k (so 
that a non-existent hash is indicated by PyObject_HashNotImplemented in 
tp_hash at the C level as well as by __hash__ = None at the Python level).

Cheers,
Nick.

_______________________________________________
Python-3000 mailing list
Python-3000 at python.org
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/ncoghlan%40gmail.com

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jnoller at gmail.com  Tue Jul 15 15:36:24 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 15 Jul 2008 09:36:24 -0400
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
Message-ID: <4222a8490807150636w27d3186chd52d7266d0670b65@mail.gmail.com>

On Tue, Jul 15, 2008 at 8:32 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> A reminder: the second betas of Python 2.6 and 3.0 are schedule for
> tomorrow.  I will try to hang out on #python-dev today and will start
> looking at the trackers and buildbots.  Hopefully, we're on track to get the
> releases out!
>
> If there is anything you need a decision on, please follow up to this
> thread.  I'm inundated with email so I can't watch every thread on the
> mailing lists.  Or ping me on #python-dev.
>
> - -Barry
>

One fix I would like for the upcoming beta is the patch to issue874900
- this seems to have resolved a good chunk of the test_multiprocessing
hangs we've been having problems with. I'm in the process of
double-checking the latest patch posted by Greg.

http://bugs.python.org/issue874900

From steve at pearwood.info  Tue Jul 15 15:42:23 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 15 Jul 2008 23:42:23 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>
References: <20080713093936.GA3623@benfinney.id.au>
	<87r69wa8hs.fsf@benfinney.id.au>
	<pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>
Message-ID: <200807152342.24160.steve@pearwood.info>

On Tue, 15 Jul 2008 04:55:59 pm Thomas Lotze wrote:

> I'm surprised nobody (that I've noticed) has brought up the point yet
> that test code is a lot easier to read if it makes positive
> assertions. 

Please don't claim that your subjective opinion is an objective fact.


> When reading failure conditions, one has to constantly 
> invert them in order to deduce the behaviour that is tested.

You might have to. Don't assume that everyone else has your difficulty.


> failUnless and friends aren't better either IMO since while they do
> work with positive assertions, the method names themselves are doubly
> negative. assert* methods are so much more straightforward to
> comprehend.

Maybe for you. That's not a human universal. Please don't assume that 
your favourite bike-shed colour must be the favourite colour of 
everyone else too.



-- 
Steven

From p.f.moore at gmail.com  Tue Jul 15 15:43:03 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Jul 2008 14:43:03 +0100
Subject: [Python-Dev] Mercurial mirrors
In-Reply-To: <loom.20080715T131942-636@post.gmane.org>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com>
	<loom.20080715T131942-636@post.gmane.org>
Message-ID: <79990c6b0807150643s6fdcc3fes68511e737768df23@mail.gmail.com>

On 15/07/2008, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
> >
> > If we're setting up a variety of DVCS systems there, I'd be willing to
> > set up Mercurial repos (I have my own local one, just trunk at the
> > moment but py3k would be easy enough to add).
>
> Using the convert extension or using hgsvn?
> I already have public mirrors using hgsvn (synced every 10 minutes):
> http://hg.pitrou.net/public/py3k/py3k/
> http://hg.pitrou.net/public/cpython/trunk/

Personally, I use convert, because it's more robust on WIndows (there
were a few commits with case clashes which hgsvn can't get past on a
Windows box).

For a central repo, I don't care - but I'd prefer it if all the
options were hosted on code.python.org, just so that no one option
feels more "official" than any other. If I do the work, I'd use
convert, simply because I'm more familiar with it, but that's all.
OTOH, if the process is just copying a local mirror up to the server,
copying your repo might be better (my PC is not always on, and I don't
have frequent syncs set up at the moment).

Paul.

From musiccomposition at gmail.com  Tue Jul 15 15:43:58 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 15 Jul 2008 08:43:58 -0500
Subject: [Python-Dev] Default metaclass in Python 3.0 modules (was: PEP:
	Consolidating names and classes in the `unittest` module
	(updated 2008-07-15))
In-Reply-To: <87skub8nv2.fsf_-_@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com>
	<87skub8nv2.fsf_-_@benfinney.id.au>
Message-ID: <1afaf6160807150643o2ba28d1awf993cd4594f0036f@mail.gmail.com>

On Tue, Jul 15, 2008 at 5:04 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
>>
>> Line 94-95 in unittest.py (trunk):
>> # All classes defined herein are 'new-style' classes, allowing use of 'super()'
>> __metaclass__ = type
>
> Hmm, you're right; I see that in Python 2.5.2 'unittest.py'.
>
> Why is it not there in 3.0's 'unittest.py'? Is this achieved some
> other way?

All classes are new-style classes in 3.0!




-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From ben+python at benfinney.id.au  Tue Jul 15 15:48:08 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 23:48:08 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
Message-ID: <87y7436yxz.fsf@benfinney.id.au>

Andrew Bennetts <andrew-pythondev at puzzling.org> writes:

> Ben Finney wrote:
> > "Stephen J. Turnbull" <stephen at xemacs.org> writes:
> > > Message-ID: <loom.20080714T230912-310 at post.gmane.org>
> > > From: Antoine Pitrou <solipsis at pitrou.net>
> > 
> > That measured only usage of unittest *within the Python standard
> > library*. Is that the only body of unittest-using code we need
> > consider?
> 
> Three more data points then:
> 
> bzr: 13228 assert* vs. 770 fail*.
> Twisted: 6149 assert* vs. 1666 fail*.
> paramiko: 431 assert* vs. 4 fail*.
> 
> The data seems pretty overwhelmingly in favour of keeping assert*. 

Noted, thanks.

So far I have "precedent and tradition" and "positive admonition looks
better" in support of preferring the 'assert*' names. Are there any
others?

I believe I've stated (in the most-recent PEP revision) the strongest
reasons in favour of the 'fail*' names.

This all gets summarised in the Rationale section for the PEP.

-- 
 \           ?Killing the creator was the traditional method of patent |
  `\                        protection? ?Terry Pratchett, _Small Gods_ |
_o__)                                                                  |
Ben Finney


From steve at pearwood.info  Tue Jul 15 15:51:41 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 15 Jul 2008 23:51:41 +1000
Subject: [Python-Dev]
	=?utf-8?q?PEP=3A_Consolidating_names_and_classes_in_?=
	=?utf-8?q?the=09=60unittest=60_module_=28updated_2008-07-15=29?=
In-Reply-To: <20080715105425.GI60130@nexus.in-nomine.org>
References: <87vdz8a983.fsf@benfinney.id.au> <87fxqb8mlp.fsf@benfinney.id.au>
	<20080715105425.GI60130@nexus.in-nomine.org>
Message-ID: <200807152351.41909.steve@pearwood.info>

On Tue, 15 Jul 2008 08:54:25 pm Jeroen Ruigrok van der Werven wrote:

> Some greps on random Python projects give me a 4-10:1 ratio for
> assert* versus fail*.

Without knowing what those "random" projects are, or what the grep was, 
it's impossible to interpret that statistic. For example, is it biased 
by the existence of the assert keyword? Do they represent programmers' 
free choices, or the projects' compulsory style guides?

> Personally I also find the assert* syntax preferable over fail*.

Thank you for acknowledging that as a personal preference rather than 
making another spurious claim of objective superiority.



-- 
Steven

From ben+python at benfinney.id.au  Tue Jul 15 15:58:02 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 15 Jul 2008 23:58:02 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
References: <87vdz8a983.fsf@benfinney.id.au>
Message-ID: <87skub6yhh.fsf@benfinney.id.au>

Significant updates include removing all reference to the
(already-resolved) new-style class issue, adding footnotes and
references, and a Rationale summary of discussion on both sides of the
divide for 'assert*' versus 'fail*' names.


:PEP:               XXX
:Title:             Consolidating names in the `unittest` module
:Version:           0.2
:Last-Modified:     2008-07-15
:Author:            Ben Finney <ben+python at benfinney.id.au>
:Status:            Draft
:Type:              Standards Track
:Content-Type:      test/x-rst
:Created:           2008-07-14
:Python-Version:    2.7, 3.1
:Post-History:


..  contents::


Abstract
========

This PEP proposes to consolidate the names that constitute the API of
the standard library `unittest` module, with the goal of removing
redundant names, and conforming with PEP 8.


Motivation
==========

The normal use case for the `unittest` module is to subclass its
classes, overriding and re-using its functios and methods. This draws
constant attention to the fact that the existing implementation fails
several current Python standards:

* It does not conform to PEP 8 [#PEP-8]_, requiring users to write
  their own non-PEP-8 conformant names when overriding methods, and
  encouraging extensions to further depart from PEP 8.

* It has many synonyms in its API, which goes against the Zen of
  Python [#PEP-20]_ (specifically, that "there should be one, and
  preferably only one, obvious way to do it").


Specification
=============

Remove obsolete names
---------------------

The following module attributes are not documented as part of the API
and are marked as obsolete in the implementation. They will be
removed.

* ``_makeLoader``
* ``getTestCaseNames``
* ``makeSuite``
* ``findTestCases``

Remove redundant names
----------------------

The following attribute names exist only as synonyms for other names.
They are to be removed, leaving only one name for each attribute in
the API.

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

* ``assertEqual``
* ``assertEquals``
* ``assertNotEqual``
* ``assertNotEquals``
* ``assertAlmostEqual``
* ``assertAlmostEquals``
* ``assertNotAlmostEqual``
* ``assertNotAlmostEquals``
* ``assertRaises``
* ``assert_``
* ``assertTrue``
* ``assertFalse``

Conform API with PEP 8
----------------------

The following names are to be introduced, each replacing an existing
name, to make all names in the module conform with PEP 8 [#PEP-8]_.
Each name is shown with the existing name that it replaces.

Where function parameters are to be renamed also, they are shown.
Where function parameters are not to be renamed, they are elided with
the ellipse ("?") symbol.

Module attributes
~~~~~~~~~~~~~~~~~

``default_test_loader``
  Replaces ``defaultTestLoader``

``TestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``add_error(?)``
  Replaces ``addError(?)``

``add_result(?)``
  Replaces ``addResult(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``should_stop``
  Replaces ``shouldStop``

``start_test(?)``
  Replaces ``startTest(?)``

``stop_test(?)``
  Replaces ``stopTest(?)``

``tests_run``
  Replaces ``testsRun``

``was_successful(?)``
  Replaces ``wasSuccessful(?)``

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, method_name='run_test')``
  Replaces ``__init__(self, methodName='runTest')``

``_test_method_doc``
  Replaces ``_testMethodDoc``

``_test_method_name``
  Replaces ``_testMethodName``

``failure_exception``
  Replaces ``failureException``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``default_test_result(?)``
  Replaces ``defaultTestResult(?)``

``fail_if(?)``
  Replaces ``failIf(?)``

``fail_if_almost_equal(?)``
  Replaces ``failIfAlmostEqual(?)``

``fail_if_equal(?)``
  Replaces ``failIfEqual(?)``

``fail_unless(?)``
  Replaces ``failUnless(?)``

``fail_unless_almost_equal(?)`` 
  Replaces ``failUnlessAlmostEqual(?)``

``fail_unless_equal(?)``
  Replaces ``failUnlessEqual(?)``

``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)``
  Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``FunctionTestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, test_func, set_up, tear_down, description)``
  Replaces ``__init__(self, testFunc, setUp, tearDown, description)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``TestSuite`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~

``add_test(?)``
  Replaces ``addTest(?)``

``add_tests(?)``
  Replaces ``addTests(?)``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``TestLoader`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``sort_test_methods_using``
  Replaces ``sortTestMethodsUsing``

``suite_class``
  Replaces ``suiteClass``

``test_method_prefix``
  Replaces ``testMethodPrefix``

``get_test_case_names(self, test_case_class)``
  Replaces ``getTestCaseNames(self, testCaseClass)``

``load_tests_from_module(?)``
  Replaces ``loadTestsFromModule(?)``

``load_tests_from_name(?)``
  Replaces ``loadTestsFromName(?)``

``load_tests_from_names(?)``
  Replaces ``loadTestsFromNames(?)``

``load_tests_from_test_case(self, test_case_class)``
  Replaces ``loadTestsFromTestCase(self, testCaseClass)``

``_TextTestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``show_all``
  Replaces ``showAll``

``add_error(?)``
  Replaces ``addError(?)``

``add_failure(?)``
  Replaces ``addFailure(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``get_description(?)``
  Replaces ``getDescription(?)``

``print_error_list(?)``
  Replaces ``printErrorList(?)``

``print_errors(?)``
  Replaces ``printErrors(?)``

``start_test(?)``
  Replaces ``startTest(?)``

``TextTestRunner`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``_make_result(?)``
  Replaces ``_makeResult(?)``

``TestProgram`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, module, default_test, argv, test_runner, test_loader)``
  Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)``

``create_tests(?)``
  Replaces ``createTests(?)``

``parse_args(?)``
  Replaces ``parseArgs(?)``

``run_tests(?)``
  Replaces ``runTests(?)``

``usage_exit(?)``
  Replaces ``usageExit(?)``


Rationale
=========

Redundant names
---------------

The current API, with two or in some cases three different names
referencing exactly the same function, leads to an overbroad and
redundant API that violates PEP 20 [#PEP-20]_ ("there should be one,
and preferably only one, obvious way to do it").

Removal of ``assert*`` names
----------------------------

While there is consensus support to `remove redundant names`_ for the
``TestCase`` test methods, the issue of which set of names should be
retained is controversial.

Arguments in favour of retaining only the ``assert*`` names:

* BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference
  for the ``assert*`` names.

* Precedent: The Python standard library currently uses the
  ``assert*`` names by a roughly 8:1 majority over the ``fail*``
  names. (Counting unit tests in the py3k tree at 2008-07-15
  [#pitrou-1]_.)

  An ad-hoc sampling of other projects that use `unittest` also
  demonstrates strong preference for use of the ``assert*`` names
  [#bennetts-1]_.

* Positive admonition: The ``assert*`` names state the intent of how
  the code under test *should* behave, while the ``fail*`` names are
  phrased in terms of how the code *should not* behave.

Arguments in favour of retaining only the ``fail*`` names:

* Explicit is better than implicit: The ``fail*`` names state *what
  the function will do* explicitly: fail the test. With the
  ``assert*`` names, the action to be taken is only implicit.

* Avoid false implication: The test methods do not have any necessary
  connection with the built-in ``assert`` statement. Even the
  exception raised, while it defaults to ``AssertionException``, is
  explicitly customisable via the documented ``failure_exception``
  attribute. Choosing the ``fail*`` names avoids the false association
  with either of these.

  This is exacerbated by the plain-boolean test using a name of
  ``assert_`` (with a trailing underscore) to avoid a name collision
  with the built-in ``assert`` statement. The corresponding
  ``fail_if`` name has no such issue.

PEP 8 names
-----------

Although `unittest` (and its predecessor `PyUnit`) are intended to be
familiar to users of other xUnit interfaces, there is no attempt at
direct API compatibility since the only code that Python's `unittest`
interfaces with is other Python code. The module is in the standard
library and its names should all conform with PEP 8 [#PEP-8]_.


Backwards Compatibility
=======================

The names to be obsoleted should be deprecated and removed according
to the schedule for modules in PEP 4 [#PEP-4]_.

While deprecated, use of the deprecated attributes should raise a
``DeprecationWarning``, with a message stating which replacement name
should be used.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..  [#PEP-4] http://www.python.org/dev/peps/pep-0004
..  [#PEP-8] http://www.python.org/dev/peps/pep-0008
..  [#PEP-20] http://www.python.org/dev/peps/pep-0020
..  [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-April/078485.html
..  [#pitrou-1] http://mail.python.org/pipermail/python-dev/2008-July/081090.html
..  [#bennetts-1] http://mail.python.org/pipermail/python-dev/2008-July/081141.html


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From solipsis at pitrou.net  Tue Jul 15 16:01:20 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Jul 2008 14:01:20 +0000 (UTC)
Subject: [Python-Dev] Mercurial mirrors
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com>
	<loom.20080715T131942-636@post.gmane.org>
	<79990c6b0807150643s6fdcc3fes68511e737768df23@mail.gmail.com>
Message-ID: <loom.20080715T135736-490@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:
> 
> Personally, I use convert, because it's more robust on WIndows (there
> were a few commits with case clashes which hgsvn can't get past on a
> Windows box).

If the convert extension is finally usable for incremental mirroring, then it's
good news. I don't want to maintain hgsvn all my life :-)

> For a central repo, I don't care - but I'd prefer it if all the
> options were hosted on code.python.org, just so that no one option
> feels more "official" than any other.

I completely agree.

Regards

Antoine.



From R.W.Thomas.02 at cantab.net  Tue Jul 15 16:00:30 2008
From: R.W.Thomas.02 at cantab.net (Richard Thomas)
Date: Tue, 15 Jul 2008 15:00:30 +0100
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
In-Reply-To: <87y7436yxz.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<87y7436yxz.fsf@benfinney.id.au>
Message-ID: <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com>

On Tue, Jul 15, 2008 at 2:48 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> So far I have "precedent and tradition" and "positive admonition looks
> better" in support of preferring the 'assert*' names. Are there any
> others?

I've been told by a couple of non-programmers that "failUnless" is
more intuitive than "assert" if only for the reason that its unclear
what "assert" might do. This is similar to one of the arguments raised
in the PEP, but considered from the point of view of someone new to
test frameworks it could be all the more important.

On another note, while I understand that consistency is a good thing
is it really that important in test suites? Obviously the unittest
module itself should be internally consistent but why not provide
people using the all the synonyms they might want? I can see people
just wrapping TestCase to add the synonyms back in.

Richard

From ncoghlan at gmail.com  Tue Jul 15 16:16:36 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Jul 2008 00:16:36 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
 the	`unittest`module (updated 2008-07-15)
In-Reply-To: <487C82F1.9010506@gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
Message-ID: <487CB144.1080200@gmail.com>

Nick Coghlan wrote:
> One option for rationalising the API would be to merely keep the
> shortest version of each phrase (i.e. use assert* instead of
> fail_unless* for the positive tests and fail_if* instead of
> assert_not* for the negative tests, and always drop the trailing 's'
> from 'equals').

Disclaimer: I'm not convinced the ideas in this message are actually a 
good idea myself. But I found them intriguing enough to bother posting 
them. To give some idea of how the different styles would look, here's 
an example (based on one of the tests in test_runpy) using a combination 
of assert* and fail_if* names:

         self.fail_if_contains(d1, "result")
         self.assert_identical(d2["initial"], initial)
         self.assert_equal(d2["result"], self.expected_result)
         self.assert_equal(d2["nested"]["x"], 1)
         self.assert_(d2["run_name_in_sys_modules"])
         self.assert_(d2["module_in_sys_modules"])
         self.assert_identical(d2["run_argv0"], file)
         self.assert_identical(sys.argv[0], saved_argv0)
         self.fail_if_contains(sys.modules, name)

A somewhat odd thought that occurred to me is that the shortest possible 
way of writing negative assertions (i.e. asserting that something is not 
the case) is to treat them as denials and use the single word 'deny'.

This approach would give:

Test assertions:
   assert_almost_equal
   assert_identical
   assert_contains
   assert_raises
   assert_equal
   assert_

Test denials (negative assertions):
   deny_almost_equal   (17)
   deny_identical      (14)
   deny_contains       (13)
   deny_equal          (10)
   deny                 (4)

Explicit failure:
   fail                 (4)

Using the test_runpy example assertions:
         self.deny_contains(d1, "result")
         self.assert_identical(d2["initial"], initial)
         self.assert_equal(d2["result"], self.expected_result)
         self.assert_equal(d2["nested"]["x"], 1)
         self.assert_(d2["run_name_in_sys_modules"])
         self.assert_(d2["module_in_sys_modules"])
         self.assert_identical(d2["run_argv0"], file)
         self.assert_identical(sys.argv[0], saved_argv0)
         self.deny_contains(sys.modules, name)

I actually quite like that - and it saves not only several characters, 
but also an underscore over the fail_if* and assert_not* variants.

The second odd thought was what happens if the 'assert' is made implicit?

Test assertions:
   are_almost_equal
   are_identical
   does_contain
   does_raise
   are_equal
   assert_

Test negative assertions:
   not_almost_equal
   not_identical
   not_contains
   not_equal
   not_

Explicit failure:
   fail

Using the test_runpy example assertions:
         self.not_contains(d1, "result")
         self.are_identical(d2["initial"], initial)
         self.are_equal(d2["result"], self.expected_result)
         self.are_equal(d2["nested"]["x"], 1)
         self.assert_(d2["run_name_in_sys_modules"])
         self.assert_(d2["module_in_sys_modules"])
         self.are_identical(d2["run_argv0"], file)
         self.are_identical(sys.argv[0], saved_argv0)
         self.not_contains(sys.modules, name)

Yecch, I think that idea can be safely consigned to the mental trash heap.
Another late night API concept: create a "check" object with the LHS of 
the binary operation being asserted, then use methods on that check 
object to supply the RHS:

         self.check("result").not_in(d1)
         self.check(d2["initial"]).is_(initial)
         self.check(d2["result"]).equals(self.expected_result)
         self.check(d2["nested"]["x"]).equals(1)
         self.assert_(d2["run_name_in_sys_modules"])
         self.assert_(d2["module_in_sys_modules"])
         self.check(d2["run_argv0"]).is_(file)
         self.check(sys.argv[0]).is_(saved_argv0)
         self.check(name).not_in(sys.modules)

This approach would also be handy if you needed to check multiple 
conditions on a single result:

     check = self.check(result)
     check.is_equal(expected_result) # Right answer
     check.is_not(expected_result)   # Fresh object
     check.is_(recent_results[-1])   # Recorded properly

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ctb at msu.edu  Tue Jul 15 16:23:35 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Tue, 15 Jul 2008 07:23:35 -0700
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
In-Reply-To: <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<87y7436yxz.fsf@benfinney.id.au>
	<1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com>
Message-ID: <20080715142335.GG2873@caltech.edu>

On Tue, Jul 15, 2008 at 03:00:30PM +0100, Richard Thomas wrote:
-> On Tue, Jul 15, 2008 at 2:48 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
-> > So far I have "precedent and tradition" and "positive admonition looks
-> > better" in support of preferring the 'assert*' names. Are there any
-> > others?
-> 
-> I've been told by a couple of non-programmers that "failUnless" is
-> more intuitive than "assert" if only for the reason that its unclear
-> what "assert" might do. This is similar to one of the arguments raised
-> in the PEP, but considered from the point of view of someone new to
-> test frameworks it could be all the more important.

Maybe this is an unnecessarily hard line, but if someone doesn't know
what "assert" does, then they should go look up the keyword -- it's
part of Python (and present in C, C++, Perl, and PHP as well).  So
unless the person is new to Python AND testing, they should know it.

-> On another note, while I understand that consistency is a good thing
-> is it really that important in test suites? Obviously the unittest
-> module itself should be internally consistent but why not provide
-> people using the all the synonyms they might want? I can see people
-> just wrapping TestCase to add the synonyms back in.

While I don't have a strong position on the assert* vs fail*, I think
consistency and simplicity in test suites is at least as important as in
code: if you have to work hard to understand and debug your test suites,
you've done something seriously wrong in building your tests.

Tests should be as simple as possible, and no simpler.

--titus
-- 
C. Titus Brown, ctb at msu.edu

From paul at rudin.co.uk  Tue Jul 15 16:35:57 2008
From: paul at rudin.co.uk (Paul Rudin)
Date: Tue, 15 Jul 2008 15:35:57 +0100
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
Message-ID: <87fxqb5i5u.fsf@rudin.co.uk>

Nick Coghlan <ncoghlan at gmail.com> writes:


> While true, that doesn't change the fact that fail_if_almost_equal is
> an undesirably long method name.

fail_if_near ?


From p.f.moore at gmail.com  Tue Jul 15 17:09:13 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Jul 2008 16:09:13 +0100
Subject: [Python-Dev] Mercurial mirrors
In-Reply-To: <loom.20080715T135736-490@post.gmane.org>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com>
	<loom.20080715T131942-636@post.gmane.org>
	<79990c6b0807150643s6fdcc3fes68511e737768df23@mail.gmail.com>
	<loom.20080715T135736-490@post.gmane.org>
Message-ID: <79990c6b0807150809o1e01d5d1odc6b6ef1e83d8df8@mail.gmail.com>

On 15/07/2008, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
> >
> > Personally, I use convert, because it's more robust on WIndows (there
> > were a few commits with case clashes which hgsvn can't get past on a
> > Windows box).
>
> If the convert extension is finally usable for incremental mirroring, then it's
> good news. I don't want to maintain hgsvn all my life :-)

It works for me. I think a couple of very recently landed patches[1]
may be needed for certain case issues on Windows, but that's all.
Also, it requires the Subversion bindings, and the "official" Windows
binaries don't include those - but it's easy to build your own copy
that does.

Paul.

[1] In other words, not in 1.0 or even 1.0.1.

From ncoghlan at gmail.com  Tue Jul 15 17:59:17 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Jul 2008 01:59:17 +1000
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <487CA63E.4000207@gmail.com>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
	<487CA63E.4000207@gmail.com>
Message-ID: <487CC955.1000305@gmail.com>

Nick Coghlan wrote:
> Barry Warsaw wrote:
>> A reminder: the second betas of Python 2.6 and 3.0 are schedule for 
>> tomorrow.  I will try to hang out on #python-dev today and will start 
>> looking at the trackers and buildbots.  Hopefully, we're on track to 
>> get the releases out!
>>
>> If there is anything you need a decision on, please follow up to this 
>> thread.  I'm inundated with email so I can't watch every thread on the 
>> mailing lists.  Or ping me on #python-dev.
> 
> I'll be checking in the fix for issue 2235 shortly (the problem with 
> __hash__ not being inherited in 2.x). A pre-review from Guido would have 
> been nice (since monkeying with typeobject.c is always a somewhat 
> delicate operation), but it looks like he didn't get a chance to get 
> back to it after Europython.
> 
> I'm also going to forward port the underlying implementation to Py3k (so 
> that a non-existent hash is indicated by PyObject_HashNotImplemented in 
> tp_hash at the C level as well as by __hash__ = None at the Python level).

This is now done. There's some documentation updates still to do, as 
well as adding the Py3k warning back in to the 2.6 version, but I won't 
have time to do that for this release - I dropped the tracker item down 
to deferred blocker so it gets back on the list for beta 3.

Looking at the buildbots, getting the multiprocessing fixes in looks 
like it will be a major help in getting more of them to go green, and 
the import related lib2to3 tests also appear to need some attention.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Tue Jul 15 18:26:09 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Jul 2008 09:26:09 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com>
	<87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com>
	<874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org>
	<87r69wa8hs.fsf@benfinney.id.au>
	<pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>
Message-ID: <ca471dc20807150926x20e4b0f5n77f9a4af355b3d17@mail.gmail.com>

On Mon, Jul 14, 2008 at 11:55 PM, somebody wrote:
> I'm surprised nobody (that I've noticed) has brought up the point yet that
[...]

Not picking on whoever wrote that specifically, but if there's
anything that surprises me, it's how many people have voiced opinions
already (including many of them that I hadn't heard in this group
before). There doesn't seem to be an end to this debate, and it is
awfully close to deteriorating to pure bikeshedding and attempted
ad-hominem attacks. I really don't have time to participate in detail,
since all the time I have for Python I need to spend on trying to help
review the 2.6 and 3.0 beta releases. But I want to remind people that
radical changes to the unittest infrastructure will inconvenience many
large 3rd party projects using Python, and I urge folks to look for
ways to improve the unittest APIs in other ways instead. It's not the
end of the world if the unittesting API uses assertEqual() instead of
assert_equal() until the end of times. It would, however, be a shame
if we couldn't agree to *add* a bunch of features, for example better
reporting when two lists or long strings differ.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Tue Jul 15 18:40:41 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 15 Jul 2008 12:40:41 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module
In-Reply-To: <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au>	<1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com>	<87abgk9hr2.fsf@benfinney.id.au>
	<1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com>
Message-ID: <g5iju9$69j$1@ger.gmane.org>



Benjamin Peterson wrote:
> On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> "Benjamin Peterson" <musiccomposition at gmail.com> writes:
>>
>>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>>> Use new-style classes throughout
>>>> --------------------------------
>>>>
>>>> The following classes will inherit explicitly from the built-in
>>>> `object` type, to make all classes in the module part of the new-style
>>>> type hierarchy.
>>>>
>>>> * ``TestResult``
>>>> * ``TestCase``
>>>> * ``TestSuite``
>>>> * ``TestLoader``
>>>> * ``_WritelnDecorator``
>>>> * ``TextTestRunner``
>>>> * ``TestProgram``
>>> They already do. __metaclass__ = type is found in unittest.py.
>> Not in the copy I have. Is that in 3.x only, or in 2.x also?
> 
> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import unittest
>>>> isinstance(unittest.TestCase, object)
> True

*Everything* is an instance of object.
The question is whether the test classes are subclasses of object.
In particular, issubclass(unittest.TestCase, object)?
Either direct inheritance from object or '__metaclass__ = type' will 
make them so.  With neither, they will not be.

tjr


From tjreedy at udel.edu  Tue Jul 15 19:09:05 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 15 Jul 2008 13:09:05 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fxqb8mlp.fsf@benfinney.id.au>
	<87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <g5iljh$cmt$1@ger.gmane.org>



Stephen J. Turnbull wrote:

> Yes, for the purposes of this PEP.  We already know that many people
> want various different things.  You want fail* /rather than/ assert*,
> but Steven d'Aprono wants /both/, and I prefer assert* /exclusively/.
> I don't see why we all shouldn't be satisfied[1], so the content of
> unittest should not set policy for our own projects.  So there should
> be other modules (perhaps in the stdlib, perhaps not) to satisfy those
> preferences not catered to by stdlib's unittest.

It should be trivial to write a module 'unitfail' that would 'from 
unittest import *' and then flip the assert names to fail names.  The 
only question is whether it needs to be in the stdlib or merely PyPI.

I word this this way because Guido has already blessed the assert forms 
for unittest.  If he is persuaded otherwise, revise accordingly.

> Thus this PEP should restrict it's concern to revising unittest to
> conform to PEPs and help standardize Python's own testing, without
> trying to impose standards on the whole community of Python users.

For the community as a whole, all stdlib modules are suggestions and 
examples, not commands.

tjr


From tjreedy at udel.edu  Tue Jul 15 19:17:47 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 15 Jul 2008 13:17:47 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <20080715044253.GA31517@caltech.edu>
References: <87mykka8el.fsf@benfinney.id.au>	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>	<487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>	<487BE490.8080406@voidspace.org.uk>	<A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1>	<ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com>	<487C1ABA.4010701@voidspace.org.uk>	<4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1>
	<20080715044253.GA31517@caltech.edu>
Message-ID: <g5im3r$eoa$1@ger.gmane.org>


> http://lists.idyll.org/pipermail/testing-in-python/2007-November/000406.html
> 
> & associated thread, for those interested in the variety of mock
> libraries...

That might be a good beginning for an updateable wiki page on mock 
libraries.


From guido at python.org  Tue Jul 15 19:37:54 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Jul 2008 10:37:54 -0700
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
Message-ID: <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>

On Fri, Jul 11, 2008 at 1:16 PM, Brett Cannon <brett at python.org> wrote:
> No, we should eat our own dog food and transition the code over. If
> anything it will help with code maintenance between 2.x and 3.x.

Agreed. It would be annoying to users trying to clear their own code
of -3 warnings if the stdlib emitted warnings they couldn't easily
suppress.

It should be done with extreme caution though, and preferably on a
case-by-case basis rather than in a sweeping pass. Caution should also
be taken that the changes aren't merged into 3.0 (where presumably
they would conflict with already 3.0-compatible code).

I wonder if it might not be simpler (at least in some cases) to just
disable the warnings for certain modules? I imagine in many cases
fixing up the 2.6 code to suppress -3 warnings would be mere busywork
-- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for
such code there's no need to suppress the -3 warnings, as they serve
as a warning to any user of the module that they are using something
that won't survive into 3.0.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at ctypes.org  Tue Jul 15 19:38:28 2008
From: theller at ctypes.org (Thomas Heller)
Date: Tue, 15 Jul 2008 19:38:28 +0200
Subject: [Python-Dev] ctypes assertion failure
In-Reply-To: <486E40B0.4020608@timgolden.me.uk>
References: <486E40B0.4020608@timgolden.me.uk>
Message-ID: <g5in9o$igt$1@ger.gmane.org>

Tim Golden schrieb:
> This problem was raised on the comtypes-users list
> as it prevents comtypes from being imported on Python 2.6
> at the moment.
> 
> http://bugs.python.org/issue3258
> 
> I'll try to find the time to step through to code to work out
> what's going on, but it's inside the innards of ctypes which
> I've never looked into before.
> 
> Could someone confirm at a glance whether this should be
> given a high priority, please? It results in an assertion error
> in debug mode, a SystemError in non-debug referring to
> a NULL return without an Exception set.
> 

The problem is now fixed.

-- 
Thanks,
Thomas


From musiccomposition at gmail.com  Tue Jul 15 19:41:32 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 15 Jul 2008 12:41:32 -0500
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
	<ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
Message-ID: <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>

On Tue, Jul 15, 2008 at 12:37 PM, Guido van Rossum <guido at python.org> wrote:
>
> I wonder if it might not be simpler (at least in some cases) to just
> disable the warnings for certain modules? I imagine in many cases
> fixing up the 2.6 code to suppress -3 warnings would be mere busywork
> -- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for
> such code there's no need to suppress the -3 warnings, as they serve
> as a warning to any user of the module that they are using something
> that won't survive into 3.0.)

Very true. It might be easiest to just throw (maybe even just temporarily)

if sys.py3kwarning:
    warnings.filterwarnings("ignore", category=DeprecationWarning,
module=__name__)

at the top of the offending module.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From tseaver at palladion.com  Tue Jul 15 19:45:44 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 15 Jul 2008 13:45:44 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <20080715130533.GB17917@steerpike.home.puzzling.org>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
Message-ID: <487CE248.9030400@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Bennetts wrote:
> Ben Finney wrote:
>> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
>>
>>> Ben Finney writes:
>>>
>>>  > Removal of ``assert*`` names
>>>  > ----------------------------
>>>  > 
>>>  > There is no overwhelming consensus on whether to remove the
>>>  > ``assert*`` names or the ``fail*`` names;
>>>
>>> 7 to 1 is overwhelming in my book.  See
>>> Message-ID: <loom.20080714T230912-310 at post.gmane.org>
>>> From: Antoine Pitrou <solipsis at pitrou.net>
>> That measured only usage of unittest *within the Python standard
>> library*. Is that the only body of unittest-using code we need
>> consider?
> 
> Three more data points then:
> 
> bzr: 13228 assert* vs. 770 fail*.
> 
> Twisted: 6149 assert* vs. 1666 fail*.
> 
> paramiko: 431 assert* vs. 4 fail*.
> 
> The data seems pretty overwhelmingly in favour of keeping assert*. 

FWIW:  Zope2:  16878 'assert*', 2892 'fail*'.  I would keep both by
preference, rather than insist on a "foolish consistency."


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIfOJI+gerLs4ltQ4RArw5AJ0fUzXiMefC4CQZDk4K0mlQFn3nAACfRLya
CuhfeNQd8OUPFi5+E5aJN9M=
=2Hsl
-----END PGP SIGNATURE-----


From brett at python.org  Tue Jul 15 20:38:27 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Jul 2008 11:38:27 -0700
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
Message-ID: <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com>

On Tue, Jul 15, 2008 at 5:32 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> A reminder: the second betas of Python 2.6 and 3.0 are schedule for
> tomorrow.  I will try to hang out on #python-dev today and will start
> looking at the trackers and buildbots.  Hopefully, we're on track to get the
> releases out!
>
> If there is anything you need a decision on, please follow up to this
> thread.  I'm inundated with email so I can't watch every thread on the
> mailing lists.  Or ping me on #python-dev.
>

The new urllib package landed between the last beta and now, but
without fixers. Issue3316 has potential ones, but they have some
tweaks that need to be made before they are release quality. If the
beta goes out 2to3 will not be able to fix imports for urllib and
urllib2. Don't know if that is enough to hold up the release or just
something to put in the release notes that this will be resolved by
the next beta; made it a release blocker to catch your eye, Barry. =)

Oh, and fixers for test.test_support to test.support is not in either,
but this is probably such a small use case outside the core that I am
not sweating bullets for writing a new fixer just for this one case.

-Brett

From mike.klaas at gmail.com  Tue Jul 15 21:16:22 2008
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 15 Jul 2008 12:16:22 -0700
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
In-Reply-To: <20080715130533.GB17917@steerpike.home.puzzling.org>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
Message-ID: <AD17D4F4-6CDD-47DF-BD6E-8F88D6E96086@gmail.com>

On 15-Jul-08, at 6:05 AM, Andrew Bennetts wrote:

> Ben Finney wrote:
>> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
>>
>> That measured only usage of unittest *within the Python standard
>> library*. Is that the only body of unittest-using code we need
>> consider?
>
> Three more data points then:
>
> bzr: 13228 assert* vs. 770 fail*.
>
> Twisted: 6149 assert* vs. 1666 fail*.
>
> paramiko: 431 assert* vs. 4 fail*.

Our internal code base:

$ ack self.assert. | wc -l
3232
$ ack self.fail. | wc -l
1124

-Mike


From nas at arctrix.com  Tue Jul 15 22:04:11 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 15 Jul 2008 14:04:11 -0600
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
Message-ID: <20080715200411.GA26998@arctrix.com>

On Mon, Jul 14, 2008 at 09:31:47PM -0400, Barry Warsaw wrote:
> Neil, we should try to host them on code.python.org.

I was hoping to get a sense of the interest.  Oh well, if you build
it they might come. ;-) I've written draft instructions, temporarily
at <http://python.ca/nas/tmp/git-notes.txt>.  The trunk and py3k
repositories are now avaliable at git://code.python.org.  They
currently get updated every 30 minutes.

I'm definitely not a git guru so I hope the instructions work for
people.  The setup is a little complicated because I want updates to
use the more efficient git protocol rather than hammering the SVN
server.  I've tried making a check-in using dcommit and everything
seems okay.  I'd gladly receive suggestions on improving the
instructions.

  Neil

From musiccomposition at gmail.com  Tue Jul 15 22:26:24 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 15 Jul 2008 15:26:24 -0500
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <20080715200411.GA26998@arctrix.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
Message-ID: <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>

On Tue, Jul 15, 2008 at 3:04 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> On Mon, Jul 14, 2008 at 09:31:47PM -0400, Barry Warsaw wrote:
>> Neil, we should try to host them on code.python.org.
>
> I was hoping to get a sense of the interest.  Oh well, if you build
> it they might come. ;-) I've written draft instructions, temporarily
> at <http://python.ca/nas/tmp/git-notes.txt>.  The trunk and py3k
> repositories are now avaliable at git://code.python.org.  They
> currently get updated every 30 minutes.

Can we push branches?



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From brett at python.org  Tue Jul 15 23:01:14 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Jul 2008 14:01:14 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <20080715200411.GA26998@arctrix.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
Message-ID: <bbaeab100807151401q1439df32re0996716505c1193@mail.gmail.com>

On Tue, Jul 15, 2008 at 1:04 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> On Mon, Jul 14, 2008 at 09:31:47PM -0400, Barry Warsaw wrote:
>> Neil, we should try to host them on code.python.org.
>
> I was hoping to get a sense of the interest.  Oh well, if you build
> it they might come. ;-) I've written draft instructions, temporarily
> at <http://python.ca/nas/tmp/git-notes.txt>.  The trunk and py3k
> repositories are now avaliable at git://code.python.org.  They
> currently get updated every 30 minutes.
>
> I'm definitely not a git guru so I hope the instructions work for
> people.  The setup is a little complicated because I want updates to
> use the more efficient git protocol rather than hammering the SVN
> server.  I've tried making a check-in using dcommit and everything
> seems okay.  I'd gladly receive suggestions on improving the
> instructions.
>

If you are up to keep this up and running, Neil, I (or anyone else
with pydotorg access) can put your instructions online like the bzr
instructions at http://www.python.org/dev/bazaar/ .

-Brett

From stephen at xemacs.org  Tue Jul 15 23:31:51 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Jul 2008 06:31:51 +0900
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <g5iljh$cmt$1@ger.gmane.org>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>
	<g5iljh$cmt$1@ger.gmane.org>
Message-ID: <87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp>

Terry Reedy writes:

 > For the community as a whole, all stdlib modules are suggestions and 
 > examples, not commands.

Well, even if "standard" is too strong a word, the DeprecationWarnings
and eventual removal of the methods surely constitute an imposition.

From nas at arctrix.com  Tue Jul 15 23:31:15 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 15 Jul 2008 21:31:15 +0000 (UTC)
Subject: [Python-Dev] git repositories for trunk and py3k
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
Message-ID: <g5j4v3$6hi$1@ger.gmane.org>

Benjamin Peterson <musiccomposition at gmail.com> wrote:
> Can we push branches?

The git-daemon is setup as read-only.  If you have write access to
the SVN repository then you can push back changes using git-svn.
That's quite a nice way to work, IMHO and provides an easy path for
people who are used to Subversion and want to dip their toe in the
dvcs waters.

While there is no support on code.python.org for publishing your own
git branches, there's no reason why you couldn't publish a branch
using some other host (it's a dvcs after all).  In the short term,
perhaps using private branches in combination with "git
format-patch" and email is the easiest process.

Regards,

  Neil


From solipsis at pitrou.net  Tue Jul 15 23:52:35 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Jul 2008 23:52:35 +0200
Subject: [Python-Dev] [Python-3000] commit access request
In-Reply-To: <bbaeab100807151357p75e46a0fi4e05663368d74cb5@mail.gmail.com>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
	<loom.20080715T134107-337@post.gmane.org>
	<1afaf6160807150647n3fa5d6c2lbaf0db42bad76fd@mail.gmail.com>
	<bbaeab100807151134l1ed88favcd2889571cfdd405@mail.gmail.com>
	<loom.20080715T200625-20@post.gmane.org>
	<bbaeab100807151357p75e46a0fi4e05663368d74cb5@mail.gmail.com>
Message-ID: <1216158755.6020.29.camel@fsol>

Le mardi 15 juillet 2008 ? 13:57 -0700, Brett Cannon a ?crit :
> I can't give the privileges, Antoine, but I know that an SSH 2 public
> key is going to be needed (see the dev FAQ at
> http://www.python.org/dev/faq/ if you don't know how to generate one).

It's attached.

> They will also need to know how you want your first.last name spelled
> for your user account.

Please make that antoine.pitrou.

> Just send that all in a separate email to python-dev and someone who
> can add you will get to it at some point (probably soon).

Thanks !

Antoine.

-------------- next part --------------
ssh-dss AAAAB3NzaC1kc3MAAACBAJiwbeZEP4oPxn/mpbcjPIfmyqDRoyut/AAE76coOCKB3BouwYJPns4Osas6NjR3m9TErNrvcFcm2jCXYTxbrIQmnz1oLwxzmjPK36AUpXtqOy5GPdFUJiVe7iqEBslwtaPVGweXrYdv7ZaI7G6m1ZJDh03uqP6HjNIA1y3yeJFDAAAAFQCxc0R9pwTFUnB4HeHJnx0xE1qg3wAAAIAyxftjIkEAghCBeqbc7W5GRHt2AQ6nczrMCn76pc25+2SkbK750Zk5dAZtIiGkvwNBH2sDdKUzccWkKN7LcyW8ChPLLM1VNVHfgwqQEDbopjd7FkLsOwuQGSxpPoZpTPmewmfJxBJN3kNE5MRVYpzihmqCCoNRi4peMBonuevm4QAAAIEAgCRFJydgXmtk3MiGkX2MyU0DAoAmRzUtJtstYLY0ZqdUvd/0fD4JkpIyZu1N7ROYwzsVCAoU/H6gNGuUFY/NcvfKBUBPZ/yYHECTmstJIWY2X1yVSx59s4lJBZzVUgrP++KTLS+F7n6bil5rsmAaUqg/MoxoMJVLhkvdLvyVPZY= antoine.pitrou

From stephen at xemacs.org  Wed Jul 16 00:20:40 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Jul 2008 07:20:40 +0900
Subject: [Python-Dev]  PEP: Consolidating names in the `unittest` module
In-Reply-To: <87skub6yhh.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
Message-ID: <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:

 > Removal of ``assert*`` names
 > ----------------------------

 > Arguments in favour of retaining only the ``assert*`` names:
 > 
 > * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference
 >   for the ``assert*`` names.
 > 
 > * Precedent: The Python standard library currently uses the
 >   ``assert*`` names by a roughly 8:1 majority over the ``fail*``
 >   names. (Counting unit tests in the py3k tree at 2008-07-15
 >   [#pitrou-1]_.)
 > 
 >   An ad-hoc sampling of other projects that use `unittest` also
 >   demonstrates strong preference for use of the ``assert*`` names
 >   [#bennetts-1]_.
 > 
 > * Positive admonition: The ``assert*`` names state the intent of how
 >   the code under test *should* behave, while the ``fail*`` names are
 >   phrased in terms of how the code *should not* behave.

FWIW, I think these are fairly stated.  So fairly that I'm surprised
you haven't been persuaded!<wink>  Nitpick: the second point is not
just "precedent", there's an economic reason there too.  Tests in the
standard distribution which use the deprecated style will need to be
converted.  Steven d'Aprano claims this is nontrivial (and thus error-
prone) in some cases.  I haven't seen that claim denied, and it seems
plausible to me.

 > PEP 8 names
 > -----------
 > 
 > Although `unittest` (and its predecessor `PyUnit`) are intended to be
 > familiar to users of other xUnit interfaces, there is no attempt at
 > direct API compatibility since the only code that Python's `unittest`
 > interfaces with is other Python code. The module is in the standard
 > library and its names should all conform with PEP 8 [#PEP-8]_.

You should mention Raymond's concern that PEP 8 names are quite a bit
longer.


From guido at python.org  Wed Jul 16 00:13:22 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Jul 2008 15:13:22 -0700
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <ca471dc20807151513i5fede9ccjeaac48a50a05f6b6@mail.gmail.com>

On Tue, Jul 15, 2008 at 3:20 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>  > * Positive admonition: The ``assert*`` names state the intent of how
>  >   the code under test *should* behave, while the ``fail*`` names are
>  >   phrased in terms of how the code *should not* behave.
>
> FWIW, I think these are fairly stated.  So fairly that I'm surprised
> you haven't been persuaded!<wink>  Nitpick: the second point is not
> just "precedent", there's an economic reason there too.  Tests in the
> standard distribution which use the deprecated style will need to be
> converted.  Steven d'Aprano claims this is nontrivial (and thus error-
> prone) in some cases.  I haven't seen that claim denied, and it seems
> plausible to me.

I'd like to see examples of that (this would be Steven's task if he's
serious about his assertion). Since the fail and assert names are
mapped to each other using aliasing I don't see how it could be
nontrivial to map e.g. self.failIf(x) to self.assertFalse(x) -- these
are the same function!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Wed Jul 16 00:23:11 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 15 Jul 2008 18:23:11 -0400
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>	<g57ll0$kqn$1@ger.gmane.org>	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>	<ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
	<1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>
Message-ID: <g5j80e$g4d$1@ger.gmane.org>



Benjamin Peterson wrote:
> On Tue, Jul 15, 2008 at 12:37 PM, Guido van Rossum <guido at python.org> wrote:
>> I wonder if it might not be simpler (at least in some cases) to just
>> disable the warnings for certain modules? I imagine in many cases
>> fixing up the 2.6 code to suppress -3 warnings would be mere busywork
>> -- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for
>> such code there's no need to suppress the -3 warnings, as they serve
>> as a warning to any user of the module that they are using something
>> that won't survive into 3.0.)
> 
> Very true. It might be easiest to just throw (maybe even just temporarily)
> 
> if sys.py3kwarning:
>     warnings.filterwarnings("ignore", category=DeprecationWarning,
> module=__name__)
> 
> at the top of the offending module.

Is it possible to *first* 'raise' a DeprecationWarning("This module will 
disappear") before turning the rest off?  Exactly 1 seems the right 
number to me for modules that will go.


From ben+python at benfinney.id.au  Wed Jul 16 00:34:00 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 08:34:00 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<87y7436yxz.fsf@benfinney.id.au>
	<1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com>
	<20080715142335.GG2873@caltech.edu>
Message-ID: <87d4le7p5z.fsf@benfinney.id.au>

"C. Titus Brown" <ctb at msu.edu> writes:

> On Tue, Jul 15, 2008 at 03:00:30PM +0100, Richard Thomas wrote:
> -> I've been told by a couple of non-programmers that "failUnless"
> -> is more intuitive than "assert" if only for the reason that its
> -> unclear what "assert" might do. This is similar to one of the
> -> arguments raised in the PEP, but considered from the point of
> -> view of someone new to test frameworks it could be all the more
> -> important.
> 
> Maybe this is an unnecessarily hard line, but if someone doesn't know
> what "assert" does, then they should go look up the keyword -- it's
> part of Python (and present in C, C++, Perl, and PHP as well).  So
> unless the person is new to Python AND testing, they should know it.

That's exactly the problem with the 'assert*' names: The test methods
of TestCase *don't* do the same thing as the Python 'assert'
statement, and aren't meant to. The association is confusing, even
(especially?) if one knows what the 'assert' statement does.

> While I don't have a strong position on the assert* vs fail*, I
> think consistency and simplicity in test suites is at least as
> important as in code: if you have to work hard to understand and
> debug your test suites, you've done something seriously wrong in
> building your tests.

Yes. While I prefer the 'fail*' names, I would prefer to lose *either*
of 'assert*' or 'fail*' than to retain redundant names in the API.

-- 
 \     ?If I had known what it would be like to have it all... I might |
  `\     have been willing to settle for less.? ?Jane Wagner, via Lily |
_o__)                                                           Tomlin |
Ben Finney


From guido at python.org  Wed Jul 16 00:37:37 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Jul 2008 15:37:37 -0700
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
In-Reply-To: <87d4le7p5z.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<87y7436yxz.fsf@benfinney.id.au>
	<1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com>
	<20080715142335.GG2873@caltech.edu> <87d4le7p5z.fsf@benfinney.id.au>
Message-ID: <ca471dc20807151537n2f639ef8t7e5e6ae8b4bb7445@mail.gmail.com>

On Tue, Jul 15, 2008 at 3:34 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> That's exactly the problem with the 'assert*' names: The test methods
> of TestCase *don't* do the same thing as the Python 'assert'
> statement, and aren't meant to. The association is confusing, even
> (especially?) if one knows what the 'assert' statement does.

The parallel is intentional. They even raise the same exception. Too
bad it doesn't work for you; get over it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Wed Jul 16 00:38:58 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Jul 2008 15:38:58 -0700
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <g5j80e$g4d$1@ger.gmane.org>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
	<ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
	<1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>
	<g5j80e$g4d$1@ger.gmane.org>
Message-ID: <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com>

On Tue, Jul 15, 2008 at 3:23 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>
> Benjamin Peterson wrote:
>>
>> On Tue, Jul 15, 2008 at 12:37 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>>
>>> I wonder if it might not be simpler (at least in some cases) to just
>>> disable the warnings for certain modules? I imagine in many cases
>>> fixing up the 2.6 code to suppress -3 warnings would be mere busywork
>>> -- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for
>>> such code there's no need to suppress the -3 warnings, as they serve
>>> as a warning to any user of the module that they are using something
>>> that won't survive into 3.0.)
>>
>> Very true. It might be easiest to just throw (maybe even just temporarily)
>>
>> if sys.py3kwarning:
>>    warnings.filterwarnings("ignore", category=DeprecationWarning,
>> module=__name__)
>>
>> at the top of the offending module.
>
> Is it possible to *first* 'raise' a DeprecationWarning("This module will
> disappear") before turning the rest off?  Exactly 1 seems the right number
> to me for modules that will go.
>

Wouldn't you just add the filter after the DeprecationWarning is
raised to have that behavior?

-Brett

From ben+python at benfinney.id.au  Wed Jul 16 00:36:56 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 08:36:56 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>
	<g5iljh$cmt$1@ger.gmane.org>
	<87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <878ww27p13.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:

> Terry Reedy writes:
> 
>  > For the community as a whole, all stdlib modules are suggestions
>  > and examples, not commands.
> 
> Well, even if "standard" is too strong a word, the DeprecationWarnings
> and eventual removal of the methods surely constitute an imposition.

I understood Terry's statement as meaning that there is no commandment
to use the standard library 'unittest' module at all. If a user
doesn't like the behaviour of a module (such as 'unittest') in the
standard library, they can accept the burden of implementing one that
behaves the way they like.

-- 
 \       ?I never forget a face, but in your case I'll be glad to make |
  `\                                      an exception.? ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 00:41:47 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 08:41:47 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
	<487CB144.1080200@gmail.com>
Message-ID: <87zloi6a8k.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> A somewhat odd thought that occurred to me is that the shortest
> possible way of writing negative assertions (i.e. asserting that
> something is not the case) is to treat them as denials and use the
> single word 'deny'.

-1

This, to me, is neither intuitive nor meaningful in context. The term
"deny" is strongly linked to its antonym, "permit".

Whom is being denied? What have they asked to do that I am denying in
my test? I think in terms of "true or false", or "pass or fail". I'm
making statements about behaviour of the program, not about permitting
or denying something.

-- 
 \        ?The industrial system is profoundly dependent on commercial |
  `\   television and could not exist in its present form without it.? |
_o__)        ?John Kenneth Galbraith, _The New Industrial State_, 1967 |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 00:38:59 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 08:38:59 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<487CE248.9030400@palladion.com>
Message-ID: <874p6q7oxo.fsf@benfinney.id.au>

Tres Seaver <tseaver at palladion.com> writes:

> FWIW:  Zope2:  16878 'assert*', 2892 'fail*'.

Thanks.

> I would keep both by preference, rather than insist on a "foolish
> consistency."

The "consistency" argument leads to the PEP 8 names. The removal of
redundant names is not made in the name of consistency, but of API
simplicity.

-- 
 \     ?Under democracy one party always devotes its chief energies to |
  `\   trying to prove that the other party is unfit to rule--and both |
_o__)              commonly succeed, and are right.? ?Henry L. Mencken |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 00:44:47 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 08:44:47 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
	<487A8F7B.1090901@holdenweb.com>
	<87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com>
	<874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org>
	<87r69wa8hs.fsf@benfinney.id.au>
	<pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de>
	<ca471dc20807150926x20e4b0f5n77f9a4af355b3d17@mail.gmail.com>
Message-ID: <87vdz66a3k.fsf@benfinney.id.au>

"Guido van Rossum" <guido at python.org> writes:

> It would, however, be a shame if we couldn't agree to *add* a bunch
> of features, for example better reporting when two lists or long
> strings differ.

I intend to phrase such additions in terms of PEP-8-only names, so
this name consolidation seems a natural prerequisite.

-- 
 \           ?Are you pondering what I'm pondering?? ?Well, I think so |
  `\      Brain, but what if we stick to the seat covers?? ?_Pinky and |
_o__)                                                       The Brain_ |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 00:59:08 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 08:59:08 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <87r69u69fn.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:

> FWIW, I think these are fairly stated.  So fairly that I'm surprised
> you haven't been persuaded!<wink>

I'm not persuaded because I find the arguments for 'fail*' more
persuasive :-)

I am, however, convinced that the consensus of the community is that
'assert*' is the right choice. The PEP will be revised accordingly.

> Nitpick: the second point is not just "precedent", there's an
> economic reason there too.
[...]

> You should mention Raymond's concern that PEP 8 names are quite a
> bit longer.

Noted both of these, thanks.

-- 
 \          ?The best way to get information on Usenet is not to ask a |
  `\               question, but to post the wrong information.? ?Aahz |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 01:40:05 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 09:40:05 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
	(updated 2008-07-16)
References: <87vdz8a983.fsf@benfinney.id.au>
Message-ID: <87k5fm67je.fsf@benfinney.id.au>

Significant changes are resolving to remove the 'fail*' names, giving
recommended name to use for each removed name, and further summary of
related discussion.


:PEP:               XXX
:Title:             Consolidating names in the `unittest` module
:Version:           0.3
:Last-Modified:     2008-07-16
:Author:            Ben Finney <ben+python at benfinney.id.au>
:Status:            Draft
:Type:              Standards Track
:Content-Type:      test/x-rst
:Created:           2008-07-14
:Python-Version:    2.7, 3.1
:Post-History:


..  contents::


Abstract
========

This PEP proposes to consolidate the names that constitute the API of
the standard library `unittest` module, with the goal of removing
redundant names, and conforming with PEP 8.


Motivation
==========

The normal use case for the `unittest` module is to subclass its
classes, overriding and re-using its functios and methods. This draws
constant attention to the fact that the existing implementation fails
several current Python standards:

* It does not conform to PEP 8 [#PEP-8]_, requiring users to write
  their own non-PEP-8 conformant names when overriding methods, and
  encouraging extensions to further depart from PEP 8.

* It has many synonyms in its API, which goes against the Zen of
  Python [#PEP-20]_ (specifically, that "there should be one, and
  preferably only one, obvious way to do it").


Specification
=============

Remove obsolete names
---------------------

The following module attributes are not documented as part of the API
and are marked as obsolete in the implementation. They will be
removed.

* ``_makeLoader``
* ``getTestCaseNames``
* ``makeSuite``
* ``findTestCases``

Remove redundant names
----------------------

The following attribute names exist only as synonyms for other names.
They are to be removed, leaving only one name for each attribute in
the API.

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

``assert_``
  Use ``assert_true``, or an ``assert`` statement.

``assertEquals``
  Use ``assert_equal``.

``assertNotEquals``
  Use ``assert_not_equal``.

``assertAlmostEquals``
  Use ``assert_almost_equal``.

``assertNotAlmostEquals``
  Use ``assert_not_almost_equal``.

``failIf``
  Use ``assert_false``.

``failUnless``
  Use ``assert_true``.

``failIfAlmostEqual``
  Use ``assert_not_almost_equal``.

``failIfEqual``
  Use ``assert_not_equal``.

``failUnlessAlmostEqual``
  Use ``assert_almost_equal``.
  
``failUnlessEqual``
  Use ``assert_equal``.

``failUnlessRaises``
  Use ``assert_raises``.

Conform API with PEP 8
----------------------

The following names are to be introduced, each replacing an existing
name, to make all names in the module conform with PEP 8 [#PEP-8]_.
Each name is shown with the existing name that it replaces.

Where function parameters are to be renamed also, they are shown.
Where function parameters are not to be renamed, they are elided with
the ellipse ("?") symbol.

Module attributes
~~~~~~~~~~~~~~~~~

``default_test_loader``
  Replaces ``defaultTestLoader``

``TestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``add_error(?)``
  Replaces ``addError(?)``

``add_result(?)``
  Replaces ``addResult(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``should_stop``
  Replaces ``shouldStop``

``start_test(?)``
  Replaces ``startTest(?)``

``stop_test(?)``
  Replaces ``stopTest(?)``

``tests_run``
  Replaces ``testsRun``

``was_successful(?)``
  Replaces ``wasSuccessful(?)``

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, method_name='run_test')``
  Replaces ``__init__(self, methodName='runTest')``

``_test_method_doc``
  Replaces ``_testMethodDoc``

``_test_method_name``
  Replaces ``_testMethodName``

``failure_exception``
  Replaces ``failureException``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``default_test_result(?)``
  Replaces ``defaultTestResult(?)``

``assert_true(?)``
  Replaces ``assertTrue(?)``

``assert_false(?)``
  Replaces ``assertFalse(?)``

``assert_almost_equal(?)``
  Replaces ``assertAlmostEqual(?)``

``assert_equal(?)``
  Replaces ``assertEqual(?)``

``assert_not_almost_equal(?)``
  Replaces ``assertNotAlmostEqual(?)``

``assert_not_equal(?)``
  Replaces ``assertNotEqual(?)``

``assert_raises(exc_class, callable_obj, *args, **kwargs)``
  Replaces ``assertRaises(excClass, callableObj, *args, **kwargs)``

``run_test(?)``
  Replaces ``runTest(?)``

``setup(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``teardown(?)``
  Replaces ``tearDown(?)``

``FunctionTestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, test_func, set_up, tear_down, description)``
  Replaces ``__init__(self, testFunc, setUp, tearDown, description)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``TestSuite`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~

``add_test(?)``
  Replaces ``addTest(?)``

``add_tests(?)``
  Replaces ``addTests(?)``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``TestLoader`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``sort_test_methods_using``
  Replaces ``sortTestMethodsUsing``

``suite_class``
  Replaces ``suiteClass``

``test_method_prefix``
  Replaces ``testMethodPrefix``

``get_test_case_names(self, test_case_class)``
  Replaces ``getTestCaseNames(self, testCaseClass)``

``load_tests_from_module(?)``
  Replaces ``loadTestsFromModule(?)``

``load_tests_from_name(?)``
  Replaces ``loadTestsFromName(?)``

``load_tests_from_names(?)``
  Replaces ``loadTestsFromNames(?)``

``load_tests_from_test_case(self, test_case_class)``
  Replaces ``loadTestsFromTestCase(self, testCaseClass)``

``_TextTestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``show_all``
  Replaces ``showAll``

``add_error(?)``
  Replaces ``addError(?)``

``add_failure(?)``
  Replaces ``addFailure(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``get_description(?)``
  Replaces ``getDescription(?)``

``print_error_list(?)``
  Replaces ``printErrorList(?)``

``print_errors(?)``
  Replaces ``printErrors(?)``

``start_test(?)``
  Replaces ``startTest(?)``

``TextTestRunner`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``_make_result(?)``
  Replaces ``_makeResult(?)``

``TestProgram`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, module, default_test, argv, test_runner, test_loader)``
  Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)``

``create_tests(?)``
  Replaces ``createTests(?)``

``parse_args(?)``
  Replaces ``parseArgs(?)``

``run_tests(?)``
  Replaces ``runTests(?)``

``usage_exit(?)``
  Replaces ``usageExit(?)``


Rationale
=========

Redundant names
---------------

The current API, with two or in some cases three different names
referencing exactly the same function, leads to an overbroad and
redundant API that violates PEP 20 [#PEP-20]_ ("there should be one,
and preferably only one, obvious way to do it").

Removal of ``fail*`` names
--------------------------

While there is consensus support to `remove redundant names`_ for the
``TestCase`` test methods, the issue of which set of names should be
retained is controversial (it was several times characterised as
"bike-shedding" [#bikeshed]_ by the BDFL and others).

With good arguments made in favour of each of "retain ``assert*``" and
"retain ``fail*``", this PEP resolves in favour of stated BDFL
preference, and de facto community preference.

Arguments in favour of retaining only the ``assert*`` names:

* BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference
  for the ``assert*`` names.

* Precedent: The Python standard library currently uses the
  ``assert*`` names by a roughly 8:1 majority over the ``fail*``
  names. (Counting unit tests in the py3k tree at 2008-07-15
  [#pitrou-1]_.)

  An ad-hoc sampling of other projects that use `unittest` also
  demonstrates strong preference for use of the ``assert*`` names
  [#bennetts-1]_, [#seaver-1]_.

* Economic: The above precedent indicates a simple economic argument
  [#turnbull-1]_: the majority of tests will not need their statements
  re-phrased, so updating in favour of them will involve less
  disruption and risk of error.

* Positive admonition: The ``assert*`` names state the intent of how
  the code under test *should* behave, while the ``fail*`` names are
  phrased in terms of how the code *should not* behave.

Arguments in favour of retaining only the ``fail*`` names:

* Explicit is better than implicit: The ``fail*`` names state *what
  the function will do* explicitly: fail the test. With the
  ``assert*`` names, the action to be taken is only implicit.

* Avoid false implication: The test methods do not have any necessary
  connection with the built-in ``assert`` statement. Even the
  exception raised, while it defaults to ``AssertionException``, is
  explicitly customisable via the documented ``failure_exception``
  attribute. Choosing the ``fail*`` names avoids the false association
  with either of these.

  It has been argued [#thomas-1]_ that newcomers to `unittest` who
  already know what ``assert`` does can be confused by the behaviour
  of the ``assert*`` names. This confusion does not exist for the
  ``fail*`` names, which don't conflict with existing concepts.

* Avoid name collision: The above confusion with ``assert`` is
  exacerbated by the plain-boolean test using a name of ``assert_``
  (with a trailing underscore) to avoid a name collision with the
  built-in ``assert`` statement. The corresponding ``fail_if`` name
  has no such issue.

PEP 8 names
-----------

Although `unittest` (and its predecessor `PyUnit`) are intended to be
familiar to users of other xUnit interfaces, there is no attempt at
direct API compatibility since the only code that Python's `unittest`
interfaces with is other Python code. The module is in the standard
library and its names should all conform with PEP 8 [#PEP-8]_.

While argument was raised [#hettinger-1]_ over the length of names
resulting from a simple conversion of names to
multi_word_with_underscores, this PEP chooses names that are
consistent in wording with the existing names.

An exception was made in the case of ``setup`` and ``teardown``. The
two-word form of these names was judged [#hettinger-1]_ too cumbersome
compared to the new single-word form.
 

Backwards Compatibility
=======================

The names to be obsoleted should be deprecated and removed according
to the schedule for modules in PEP 4 [#PEP-4]_.

While deprecated, use of the deprecated attributes should raise a
``DeprecationWarning``, with a message stating which replacement name
should be used.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..  [#PEP-4] http://www.python.org/dev/peps/pep-0004
..  [#PEP-8] http://www.python.org/dev/peps/pep-0008
..  [#PEP-20] http://www.python.org/dev/peps/pep-0020
..  [#bikeshed] http://www.catb.org/~esr/jargon/html/B/bikeshedding.html
..  [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-April/078485.html
..  [#pitrou-1] http://mail.python.org/pipermail/python-dev/2008-July/081090.html
..  [#bennetts-1] http://mail.python.org/pipermail/python-dev/2008-July/081141.html
..  [#seaver-1] http://mail.python.org/pipermail/python-dev/2008-July/081166.html
..  [#thomas-1] http://mail.python.org/pipermail/python-dev/2008-July/081153.html
..  [#hettinger-1] http://mail.python.org/pipermail/python-dev/2008-July/081107.html
..  [#turnbull-1] http://mail.python.org/pipermail/python-dev/2008-July/081175.html


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From barry at python.org  Wed Jul 16 01:42:17 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Jul 2008 19:42:17 -0400
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
	<bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com>
Message-ID: <9CD62042-4AA1-421E-BB63-129DEC30175A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I realized that the previous schedule really called for a release  
today 7/15 because I'm a bit busy tomorrow night.  In any event, let's  
try to stick to doing it on 7/16.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSH012XEjvBPtnXfVAQLJiQQAiP+/S1KVuv9ZgzyS7XdQNW2USTcBGkCh
LIQs3Z+ueuZ9MORdwuDnuZmJWpxataYe26UiPKxt8BFJO81x1i7hy4/uvrO4aE+z
doksz9eJtclNWQNAcvgRX2bKM0OBO57egJrqxYFUgqlziELxOT/U2//josdl3jR1
ovOb/C0nWVU=
=LBea
-----END PGP SIGNATURE-----

From barry at python.org  Wed Jul 16 01:45:42 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Jul 2008 19:45:42 -0400
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
	<bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com>
Message-ID: <AD5D714F-3513-4BF5-A72C-2ADF6C62181E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 15, 2008, at 2:38 PM, Brett Cannon wrote:

> On Tue, Jul 15, 2008 at 5:32 AM, Barry Warsaw <barry at python.org>  
> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> A reminder: the second betas of Python 2.6 and 3.0 are schedule for
>> tomorrow.  I will try to hang out on #python-dev today and will start
>> looking at the trackers and buildbots.  Hopefully, we're on track  
>> to get the
>> releases out!
>>
>> If there is anything you need a decision on, please follow up to this
>> thread.  I'm inundated with email so I can't watch every thread on  
>> the
>> mailing lists.  Or ping me on #python-dev.
>>
>
> The new urllib package landed between the last beta and now, but
> without fixers. Issue3316 has potential ones, but they have some
> tweaks that need to be made before they are release quality. If the
> beta goes out 2to3 will not be able to fix imports for urllib and
> urllib2. Don't know if that is enough to hold up the release or just
> something to put in the release notes that this will be resolved by
> the next beta; made it a release blocker to catch your eye, Barry. =)

I knocked this down to a deferred blocker, so I won't let it hold up  
beta2, though I'd /really/ like to get this cleaned up and committed  
in time.  If I get enough deferred blockers though, I might hold  
things up after all.

> Oh, and fixers for test.test_support to test.support is not in either,
> but this is probably such a small use case outside the core that I am
> not sweating bullets for writing a new fixer just for this one case.

Ok.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSH02pnEjvBPtnXfVAQL1qgP/a2yf6fagOS7Hvbwd08ZhYkaB+GDTcFMy
mo/bJtvrdCRwPm+60q5mZYuMBIz/I7IBSQmSI09IOdEi0RPoXH6Los3LMrt+Aa6v
XgPu7nYcO1vNa/b+6vNvsBAEPfEuaMJsSRpHNJfAWsjF82BsiNEpJ0D5906CAhyW
MrjaPUb47bs=
=/qoQ
-----END PGP SIGNATURE-----

From greg.ewing at canterbury.ac.nz  Wed Jul 16 01:46:02 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Jul 2008 11:46:02 +1200
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest`module (updated 2008-07-15)
In-Reply-To: <d06a5cd30807141918w3d574c36lb9c84435e014fe36@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<d06a5cd30807141918w3d574c36lb9c84435e014fe36@mail.gmail.com>
Message-ID: <487D36BA.7060807@canterbury.ac.nz>

Jonathan Lange wrote:

> My name's Jonathan, and I spell "set up" as "set up" and "tear down"
> as "tear down".

In English, it depends on how they're being used. As
nouns they're single words, as verbs they're two
words.

As function names, they could be read either way, so
it comes down to readability. To my eyes there is no
loss of readability when omitting the underscores,
so simplicity argues for leaving them out.

-- 
Greg

From steve at pearwood.info  Wed Jul 16 01:54:56 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 16 Jul 2008 09:54:56 +1000
Subject: [Python-Dev]
	=?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?=
	=?iso-8859-1?q?asserts=09vs=2E=09failIf/Unlesses?=
In-Reply-To: <8763r7sg22.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<200807150940.44919.steve@pearwood.info>
	<8763r7sg22.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <200807160954.56875.steve@pearwood.info>

On Tue, 15 Jul 2008 06:32:53 pm Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
>  > On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote:
>  > > FWIW, I meant 10 != not not 10.
>  > >
>  > >>> 10 != not not 10
>  >
>  >   File "<stdin>", line 1
>  >     10 != not not 10
>  >             ^
>  > SyntaxError: invalid syntax
>  >
>  > With respect, I think that the fact that you made an analogy with
>  > Python code that you hadn't tested, got it wrong, then corrected
>  > it, and *still* got it wrong, is telling.
>
> It's telling only if you ignore the fact that it's the ad hominem
> fallacy.

The ad hominem fallacy does not mean "somebody pointed out I made a 
silly mistake". It is not a fallacy to reject a person's argument 
because it is hastily or carelessly made.

However, fair is fair -- I too have to accept a slice of humble pie. In 
*my* haste to make my point, I neglected to actually make the point I 
intended to. I went on to write "It's part of the pattern of this 
thread" but didn't explain why. Namely that with so many people trying 
to push their personal preference, and signs of frustration on both 
sides, people are making unsupported statements of opinion as if they 
were facts, and generally getting sloppy. Ironic, yes?

Having communicated poorly, and by doing so giving you offence, I will 
accept an appropriate fish-slapping.


>  > When it comes to assert* versus fail* tests, this is one case
>  > where I don't believe "one obvious way to do it" should apply.
>
> In the context you were talking about, where you don't need to show
> anybody your tests, I don't see any reason why TOOWTDI should apply.
> In a debugging context, I don't see any reason why it should, either.
>
> But here we're talking about the standard unit-testing framework,
> which is used by Python itself.

But I'm using that same framework, so decisions made regarding that 
framework are going to impact me. Terry Reedy says that "stdlib modules 
are suggestions and examples, not commands", which is strictly true but 
I think he discounts just how strong the stdlib's influence is. Perhaps 
large projects, and tiny ones, are free to go their own way, but in my 
experience (such that it is), small projects tend to follow the stdlib.


> So it *is* about communication with 
> other developers, and it *does* make sense to avoid proliferation of
> redundant vocabulary.

Certainly redundant vocabulary has a cost, but it also has a benefit. At 
least for those people (including myself) who frequently think of a 
unit test as a sequence of potential failures rather than expected 
successes. Only if the code "fails to fail" does it pass the test, and 
so I often find fail* tests more suitable. YMMV.



-- 
Steven

From solipsis at pitrou.net  Tue Jul 15 19:55:00 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Jul 2008 19:55:00 +0200
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <87y7436yxz.fsf@benfinney.id.au>
Message-ID: <1216144500.6020.5.camel@fsol>


> So far I have "precedent and tradition" and "positive admonition looks
> better" in support of preferring the 'assert*' names. Are there any
> others?

Avoiding double negatives.
(and don't tell me "fail" is not a negative word, because you just used
the phrase "positive admonition" to refer to the assert* variants, which
implies quite clearly that the fail* variants are on the negative
side ;-))

Regards

Antoine.



From brett at python.org  Wed Jul 16 02:05:26 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Jul 2008 17:05:26 -0700
Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for
	tomorrow
In-Reply-To: <AD5D714F-3513-4BF5-A72C-2ADF6C62181E@python.org>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
	<bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com>
	<AD5D714F-3513-4BF5-A72C-2ADF6C62181E@python.org>
Message-ID: <bbaeab100807151705q56cade4cpc9b6417d8f70d34c@mail.gmail.com>

On Tue, Jul 15, 2008 at 4:45 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jul 15, 2008, at 2:38 PM, Brett Cannon wrote:
>
>> On Tue, Jul 15, 2008 at 5:32 AM, Barry Warsaw <barry at python.org> wrote:
>>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> A reminder: the second betas of Python 2.6 and 3.0 are schedule for
>>> tomorrow.  I will try to hang out on #python-dev today and will start
>>> looking at the trackers and buildbots.  Hopefully, we're on track to get
>>> the
>>> releases out!
>>>
>>> If there is anything you need a decision on, please follow up to this
>>> thread.  I'm inundated with email so I can't watch every thread on the
>>> mailing lists.  Or ping me on #python-dev.
>>>
>>
>> The new urllib package landed between the last beta and now, but
>> without fixers. Issue3316 has potential ones, but they have some
>> tweaks that need to be made before they are release quality. If the
>> beta goes out 2to3 will not be able to fix imports for urllib and
>> urllib2. Don't know if that is enough to hold up the release or just
>> something to put in the release notes that this will be resolved by
>> the next beta; made it a release blocker to catch your eye, Barry. =)
>
> I knocked this down to a deferred blocker, so I won't let it hold up beta2,
> though I'd /really/ like to get this cleaned up and committed in time.  If I
> get enough deferred blockers though, I might hold things up after all.
>

Nick is actively working on it, so it might make it in b2.

-Brett

From steve at pearwood.info  Wed Jul 16 02:10:57 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 16 Jul 2008 10:10:57 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <ca471dc20807151513i5fede9ccjeaac48a50a05f6b6@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au>
	<87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ca471dc20807151513i5fede9ccjeaac48a50a05f6b6@mail.gmail.com>
Message-ID: <200807161010.57836.steve@pearwood.info>

On Wed, 16 Jul 2008 08:13:22 am Guido van Rossum wrote:

> > Tests in the standard distribution which use the deprecated 
> > style will need to be converted.  Steven d'Aprano claims this is
> > nontrivial (and thus error- prone) in some cases.  I haven't seen
> > that claim denied, and it seems plausible to me.
>
> I'd like to see examples of that (this would be Steven's task if he's
> serious about his assertion). Since the fail and assert names are
> mapped to each other using aliasing I don't see how it could be
> nontrivial to map e.g. self.failIf(x) to self.assertFalse(x) -- these
> are the same function!

I have not knowingly claimed that mechanically swapping fail* to assert* 
tests was difficult. The difficulty I refer to is about readability and 
understanding of the code. I often think about tests as sequences of 
possible failures, and as such my unit tests are most naturally written 
as fail*. It is that mental effort of reversing the sense of the tests 
when reading and writing assert* tests that I refer to.

If you want to declare that fail* must go, I'll be disappointed but life 
will go on. But despite the claims of those who have asserted (pun 
intended) that fail* tests are always more difficult to understand, 
that's not the case for everyone.



-- 
Steven

From grflanagan at gmail.com  Wed Jul 16 02:12:02 2008
From: grflanagan at gmail.com (Gerard flanagan)
Date: Wed, 16 Jul 2008 02:12:02 +0200
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest`module (updated 2008-07-15)
In-Reply-To: <877ibn8flm.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>	<87od4z8n9o.fsf@benfinney.id.au>
	<487C82F1.9010506@gmail.com> <877ibn8flm.fsf@benfinney.id.au>
Message-ID: <g5jecg$191$1@ger.gmane.org>

Ben Finney wrote:
> Nick Coghlan <ncoghlan at gmail.com> writes:
>> One option for rationalising the API would be to merely keep the
>> shortest version of each phrase (i.e. use assert* instead of
>> fail_unless* for the positive tests and fail_if* instead of
>> assert_not* for the negative tests, and always drop the trailing 's'
>> from 'equals').
> 
> I think clarity, consistency, and discoverability are more important
> criteria than saving a few characters in a method name.
> 
>> Adding in possible positive and negative forms of the proposed new
>> methods
> 
> A major point of this PEP is to *remove* redundant names in the API.
> 


     def it_is_a_truth_universally_recognised_that(...):

     def we_are_all_feckin_doomed_unless(...):

should be quite sufficient ;-)

entia-non-sunt-multiplicanda-ly y'rs

G.




From collinw at gmail.com  Wed Jul 16 02:20:30 2008
From: collinw at gmail.com (Collin Winter)
Date: Tue, 15 Jul 2008 17:20:30 -0700
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <87skub6yhh.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
Message-ID: <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>

On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Significant updates include removing all reference to the
> (already-resolved) new-style class issue, adding footnotes and
> references, and a Rationale summary of discussion on both sides of the
> divide for 'assert*' versus 'fail*' names.
>
>
> :PEP:               XXX
> :Title:             Consolidating names in the `unittest` module
> :Version:           0.2
> :Last-Modified:     2008-07-15
> :Author:            Ben Finney <ben+python at benfinney.id.au>
> :Status:            Draft
> :Type:              Standards Track
> :Content-Type:      test/x-rst
> :Created:           2008-07-14
> :Python-Version:    2.7, 3.1
> :Post-History:
>
>
> ..  contents::
>
>
> Abstract
> ========
>
> This PEP proposes to consolidate the names that constitute the API of
> the standard library `unittest` module, with the goal of removing
> redundant names, and conforming with PEP 8.
>
>
> Motivation
> ==========
>
> The normal use case for the `unittest` module is to subclass its
> classes, overriding and re-using its functios and methods. This draws
> constant attention to the fact that the existing implementation fails
> several current Python standards:
>
> * It does not conform to PEP 8 [#PEP-8]_, requiring users to write
>  their own non-PEP-8 conformant names when overriding methods, and
>  encouraging extensions to further depart from PEP 8.
>
> * It has many synonyms in its API, which goes against the Zen of
>  Python [#PEP-20]_ (specifically, that "there should be one, and
>  preferably only one, obvious way to do it").
>
>
> Specification
> =============
>
> Remove obsolete names
> ---------------------
>
> The following module attributes are not documented as part of the API
> and are marked as obsolete in the implementation. They will be
> removed.
>
> * ``_makeLoader``
> * ``getTestCaseNames``
> * ``makeSuite``
> * ``findTestCases``
>
> Remove redundant names
> ----------------------
>
> The following attribute names exist only as synonyms for other names.
> They are to be removed, leaving only one name for each attribute in
> the API.
>
> ``TestCase`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~
>
> * ``assertEqual``
> * ``assertEquals``
> * ``assertNotEqual``
> * ``assertNotEquals``
> * ``assertAlmostEqual``
> * ``assertAlmostEquals``
> * ``assertNotAlmostEqual``
> * ``assertNotAlmostEquals``
> * ``assertRaises``
> * ``assert_``
> * ``assertTrue``
> * ``assertFalse``
>
> Conform API with PEP 8
> ----------------------
>
> The following names are to be introduced, each replacing an existing
> name, to make all names in the module conform with PEP 8 [#PEP-8]_.
> Each name is shown with the existing name that it replaces.
>
> Where function parameters are to be renamed also, they are shown.
> Where function parameters are not to be renamed, they are elided with
> the ellipse ("?") symbol.
>
> Module attributes
> ~~~~~~~~~~~~~~~~~
>
> ``default_test_loader``
>  Replaces ``defaultTestLoader``
>
> ``TestResult`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~~~
>
> ``add_error(?)``
>  Replaces ``addError(?)``
>
> ``add_result(?)``
>  Replaces ``addResult(?)``
>
> ``add_success(?)``
>  Replaces ``addSuccess(?)``
>
> ``should_stop``
>  Replaces ``shouldStop``
>
> ``start_test(?)``
>  Replaces ``startTest(?)``
>
> ``stop_test(?)``
>  Replaces ``stopTest(?)``
>
> ``tests_run``
>  Replaces ``testsRun``
>
> ``was_successful(?)``
>  Replaces ``wasSuccessful(?)``
>
> ``TestCase`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~
>
> ``__init__(self, method_name='run_test')``
>  Replaces ``__init__(self, methodName='runTest')``
>
> ``_test_method_doc``
>  Replaces ``_testMethodDoc``
>
> ``_test_method_name``
>  Replaces ``_testMethodName``
>
> ``failure_exception``
>  Replaces ``failureException``
>
> ``count_test_cases(?)``
>  Replaces ``countTestCases(?)``
>
> ``default_test_result(?)``
>  Replaces ``defaultTestResult(?)``
>
> ``fail_if(?)``
>  Replaces ``failIf(?)``
>
> ``fail_if_almost_equal(?)``
>  Replaces ``failIfAlmostEqual(?)``
>
> ``fail_if_equal(?)``
>  Replaces ``failIfEqual(?)``
>
> ``fail_unless(?)``
>  Replaces ``failUnless(?)``
>
> ``fail_unless_almost_equal(?)``
>  Replaces ``failUnlessAlmostEqual(?)``
>
> ``fail_unless_equal(?)``
>  Replaces ``failUnlessEqual(?)``
>
> ``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)``
>  Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)``
>
> ``run_test(?)``
>  Replaces ``runTest(?)``
>
> ``set_up(?)``
>  Replaces ``setUp(?)``
>
> ``short_description(?)``
>  Replaces ``shortDescription(?)``
>
> ``tear_down(?)``
>  Replaces ``tearDown(?)``
>
> ``FunctionTestCase`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> ``__init__(self, test_func, set_up, tear_down, description)``
>  Replaces ``__init__(self, testFunc, setUp, tearDown, description)``
>
> ``run_test(?)``
>  Replaces ``runTest(?)``
>
> ``set_up(?)``
>  Replaces ``setUp(?)``
>
> ``short_description(?)``
>  Replaces ``shortDescription(?)``
>
> ``tear_down(?)``
>  Replaces ``tearDown(?)``
>
> ``TestSuite`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~~
>
> ``add_test(?)``
>  Replaces ``addTest(?)``
>
> ``add_tests(?)``
>  Replaces ``addTests(?)``
>
> ``count_test_cases(?)``
>  Replaces ``countTestCases(?)``
>
> ``TestLoader`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~~~
>
> ``sort_test_methods_using``
>  Replaces ``sortTestMethodsUsing``
>
> ``suite_class``
>  Replaces ``suiteClass``
>
> ``test_method_prefix``
>  Replaces ``testMethodPrefix``
>
> ``get_test_case_names(self, test_case_class)``
>  Replaces ``getTestCaseNames(self, testCaseClass)``
>
> ``load_tests_from_module(?)``
>  Replaces ``loadTestsFromModule(?)``
>
> ``load_tests_from_name(?)``
>  Replaces ``loadTestsFromName(?)``
>
> ``load_tests_from_names(?)``
>  Replaces ``loadTestsFromNames(?)``
>
> ``load_tests_from_test_case(self, test_case_class)``
>  Replaces ``loadTestsFromTestCase(self, testCaseClass)``
>
> ``_TextTestResult`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> ``show_all``
>  Replaces ``showAll``
>
> ``add_error(?)``
>  Replaces ``addError(?)``
>
> ``add_failure(?)``
>  Replaces ``addFailure(?)``
>
> ``add_success(?)``
>  Replaces ``addSuccess(?)``
>
> ``get_description(?)``
>  Replaces ``getDescription(?)``
>
> ``print_error_list(?)``
>  Replaces ``printErrorList(?)``
>
> ``print_errors(?)``
>  Replaces ``printErrors(?)``
>
> ``start_test(?)``
>  Replaces ``startTest(?)``
>
> ``TextTestRunner`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> ``_make_result(?)``
>  Replaces ``_makeResult(?)``
>
> ``TestProgram`` attributes
> ~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> ``__init__(self, module, default_test, argv, test_runner, test_loader)``
>  Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)``
>
> ``create_tests(?)``
>  Replaces ``createTests(?)``
>
> ``parse_args(?)``
>  Replaces ``parseArgs(?)``
>
> ``run_tests(?)``
>  Replaces ``runTests(?)``
>
> ``usage_exit(?)``
>  Replaces ``usageExit(?)``
>
>
> Rationale
> =========
>
> Redundant names
> ---------------
>
> The current API, with two or in some cases three different names
> referencing exactly the same function, leads to an overbroad and
> redundant API that violates PEP 20 [#PEP-20]_ ("there should be one,
> and preferably only one, obvious way to do it").
>
> Removal of ``assert*`` names
> ----------------------------
>
> While there is consensus support to `remove redundant names`_ for the
> ``TestCase`` test methods, the issue of which set of names should be
> retained is controversial.
>
> Arguments in favour of retaining only the ``assert*`` names:
>
> * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference
>  for the ``assert*`` names.
>
> * Precedent: The Python standard library currently uses the
>  ``assert*`` names by a roughly 8:1 majority over the ``fail*``
>  names. (Counting unit tests in the py3k tree at 2008-07-15
>  [#pitrou-1]_.)
>
>  An ad-hoc sampling of other projects that use `unittest` also
>  demonstrates strong preference for use of the ``assert*`` names
>  [#bennetts-1]_.
>
> * Positive admonition: The ``assert*`` names state the intent of how
>  the code under test *should* behave, while the ``fail*`` names are
>  phrased in terms of how the code *should not* behave.
>
> Arguments in favour of retaining only the ``fail*`` names:
>
> * Explicit is better than implicit: The ``fail*`` names state *what
>  the function will do* explicitly: fail the test. With the
>  ``assert*`` names, the action to be taken is only implicit.
>
> * Avoid false implication: The test methods do not have any necessary
>  connection with the built-in ``assert`` statement. Even the
>  exception raised, while it defaults to ``AssertionException``, is
>  explicitly customisable via the documented ``failure_exception``
>  attribute. Choosing the ``fail*`` names avoids the false association
>  with either of these.
>
>  This is exacerbated by the plain-boolean test using a name of
>  ``assert_`` (with a trailing underscore) to avoid a name collision
>  with the built-in ``assert`` statement. The corresponding
>  ``fail_if`` name has no such issue.
>
> PEP 8 names
> -----------
>
> Although `unittest` (and its predecessor `PyUnit`) are intended to be
> familiar to users of other xUnit interfaces, there is no attempt at
> direct API compatibility since the only code that Python's `unittest`
> interfaces with is other Python code. The module is in the standard
> library and its names should all conform with PEP 8 [#PEP-8]_.
>
>
> Backwards Compatibility
> =======================
>
> The names to be obsoleted should be deprecated and removed according
> to the schedule for modules in PEP 4 [#PEP-4]_.
>
> While deprecated, use of the deprecated attributes should raise a
> ``DeprecationWarning``, with a message stating which replacement name
> should be used.

Is any provision being made for a 2to3 fixer/otherwise-automated
transition for the changes you propose here?

Collin Winter

From barry at python.org  Wed Jul 16 02:22:09 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Jul 2008 20:22:09 -0400
Subject: [Python-Dev] Bug 3139
Message-ID: <585A47BF-246A-416D-B83C-E74933B0EB0E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

But 3139 appears important enough to hold up beta 2.

http://bugs.python.org/issue3139
bytearrays are not thread safe

Can we get this fixed by tomorrow?  Does anybody disagree that we  
should hold up the release for this one?  We don't have much time left  
to fix things like this.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSH0/MXEjvBPtnXfVAQJoHwP8CFYbafuSTbnVdBBydZJ5k8Fb1L4YYVG/
0ee81N4AiQZ5uGQlk4ywAVBycPa+DMWE+cdrkYiGXkPaarhcOd55dfdvL1Ww6OgO
tor2fh2IxT0ee7gO/vgbyv4ybWsxuPH/9W5O+W2hZkHd60NkJCqpiHlMKGz6caOX
+rwbgqaQOU4=
=8lO1
-----END PGP SIGNATURE-----

From greg.ewing at canterbury.ac.nz  Wed Jul 16 02:22:45 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Jul 2008 12:22:45 +1200
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<87y7436yxz.fsf@benfinney.id.au>
	<1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com>
Message-ID: <487D3F55.5010309@canterbury.ac.nz>

Richard Thomas wrote:

> I've been told by a couple of non-programmers that "failUnless" is
> more intuitive than "assert" if only for the reason that its unclear
> what "assert" might do.

But test frameworks are for use by programmers, not
non-programmers. Given that it's a test framework, would
a programmer really find it hard to guess what these
mean?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Jul 16 02:53:54 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Jul 2008 12:53:54 +1200
Subject: [Python-Dev] PEP: Consolidating names and classes in	the
 `unittest`module (updated 2008-07-15)
In-Reply-To: <87zloi6a8k.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
	<487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au>
Message-ID: <487D46A2.4050207@canterbury.ac.nz>

Ben Finney wrote:
> Nick Coghlan <ncoghlan at gmail.com> writes:
> 
> > the shortest
> > possible way of writing negative assertions (i.e. asserting that
> > something is not the case) is to treat them as denials and use the
> > single word 'deny'.
> 
> This, to me, is neither intuitive nor meaningful in context. The term
> "deny" is strongly linked to its antonym, "permit".

"Deny" also has the meaning of claiming that something is
not true (as in "deny an allegation"). When used that way,
it's not an antonym of "permit".

However, that meaning doesn't quite seem to fit here, as
we don't just want to claim that the condition is false,
but *ensure* that it's false. I can't think of a single
word offhand that means that.

-- 
Greg

From fuzzyman at voidspace.org.uk  Wed Jul 16 03:03:53 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 02:03:53 +0100
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
Message-ID: <487D48F9.3000903@voidspace.org.uk>

Collin Winter wrote:
> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>
>> Backwards Compatibility
>> =======================
>>
>> The names to be obsoleted should be deprecated and removed according
>> to the schedule for modules in PEP 4 [#PEP-4]_.
>>
>> While deprecated, use of the deprecated attributes should raise a
>> ``DeprecationWarning``, with a message stating which replacement name
>> should be used.
>>     
>
> Is any provision being made for a 2to3 fixer/otherwise-automated
> transition for the changes you propose here?
>   

As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed?

Michael
> Collin Winter
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From ben+python at benfinney.id.au  Wed Jul 16 03:19:44 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 11:19:44 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for the
	`unittest` module
Message-ID: <87fxqa62xb.fsf@benfinney.id.au>

:PEP:           XXX
:Title:         Frequently-requested additional features for the `unittest` module
:Version:       0.3
:Last-Modified: 2008-07-16
:Author:        Ben Finney <ben+python at benfinney.id.au>
:Status:        Draft
:Type:          Standards Track
:Content-Type:  test/x-rst
:Requires:      PEP XXX (Consolidating names in the `unittest` module)
:Created:       2008-07-14
:Post-History:


..  contents::


Abstract
========

This PEP proposes frequently-requested additions to the standard
library `unittest` module that are natural extensions of its existing
functionality.


Motivation
==========

The `unittest` module is functionally complete. Nevertheless, many
users request and devise extensions to perform functions that are so
common they deserve to be in the standard module.


Specification
=============

New condition tests
-------------------

The following test methods will be added to the ``TestCase`` class.
The function signature is part of this specification. The body is to
be treated as a reference implementation only; any functionally
identical implementation also meets this specification.

::

    import operator
    import re

    class TestCase(object):
        # ?

        def assert_compare_true(op, first, second, msg=None):
            if msg is None:
                msg = "%(first)r %(op)r %(second)" % vars()
            if not op(first, second):
                raise self.failure_exception(msg)

        def assert_in(container, member, msg=None):
            op = operator.__contains__
            self.assert_compare_true(op, container, member, msg)

        def assert_is(first, second, msg=None):
            op = operator.is_
            self.assert_compare_true(op, first, second, msg)

        def assert_less_than(first, second, msg=None):
            op = operator.lt
            self.assert_compare_true(op, first, second, msg)

        def assert_greater_than(first, second, msg=None):
            op = operator.gt
            self.assert_compare_true(op, first, second, msg)

        def assert_less_than_or_equal(first, second, msg=None):
            op = operator.le
            self.assert_compare_true(op, first, second, msg)

        def assert_greater_than_or_equal(first, second, msg=None):
            op = operator.ge
            self.assert_compare_true(op, first, second, msg)

        def assert_members_equal(first, second, msg=None):
            self.assert_equal(set(first), set(second), msg)

        def assert_sequence_equal(first, second, msg=None):
            self.assert_equal(list(first), list(second), msg)

        def assert_raises_with_message_regex(
            exc_class, message_regex, callable_obj, *args, **kwargs):
            exc_name = exc_class.__name__
            message_pattern = re.compile(message_regex)
            try:
                callable_obj(*args, **kwargs)
            except exc_class, exc:
                exc_message = str(exc)
                if not message_pattern.match(exc_message):
                    msg = (
                        "%(exc_name)s raised"
                        " without message matching %(message_regex)r"
                        " (got message %(exc_message)r)"
                        ) % vars()
                    raise self.failure_exception(msg)
            else:
                msg = "%(exc_name)s not raised" % vars()
                raise self.failure_exception(msg)

The following test methods are also added. Their implementation in
each case is simply the logical inverse of a corresponding method
above.

::

        def assert_compare_false(op, first, second, msg=None):
            # Logical inverse of assert_compare_true

        def assert_not_in(container, member, msg=None):
            # Logical inverse of assert_in

        def assert_is_not(first, second, msg=None)
            # Logical inverse of assert_is

        def assert_not_less_than(first, second, msg=None)
            # Logical inverse of assert_less_than

        def assert_not_greater_than(first, second, msg=None)
            # Logical inverse of assert_greater_than

        def assert_not_less_than_or_equal(first, second, msg=None)
            # Logical inverse of assert_less_than_or_equal

        def assert_not_greater_than_or_equal(first, second, msg=None)
            # Logical inverse of assert_greater_than_or_equal

        def assert_members_not_equal(first, second, msg=None)
            # Logical inverse of assert_members_equal

        def assert_sequence_not_equal(first, second, msg=None)
            # Logical inverse of assert_sequence_equal

Enhanced failure message for equality tests
-------------------------------------------

The equality tests will change their behaviour such that the message
always, even if overridden with a specific message when called,
includes extra information:

* For both ``assert_equal`` and ``assert_not_equal``: the ``repr()``
  output of the objects that were compared.

* For ``assert_equal`` comparisons of ``basestring`` instances that
  are multi-line text: the output of ``diff`` comparing the two texts.

* For membership comparisons with ``assert_*_equal``: the ``repr()``
  output of the members that were not equal in each collection. (This
  change is not done for the corresponding ``assert_*_not_equal``
  tests, which only fail if the collection members are equal.)

Simple invocation of test collection
------------------------------------

The following new functionality will be added to the `unittest`
module. The function signature for ``run_tests`` is part of this
specification; the implementation is to be considered a reference
implementation only.

::

    def run_tests(
        tests,
        loader_class=TestLoader, suite_class=TestSuite,
        runner_class=TextTestRunner):
        """ Run a collection of tests with a test runner

            :param tests:
                A sequence of objects that can contain test cases:
                modules, `TestSuite` instances, or `TestCase`
                subclasses.
            :param loader_class:
                The type of test loader to use for collecting tests
                from the `tests` collection.
            :param suite_class:
                The type of test suite to use for accumulating the
                collected test cases.
            :param runner_class:
                The type of test runner to instantiate for running the
                collected test cases.
            :return:
                None.

            """
        def iter_testcases_recursively(collection, loader):
            # Flatten and iterate over collection, generating
            # instances of TestCase
        loader = loader_class()
        suite = suite_class()
        for testcase in iter_testcases_recursively(tests, loader):
            suite.add_tests(testcase)
        runner = runner_class()
        runner.run(suite)


Rationale
=========

Names for logical-inverse tests
-------------------------------

The simple pattern established by ``assert_foo`` having a logical
inverse named ``assert_not_foo`` sometimes results in gramatically
awkward names. The following names were chosen in exception of this
pattern, in the interest of the API names being more intuitive:

* ``assert_is_not``
* ``assert_members_not_equal``
* ``assert_sequence_not_equal``

Order of method parameters
--------------------------

The methods ``assert_in``, ``assert_not_in`` have the container as the
first parameter. This makes the grammatical object of the function
name come immediately after the function name: "Assert in
`container`". This matches the convention set by the existing
``assert_raises(exception, callable_obj, ?)`` "(Assert the code
raises `exception`").


Backwards Compatibility
=======================

This PEP proposes only additional features. There are no
backward-incompatible changes.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From scott+python-dev at scottdial.com  Wed Jul 16 03:41:54 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 15 Jul 2008 21:41:54 -0400
Subject: [Python-Dev] PEP: Frequently-requested additional features for
 the	`unittest` module
In-Reply-To: <87fxqa62xb.fsf@benfinney.id.au>
References: <87fxqa62xb.fsf@benfinney.id.au>
Message-ID: <487D51E2.8060908@scottdial.com>

Ben Finney wrote:
> New condition tests
> -------------------
> 
>         def assert_less_than(first, second, msg=None):
>             op = operator.lt
>             self.assert_compare_true(op, first, second, msg)
> 
>         def assert_greater_than(first, second, msg=None):
>             op = operator.gt
>             self.assert_compare_true(op, first, second, msg)
> 
>         def assert_less_than_or_equal(first, second, msg=None):
>             op = operator.le
>             self.assert_compare_true(op, first, second, msg)
> 
>         def assert_greater_than_or_equal(first, second, msg=None):
>             op = operator.ge
>             self.assert_compare_true(op, first, second, msg)
> 
> The following test methods are also added. Their implementation in
> each case is simply the logical inverse of a corresponding method
> above.
> 
>         def assert_not_less_than(first, second, msg=None)
>             # Logical inverse of assert_less_than
> 
>         def assert_not_greater_than(first, second, msg=None)
>             # Logical inverse of assert_greater_than
> 
>         def assert_not_less_than_or_equal(first, second, msg=None)
>             # Logical inverse of assert_less_than_or_equal
> 
>         def assert_not_greater_than_or_equal(first, second, msg=None)
>             # Logical inverse of assert_greater_than_or_equal

Why?! Your other PEP is about removing redundant names, and here you 
have introduced 4 of them:

assert_not_less_than = assert_greater_than_or_equal
assert_not_greater_than = assert_less_than_or_equal
assert_not_less_than_or_equal = assert_greater_than
assert_not_greater_than_or_equal = assert_less_than

Besides, ``assert_not_greater_than_or_equal`` is god-awful-long, along 
with the complaints about PEP-8-ifying. I wonder if it would be better 
to abbreviate these names with the *same name* that was used for the 
attribute in the operator module. Let's not reinvent the wheel here..

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From ben+python at benfinney.id.au  Wed Jul 16 04:05:53 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 12:05:53 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for
	the	`unittest` module
References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com>
Message-ID: <87bq0y60se.fsf@benfinney.id.au>

Scott Dial <scott+python-dev at scottdial.com> writes:

> Why [introduce redundant test names]?
> 
> assert_not_less_than = assert_greater_than_or_equal
> assert_not_greater_than = assert_less_than_or_equal
> assert_not_less_than_or_equal = assert_greater_than
> assert_not_greater_than_or_equal = assert_less_than

To answer the question: The above tests are logically equivalent, but
the failure message would be different, reporting failure in terms of
what the caller wanted to test.

I se your point though. I'd like to see what others think on this
issue.

> Besides, ``assert_not_greater_than_or_equal`` is god-awful-long,
> along with the complaints about PEP-8-ifying. I wonder if it would
> be better to abbreviate these names with the *same name* that was
> used for the attribute in the operator module. Let's not reinvent
> the wheel here..

Interesting. So you advocate collapsing the above eight tests into the
following four:

    assert_lt
    assert_gt
    assert_le
    assert_ge

-- 
 \            ?I always had a repulsive need to be something more than |
  `\                                              human.? ?David Bowie |
_o__)                                                                  |
Ben Finney


From brett at python.org  Wed Jul 16 04:29:26 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Jul 2008 19:29:26 -0700
Subject: [Python-Dev] PEP: Frequently-requested additional features for
	the `unittest` module
In-Reply-To: <87bq0y60se.fsf@benfinney.id.au>
References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com>
	<87bq0y60se.fsf@benfinney.id.au>
Message-ID: <bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com>

On Tue, Jul 15, 2008 at 7:05 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Scott Dial <scott+python-dev at scottdial.com> writes:
>
>> Why [introduce redundant test names]?
>>
>> assert_not_less_than = assert_greater_than_or_equal
>> assert_not_greater_than = assert_less_than_or_equal
>> assert_not_less_than_or_equal = assert_greater_than
>> assert_not_greater_than_or_equal = assert_less_than
>
> To answer the question: The above tests are logically equivalent, but
> the failure message would be different, reporting failure in terms of
> what the caller wanted to test.
>
> I se your point though. I'd like to see what others think on this
> issue.
>
>> Besides, ``assert_not_greater_than_or_equal`` is god-awful-long,
>> along with the complaints about PEP-8-ifying. I wonder if it would
>> be better to abbreviate these names with the *same name* that was
>> used for the attribute in the operator module. Let's not reinvent
>> the wheel here..
>
> Interesting. So you advocate collapsing the above eight tests into the
> following four:
>
>    assert_lt
>    assert_gt
>    assert_le
>    assert_ge

Is any of this really necessary? Isn't this the equivalent of
``assert_(a < b)``? It seems like the only thing you get out of this
is a nicer error message, but ``assert_(a < b, 'not %r <= %s' % (a,
b))`` is not that complex. And do these cases really come up that
often? I would want to see some numbers showing that these are really
necessary (in both usage and people even specifying an error message
in the first place).

-Brett

From scott at scottdial.com  Wed Jul 16 04:27:59 2008
From: scott at scottdial.com (Scott Dial)
Date: Tue, 15 Jul 2008 22:27:59 -0400
Subject: [Python-Dev] PEP: Frequently-requested additional features for
 the	`unittest` module
In-Reply-To: <87bq0y60se.fsf@benfinney.id.au>
References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com>
	<87bq0y60se.fsf@benfinney.id.au>
Message-ID: <487D5CAF.60009@scottdial.com>

Ben Finney wrote:
> Scott Dial <scott+python-dev at scottdial.com> writes:
> 
>> Why [introduce redundant test names]?
> 
> To answer the question: The above tests are logically equivalent, but
> the failure message would be different, reporting failure in terms of
> what the caller wanted to test.

I can see how this argument makes sense, and is distinct from the fail* 
vs. assert* discussion. As you say, I'm interested what other think 
about this.

>> Besides, ``assert_not_greater_than_or_equal`` is god-awful-long,
>> along with the complaints about PEP-8-ifying. I wonder if it would
>> be better to abbreviate these names with the *same name* that was
>> used for the attribute in the operator module. Let's not reinvent
>> the wheel here..
> 
> Interesting. So you advocate collapsing the above eight tests into the
> following four:
> 
>     assert_lt
>     assert_gt
>     assert_le
>     assert_ge

I would argue to go even further:

assertEqual = assert_eq
assertAlmostEqual = assert_almost_eq
assertNotEqual = assert_ne
assertNotAlmostEqual = assert_almost_ne

I'm not sure if there are others, but using the same abbreviations from 
operator is consistent and readable and short, in my opinion.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From ben+python at benfinney.id.au  Wed Jul 16 05:04:12 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 13:04:12 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for
	the `unittest` module
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
Message-ID: <877ibm5y37.fsf_-_@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> The full list of changes proposed [?] and not shot down was
> something like:
[?]

>    assertLessThan
>    assertGreaterThan
>    assertLessThanOrEquals
>    assertGreaterThanOrEquals
[?]

"Brett Cannon" <brett at python.org> writes:

> Is any of this really necessary? Isn't this the equivalent of
> ``assert_(a < b)``? It seems like the only thing you get out of this
> is a nicer error message, but ``assert_(a < b, 'not %r <= %s' % (a,
> b))`` is not that complex. And do these cases really come up that
> often? I would want to see some numbers showing that these are
> really necessary (in both usage and people even specifying an error
> message in the first place).

Though I'm the champion of this PEP, I'll have to refer to Michael
Foord for his reasoning (or reference to others' reasoning) on this.

-- 
 \      ?The process by which banks create money is so simple that the |
  `\     mind is repelled.? ?John Kenneth Galbraith, _Money: Whence It |
_o__)                                       Came, Where It Went_, 1975 |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 05:06:29 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 13:06:29 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in the
	`unittest` module (updated 2008-07-15)
References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol>
Message-ID: <873ama5xze.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> (and don't tell me "fail" is not a negative word, because you just
> used the phrase "positive admonition" to refer to the assert*
> variants, which implies quite clearly that the fail* variants are on
> the negative side ;-))

This "fail is a negative word" has already been rebutted, by native
speakers of English. If you don't want to hear "fail is not a negative
word", I can't help you.

This doesn't seem to introduce anything new, so I'll leave the
existing summary of this argument as is in the PEP.

-- 
 \                   ?Holy unrefillable prescriptions, Batman!? ?Robin |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From ncoghlan at gmail.com  Wed Jul 16 07:25:12 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Jul 2008 15:25:12 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in	the
 `unittest`module (updated 2008-07-15)
In-Reply-To: <487D46A2.4050207@canterbury.ac.nz>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>	<87od4z8n9o.fsf@benfinney.id.au>
	<487C82F1.9010506@gmail.com>	<487CB144.1080200@gmail.com>
	<87zloi6a8k.fsf@benfinney.id.au>
	<487D46A2.4050207@canterbury.ac.nz>
Message-ID: <487D8638.9060807@gmail.com>

Greg Ewing wrote:
> Ben Finney wrote:
>> Nick Coghlan <ncoghlan at gmail.com> writes:
>>
>> > the shortest
>> > possible way of writing negative assertions (i.e. asserting that
>> > something is not the case) is to treat them as denials and use the
>> > single word 'deny'.
>>
>> This, to me, is neither intuitive nor meaningful in context. The term
>> "deny" is strongly linked to its antonym, "permit".
> 
> "Deny" also has the meaning of claiming that something is
> not true (as in "deny an allegation"). When used that way,
> it's not an antonym of "permit".
> 
> However, that meaning doesn't quite seem to fit here, as
> we don't just want to claim that the condition is false,
> but *ensure* that it's false. I can't think of a single
> word offhand that means that.

That was the meaning I was going for, but I agree that it doesn't quite 
fit well enough to make it a good idea. There's a reason I put that 
disclaimer at the top of the message :)

What did you think of the "check" idea at the end of the email?

Test assertions:
   check(x).almost_equal(y)
   check(x).is_(y)
   check(x).in_(y)
   check(x).equals(y)

Test negative assertions:
   check(x).not_almost_equal(y)
   check(x).is_not(y)
   check(x).not_in(y)
   check(x).not_equal(y)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Jul 16 07:33:33 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Jul 2008 15:33:33 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <487D882D.4070108@gmail.com>

Stephen J. Turnbull wrote:
> Ben Finney writes:
> 
>  > Removal of ``assert*`` names
>  > ----------------------------
> 
>  > Arguments in favour of retaining only the ``assert*`` names:
>  > 
>  > * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference
>  >   for the ``assert*`` names.
>  > 
>  > * Precedent: The Python standard library currently uses the
>  >   ``assert*`` names by a roughly 8:1 majority over the ``fail*``
>  >   names. (Counting unit tests in the py3k tree at 2008-07-15
>  >   [#pitrou-1]_.)
>  > 
>  >   An ad-hoc sampling of other projects that use `unittest` also
>  >   demonstrates strong preference for use of the ``assert*`` names
>  >   [#bennetts-1]_.
>  > 
>  > * Positive admonition: The ``assert*`` names state the intent of how
>  >   the code under test *should* behave, while the ``fail*`` names are
>  >   phrased in terms of how the code *should not* behave.
> 
> FWIW, I think these are fairly stated.  So fairly that I'm surprised
> you haven't been persuaded!<wink>  Nitpick: the second point is not
> just "precedent", there's an economic reason there too.  Tests in the
> standard distribution which use the deprecated style will need to be
> converted.  Steven d'Aprano claims this is nontrivial (and thus error-
> prone) in some cases.  I haven't seen that claim denied, and it seems
> plausible to me.

I disagree with SdA's statement there: failUnless* maps directly to 
assert* and failIf* maps directly to assertNot*. There are no semantic 
changes whatsoever involved in that remapping, just a change in the 
method names.

Note that if the API is to be rationalised at all, my personal 
preference is to keep assert* and failIf* and get rid of their longer 
counterparts (failUnless* and assertNot*). Alternatively, ditch the 
binary assertion methods entirely, and use a check object to state those 
assertions:

   check(x).almost_equal(y) instead assertAlmostEqual(x, y)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ben+python at benfinney.id.au  Wed Jul 16 07:48:09 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 15:48:09 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for
	the `unittest` module
References: <87fxqa62xb.fsf@benfinney.id.au>
Message-ID: <87abgi4bxi.fsf@benfinney.id.au>

Significant changes: Add a new 'TestLoader.load_tests_from_collection'
method, with full reference implementation. This makes the 'run_tests'
reference implementation straightforward.


:PEP:           XXX
:Title:         Frequently-requested additional features for the `unittest` module
:Version:       0.4
:Last-Modified: 2008-07-16
:Author:        Ben Finney <ben+python at benfinney.id.au>
:Status:        Draft
:Type:          Standards Track
:Content-Type:  test/x-rst
:Requires:      PEP XXX (Consolidating names in the `unittest` module)
:Created:       2008-07-14
:Post-History:


..  contents::


Abstract
========

This PEP proposes frequently-requested additions to the standard
library `unittest` module that are natural extensions of its existing
functionality.


Motivation
==========

The `unittest` module is functionally complete. Nevertheless, many
users request and devise extensions to perform functions that are so
common they deserve to have a standard implementation.


Specification
=============

New condition tests
-------------------

The following test methods will be added to the ``TestCase`` class.
The function signature is part of this specification. The body is to
be treated as a reference implementation only; any functionally
identical implementation also meets this specification.

::

    import operator
    import re

    class TestCase(object):
        # ?

        def assert_compare_true(op, first, second, msg=None):
            if msg is None:
                msg = "%(first)r %(op)r %(second)" % vars()
            if not op(first, second):
                raise self.failure_exception(msg)

        def assert_in(container, member, msg=None):
            op = operator.__contains__
            self.assert_compare_true(op, container, member, msg)

        def assert_is(first, second, msg=None):
            op = operator.is_
            self.assert_compare_true(op, first, second, msg)

        def assert_less_than(first, second, msg=None):
            op = operator.lt
            self.assert_compare_true(op, first, second, msg)

        def assert_greater_than(first, second, msg=None):
            op = operator.gt
            self.assert_compare_true(op, first, second, msg)

        def assert_less_than_or_equal(first, second, msg=None):
            op = operator.le
            self.assert_compare_true(op, first, second, msg)

        def assert_greater_than_or_equal(first, second, msg=None):
            op = operator.ge
            self.assert_compare_true(op, first, second, msg)

        def assert_members_equal(first, second, msg=None):
            self.assert_equal(set(first), set(second), msg)

        def assert_sequence_equal(first, second, msg=None):
            self.assert_equal(list(first), list(second), msg)

        def assert_raises_with_message_regex(
            exc_class, message_regex, callable_obj, *args, **kwargs):
            exc_name = exc_class.__name__
            message_pattern = re.compile(message_regex)
            try:
                callable_obj(*args, **kwargs)
            except exc_class, exc:
                exc_message = str(exc)
                if not message_pattern.match(exc_message):
                    msg = (
                        "%(exc_name)s raised"
                        " without message matching %(message_regex)r"
                        " (got message %(exc_message)r)"
                        ) % vars()
                    raise self.failure_exception(msg)
            else:
                msg = "%(exc_name)s not raised" % vars()
                raise self.failure_exception(msg)

The following test methods are also added. Their implementation in
each case is simply the logical inverse of a corresponding method
above.

::

        def assert_compare_false(op, first, second, msg=None):
            # Logical inverse of assert_compare_true

        def assert_not_in(container, member, msg=None):
            # Logical inverse of assert_in

        def assert_is_not(first, second, msg=None)
            # Logical inverse of assert_is

        def assert_not_less_than(first, second, msg=None)
            # Logical inverse of assert_less_than

        def assert_not_greater_than(first, second, msg=None)
            # Logical inverse of assert_greater_than

        def assert_not_less_than_or_equal(first, second, msg=None)
            # Logical inverse of assert_less_than_or_equal

        def assert_not_greater_than_or_equal(first, second, msg=None)
            # Logical inverse of assert_greater_than_or_equal

        def assert_members_not_equal(first, second, msg=None)
            # Logical inverse of assert_members_equal

        def assert_sequence_not_equal(first, second, msg=None)
            # Logical inverse of assert_sequence_equal

Enhanced failure message for equality tests
-------------------------------------------

The equality tests will change their behaviour such that the message
always, even if overridden with a specific message when called,
includes extra information:

* For both ``assert_equal`` and ``assert_not_equal``: the ``repr()``
  output of the objects that were compared.

* For ``assert_equal`` comparisons of ``basestring`` instances that
  are multi-line text: the output of ``diff`` comparing the two texts.

* For membership comparisons with ``assert_*_equal``: the ``repr()``
  output of the members that were not equal in each collection. (This
  change is not done for the corresponding ``assert_*_not_equal``
  tests, which only fail if the collection members are equal.)

Simple invocation of test collection
------------------------------------

To allow simpler loading and running of test cases from an arbitrary
collection, the following new functionality will be added to the
`unittest` module. The function signatures are part of this
specification; the implementation is to be considered a reference
implementation only.

::

    class TestLoader(object):
        # ?

        def load_tests_from_collection(self, collection):
            """ Return a suite of all test cases found in `collection`

                :param collection:
                    One of the following:
                    * a `TestSuite` instance
                    * a `TestCase` subclass
                    * a module
                    * an iterable producing any of these types
                :return:
                    A `TestSuite` instance containing all the test
                    cases found recursively within `collection`.

                """
            suite = None
            if isinstance(collection, TestSuite):
                suite = collection
            elif isinstance(collection, type):
                if issubclass(collection, TestCase):
                    suite = self.load_tests_from_test_case(collection)
            elif isinstance(collection, types.ModuleType):
                suite = self.load_tests_from_module(collection)
            elif hasattr(collection, '__iter__'):
                suite = self.suite_class()
                for item in collection:
                    suite.add_test(self.load_tests_from_collection(item))
            else:
                msg = "not a test collection: %(collection)r" % vars()
                raise TypeError(msg)
            return suite


    def run_tests(
        tests,
        loader_class=TestLoader, runner_class=TextTestRunner):
        """ Run a collection of tests with a test runner

            :param tests:
                A test collection as defined by the `loader_class`
                method `load_tests_from_collection`.
            :param loader_class:
                The type of test loader to use for collecting tests
                from the `tests` collection.
            :param runner_class:
                The type of test runner to instantiate for running the
                collected test suite.
            :return:
                None.

            """
        loader = loader_class()
        suite = loader.load_tests_from_collection(tests)
        runner = runner_class()
        runner.run(suite)


Rationale
=========

Names for logical-inverse tests
-------------------------------

The simple pattern established by ``assert_foo`` having a logical
inverse named ``assert_not_foo`` sometimes results in gramatically
awkward names. The following names were chosen in exception of this
pattern, in the interest of the API names being more intuitive:

* ``assert_is_not``
* ``assert_members_not_equal``
* ``assert_sequence_not_equal``

Order of method parameters
--------------------------

The methods ``assert_in``, ``assert_not_in`` have the container as the
first parameter. This makes the grammatical object of the function
name come immediately after the function name: "Assert in
`container`". This matches the convention set by the existing
``assert_raises(exception, callable_obj, ?)`` "(Assert the code
raises `exception`").


Backwards Compatibility
=======================

This PEP proposes only additional features. There are no
backward-incompatible changes.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From tjreedy at udel.edu  Wed Jul 16 08:10:54 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Jul 2008 02:10:54 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <878ww27p13.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fxqb8mlp.fsf@benfinney.id.au>	<87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>	<g5iljh$cmt$1@ger.gmane.org>	<87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp>
	<878ww27p13.fsf@benfinney.id.au>
Message-ID: <g5k3dc$hgk$1@ger.gmane.org>



Ben Finney wrote:
> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
> 
>> Terry Reedy writes:
>>
>>  > For the community as a whole, all stdlib modules are suggestions
>>  > and examples, not commands.
>>
>> Well, even if "standard" is too strong a word, the DeprecationWarnings
>> and eventual removal of the methods surely constitute an imposition.
> 
> I understood Terry's statement as meaning that there is no commandment
> to use the standard library 'unittest' module at all. If a user
> doesn't like the behaviour of a module (such as 'unittest') in the
> standard library, they can accept the burden of implementing one that
> behaves the way they like.

I have written the few specialized test functions that I need my current 
project.  I did this after considering (and taking ideas from) py.test. 
  I will also use doctest as a supplement.

I also meant that one who does use something is free to rename it.  I 
commonly import xskjfl as x;-)


From tjreedy at udel.edu  Wed Jul 16 08:17:44 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Jul 2008 02:17:44 -0400
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <487D48F9.3000903@voidspace.org.uk>
References: <87vdz8a983.fsf@benfinney.id.au>
	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
	<487D48F9.3000903@voidspace.org.uk>
Message-ID: <g5k3q7$ig7$1@ger.gmane.org>



Michael Foord wrote:
> Collin Winter wrote:

>> Is any provision being made for a 2to3 fixer/otherwise-automated
>> transition for the changes you propose here?
>>   
> 
> As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed?

A fixer will only be needed when it actually is needed, but when it is, 
it should be a unittest-name fixer since previous 3.x code will also 
need fixing.  Since the duplicates are multiples names for the same 
objects, the fixer should be a trivial name substitution.


From ben+python at benfinney.id.au  Wed Jul 16 08:25:06 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 16:25:06 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in	the
	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
	<487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au>
	<487D46A2.4050207@canterbury.ac.nz> <487D8638.9060807@gmail.com>
Message-ID: <8763r64a7x.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> What did you think of the "check" idea at the end of the email?
> 
> Test assertions:
>   check(x).almost_equal(y)
>   check(x).is_(y)
>   check(x).in_(y)
>   check(x).equals(y)
> 
> Test negative assertions:
>   check(x).not_almost_equal(y)
>   check(x).is_not(y)
>   check(x).not_in(y)
>   check(x).not_equal(y)

-1

'check' is even less explicit about what will happen than 'assert'. At
least the latter has existing programming-language connotations of
"fail immediately if not true", which 'check' lacks.

-- 
 \        ?I used to work in a fire hydrant factory. You couldn't park |
  `\                          anywhere near the place.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From grubert at users.sourceforge.net  Wed Jul 16 08:50:02 2008
From: grubert at users.sourceforge.net (engelbert gruber)
Date: Wed, 16 Jul 2008 08:50:02 +0200
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
	<ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
	<1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>
	<g5j80e$g4d$1@ger.gmane.org>
	<bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com>
Message-ID: <ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com>

I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes
``DeprecationWarning: callable() not supported in 3.x;`` .

and if someone with commit rights would take i would volunteer to
fix these two on case by case basis, even if it is not my case.

all the best

From andrew-pythondev at puzzling.org  Wed Jul 16 09:00:56 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed, 16 Jul 2008 17:00:56 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`module (updated 2008-07-15)
In-Reply-To: <487D8638.9060807@gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
	<487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au>
	<487D46A2.4050207@canterbury.ac.nz> <487D8638.9060807@gmail.com>
Message-ID: <20080716070056.GD26808@steerpike.home.puzzling.org>

Nick Coghlan wrote:
[...]
>
> What did you think of the "check" idea at the end of the email?
>
> Test assertions:
>   check(x).almost_equal(y)
>   check(x).is_(y)
>   check(x).in_(y)
>   check(x).equals(y)
>
> Test negative assertions:
>   check(x).not_almost_equal(y)
>   check(x).is_not(y)
>   check(x).not_in(y)
>   check(x).not_equal(y)

Wow.

This is a textbook example of a bikeshed discussion.  The names (and now
syntax!) of assertions are the most cosmetic issue there is with the unittest
module, yet I see over *200* messages about it.

Deeper, but important issues, such as how to raise structured exceptions that a
GUI test runner could usefully display and introspect, are ignored.  I can list
lots of others.

For assertions, just flip a coin already (or a roll a d20, perhaps...).  Can we
perhaps devote this considerable energy to significant improvements, please?

-Andrew.


From ben+python at benfinney.id.au  Wed Jul 16 09:01:30 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 17:01:30 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for
	the	`unittest` module
References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com>
	<87bq0y60se.fsf@benfinney.id.au> <487D5CAF.60009@scottdial.com>
Message-ID: <871w1u48j9.fsf@benfinney.id.au>

Scott Dial <scott at scottdial.com> writes:

> I would argue to go even further:
> 
> assertEqual = assert_eq
> assertAlmostEqual = assert_almost_eq
> assertNotEqual = assert_ne
> assertNotAlmostEqual = assert_almost_ne
> 
> I'm not sure if there are others, but using the same abbreviations
> from operator is consistent and readable and short, in my opinion.

This doesn't seem to be a good fit for either of "Consolidate names in
unittest API" (which seeks to make the renames meet standards, not
find better names), nor "Frequently-requested additions to unittest"
(which seeks *only* to add functionality, not change existing names).

-- 
 \      ?An expert is a man who has made all the mistakes which can be |
  `\                         made in a very narrow field.? ?Niels Bohr |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 09:53:22 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 17:53:22 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest`module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1>
	<87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com>
	<487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au>
	<487D46A2.4050207@canterbury.ac.nz> <487D8638.9060807@gmail.com>
	<20080716070056.GD26808@steerpike.home.puzzling.org>
Message-ID: <87skua2rkd.fsf@benfinney.id.au>

Andrew Bennetts <andrew-pythondev at puzzling.org> writes:

> This is a textbook example of a bikeshed discussion. The names (and
> now syntax!) of assertions are the most cosmetic issue there is with
> the unittest module, yet I see over *200* messages about it.

I've found it much more insightful than you seem to. I'm sorry it
hasn't been of more interest to you, but please don't discount the
value of this discussion for the rest of us.

> Deeper, but important issues, such as how to raise structured
> exceptions that a GUI test runner could usefully display and
> introspect, are ignored. I can list lots of others.

I look forward to you writing, promoting, and championing the PEP
document(s) for these issues that you consider important.

-- 
 \      ?Some forms of reality are so horrible we refuse to face them, |
  `\     unless we are trapped into it by comedy. To label any subject |
_o__)        unsuitable for comedy is to admit defeat.? ?Peter Sellers |
Ben Finney


From mal at egenix.com  Wed Jul 16 10:00:32 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Jul 2008 10:00:32 +0200
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
Message-ID: <487DAAA0.1040604@egenix.com>

On 2008-07-16 02:20, Collin Winter wrote:
> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> Significant updates include removing all reference to the
>> (already-resolved) new-style class issue, adding footnotes and
>> references, and a Rationale summary of discussion on both sides of the
>> divide for 'assert*' versus 'fail*' names.
>>
>>
>> :PEP:               XXX
>> :Title:             Consolidating names in the `unittest` module
>> :Version:           0.2
>> :Last-Modified:     2008-07-15
>> :Author:            Ben Finney <ben+python at benfinney.id.au>
>> :Status:            Draft
>> :Type:              Standards Track
>> :Content-Type:      test/x-rst
>> :Created:           2008-07-14
>> :Python-Version:    2.7, 3.1

+1 for doing this in 3.1.

-1 for Python 2.7.

The main reason is that there's just too much 2.x code out there
using the API names you are suggesting to change and/or remove
from the module.

Since this is a major change in the unit test API, I'd also like
to suggest that you use a new module name.

This is both a precaution to prevent tests failing due to not having
been upgraded and a way for old code to continue working by adding
the old unittest module on sys.path.

Please note that the required renaming of the methods in existing
tests is not going to be as straight forward as you may think,
since you may well rename method calls into the tested application
rather than just the unit test class method calls if you're not
careful.

>> Abstract
>> ========
>>
>> This PEP proposes to consolidate the names that constitute the API of
>> the standard library `unittest` module, with the goal of removing
>> redundant names, and conforming with PEP 8.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 16 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From ben+python at benfinney.id.au  Wed Jul 16 10:14:37 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 18:14:37 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
	<487DAAA0.1040604@egenix.com>
Message-ID: <87od4y2qky.fsf@benfinney.id.au>

"M.-A. Lemburg" <mal at egenix.com> writes:

> Since this is a major change in the unit test API, I'd also like
> to suggest that you use a new module name.
> 
> This is both a precaution to prevent tests failing due to not having
> been upgraded and a way for old code to continue working by adding
> the old unittest module on sys.path.

Do you have a specific argument against the provisions already stated
in the PEP for backward compatibility? They seem to address your
concerns already.

-- 
 \       ?If you don't know what your program is supposed to do, you'd |
  `\                 better not start writing it.? ?Edsger W. Dijkstra |
_o__)                                                                  |
Ben Finney


From stephen at xemacs.org  Wed Jul 16 12:21:21 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Jul 2008 19:21:21 +0900
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
In-Reply-To: <873ama5xze.fsf@benfinney.id.au>
References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol>
	<873ama5xze.fsf@benfinney.id.au>
Message-ID: <87od4yw2n2.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:

 > This "fail is a negative word" has already been rebutted, by native
 > speakers of English.

Not successfully, it hasn't.  Steven d'Aprano describes one style of
testing as "the test passes if it fails to fail in each of a sequence
of cases."  That is perfectly good English, which makes no sense if
"fail" completely lacks the semantics of negation.  The intuition that
"fail" is a negative word is thus well-founded in standard usage.

By the way, a native speaker is a person who has no need to understand
how his language works; he just uses it.  Being a native speaker
doesn't qualify one as an authority on her language.

From solipsis at pitrou.net  Wed Jul 16 12:16:26 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Jul 2008 10:16:26 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP=3A_Consolidating_names_and_classes_in_?=
	=?utf-8?q?the=09=60unittest=60_module_=28updated_2008-07-15=29?=
References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol>
	<873ama5xze.fsf@benfinney.id.au>
Message-ID: <loom.20080716T100023-565@post.gmane.org>

Ben Finney <ben+python <at> benfinney.id.au> writes:
> 
> This "fail is a negative word" has already been rebutted, by native
> speakers of English.

Well, Stephen's and Greg's own answers notwithstanding, if you really want an
authoritative answer, the best would be to open a dictionary and contrast the
given definitions...

* http://www.thefreedictionary.com/fail
"To prove deficient or lacking; perform ineffectively or inadequately; To be
unsuccessful"

* http://www.thefreedictionary.com/assert
"To state or express positively"

But perhaps there is enough of this fail/assert discussion now :-)

Regards

Antoine.



From mal at egenix.com  Wed Jul 16 13:20:19 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Jul 2008 13:20:19 +0200
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <87od4y2qky.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>	<487DAAA0.1040604@egenix.com>
	<87od4y2qky.fsf@benfinney.id.au>
Message-ID: <487DD973.7010007@egenix.com>

On 2008-07-16 10:14, Ben Finney wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>> Since this is a major change in the unit test API, I'd also like
>> to suggest that you use a new module name.
>>
>> This is both a precaution to prevent tests failing due to not having
>> been upgraded and a way for old code to continue working by adding
>> the old unittest module on sys.path.
> 
> Do you have a specific argument against the provisions already stated
> in the PEP for backward compatibility? They seem to address your
> concerns already.

The PEP doesn't mention changing the module name and deprecating
the old one. Instead it wants to deprecate all the old names (and cites
PEP 4 for this), but keeping the module name.

Note that PEP 4 targets deprecating use of whole modules, not single
APIs, or - like in your case - more or less the complete existing API
of a module.

Given the scope of the changes, you are really creating a completely
new API and as a result should also get a new module name. You can then
deprecate use of the old "unittest" module name and point users to the
new one.

Developers who don't feel like changing 10000+ tests can then continue
to use the old module and start using the new module for new projects.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 16 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From fuzzyman at voidspace.org.uk  Wed Jul 16 14:02:38 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 13:02:38 +0100
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <487DD973.7010007@egenix.com>
References: <87vdz8a983.fsf@benfinney.id.au>	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>	<487DAAA0.1040604@egenix.com>	<87od4y2qky.fsf@benfinney.id.au>
	<487DD973.7010007@egenix.com>
Message-ID: <487DE35E.2070501@voidspace.org.uk>

M.-A. Lemburg wrote:
> On 2008-07-16 10:14, Ben Finney wrote:
>> "M.-A. Lemburg" <mal at egenix.com> writes:
>>
>>> Since this is a major change in the unit test API, I'd also like
>>> to suggest that you use a new module name.
>>>
>>> This is both a precaution to prevent tests failing due to not having
>>> been upgraded and a way for old code to continue working by adding
>>> the old unittest module on sys.path.
>>
>> Do you have a specific argument against the provisions already stated
>> in the PEP for backward compatibility? They seem to address your
>> concerns already.
>
> The PEP doesn't mention changing the module name and deprecating
> the old one. Instead it wants to deprecate all the old names (and cites
> PEP 4 for this), but keeping the module name.
>
> Note that PEP 4 targets deprecating use of whole modules, not single
> APIs, or - like in your case - more or less the complete existing API
> of a module.

Which PEP is usually referenced for the deprecation of individual APIs?

>
> Given the scope of the changes, you are really creating a completely
> new API and as a result should also get a new module name. You can then
> deprecate use of the old "unittest" module name and point users to the
> new one.

You propose that we duplicate the entire module with a new name, 
maintaining both in parallel but with different method names?

That doesn't sound wise to me.

>
> Developers who don't feel like changing 10000+ tests can then continue
> to use the old module and start using the new module for new projects.
>

So we shouldn't bring the API inline with PEP 8 because it is widely used?

Even if it causes some pain (and the methods won't be removed for 
several years if we follow the normal deprecation schedule), the fact 
that the API is widely used would seem to be an argument in favor of 
making it follow the Python style guidelines.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From rrr at ronadam.com  Wed Jul 16 14:13:20 2008
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 16 Jul 2008 07:13:20 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
Message-ID: <487DE5E0.4060406@ronadam.com>



Raymond Hettinger wrote:
> From: "Ben Finney" <ben+python at benfinney.id.au>
>> Right, so I'm putting up a separate PEP just for the renaming. Should
>> be arriving on this list soon.
> 
> I would like to work with you or someone else who is interested
> on an alternative PEP for a separate, simpler test module
> using the py.test syntax.  That is much simpler to learn and use.
> Instead of self.assertIsNot and whatnot, you write:
>   assert a is not b
> No need for tons of word-by-word spellings on things we already
> have syntax for.  Almost anyone who has used py.test can attest
> its syntax is much more natural, easy to learn, easy to both
> read and write, and is much lighter weight.  I think some variant
> of py.test could be done that is compatable with unittest
> and the did not have the "magic" present in earlier versions of py.test.
> I wrote a recipe (somewhat rough and incomplete) that shows how
> easily this could be done:
> 
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194
> 
> Raymond

+1 for a simpler testing module.

Just letting you know there is interest in a lighter weight testing suite.

Looking at the unittest discussions, it doesn't look like it is getting 
easier to use, but more complex. Py.test looks very interesting, especially 
the test discovery parts.  I also agree we don't need special methods for 
every variation of assertedness.


I've been thinking that a few decorators may go a long way to making 
writing tests easy.  It also reduces the level of indentation needed.

Ron









From rrr at ronadam.com  Wed Jul 16 14:13:20 2008
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 16 Jul 2008 07:13:20 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
Message-ID: <487DE5E0.4060406@ronadam.com>



Raymond Hettinger wrote:
> From: "Ben Finney" <ben+python at benfinney.id.au>
>> Right, so I'm putting up a separate PEP just for the renaming. Should
>> be arriving on this list soon.
> 
> I would like to work with you or someone else who is interested
> on an alternative PEP for a separate, simpler test module
> using the py.test syntax.  That is much simpler to learn and use.
> Instead of self.assertIsNot and whatnot, you write:
>   assert a is not b
> No need for tons of word-by-word spellings on things we already
> have syntax for.  Almost anyone who has used py.test can attest
> its syntax is much more natural, easy to learn, easy to both
> read and write, and is much lighter weight.  I think some variant
> of py.test could be done that is compatable with unittest
> and the did not have the "magic" present in earlier versions of py.test.
> I wrote a recipe (somewhat rough and incomplete) that shows how
> easily this could be done:
> 
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194
> 
> Raymond

+1 for a simpler testing module.

Just letting you know there is interest in a lighter weight testing suite.

Looking at the unittest discussions, it doesn't look like it is getting 
easier to use, but more complex. Py.test looks very interesting, especially 
the test discovery parts.  I also agree we don't need special methods for 
every variation of assertedness.


I've been thinking that a few decorators may go a long way to making 
writing tests easy.  It also reduces the level of indentation needed.

Ron









From fuzzyman at voidspace.org.uk  Wed Jul 16 14:21:44 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 13:21:44 +0100
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <g5k3q7$ig7$1@ger.gmane.org>
References: <87vdz8a983.fsf@benfinney.id.au>	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>	<487D48F9.3000903@voidspace.org.uk>
	<g5k3q7$ig7$1@ger.gmane.org>
Message-ID: <487DE7D8.6050308@voidspace.org.uk>

Terry Reedy wrote:
>
>
> Michael Foord wrote:
>> Collin Winter wrote:
>
>>> Is any provision being made for a 2to3 fixer/otherwise-automated
>>> transition for the changes you propose here?
>>>   
>>
>> As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed?
>
> A fixer will only be needed when it actually is needed, but when it 
> is, it should be a unittest-name fixer since previous 3.x code will 
> also need fixing.  Since the duplicates are multiples names for the 
> same objects, the fixer should be a trivial name substitution.

Can 2to3 fixers be used for 2to2 and 3to3 translation then?

Michael

>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From ben+python at benfinney.id.au  Wed Jul 16 14:22:48 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 22:22:48 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol>
	<873ama5xze.fsf@benfinney.id.au>
	<87od4yw2n2.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <877ibm2f3b.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:

> The intuition that "fail" is a negative word is thus well-founded in
> standard usage.

That's not the same thing as "fail" being a negative word in the sense
meant by "double negative".

That is, "not fail" is not a double negative; nor is "fail if X is not
inside Y" a double negative. The use of "fail" in those phrases is a
action, a verb, not a "negative".

So, this issue of avoiding "fail" in order to "avoid double negatives"
has no basis in the use of "fail" in unittest.

-- 
 \          ?It was half way to Rivendell when the drugs began to take |
  `\        hold? ?Hunter S. Tolkien, _Fear and Loathing in Barad-D?r_ |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 14:24:29 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 22:24:29 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol>
	<873ama5xze.fsf@benfinney.id.au>
	<loom.20080716T100023-565@post.gmane.org>
Message-ID: <873ama2f0i.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> * http://www.thefreedictionary.com/fail
> "To prove deficient or lacking; perform ineffectively or inadequately; To be
> unsuccessful"

Yes. It's a verb, not a negative modifer. To use it with a negative
like "not" is not creating a "double negative".

-- 
 \          ?It's dangerous to be right when the government is wrong.? |
  `\                                   ?Francois Marie Arouet Voltaire |
_o__)                                                                  |
Ben Finney


From fuzzyman at voidspace.org.uk  Wed Jul 16 14:30:58 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 13:30:58 +0100
Subject: [Python-Dev] PEP: Frequently-requested additional features for
 the `unittest` module
In-Reply-To: <bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com>
References: <87fxqa62xb.fsf@benfinney.id.au>
	<487D51E2.8060908@scottdial.com>	<87bq0y60se.fsf@benfinney.id.au>
	<bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com>
Message-ID: <487DEA02.2070508@voidspace.org.uk>

Brett Cannon wrote:
> On Tue, Jul 15, 2008 at 7:05 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>   
>> Scott Dial <scott+python-dev at scottdial.com> writes:
>>
>>     
>>> Why [introduce redundant test names]?
>>>
>>> assert_not_less_than = assert_greater_than_or_equal
>>> assert_not_greater_than = assert_less_than_or_equal
>>> assert_not_less_than_or_equal = assert_greater_than
>>> assert_not_greater_than_or_equal = assert_less_than
>>>       
>> To answer the question: The above tests are logically equivalent, but
>> the failure message would be different, reporting failure in terms of
>> what the caller wanted to test.
>>
>> I se your point though. I'd like to see what others think on this
>> issue.
>>
>>     
>>> Besides, ``assert_not_greater_than_or_equal`` is god-awful-long,
>>> along with the complaints about PEP-8-ifying. I wonder if it would
>>> be better to abbreviate these names with the *same name* that was
>>> used for the attribute in the operator module. Let's not reinvent
>>> the wheel here..
>>>       
>> Interesting. So you advocate collapsing the above eight tests into the
>> following four:
>>
>>    assert_lt
>>    assert_gt
>>    assert_le
>>    assert_ge
>>     
>
> Is any of this really necessary? Isn't this the equivalent of
> ``assert_(a < b)``? It seems like the only thing you get out of this
> is a nicer error message, but ``assert_(a < b, 'not %r <= %s' % (a,
> b))`` is not that complex. And do these cases really come up that
> often? I would want to see some numbers showing that these are really
> necessary (in both usage and people even specifying an error message
> in the first place).
>   

I'm inclined to agree. It was part of a set of additions suggested by 
Guido. From here I think (as part of the unittest extensions that google 
maintains):

http://mail.python.org/pipermail/python-dev/2008-April/078702.html

I've used tests like that when implementing numeric objects, which has 
been several times - but only a tiny proportion of the tests I've written.

I wouldn't be sorry not to include them.

Michael



> -Brett
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From mal at egenix.com  Wed Jul 16 14:36:45 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Jul 2008 14:36:45 +0200
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <487DE35E.2070501@voidspace.org.uk>
References: <87vdz8a983.fsf@benfinney.id.au>	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>	<487DAAA0.1040604@egenix.com>	<87od4y2qky.fsf@benfinney.id.au>	<487DD973.7010007@egenix.com>
	<487DE35E.2070501@voidspace.org.uk>
Message-ID: <487DEB5D.7010507@egenix.com>

On 2008-07-16 14:02, Michael Foord wrote:
> M.-A. Lemburg wrote:
>> On 2008-07-16 10:14, Ben Finney wrote:
>>> "M.-A. Lemburg" <mal at egenix.com> writes:
>>>
>>>> Since this is a major change in the unit test API, I'd also like
>>>> to suggest that you use a new module name.
>>>>
>>>> This is both a precaution to prevent tests failing due to not having
>>>> been upgraded and a way for old code to continue working by adding
>>>> the old unittest module on sys.path.
>>>
>>> Do you have a specific argument against the provisions already stated
>>> in the PEP for backward compatibility? They seem to address your
>>> concerns already.
>>
>> The PEP doesn't mention changing the module name and deprecating
>> the old one. Instead it wants to deprecate all the old names (and cites
>> PEP 4 for this), but keeping the module name.
>>
>> Note that PEP 4 targets deprecating use of whole modules, not single
>> APIs, or - like in your case - more or less the complete existing API
>> of a module.
> 
> Which PEP is usually referenced for the deprecation of individual APIs?

PEP 5 could be used for that.

Adding several 10s of deprecation warnings to the unittest module
is not going to make life easier for anyone. Adding just a single
one on import and following PEP 4 is.

If you do want to apply major changes to a module without changing
the name, then this could be done as part of the 2.x -> 3.x transition.
The 2.x branch is not the right place for such breakage.

>> Given the scope of the changes, you are really creating a completely
>> new API and as a result should also get a new module name. You can then
>> deprecate use of the old "unittest" module name and point users to the
>> new one.
> 
> You propose that we duplicate the entire module with a new name, 
> maintaining both in parallel but with different method names?

No, I'm proposing to apply all the name changes to a new module
and deprecate the old one. unittest will then go unmaintained
until it is removed.

> That doesn't sound wise to me.
> 
>>
>> Developers who don't feel like changing 10000+ tests can then continue
>> to use the old module and start using the new module for new projects.
>>
> 
> So we shouldn't bring the API inline with PEP 8 because it is widely used?

I didn't say that. However, if it's not required, then breaking
a complete module API isn't necessary - "practicality beats purity".

Instead add a new module with all the changes and have developers
gradually migrate to the new code.

> Even if it causes some pain (and the methods won't be removed for 
> several years if we follow the normal deprecation schedule), the fact 
> that the API is widely used would seem to be an argument in favor of 
> making it follow the Python style guidelines.

So you're saying that because many people use the code, we should be
more inclined to make their life harder. That's an interesting
argument :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 16 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From ben+python at benfinney.id.au  Wed Jul 16 14:37:31 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 22:37:31 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
	<487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au>
	<487DD973.7010007@egenix.com>
Message-ID: <87y7420zuc.fsf@benfinney.id.au>

"M.-A. Lemburg" <mal at egenix.com> writes:

> The PEP doesn't mention changing the module name and deprecating
> the old one.

Right. The intention is to have a PEP-8-conformant 'unittest' module,
not an entirely new module.

> Instead it wants to deprecate all the old names (and cites PEP 4 for
> this), but keeping the module name.

Yes, because all the existing documented API is retained as is, just
with new names that conform with PEP 8.

> Note that PEP 4 targets deprecating use of whole modules, not single
> APIs, or - like in your case - more or less the complete existing
> API of a module.

This is true, I tried to be specific about what was to be done to
deprecate individual attributes, and could only find PEP 4. Is there a
better reference for deprecation of Python features?

> Given the scope of the changes, you are really creating a completely
> new API and as a result should also get a new module name.

I disagree. The API is not "completely new"; it has exactly the same
functionality and expected behaviour, just with a different set of
names.

> Developers who don't feel like changing 10000+ tests can then
> continue to use the old module and start using the new module for
> new projects.

I agree with Michael Foord that an extensively-used standard library
module is an argument in favour of re-working that module to be in
line with the standard library guidelines.

The change is, as has been pointed out elsewhere, a replacement of one
set of names with another, retaining exactly the same expected
behaviour. It's not on the scale of deprecating usage of an entire
module.

-- 
 \         ?I went to the museum where they had all the heads and arms |
  `\      from the statues that are in all the other museums.? ?Steven |
_o__)                                                           Wright |
Ben Finney


From solipsis at pitrou.net  Wed Jul 16 14:40:22 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Jul 2008 12:40:22 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP=3A_Consolidating_names_and_classes_in?=
	=?utf-8?q?=09the=09=60unittest=60_module_=28updated_2008-07-15=29?=
References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol>
	<873ama5xze.fsf@benfinney.id.au>
	<loom.20080716T100023-565@post.gmane.org>
	<873ama2f0i.fsf@benfinney.id.au>
Message-ID: <loom.20080716T123508-851@post.gmane.org>

Ben Finney <ben+python <at> benfinney.id.au> writes:
> > * http://www.thefreedictionary.com/fail
> > "To prove deficient or lacking; perform ineffectively or inadequately; To be
> > unsuccessful"
> 
> Yes. It's a verb, not a negative modifer. To use it with a negative
> like "not" is not creating a "double negative".

Ok, let's stop it there. I'm tired of trying to point the obvious.




From ben+python at benfinney.id.au  Wed Jul 16 15:00:51 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 23:00:51 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<487A8700.8000200@voidspace.org.uk>
	<87mykka8el.fsf@benfinney.id.au>
	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>
	<487B60FB.5010602@voidspace.org.uk>
	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<487DE5E0.4060406@ronadam.com>
Message-ID: <87prpe0yrg.fsf@benfinney.id.au>

Ron Adam <rrr at ronadam.com> writes:

> +1 for a simpler testing module.

I've no objection.

> Just letting you know there is interest in a lighter weight testing
> suite.

'doctest' is a very simple testing module, that is a very useful tool.

> Looking at the unittest discussions, it doesn't look like it is
> getting easier to use, but more complex.

How so?

One PEP proposed this week specifies to do nothing but conform
'unittest' with the standard library guidelines, and remove redundant
names. That surely makes it simpler to use.

Another PEP specifies to add helper methods that simplify a number of
common cases.

What is it you see making unittest "more complex to use"?

> Py.test looks very interesting, especially the test discovery parts.
> I also agree we don't need special methods for every variation of
> assertedness.

My main complaint about 'py.test' is that it's yet-another-framework.
We already have 'doctest' and 'unittest', and they play together
reasonably well.

'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks
better for consideration, especially since it integrates well with
'unittest'.

> I've been thinking that a few decorators may go a long way to making
> writing tests easy. It also reduces the level of indentation needed.

There are a number of these already in 'nose'.

-- 
 \        ?I fly Air Bizarre. You buy a combination one-way round-trip |
  `\    ticket. Leave any Monday, and they bring you back the previous |
_o__)     Friday. That way you still have the weekend.? ?Steven Wright |
Ben Finney


From Scott.Daniels at Acm.Org  Wed Jul 16 15:13:53 2008
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 16 Jul 2008 06:13:53 -0700
Subject: [Python-Dev] PEP: Frequently-requested additional features for
 the `unittest` module
In-Reply-To: <87abgi4bxi.fsf@benfinney.id.au>
References: <87fxqa62xb.fsf@benfinney.id.au> <87abgi4bxi.fsf@benfinney.id.au>
Message-ID: <g5krih$4f8$1@ger.gmane.org>

Ben Finney wrote:
...

>         def assert_compare_true(op, first, second, msg=None):
>             if msg is None:
>                 msg = "%(first)r %(op)r %(second)" % vars()
>             if not op(first, second):
>                 raise self.failure_exception(msg)

I would rather something more like:

       def assert_compare_true(op, first, second, msg=None):
           if op(first, second):
               return
           raise self.failure_exception(msg)
           if msg is None:
               self.failure_exception("%(first)r %(op)r %(second)"
                                          % vars())
           self.failure_exception("%(first)r %(op)r %(second): %(msg)"
                                  % vars())


--Scott David Daniels
Scott.Daniels at Acm.Org


From ben+python at benfinney.id.au  Wed Jul 16 15:12:03 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 23:12:03 +1000
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
	<487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au>
	<487DD973.7010007@egenix.com> <487DE35E.2070501@voidspace.org.uk>
	<487DEB5D.7010507@egenix.com>
Message-ID: <87lk020y8s.fsf@benfinney.id.au>

"M.-A. Lemburg" <mal at egenix.com> writes:

> On 2008-07-16 14:02, Michael Foord wrote:
> > M.-A. Lemburg wrote:
> >> Note that PEP 4 targets deprecating use of whole modules, not
> >> single APIs, or - like in your case - more or less the complete
> >> existing API of a module.
> >
> > Which PEP is usually referenced for the deprecation of individual
> > APIs?
> 
> PEP 5 could be used for that.

That seems an even worse fit; it speaks of changing language features,
not library modules. At least PEP 4 talks about when to raise
DeprecationWarning.

> Adding several 10s of deprecation warnings to the unittest module is
> not going to make life easier for anyone. Adding just a single one
> on import and following PEP 4 is.

I don't see how the first "is not going to make life easier" if the
second somehow is. Is a programmer going to be helpless in the face of
some DeprecationWarnings but not others?

> If you do want to apply major changes to a module without changing
> the name, then this could be done as part of the 2.x -> 3.x
> transition.

This has already been rejected
<URL:http://mail.python.org/pipermail/python-dev/2008-April/078485.html>.

I'm inclined to agree that it's not right for 2.x. I'll revise the PEP
accordingly.

-- 
 \     ?First they came for the verbs, and I said nothing, for verbing |
  `\    weirds language. Then, they arrival for the nouns and I speech |
_o__)                           nothing, for I no verbs.? ?Peter Ellis |
Ben Finney


From barry at python.org  Wed Jul 16 15:18:19 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 16 Jul 2008 09:18:19 -0400
Subject: [Python-Dev] Reminder: beta 2's schedule for tomorrow
In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org>
Message-ID: <53E31F7D-875C-466C-B5F6-2D3D274847D4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 15, 2008, at 8:32 AM, Barry Warsaw wrote:

> If there is anything you need a decision on, please follow up to  
> this thread.  I'm inundated with email so I can't watch every thread  
> on the mailing lists.  Or ping me on #python-dev.

I'm not currently optimistic that we can release today.

As I see it we have several problems serious enough to delay a  
release.  First, we have no green 2.6 or 3.0 buildbots

http://www.python.org/dev/buildbot/stable/

It looks like few of the buildbots for 2.6 are actually online, but  
still, the 3.0 buildbots are all failing.

We still have, and have had for a long time now, serious problems with  
the multiprocessing module:

http://bugs.python.org/issue?@columns=title,id,activity,versions,status&@sort=activity&@filter=priority,status&@pagesize=50&@startwith=0&priority=1&status=1&@dispname=Showstoppers

I noticed today that _multiprocessing.so has build problems on OS X  
10.5 and Ubuntu 8.04.  I opened a separate issue about that so as not  
to get lost in the other bug about test_multiprocessing hanging.  As  
evidenced by the thread for issue 3088, lots of people have put a lot  
of work into this module, but unfortunately we're still not there.   
These seem serious enough to me to hold up the release, especially  
since we have only one more beta left.

Speaking of which, we have a number of deferred blockers:

http://bugs.python.org/issue?@columns=title,id,activity,status&@sort=activity&@group=priority&@filter=priority,status&@pagesize=50&@startwith=0&priority=6&status=1&@dispname=Deferred%20Blockers

These will not block beta 2, but they /will/ block beta 3, so they  
really need some attention.

Please everyone, if you have only a little bit of time to work on  
Python, I hope you will attack the release critical and deferred  
blocker issues, and work on turning the buildbots green.  These are  
the top priorities in order to get 2.6 and 3.0 out on time.  And just  
as added incentive, our October 1st goal is being noted by downstream  
vendors.  I've been told that "Apple is willing to take the new Python  
if it is GM on schedule by Oct 1st, but may not if it is delayed"  
though you should not infer anything about Apple's schedule from this.

Still, we won't sacrifice quality in order to hit the October 1st  
goal.  After beta 2, I start getting mean. :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSH31HHEjvBPtnXfVAQIH+AQAtkRJo0uhvZ300jq0YSR6ezUU0tMHJwmd
pwLWwZWDqPyh3/ohKwrajdGm5hYS8gOnTo+k2XEEJjQJdLMTblzZ3ArsGfhXHqbV
GPeKK/6BYT6SOD75ubINq+jSsu6Pvjsadbk/cp/x53WwHL8kX40D0YQBhp3KQRrz
zgFmfVFPcys=
=3SDN
-----END PGP SIGNATURE-----

From ben+python at benfinney.id.au  Wed Jul 16 15:26:12 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 23:26:12 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for
	the `unittest` module
References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com>
	<87bq0y60se.fsf@benfinney.id.au>
	<bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com>
	<487DEA02.2070508@voidspace.org.uk>
Message-ID: <87hcaq0xl7.fsf@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> I'm inclined to agree. It was part of a set of additions suggested
> by Guido. From here I think (as part of the unittest extensions that
> google maintains):
> 
> http://mail.python.org/pipermail/python-dev/2008-April/078702.html
> 
> I've used tests like that when implementing numeric objects, which
> has been several times - but only a tiny proportion of the tests
> I've written.
> 
> I wouldn't be sorry not to include them.

Okay. I'll revise the PEP to remove all these lt, gt, le, ge
comparison tests, and note that they are supported entirely by
``assert_compare_true`` and ``assert_compare_false`` with a specific
comparison operator.

-- 
 \     ?Many are stubborn in pursuit of the path they have chosen, few |
  `\                     in pursuit of the goal.? ?Friedrich Nietzsche |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Wed Jul 16 15:36:32 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 23:36:32 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for
	the `unittest` module
References: <87fxqa62xb.fsf@benfinney.id.au> <87abgi4bxi.fsf@benfinney.id.au>
	<g5krih$4f8$1@ger.gmane.org>
Message-ID: <87d4le0x3z.fsf@benfinney.id.au>

Scott David Daniels <Scott.Daniels at Acm.Org> writes:

> I would rather something more like:
> 
>       def assert_compare_true(op, first, second, msg=None):
>           if op(first, second):
>               return
>           raise self.failure_exception(msg)
>           if msg is None:
>               self.failure_exception("%(first)r %(op)r %(second)"
>                                          % vars())
>           self.failure_exception("%(first)r %(op)r %(second): %(msg)"
>                                  % vars())

I'm confused. It appears to me that your version never gets past the
first 'raise' statement, which is unconditional; and the rest seems to
do nothing but instantiate exceptions without using them.

Do you perhaps mean something like this::

    def assert_compare_true(op, first, second, msg=None):
        fail_detail = "%(first)r %(op)r %(second)r" % vars()
        if msg is None:
            msg = fail_detail
        else:
            msg = "%(fail_detail)s: %(msg)s" % vars()
        if not op(first, second):
            raise self.failure_exception(msg)

If so, that sems to be in line with the "Enhanced failure message"
principle exercised elsewhere in the same PEP, i.e. that the failure
message should *always* contain certain information, even if a message
is specified by the caller.

One downside I can see is that, in optimising for this common case, it
makes the function useless to someone who wants to specify their own
failure message exactly, without the specific extra information in
that specific format.

What do others think of this?

-- 
 \    ?Holy contributing to the delinquency of minors, Batman!? ?Robin |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From rbp at isnomore.net  Wed Jul 16 15:35:39 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Wed, 16 Jul 2008 10:35:39 -0300
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
In-Reply-To: <874p6q7oxo.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<487CE248.9030400@palladion.com> <874p6q7oxo.fsf@benfinney.id.au>
Message-ID: <20080716133539.GA16835@isnomore.net>

On Tue, Jul 15 2008 at 07:38:59PM BRT, Ben Finney <ben+python at benfinney.id.au> wrote:
> Tres Seaver <tseaver at palladion.com> writes:
> 
> > I would keep both by preference, rather than insist on a "foolish
> > consistency."

+1

> The "consistency" argument leads to the PEP 8 names. The removal of
> redundant names is not made in the name of consistency, but of API
> simplicity.

Isn't this *enourmous* discussion a clear sign that a lot of people *won't*
find a trimmed-down API simpler? If assert* to fail* mapping is consistent,
then voil?, simple, everyone can remember method names (+1 for PEP-8
renaming), everyone is free to think of each test as they please.


	rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From ben+python at benfinney.id.au  Wed Jul 16 15:54:26 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 16 Jul 2008 23:54:26 +1000
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<487CE248.9030400@palladion.com> <874p6q7oxo.fsf@benfinney.id.au>
	<20080716133539.GA16835@isnomore.net>
Message-ID: <878ww20wa5.fsf@benfinney.id.au>

Rodrigo Bernardo Pimentel <rbp at isnomore.net> writes:

> On Tue, Jul 15 2008 at 07:38:59PM BRT, Ben Finney <ben+python at benfinney.id.au> wrote:
> > The "consistency" argument leads to the PEP 8 names. The removal
> > of redundant names is not made in the name of consistency, but of
> > API simplicity.
> 
> Isn't this *enourmous* discussion a clear sign that a lot of people
> *won't* find a trimmed-down API simpler?

The majority of this discussion hasn't been about whether or not to
trim the API, so I'd have to answer no. Instead, it shows that people
have different ideas of how it should be done.

> If assert* to fail* mapping is consistent, then voil?, simple,
> everyone can remember method names (+1 for PEP-8 renaming), everyone
> is free to think of each test as they please.

Thanks for the feedback.

-- 
 \          ?Pity the meek, for they shall inherit the earth.? ?Donald |
  `\                                              Robert Perry Marquis |
_o__)                                                                  |
Ben Finney


From rbp at isnomore.net  Wed Jul 16 16:26:30 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Wed, 16 Jul 2008 11:26:30 -0300
Subject: [Python-Dev] PEP: Consolidating names and classes in
	the	`unittest` module (updated 2008-07-15)
In-Reply-To: <878ww20wa5.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au>
	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87fxqb8mlp.fsf@benfinney.id.au>
	<20080715130533.GB17917@steerpike.home.puzzling.org>
	<487CE248.9030400@palladion.com> <874p6q7oxo.fsf@benfinney.id.au>
	<20080716133539.GA16835@isnomore.net>
	<878ww20wa5.fsf@benfinney.id.au>
Message-ID: <20080716142629.GA19934@isnomore.net>

On Wed, Jul 16 2008 at 10:54:26AM BRT, Ben Finney <ben+python at benfinney.id.au> wrote:
> Rodrigo Bernardo Pimentel <rbp at isnomore.net> writes:
> 
> > On Tue, Jul 15 2008 at 07:38:59PM BRT, Ben Finney <ben+python at benfinney.id.au> wrote:
> > > The "consistency" argument leads to the PEP 8 names. The removal
> > > of redundant names is not made in the name of consistency, but of
> > > API simplicity.
> > 
> > Isn't this *enourmous* discussion a clear sign that a lot of people
> > *won't* find a trimmed-down API simpler?
> 
> The majority of this discussion hasn't been about whether or not to
> trim the API, so I'd have to answer no. Instead, it shows that people
> have different ideas of how it should be done.

Ok, I've oversimplified my sentence. In more words, what I meant was that I
don't think the benefits of getting rid of assert* or fail* methods are very
clear (thus my "+1" on Tres's suggestion of leaving both), and it doesn't
seem to me like we're progressing in the direction of reaching an agreement
(though there has been great exposition of arguments). I mean, the benefits
of removing fail* seem clear for some people, but are strongly opposed by
others, and even the benefits of removing assert* seem clear for some
(IIRC).

All this lead to my previous sentence, which should read: isn't this
*enourmous* discussion a clear sign that a lot of people *won't* find an API
without fail* (or assert*) simpler? I agree there's trimming to be done, of
course. I'm just saying that there's clearly no agreement on this particular
point of whether to remove fail* completely, so maybe we should take it as
as indication that it might not actually simplify the API (in the sense of
how comfortable people are with using it).

> > If assert* to fail* mapping is consistent, then voil?, simple,
> > everyone can remember method names (+1 for PEP-8 renaming), everyone
> > is free to think of each test as they please.
> 
> Thanks for the feedback.

No problem! It's great to be in a community where these things are so openly
discussed :)


	rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From ncoghlan at gmail.com  Wed Jul 16 16:29:45 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Jul 2008 00:29:45 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for
 the `unittest` module
In-Reply-To: <87d4le0x3z.fsf@benfinney.id.au>
References: <87fxqa62xb.fsf@benfinney.id.au>
	<87abgi4bxi.fsf@benfinney.id.au>	<g5krih$4f8$1@ger.gmane.org>
	<87d4le0x3z.fsf@benfinney.id.au>
Message-ID: <487E05D9.4050600@gmail.com>

Ben Finney wrote:
> Do you perhaps mean something like this::
> 
>     def assert_compare_true(op, first, second, msg=None):
>         fail_detail = "%(first)r %(op)r %(second)r" % vars()
>         if msg is None:
>             msg = fail_detail
>         else:
>             msg = "%(fail_detail)s: %(msg)s" % vars()
>         if not op(first, second):
>             raise self.failure_exception(msg)
> 
> If so, that sems to be in line with the "Enhanced failure message"
> principle exercised elsewhere in the same PEP, i.e. that the failure
> message should *always* contain certain information, even if a message
> is specified by the caller.

I was going to suggest the same thing, so this sounds like a good idea 
to me. The other point I was going to make is that repr(operator.lt) is 
actually "<built-in function lt>". It may be better to just make a 
general method for asserting that an operation gives an expected result 
and gives the name of the operation and the details of the arguments and 
expected result if anything goes wrong:

     def assert_op(op, *args, msg=None, expected=True):
         fail_detail = "%r%r != %r" % (op.__name__, arg, expect)
         if msg is None:
             msg = fail_detail
         else:
             msg = "%s: %s" % (msg, fail_detail)
         if op(*args) != expected:
             raise self.failure_exception(msg)

> One downside I can see is that, in optimising for this common case, it
> makes the function useless to someone who wants to specify their own
> failure message exactly, without the specific extra information in
> that specific format.

If you don't want the extra details in the error messages then you can 
just use the basic self.assert_ method - the only advantage to use the 
other assertion methods is they provide additional details in the 
assertion error. (except assert_raises, where it provides the try/catch 
block as well)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From eric+python-dev at trueblade.com  Wed Jul 16 16:35:16 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Wed, 16 Jul 2008 10:35:16 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
Message-ID: <487E0724.6020002@trueblade.com>

Does anyone know why 'F' is the same as 'f'?  Wouldn't it make more 
sense to either drop it, or make it convert the exponent to upper case 
(like 'E' and 'G')?  Compatibility with %-formatting is the only reason 
I can think of to keep up, but I get the sense we've given up on an 
automatic conversion from %-formatting to str.format().  Plus, I can 
find no uses of '%F' in the standard library.

And really, if you could write something to automatically convert 
%-formatting to str.format(), you'd be able to make 'F' to 'f', anyway.

I know it's a nit, but I'm reviewing the documentation and it sticks out.

From dickinsm at gmail.com  Wed Jul 16 17:07:03 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 16 Jul 2008 16:07:03 +0100
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <487E0724.6020002@trueblade.com>
References: <487E0724.6020002@trueblade.com>
Message-ID: <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>

On Wed, Jul 16, 2008 at 3:35 PM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Does anyone know why 'F' is the same as 'f'?  Wouldn't it make more sense to
> either drop it, or make it convert the exponent to upper case

What exponent?  Isn't the point of 'f' formatting that there is no exponent?

In C, the only difference seems to be that a NaN or infinity formatted with '%F'
is turned into "NAN" or "INF" instead of "nan" or "inf".

Mark

From daniel at stutzbachenterprises.com  Wed Jul 16 17:14:23 2008
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 16 Jul 2008 10:14:23 -0500
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>
	<5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>
Message-ID: <eae285400807160814p16d3bab3w82dff27d69c9a80@mail.gmail.com>

On Wed, Jul 16, 2008 at 10:07 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Wed, Jul 16, 2008 at 3:35 PM, Eric Smith
> <eric+python-dev at trueblade.com> wrote:
>> Does anyone know why 'F' is the same as 'f'?  Wouldn't it make more sense to
>> either drop it, or make it convert the exponent to upper case
>
> What exponent?  Isn't the point of 'f' formatting that there is no exponent?

There's no exponent for small-magnitude numbers, but still an exponent
for large-magnitude numbers:

>>> '%f' % (10**100)
'1e+100'

-- 
Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC

From collinw at gmail.com  Wed Jul 16 17:15:13 2008
From: collinw at gmail.com (Collin Winter)
Date: Wed, 16 Jul 2008 08:15:13 -0700
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <487D48F9.3000903@voidspace.org.uk>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
	<487D48F9.3000903@voidspace.org.uk>
Message-ID: <43aa6ff70807160815m21fc967ah976af182f9402e95@mail.gmail.com>

On Tue, Jul 15, 2008 at 6:03 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Collin Winter wrote:
>>
>> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au>
>> wrote:
>>>
>>> Backwards Compatibility
>>> =======================
>>>
>>> The names to be obsoleted should be deprecated and removed according
>>> to the schedule for modules in PEP 4 [#PEP-4]_.
>>>
>>> While deprecated, use of the deprecated attributes should raise a
>>> ``DeprecationWarning``, with a message stating which replacement name
>>> should be used.
>>>
>>
>> Is any provision being made for a 2to3 fixer/otherwise-automated
>> transition for the changes you propose here?
>>
>
> As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed?

IMO some kind of automated transition tool is needed -- anyone who has
the time to convert their codebase by hand (for some definition of "by
hand" that involves sed) doesn't have enough to do.

Collin

From eric+python-dev at trueblade.com  Wed Jul 16 17:15:40 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Wed, 16 Jul 2008 11:15:40 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>
	<5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>
Message-ID: <487E109C.7050806@trueblade.com>

Mark Dickinson wrote:
> On Wed, Jul 16, 2008 at 3:35 PM, Eric Smith
> <eric+python-dev at trueblade.com> wrote:
>> Does anyone know why 'F' is the same as 'f'?  Wouldn't it make more sense to
>> either drop it, or make it convert the exponent to upper case
> 
> What exponent?  Isn't the point of 'f' formatting that there is no exponent?

There's no exponent until the number gets large.  I haven't looked up 
how big the number has to get.  On my Mac, it's somewhere between 1e50 
and 1e60.

$ ./python.exe
Python 3.0b1+ (py3k:64984:64985, Jul 15 2008, 20:17:06)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> '%f' % 1e100
'1e+100'
 >>> '%F' % 1e100
'1e+100'
 >>> '%f' % 1.2
'1.200000'

str.format() works the same.

> In C, the only difference seems to be that a NaN or infinity formatted with '%F'
> is turned into "NAN" or "INF" instead of "nan" or "inf".

Not so in Python.
 >>> '%f' % float('nan')
'nan'
 >>> '%F' % float('nan')
'nan'


But it is the case with 'e':
 >>> '%e' % float('nan')
'nan'
 >>> '%E' % float('nan')
'NAN'

From collinw at gmail.com  Wed Jul 16 17:16:38 2008
From: collinw at gmail.com (Collin Winter)
Date: Wed, 16 Jul 2008 08:16:38 -0700
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <487DE7D8.6050308@voidspace.org.uk>
References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au>
	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
	<487D48F9.3000903@voidspace.org.uk> <g5k3q7$ig7$1@ger.gmane.org>
	<487DE7D8.6050308@voidspace.org.uk>
Message-ID: <43aa6ff70807160816p87ee58dlca68168692053dc8@mail.gmail.com>

On Wed, Jul 16, 2008 at 5:21 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Terry Reedy wrote:
>>
>>
>> Michael Foord wrote:
>>>
>>> Collin Winter wrote:
>>
>>>> Is any provision being made for a 2to3 fixer/otherwise-automated
>>>> transition for the changes you propose here?
>>>>
>>>
>>> As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed?
>>
>> A fixer will only be needed when it actually is needed, but when it is, it
>> should be a unittest-name fixer since previous 3.x code will also need
>> fixing.  Since the duplicates are multiples names for the same objects, the
>> fixer should be a trivial name substitution.
>
> Can 2to3 fixers be used for 2to2 and 3to3 translation then?

The intention is for the infrastructure behind 2to3 to be generally
reusable for other Python source-to-source translation tools, be that
2to2 or 3to3. That hasn't fully materialized yet, but it's getting
there.

Collin

From dickinsm at gmail.com  Wed Jul 16 17:17:26 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 16 Jul 2008 16:17:26 +0100
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <eae285400807160814p16d3bab3w82dff27d69c9a80@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>
	<5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>
	<eae285400807160814p16d3bab3w82dff27d69c9a80@mail.gmail.com>
Message-ID: <5c6f2a5d0807160817u1126b13cm542b42000fddbf47@mail.gmail.com>

On Wed, Jul 16, 2008 at 4:14 PM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> There's no exponent for small-magnitude numbers, but still an exponent
> for large-magnitude numbers:
>
>>>> '%f' % (10**100)
> '1e+100'

So there is!  Thanks for the correction.

Mark

From dickinsm at gmail.com  Wed Jul 16 17:51:39 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 16 Jul 2008 16:51:39 +0100
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <487E109C.7050806@trueblade.com>
References: <487E0724.6020002@trueblade.com>
	<5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>
	<487E109C.7050806@trueblade.com>
Message-ID: <5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com>

On Wed, Jul 16, 2008 at 4:15 PM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> There's no exponent until the number gets large.  I haven't looked up how
> big the number has to get.  On my Mac, it's somewhere between 1e50 and 1e60.

I think it's around 1e50, courtesy of the rather oddly-phrased line in
unicodeobject.c
(this is in py3k) that looks like:

    if (type == 'f' && (fabs(x) / 1e25) >= 1e25)

In any case, I agree that the current 'F' is strange.  Even after
having read the
relevant line of PEP 3101 several times in the past, part of my brain still
believes that something formatted with 'F' should have all letters appearing
in upper case.

Mark

From mal at egenix.com  Wed Jul 16 18:36:58 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Jul 2008 18:36:58 +0200
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <87lk020y8s.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>	<487DAAA0.1040604@egenix.com>
	<87od4y2qky.fsf@benfinney.id.au>	<487DD973.7010007@egenix.com>
	<487DE35E.2070501@voidspace.org.uk>	<487DEB5D.7010507@egenix.com>
	<87lk020y8s.fsf@benfinney.id.au>
Message-ID: <487E23AA.9010400@egenix.com>

On 2008-07-16 15:12, Ben Finney wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>> On 2008-07-16 14:02, Michael Foord wrote:
>>> M.-A. Lemburg wrote:
>>>> Note that PEP 4 targets deprecating use of whole modules, not
>>>> single APIs, or - like in your case - more or less the complete
>>>> existing API of a module.
>>> Which PEP is usually referenced for the deprecation of individual
>>> APIs?
>> PEP 5 could be used for that.
> 
> That seems an even worse fit; it speaks of changing language features,
> not library modules. At least PEP 4 talks about when to raise
> DeprecationWarning.

Right and the methods described there are usually also applied
to language changes and API changes.

I just wanted to make clear that your "...see PEP 4 for how to handle
backwards compatibility..." statement doesn't apply to the changes
described in your PEP. However, it does point at a possible compromise
which would make the transition easier on everyone.

>> Adding several 10s of deprecation warnings to the unittest module is
>> not going to make life easier for anyone. Adding just a single one
>> on import and following PEP 4 is.
> 
> I don't see how the first "is not going to make life easier" if the
> second somehow is. Is a programmer going to be helpless in the face of
> some DeprecationWarnings but not others?

Using the first method (changing the API names), you force
developers to change existing code, which results in testing
the test code. Lots of work with no real benefit.

With the second method, they can use the new names with new test code
(which then imports the new module). They don't have to test
their existing tests for obscure search&replace errors.

>> If you do want to apply major changes to a module without changing
>> the name, then this could be done as part of the 2.x -> 3.x
>> transition.
> 
> This has already been rejected
> <URL:http://mail.python.org/pipermail/python-dev/2008-April/078485.html>

I wasn't suggesting to apply to the change to 3.0, but instead
suggesting that if you want to implement such a major API change,
this should be done only on the 3.x branch and be dealt with in the
2to3 tool.

> I'm inclined to agree that it's not right for 2.x. I'll revise the PEP
> accordingly.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 16 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From rrr at ronadam.com  Wed Jul 16 18:56:33 2008
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 16 Jul 2008 11:56:33 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <87prpe0yrg.fsf@benfinney.id.au>
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>	<87mykka8el.fsf@benfinney.id.au>	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>	<487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>	<487DE5E0.4060406@ronadam.com>
	<87prpe0yrg.fsf@benfinney.id.au>
Message-ID: <487E2841.1050101@ronadam.com>



Ben Finney wrote:
> Ron Adam <rrr at ronadam.com> writes:
> 
>> +1 for a simpler testing module.
> 
> I've no objection.
> 
>> Just letting you know there is interest in a lighter weight testing
>> suite.
> 
> 'doctest' is a very simple testing module, that is a very useful tool.
> 
>> Looking at the unittest discussions, it doesn't look like it is
>> getting easier to use, but more complex.
> 
> How so?
> 
> One PEP proposed this week specifies to do nothing but conform
> 'unittest' with the standard library guidelines, and remove redundant
> names. That surely makes it simpler to use.

No complaint here.  It's a good place to start.


> Another PEP specifies to add helper methods that simplify a number of
> common cases.

In my opinion adding 19 more methods makes it more complex.

I'd rather see more focus on handling test failures in general without the 
need for so many special helper functions.



> What is it you see making unittest "more complex to use"?

More methods and method signatures to learn and remember.


   assert_true(?)

   assert_false(?)

   assert_almost_equal(?)

   assert_equal(?)

   assert_not_almost_equal(?)

   assert_not_equal(?)

   assert_raises(exc_class, callable_obj, *args, **kwargs)

   assert_compare_true(op, first, second, msg=None)

   assert_in(container, member, msg=None)

   assert_is(first, second, msg=None)

   assert_less_than(first, second, msg=None)

   assert_greater_than(first, second, msg=None)

   assert_less_than_or_equal(first, second, msg=None)

   assert_greater_than_or_equal(first, second, msg=None)

   assert_members_equal(first, second, msg=None)

   assert_sequence_equal(first, second, msg=None)

   assert_raises_with_message_regex(exc_class, message_regex,
                callable_obj, *args, **kwargs)

   assert_compare_false(op, first, second, msg=None)

   assert_not_in(container, member, msg=None)

   assert_is_not(first, second, msg=None)

   assert_not_less_than(first, second, msg=None)

   assert_not_greater_than(first, second, msg=None)

   assert_not_less_than_or_equal(first, second, msg=None)

   assert_not_greater_than_or_equal(first, second, msg=None)

   assert_members_not_equal(first, second, msg=None)

   assert_sequence_not_equal(first, second, msg=None)



That comes to 26 variations of assert.



There are really only a small set of basic conditions to test for.

      correct values
      incorrect values
      unexpected exceptions
      correct exceptions
      incorrect exceptions
      missing exceptions


I think the unittest module could better handle testing of exceptions and 
distinguishing exception produced by test code from the code being tested. 
  That was painfully clear (to me) when we where fixing all the Python 3000 
tests.


>> Py.test looks very interesting, especially the test discovery parts.
>> I also agree we don't need special methods for every variation of
>> assertedness.
> 
> My main complaint about 'py.test' is that it's yet-another-framework.
> We already have 'doctest' and 'unittest', and they play together
> reasonably well.

I love doctest.  Because it is very different in how it works, I don't 
think it competes with more formal testing methods.


> 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks
> better for consideration, especially since it integrates well with
> 'unittest'.

Thanks for the link!  I'll give 'nose' a try next time I write tests.

>> I've been thinking that a few decorators may go a long way to making
>> writing tests easy. It also reduces the level of indentation needed.
> 
> There are a number of these already in 'nose'.

Yes, I looked at the decorator and think it is very nice and simple to use.


If something like "nose" was a submodule of unittest, unittest may be able 
to use some of it's parts.  Maybe gradually the more java like unittest 
classes and methods could be depreciated?


Cheers,
    Ron














From rrr at ronadam.com  Wed Jul 16 18:56:33 2008
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 16 Jul 2008 11:56:33 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <87prpe0yrg.fsf@benfinney.id.au>
References: <20080713093936.GA3623@benfinney.id.au>	<487A8700.8000200@voidspace.org.uk>	<87mykka8el.fsf@benfinney.id.au>	<ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1>	<487B60FB.5010602@voidspace.org.uk>	<87iqv89kj5.fsf@benfinney.id.au>	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>	<487DE5E0.4060406@ronadam.com>
	<87prpe0yrg.fsf@benfinney.id.au>
Message-ID: <487E2841.1050101@ronadam.com>



Ben Finney wrote:
> Ron Adam <rrr at ronadam.com> writes:
> 
>> +1 for a simpler testing module.
> 
> I've no objection.
> 
>> Just letting you know there is interest in a lighter weight testing
>> suite.
> 
> 'doctest' is a very simple testing module, that is a very useful tool.
> 
>> Looking at the unittest discussions, it doesn't look like it is
>> getting easier to use, but more complex.
> 
> How so?
> 
> One PEP proposed this week specifies to do nothing but conform
> 'unittest' with the standard library guidelines, and remove redundant
> names. That surely makes it simpler to use.

No complaint here.  It's a good place to start.


> Another PEP specifies to add helper methods that simplify a number of
> common cases.

In my opinion adding 19 more methods makes it more complex.

I'd rather see more focus on handling test failures in general without the 
need for so many special helper functions.



> What is it you see making unittest "more complex to use"?

More methods and method signatures to learn and remember.


   assert_true(?)

   assert_false(?)

   assert_almost_equal(?)

   assert_equal(?)

   assert_not_almost_equal(?)

   assert_not_equal(?)

   assert_raises(exc_class, callable_obj, *args, **kwargs)

   assert_compare_true(op, first, second, msg=None)

   assert_in(container, member, msg=None)

   assert_is(first, second, msg=None)

   assert_less_than(first, second, msg=None)

   assert_greater_than(first, second, msg=None)

   assert_less_than_or_equal(first, second, msg=None)

   assert_greater_than_or_equal(first, second, msg=None)

   assert_members_equal(first, second, msg=None)

   assert_sequence_equal(first, second, msg=None)

   assert_raises_with_message_regex(exc_class, message_regex,
                callable_obj, *args, **kwargs)

   assert_compare_false(op, first, second, msg=None)

   assert_not_in(container, member, msg=None)

   assert_is_not(first, second, msg=None)

   assert_not_less_than(first, second, msg=None)

   assert_not_greater_than(first, second, msg=None)

   assert_not_less_than_or_equal(first, second, msg=None)

   assert_not_greater_than_or_equal(first, second, msg=None)

   assert_members_not_equal(first, second, msg=None)

   assert_sequence_not_equal(first, second, msg=None)



That comes to 26 variations of assert.



There are really only a small set of basic conditions to test for.

      correct values
      incorrect values
      unexpected exceptions
      correct exceptions
      incorrect exceptions
      missing exceptions


I think the unittest module could better handle testing of exceptions and 
distinguishing exception produced by test code from the code being tested. 
  That was painfully clear (to me) when we where fixing all the Python 3000 
tests.


>> Py.test looks very interesting, especially the test discovery parts.
>> I also agree we don't need special methods for every variation of
>> assertedness.
> 
> My main complaint about 'py.test' is that it's yet-another-framework.
> We already have 'doctest' and 'unittest', and they play together
> reasonably well.

I love doctest.  Because it is very different in how it works, I don't 
think it competes with more formal testing methods.


> 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks
> better for consideration, especially since it integrates well with
> 'unittest'.

Thanks for the link!  I'll give 'nose' a try next time I write tests.

>> I've been thinking that a few decorators may go a long way to making
>> writing tests easy. It also reduces the level of indentation needed.
> 
> There are a number of these already in 'nose'.

Yes, I looked at the decorator and think it is very nice and simple to use.


If something like "nose" was a submodule of unittest, unittest may be able 
to use some of it's parts.  Maybe gradually the more java like unittest 
classes and methods could be depreciated?


Cheers,
    Ron














From guido at python.org  Wed Jul 16 19:02:35 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 10:02:35 -0700
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
	<ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
	<1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>
	<g5j80e$g4d$1@ger.gmane.org>
	<bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com>
	<ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com>
Message-ID: <ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com>

On Tue, Jul 15, 2008 at 11:50 PM, engelbert gruber
<grubert at users.sourceforge.net> wrote:
> I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes
> ``DeprecationWarning: callable() not supported in 3.x;`` .

Good catch, Engelbert.

But why would has_key() need a warning when 2to3 will fix it just fine?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jul 16 19:25:29 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 10:25:29 -0700
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <487E0724.6020002@trueblade.com>
References: <487E0724.6020002@trueblade.com>
Message-ID: <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>

On Wed, Jul 16, 2008 at 7:35 AM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Does anyone know why 'F' is the same as 'f'?  Wouldn't it make more sense to
> either drop it, or make it convert the exponent to upper case (like 'E' and
> 'G')?  Compatibility with %-formatting is the only reason I can think of to
> keep up, but I get the sense we've given up on an automatic conversion from
> %-formatting to str.format().  Plus, I can find no uses of '%F' in the
> standard library.

My best guess as to why 'F' is the same as 'f' is that somebody
(could've been me :-) thought, like several others in this thread,
that %f never prints an exponent. I agree that making it emit an 'E'
when an exponent is used is the right thing to do. Do it now!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric+python-dev at trueblade.com  Wed Jul 16 19:30:24 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Wed, 16 Jul 2008 13:30:24 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>
	<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>
Message-ID: <487E3030.5010203@trueblade.com>

Guido van Rossum wrote:
> On Wed, Jul 16, 2008 at 7:35 AM, Eric Smith
> <eric+python-dev at trueblade.com> wrote:
>> Does anyone know why 'F' is the same as 'f'?  Wouldn't it make more sense to
>> either drop it, or make it convert the exponent to upper case (like 'E' and
>> 'G')?  Compatibility with %-formatting is the only reason I can think of to
>> keep up, but I get the sense we've given up on an automatic conversion from
>> %-formatting to str.format().  Plus, I can find no uses of '%F' in the
>> standard library.
> 
> My best guess as to why 'F' is the same as 'f' is that somebody
> (could've been me :-) thought, like several others in this thread,
> that %f never prints an exponent. I agree that making it emit an 'E'
> when an exponent is used is the right thing to do. Do it now!
> 

It shares code with %-formatting.  Change that, too?  I couldn't find 
any occurrences of %F in the stdlib.  Not that that's the entire 
universe, of course.

The change is slightly less elegant if I don't change %-formatting, but 
still doable, especially if the betas don't get cut today.


From guido at python.org  Wed Jul 16 19:38:24 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 10:38:24 -0700
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <487E3030.5010203@trueblade.com>
References: <487E0724.6020002@trueblade.com>
	<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>
	<487E3030.5010203@trueblade.com>
Message-ID: <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com>

On Wed, Jul 16, 2008 at 10:30 AM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Guido van Rossum wrote:
>>
>> On Wed, Jul 16, 2008 at 7:35 AM, Eric Smith
>> <eric+python-dev at trueblade.com> wrote:
>>>
>>> Does anyone know why 'F' is the same as 'f'?  Wouldn't it make more sense
>>> to
>>> either drop it, or make it convert the exponent to upper case (like 'E'
>>> and
>>> 'G')?  Compatibility with %-formatting is the only reason I can think of
>>> to
>>> keep up, but I get the sense we've given up on an automatic conversion
>>> from
>>> %-formatting to str.format().  Plus, I can find no uses of '%F' in the
>>> standard library.
>>
>> My best guess as to why 'F' is the same as 'f' is that somebody
>> (could've been me :-) thought, like several others in this thread,
>> that %f never prints an exponent. I agree that making it emit an 'E'
>> when an exponent is used is the right thing to do. Do it now!
>>
>
> It shares code with %-formatting.  Change that, too?  I couldn't find any
> occurrences of %F in the stdlib.  Not that that's the entire universe, of
> course.
>
> The change is slightly less elegant if I don't change %-formatting, but
> still doable, especially if the betas don't get cut today.

Fine with me to change %F as well -- after all that's where the
misunderstanding comes from. (More and more I'm beginning to think it
was my mistake. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From grubert at users.sourceforge.net  Wed Jul 16 20:18:59 2008
From: grubert at users.sourceforge.net (engelbert gruber)
Date: Wed, 16 Jul 2008 20:18:59 +0200
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
	<ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
	<1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>
	<g5j80e$g4d$1@ger.gmane.org>
	<bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com>
	<ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com>
	<ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com>
Message-ID: <ac12bf3a0807161118v252bca49ic62a14332d5132f7@mail.gmail.com>

On Wed, Jul 16, 2008 at 7:02 PM, Guido van Rossum <guido at python.org> wrote:

> On Tue, Jul 15, 2008 at 11:50 PM, engelbert gruber
> <grubert at users.sourceforge.net> wrote:
>> I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes
>> ``DeprecationWarning: callable() not supported in 3.x;`` .
>
> Good catch, Engelbert.
>
> But why would has_key() need a warning when 2to3 will fix it just fine?

for example, if has_key is the only problem in my module, it is
possible to support py22 to py3 with one source which i think would be
quite handy.

and for this the warning is great.

From tim.peters at gmail.com  Wed Jul 16 20:28:04 2008
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 16 Jul 2008 14:28:04 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>
	<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>
Message-ID: <1f7befae0807161128k7629e2a7x28fd1b2f5bf1a139@mail.gmail.com>

[Guido]
> My best guess as to why 'F' is the same as 'f' is that somebody
> (could've been me :-) thought, like several others in this thread,
> that %f never prints an exponent. I agree that making it emit an 'E'
> when an exponent is used is the right thing to do. Do it now!

The C standard doesn't allow for %f (or %F) to produce an exponent.
That appears to be a Python innovation.  People should try their
examples under their native C compiler instead (best I can tell, the
idea that %f/%F can produce an exponent came only from running Python
examples, never from running C examples).

For example,

"""
#include <stdio.h>

int main() {
    printf("%f\n", 1e300);
}
"""

Under the Cygwin gcc, that displays (the admittedly atrocious, but
that's why people shouldn't use %f inappropriately to begin with ;-)):

100000000000000005250476025520442024870446900000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000.000000

As far as C is concerned, the only difference between %f and %F is:

    The F conversion specifier  produces INF, INFINITY, or NAN instead of inf,
    infinity, or nan, respectively

From eric+python-dev at trueblade.com  Wed Jul 16 20:30:51 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Wed, 16 Jul 2008 14:30:51 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>	
	<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>	
	<487E3030.5010203@trueblade.com>
	<ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com>
Message-ID: <487E3E5B.5070700@trueblade.com>

Guido van Rossum wrote:

>> It shares code with %-formatting.  Change that, too?  I couldn't find any
>> occurrences of %F in the stdlib.  Not that that's the entire universe, of
>> course.
>>
>> The change is slightly less elegant if I don't change %-formatting, but
>> still doable, especially if the betas don't get cut today.
> 
> Fine with me to change %F as well -- after all that's where the
> misunderstanding comes from. (More and more I'm beginning to think it
> was my mistake. :-)

I added this as 
http://mail.python.org/pipermail/python-dev/2008-July/081242.html

I should be able to get it checked in today.

Eric.


From tseaver at palladion.com  Wed Jul 16 21:36:32 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 16 Jul 2008 15:36:32 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <878ww27p13.fsf@benfinney.id.au>
References: <87vdz8a983.fsf@benfinney.id.au>
	<8763r89go5.fsf@benfinney.id.au>	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fxqb8mlp.fsf@benfinney.id.au>	<87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>	<g5iljh$cmt$1@ger.gmane.org>	<87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp>
	<878ww27p13.fsf@benfinney.id.au>
Message-ID: <487E4DC0.7030602@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Finney wrote:
> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
> 
>> Terry Reedy writes:
>>
>>  > For the community as a whole, all stdlib modules are suggestions
>>  > and examples, not commands.
>>
>> Well, even if "standard" is too strong a word, the DeprecationWarnings
>> and eventual removal of the methods surely constitute an imposition.
> 
> I understood Terry's statement as meaning that there is no commandment
> to use the standard library 'unittest' module at all. If a user
> doesn't like the behaviour of a module (such as 'unittest') in the
> standard library, they can accept the burden of implementing one that
> behaves the way they like.

One might argue that making gratutiously backward-incompatible changes
to the existing 'unittest' module should fall under that heading:
stability in that package is going to be crucial for projects which have
any hope of making it across the 2-to-3 gulf, and they won't all get
there at once.

If camelCase / duplicated names are such a pain, write a *new* module,
'unittest2', and port Python's tests to use it, thereby leaving the much
larger volume of not-in-Python tests still working.  One might then
remove the 'unittest' module in a later release, packaging it as a
standalone distibution for the projects which still need it.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIfk2/+gerLs4ltQ4RAjd7AJ4iRGnOp+Udw9Jth/VtdMJhJWL50QCePUEY
O1fn7KSSIfIGr0HbIX2xl74=
=s0m/
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Wed Jul 16 21:44:23 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 16 Jul 2008 15:44:23 -0400
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <487DAAA0.1040604@egenix.com>
References: <87vdz8a983.fsf@benfinney.id.au>
	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>
	<487DAAA0.1040604@egenix.com>
Message-ID: <487E4F97.7030407@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

M.-A. Lemburg wrote:
> On 2008-07-16 02:20, Collin Winter wrote:
>> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>> Significant updates include removing all reference to the
>>> (already-resolved) new-style class issue, adding footnotes and
>>> references, and a Rationale summary of discussion on both sides of the
>>> divide for 'assert*' versus 'fail*' names.
>>>
>>>
>>> :PEP:               XXX
>>> :Title:             Consolidating names in the `unittest` module
>>> :Version:           0.2
>>> :Last-Modified:     2008-07-15
>>> :Author:            Ben Finney <ben+python at benfinney.id.au>
>>> :Status:            Draft
>>> :Type:              Standards Track
>>> :Content-Type:      test/x-rst
>>> :Created:           2008-07-14
>>> :Python-Version:    2.7, 3.1
> 
> +1 for doing this in 3.1.
> 
> -1 for Python 2.7.
> 
> The main reason is that there's just too much 2.x code out there
> using the API names you are suggesting to change and/or remove
> from the module.
> 
> Since this is a major change in the unit test API, I'd also like
> to suggest that you use a new module name.
> 
> This is both a precaution to prevent tests failing due to not having
> been upgraded and a way for old code to continue working by adding
> the old unittest module on sys.path.
> 
> Please note that the required renaming of the methods in existing
> tests is not going to be as straight forward as you may think,
> since you may well rename method calls into the tested application
> rather than just the unit test class method calls if you're not
> careful.

+1.  I had just groped my way to that counter-proposal myself, for
exactly your reasons.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIfk+W+gerLs4ltQ4RApZlAJ47NVKXxbL/oaYyVZEUgRnnvajm+wCgyOO2
4GbVo2D1eWYcJvpx1yf8bLs=
=2HV6
-----END PGP SIGNATURE-----


From fuzzyman at voidspace.org.uk  Wed Jul 16 21:47:00 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 20:47:00 +0100
Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module
In-Reply-To: <487E4F97.7030407@palladion.com>
References: <87vdz8a983.fsf@benfinney.id.au>	<87skub6yhh.fsf@benfinney.id.au>	<43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com>	<487DAAA0.1040604@egenix.com>
	<487E4F97.7030407@palladion.com>
Message-ID: <487E5034.30008@voidspace.org.uk>

Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> M.-A. Lemburg wrote:
>   
>> On 2008-07-16 02:20, Collin Winter wrote:
>>     
>>> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>>       
>>>> Significant updates include removing all reference to the
>>>> (already-resolved) new-style class issue, adding footnotes and
>>>> references, and a Rationale summary of discussion on both sides of the
>>>> divide for 'assert*' versus 'fail*' names.
>>>>
>>>>
>>>> :PEP:               XXX
>>>> :Title:             Consolidating names in the `unittest` module
>>>> :Version:           0.2
>>>> :Last-Modified:     2008-07-15
>>>> :Author:            Ben Finney <ben+python at benfinney.id.au>
>>>> :Status:            Draft
>>>> :Type:              Standards Track
>>>> :Content-Type:      test/x-rst
>>>> :Created:           2008-07-14
>>>> :Python-Version:    2.7, 3.1
>>>>         
>> +1 for doing this in 3.1.
>>
>> -1 for Python 2.7.
>>
>> The main reason is that there's just too much 2.x code out there
>> using the API names you are suggesting to change and/or remove
>> from the module.
>>
>> Since this is a major change in the unit test API, I'd also like
>> to suggest that you use a new module name.
>>
>> This is both a precaution to prevent tests failing due to not having
>> been upgraded and a way for old code to continue working by adding
>> the old unittest module on sys.path.
>>
>> Please note that the required renaming of the methods in existing
>> tests is not going to be as straight forward as you may think,
>> since you may well rename method calls into the tested application
>> rather than just the unit test class method calls if you're not
>> careful.
>>     
>
>   

Do you have production code methods called 'assertEquals' and the like? 
It sounds pretty unlikely to me.

Michael

> +1.  I had just groped my way to that counter-proposal myself, for
> exactly your reasons.
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
> Palladion Software   "Excellence by Design"    http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFIfk+W+gerLs4ltQ4RApZlAJ47NVKXxbL/oaYyVZEUgRnnvajm+wCgyOO2
> 4GbVo2D1eWYcJvpx1yf8bLs=
> =2HV6
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From guido at python.org  Wed Jul 16 21:57:47 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 12:57:47 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
Message-ID: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>

Having skimmed much material about proposed changes to the venerable
unitest module, I'd like to set some boundaries. PEPs that don't
follow the following rules are very unlikely to be accepted.

1. The API is not going to be renamed to PEP-8 conformance. This
notwithstanding the purported outcome of earlier discussions. The
renaming will cause too much grief for 3rd party developers; tracking
down why unittests fail is hard enough without also having to consider
changes to the unittest infrastructure itself. It's not the end of the
world if some standard API doesn't follow the style guide.

2. Radical changes to the API are off the table. If a radically
different API is to be accepted, the road to such acceptance is not a
design-by-committee PEP, but adoption of a 3rd party module with a
multi-year track record. If you have radically different ideas about
how to do unittesting, by all means implement them and try them out
and try to get a large audience to use them. When you are successful
in all that, *then* we'll talk about adoption into the standard
library.

3. I like assertEqual better than failUnlessEqual (and similar for all
assert* versions in favor of their fail* alias), and I don't like that
there is both assertEqual and assertEquals. But rule #1 means we have
to live with the aliases. At best we can discourage the undesirables
by documenting them out of existence.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Wed Jul 16 22:01:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 16 Jul 2008 15:01:37 -0500
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
Message-ID: <1afaf6160807161301t43568775y46611d70cb58003d@mail.gmail.com>

On Wed, Jul 16, 2008 at 2:57 PM, Guido van Rossum <guido at python.org> wrote:
> Having skimmed much material about proposed changes to the venerable
> unitest module, I'd like to set some boundaries. PEPs that don't
> follow the following rules are very unlikely to be accepted.

So basically, discussion is open for additions and improvements?
Great, I love it!



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From fuzzyman at voidspace.org.uk  Wed Jul 16 22:03:18 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 21:03:18 +0100
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
Message-ID: <487E5406.1000802@voidspace.org.uk>

Guido van Rossum wrote:
> Having skimmed much material about proposed changes to the venerable
> unitest module, I'd like to set some boundaries. PEPs that don't
> follow the following rules are very unlikely to be accepted.
>
> 1. The API is not going to be renamed to PEP-8 conformance. This
> notwithstanding the purported outcome of earlier discussions. The
> renaming will cause too much grief for 3rd party developers; tracking
> down why unittests fail is hard enough without also having to consider
> changes to the unittest infrastructure itself. It's not the end of the
> world if some standard API doesn't follow the style guide.
>   

I'm glad to see a pronouncement.

> 2. Radical changes to the API are off the table. If a radically
> different API is to be accepted, the road to such acceptance is not a
> design-by-committee PEP, but adoption of a 3rd party module with a
> multi-year track record. If you have radically different ideas about
> how to do unittesting, by all means implement them and try them out
> and try to get a large audience to use them. When you are successful
> in all that, *then* we'll talk about adoption into the standard
> library.
>   

I assume this doesn't rule out the addition of [some of..] the new 
convenience test methods?

> 3. I like assertEqual better than failUnlessEqual (and similar for all
> assert* versions in favor of their fail* alias), and I don't like that
> there is both assertEqual and assertEquals. But rule #1 means we have
> to live with the aliases. At best we can discourage the undesirables
> by documenting them out of existence.
>
>   
Presumably new methods should *not* follow PEP8 but be internally 
consistent with the existing API?

Does this mean that new methods should be added with *both* assert* and 
fail* names?

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From python at rcn.com  Wed Jul 16 22:37:16 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 16 Jul 2008 13:37:16 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
Message-ID: <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>

From: "Michael Foord" <fuzzyman at voidspace.org.uk>
> I assume this doesn't rule out the addition of [some of..] the new 
> convenience test methods?

In Kent Beck's book on Test Driven Development, he complains that most 
unittest implementations spawned from his original work have grown far 
too complicated and would be better served by sticking to a simple 
framework for writing and running tests.

Accordlingly, if a new test method gets added, it needs to add some 
signficant new capability.  IMO, little "convenience" methods like
self.assertLessThan() do not meet the criterion.  Polluting the module
with more methods makes it harder to fit into your head and does
little to simplify the task of writing mountains of tests.

If some people want to proceed down the path of "useful additions",
I challenge them to think bigger.  Give me some test methods that
improve my life.  Don't give me thirty ways to spell something I can
already do.


Raymond

From fuzzyman at voidspace.org.uk  Wed Jul 16 22:48:54 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 21:48:54 +0100
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
Message-ID: <487E5EB6.8070306@voidspace.org.uk>

Raymond Hettinger wrote:
> From: "Michael Foord" <fuzzyman at voidspace.org.uk>
>> I assume this doesn't rule out the addition of [some of..] the new 
>> convenience test methods?
>
> In Kent Beck's book on Test Driven Development, he complains that most 
> unittest implementations spawned from his original work have grown far 
> too complicated and would be better served by sticking to a simple 
> framework for writing and running tests.
>
> Accordlingly, if a new test method gets added, it needs to add some 
> signficant new capability.  IMO, little "convenience" methods like
> self.assertLessThan() do not meet the criterion.  Polluting the module
> with more methods makes it harder to fit into your head and does
> little to simplify the task of writing mountains of tests.
>
> If some people want to proceed down the path of "useful additions",
> I challenge them to think bigger.  Give me some test methods that
> improve my life.  Don't give me thirty ways to spell something I can
> already do.
>

I assert that... the following changes do meet those conditions:

assertRaisesWithMessage - for testing the error messages from library 
functions, where the error message is part of the API under test (I'm 
less convinced with the need for a regex matching version myself.)

Changes to assertEquals so that the failure messages are more useful 
(telling you which member failed the comparison for collections and 
showing a diff for long strings). This improves rather than adds.

assertIn / assertNotIn I use very regularly for collection membership 
tests (although personally I find member, container to be a more 
memorable order for the arguments - assert this is in that. The 
comparison with assertRaises in the PEP is wrong - those parameters are 
ordered so that the arbitrary number of arguments immediately follow the 
callable they belong to.)

The run_tests function for running collections of tests. Almost every 
project I've worked on has had an ad-hoc imnplementation of this, 
collecting test modules and turning them into a suitable collection for 
use with unittest.

These are the most important changes in my opinion.

assertIs / assertIsNot also sounds good, but is not something I would 
miss if they weren't added.

Michael

>
> Raymond


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From python at rcn.com  Wed Jul 16 23:03:29 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 16 Jul 2008 14:03:29 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
Message-ID: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>

>> If some people want to proceed down the path of "useful additions",
>> I challenge them to think bigger.  Give me some test methods that
>> improve my life.  Don't give me thirty ways to spell something I can
>> already do.

From: "Michael Foord" <fuzzyman at voidspace.org.uk>
> I assert that... the following changes do meet those conditions:
> 
> assertRaisesWithMessage
 . . .
> Changes to assertEquals so that the failure messages are more useful 
 ...
> assertIn / assertNotIn I use very regularly for collection membership 

- self.assert_(func(x) in result_set)
+ self.assertIn(func(x), result_set)

Yawn.  The gain is zero.  Actually, it's negative because the second
doesn't read as nicely as the pure python expression.

Think bigger!  No fat APIs.  Do something cool!  Checkout the
dynamic test creation in test_decimal to see if it can be generalized.
Give me some cool test runners.  Maybe find a way to automatically
launch pdb or to dump the locals variables at the time of failure.
Maybe move the "test_*.py" search into the unittest module.

We want *small* and powerful.  The api for TestCase instances is
already way too fat.  See an old discussion on the subject at:
       http://bugs.python.org/issue2578


> The run_tests function for running collections of tests. Almost every 
> project I've worked on has had an ad-hoc imnplementation of this, 
> collecting test modules and turning them into a suitable collection for 
> use with unittest.

Now, that's more like it.    Propose more cool stuff like this and
the module really will be improved.
 

> assertIs / assertIsNot also sounds good, but is not something I would 
> miss if they weren't added.

Doh!  We're back to replacing clean expressions using pure python syntax
with a method name equivalent.  That's a step backwards.



Raymond

From guido at python.org  Wed Jul 16 23:12:58 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 14:12:58 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <487E5406.1000802@voidspace.org.uk>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
Message-ID: <ca471dc20807161412s57bd0b34k412539d2122c87c1@mail.gmail.com>

On Wed, Jul 16, 2008 at 1:03 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Guido van Rossum wrote:
>> 2. Radical changes to the API are off the table. If a radically
>> different API is to be accepted, the road to such acceptance is not a
>> design-by-committee PEP, but adoption of a 3rd party module with a
>> multi-year track record. If you have radically different ideas about
>> how to do unittesting, by all means implement them and try them out
>> and try to get a large audience to use them. When you are successful
>> in all that, *then* we'll talk about adoption into the standard
>> library.

> I assume this doesn't rule out the addition of [some of..] the new
> convenience test methods?

Correct.

>> 3. I like assertEqual better than failUnlessEqual (and similar for all
>> assert* versions in favor of their fail* alias), and I don't like that
>> there is both assertEqual and assertEquals. But rule #1 means we have
>> to live with the aliases. At best we can discourage the undesirables
>> by documenting them out of existence.

> Presumably new methods should *not* follow PEP8 but be internally consistent
> with the existing API?

Right again.

> Does this mean that new methods should be added with *both* assert* and
> fail* names?

No, I don't see any reason to perpetuate that particular design snafu.
I said "live with the aliases", not "add more of them".

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ctb at msu.edu  Wed Jul 16 23:15:29 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 16 Jul 2008 14:15:29 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
Message-ID: <20080716211529.GA30109@caltech.edu>

On Wed, Jul 16, 2008 at 02:03:29PM -0700, Raymond Hettinger wrote:
-> - self.assert_(func(x) in result_set)
-> + self.assertIn(func(x), result_set)
-> 
-> Yawn.  The gain is zero.  Actually, it's negative because the second
-> doesn't read as nicely as the pure python expression.

People are proposing these craptastic methods because 'assert' doesn't
do good reporting by default.  Maybe we can fix that instead??

-> >The run_tests function for running collections of tests. Almost every 
-> >project I've worked on has had an ad-hoc imnplementation of this, 
-> >collecting test modules and turning them into a suitable collection for 
-> >use with unittest.
-> 
-> Now, that's more like it.    Propose more cool stuff like this and
-> the module really will be improved.

At this point I might suggest taking a look at the nose and py.test
discovery rules and writing a simple test discovery system to find &
wrap 'test_' functions/classes and doctests in a unittest wrapper.

Many people use nose and py.test (which use remarkably similar test
discovery procedures, note) and the basic algorithm is pretty well
worked out.  And, since nose wraps such tests in unittests anyway, it
can be made entirely compatible with pre-existing TestRunner
derivatives.

This steers a middle ground between trying to co-opt py.test and nose
entirely (which no one would be happy with) and leaving the existing
(and hideous, IMO) unittest as the only "standard" option.  It also
helps out people who want to take advantage of some of the niftier
py.test/nosetests features during development but *also* want clean
test code that uses a standard library module.

Let me know if I should expand on this proposal...

Paranthetically, wrt unittest, the world seems to be divided into two
kinds of people : those who find the current API uninspiring but ok, and
those who absolutely hate it.  Has anyone said that they *love* the
current unittest API with all of its boilerplate?  If not, then I think
that says something, no?

--titus
-- 
C. Titus Brown, ctb at msu.edu

From fuzzyman at voidspace.org.uk  Wed Jul 16 23:18:20 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 22:18:20 +0100
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <20080716211529.GA30109@caltech.edu>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<20080716211529.GA30109@caltech.edu>
Message-ID: <487E659C.2000505@voidspace.org.uk>

C. Titus Brown wrote:
> [snip..]
> Paranthetically, wrt unittest, the world seems to be divided into two
> kinds of people : those who find the current API uninspiring but ok, and
> those who absolutely hate it.  Has anyone said that they *love* the
> current unittest API with all of its boilerplate?  If not, then I think
> that says something, no?
>
> --titus
>   

Collecting testcases from the filesystem is a pain. But actually writing 
tests (including custom TestCases) using the unittest API is fine. I 
find unittest straightforward and readable, I like it.

I don't understand a lot of the criticism comes in for.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From ctb at msu.edu  Wed Jul 16 23:20:07 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 16 Jul 2008 14:20:07 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <20080716211529.GA30109@caltech.edu>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<20080716211529.GA30109@caltech.edu>
Message-ID: <20080716212007.GB30109@caltech.edu>

On Wed, Jul 16, 2008 at 02:15:29PM -0700, C. Titus Brown wrote:
-> At this point I might suggest taking a look at the nose and py.test
-> discovery rules and writing a simple test discovery system to find &
-> wrap 'test_' functions/classes and doctests in a unittest wrapper.
-> 
-> Many people use nose and py.test (which use remarkably similar test
-> discovery procedures, note) and the basic algorithm is pretty well
-> worked out.  And, since nose wraps such tests in unittests anyway, it
-> can be made entirely compatible with pre-existing TestRunner
-> derivatives.

Sorry for the second message, but... let's compare:

test_sort.py:
 #! /usr/bin/env python
 import unittest
 class Test(unittest.TestCase):
  def test_me(self):
     seq = [ 5, 4, 1, 3, 2 ]
     seq.sort()
     self.assertEqual(seq, [1, 2, 3, 4, 5])

 if __name__ == '__main__':
    unittest.main()

with

test_sort2.py :

 def test_me():
    seq = [ 5, 4, 1, 3 2 ]
    seq.sort()
    assert seq == [1, 2, 3, 4, 5]

The *only value* that unittest adds here is in the 'assertEqual'
statement, which (I think) returns a richer error message than 'assert'.

If I could run the second program by doing

	unittest.discover_tests('test_sort2.py')

I would be a very happy man... right now it requires installing nose or
py.test.

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From guido at python.org  Wed Jul 16 23:24:37 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 14:24:37 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
Message-ID: <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com>

On Wed, Jul 16, 2008 at 2:03 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Michael Foord" <fuzzyman at voidspace.org.uk>
>> assertIn / assertNotIn I use very regularly for collection membership
>
> - self.assert_(func(x) in result_set)
> + self.assertIn(func(x), result_set)
>
> Yawn.  The gain is zero.  Actually, it's negative because the second
> doesn't read as nicely as the pure python expression.

I disagree. The reason why we have assertEquals(x, y) is that the
error message can show the values of x and y, whereas assert x == y
can't show those. Showing the values can be tremendously useful to
debugging the failure. (Doing an intelligent comparison, e.g. a string
or list "diff" instead of showing the two values, can be even more
useful, and I'd be in favor of that rather than adding new methods
like assertListsEqual.)

(Titus asks if the assert statement could be adjusted to do better
reporting. But that's not going to happen -- it would require a
tremendous amount of compiler support that would have to be
implemented in every Python implementation (last I counted there were
at least five). In addition, python -O removes all asserts from your
code -- that's why we use assertXxx functions in the first place.)

> Think bigger!  No fat APIs.  Do something cool!  Checkout the
> dynamic test creation in test_decimal to see if it can be generalized.
> Give me some cool test runners.  Maybe find a way to automatically
> launch pdb or to dump the locals variables at the time of failure.
> Maybe move the "test_*.py" search into the unittest module.

Those ideas are cool too.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From stephen at xemacs.org  Wed Jul 16 23:45:36 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 17 Jul 2008 06:45:36 +0900
Subject: [Python-Dev] PEP: Consolidating names and classes
	in	the	`unittest` module (updated 2008-07-15)
In-Reply-To: <877ibm2f3b.fsf@benfinney.id.au>
References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol>
	<873ama5xze.fsf@benfinney.id.au>
	<87od4yw2n2.fsf@uwakimon.sk.tsukuba.ac.jp>
	<877ibm2f3b.fsf@benfinney.id.au>
Message-ID: <87fxq9wlj3.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:
 > "Stephen J. Turnbull" <stephen at xemacs.org> writes:
 > 
 > > The intuition that "fail" is a negative word is thus well-founded in
 > > standard usage.
 > 
 > That's not the same thing as "fail" being a negative word in the sense
 > meant by "double negative".

So what?  This whole exercise is about human psychology.  If it were
just a matter of defining things and we're done, it wouldn't matter
how many identifiers we use or how they're spelled, right?

You should treat those perceptions with respect when writing this kind
of PEP, not deny them outright.


From fuzzyman at voidspace.org.uk  Wed Jul 16 23:37:46 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Jul 2008 22:37:46 +0100
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <20080716212007.GB30109@caltech.edu>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>	<20080716211529.GA30109@caltech.edu>
	<20080716212007.GB30109@caltech.edu>
Message-ID: <487E6A2A.9050107@voidspace.org.uk>

C. Titus Brown wrote:
> On Wed, Jul 16, 2008 at 02:15:29PM -0700, C. Titus Brown wrote:
> -> At this point I might suggest taking a look at the nose and py.test
> -> discovery rules and writing a simple test discovery system to find &
> -> wrap 'test_' functions/classes and doctests in a unittest wrapper.
> -> 
> -> Many people use nose and py.test (which use remarkably similar test
> -> discovery procedures, note) and the basic algorithm is pretty well
> -> worked out.  And, since nose wraps such tests in unittests anyway, it
> -> can be made entirely compatible with pre-existing TestRunner
> -> derivatives.
>
> Sorry for the second message, but... let's compare:
>
> test_sort.py:
>  #! /usr/bin/env python
>  import unittest
>  class Test(unittest.TestCase):
>   def test_me(self):
>      seq = [ 5, 4, 1, 3, 2 ]
>      seq.sort()
>      self.assertEqual(seq, [1, 2, 3, 4, 5])
>
>  if __name__ == '__main__':
>     unittest.main()
>
> with
>
> test_sort2.py :
>
>  def test_me():
>     seq = [ 5, 4, 1, 3 2 ]
>     seq.sort()
>     assert seq == [1, 2, 3, 4, 5]
>
> The *only value* that unittest adds here is in the 'assertEqual'
> statement, which (I think) returns a richer error message than 'assert'.
>
>   

But if you exclude the import and class definition (two lines), then the 
test methods themselves are identical - except with unittest you have 
the advantage of the 'assertEquals' error collecting and reporting.

The boilerplate at the end is useful for running the test file in 
isolation, but you don't include anything comparable in the second example.


> If I could run the second program by doing
>
> 	unittest.discover_tests('test_sort2.py')
>
> I would be a very happy man... right now it requires installing nose or
> py.test.
>
>   
What about if you could run all tests in a project (of the first kind) with:

tests = unittest.discover_tests('path/', filter='*test.py')
unittest.run_tests(tests)

(or even just the first line).

With 'discover_tests' recursively globbing the path provided and 
collecting test files as modules.

Michael



> cheers,
> --titus
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From g.brandl at gmx.net  Wed Jul 16 23:46:46 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Jul 2008 23:46:46 +0200
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <20080716212007.GB30109@caltech.edu>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>	<20080716211529.GA30109@caltech.edu>
	<20080716212007.GB30109@caltech.edu>
Message-ID: <g5lq87$pca$1@ger.gmane.org>

C. Titus Brown schrieb:

> Sorry for the second message, but... let's compare:
> 
> test_sort.py:
>  #! /usr/bin/env python
>  import unittest
>  class Test(unittest.TestCase):
>   def test_me(self):
>      seq = [ 5, 4, 1, 3, 2 ]
>      seq.sort()
>      self.assertEqual(seq, [1, 2, 3, 4, 5])
> 
>  if __name__ == '__main__':
>     unittest.main()
> 
> with
> 
> test_sort2.py :
> 
>  def test_me():
>     seq = [ 5, 4, 1, 3 2 ]
>     seq.sort()
>     assert seq == [1, 2, 3, 4, 5]
> 
> The *only value* that unittest adds here is in the 'assertEqual'
> statement, which (I think) returns a richer error message than 'assert'.

If you use py.test, it does some magic to find out your test is an
equality comparison and displays both operands' repr().  Don't know about
nose.

Georg


From g.brandl at gmx.net  Wed Jul 16 23:48:52 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Jul 2008 23:48:52 +0200
Subject: [Python-Dev] accepted bytearray items -- integers or numbers?
Message-ID: <g5lqc5$pca$2@ger.gmane.org>

Currently, most mutating bytearray methods only accept integers
as items (in 3k, in 2.6 they also accept single-char strings, for
a reason I can't remember).

Single-index assignment accepts anything compatible with
operator.index(). This should be made consistent, but in which
direction?

Georg


From guido at python.org  Thu Jul 17 00:08:31 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 15:08:31 -0700
Subject: [Python-Dev] accepted bytearray items -- integers or numbers?
In-Reply-To: <g5lqc5$pca$2@ger.gmane.org>
References: <g5lqc5$pca$2@ger.gmane.org>
Message-ID: <ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com>

On Wed, Jul 16, 2008 at 2:48 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Currently, most mutating bytearray methods only accept integers
> as items (in 3k, in 2.6 they also accept single-char strings, for
> a reason I can't remember).
>
> Single-index assignment accepts anything compatible with
> operator.index(). This should be made consistent, but in which
> direction?

I think they should all made to go through operator.index().

BTW, looking at the 3.0 code, I was initially a little confused. While
the term 'item' generally refers to the value of a list or sequence,
several functions in bytearrayobject.c seem to be using it for an
argument that can be either an index or a slice. Notably
bytes_subscript() and bytes_ass-subscript() have this confusing
terminology. This is unrelated but you might want to fix this too.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From santagada at gmail.com  Thu Jul 17 00:09:03 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Wed, 16 Jul 2008 19:09:03 -0300
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com>
Message-ID: <6C103619-2556-4A55-88A6-CABEFDC9F4F2@gmail.com>


On 16/07/2008, at 18:24, Guido van Rossum wrote:

>>
>> Think bigger!  No fat APIs.  Do something cool!  Checkout the
>> dynamic test creation in test_decimal to see if it can be  
>> generalized.
>> Give me some cool test runners.  Maybe find a way to automatically
>> launch pdb or to dump the locals variables at the time of failure.
>> Maybe move the "test_*.py" search into the unittest module.
>
> Those ideas are cool too.


all those features are already implemented in py.test and most  
probably also on nose. Why not just port those to unittest? is this  
even being considered?


--
Leonardo Santagada




From ctb at msu.edu  Thu Jul 17 00:11:54 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 16 Jul 2008 15:11:54 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <487E6A2A.9050107@voidspace.org.uk>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<20080716211529.GA30109@caltech.edu>
	<20080716212007.GB30109@caltech.edu>
	<487E6A2A.9050107@voidspace.org.uk>
Message-ID: <20080716221154.GA7494@caltech.edu>

On Wed, Jul 16, 2008 at 10:37:46PM +0100, Michael Foord wrote:
-> >test_sort2.py :
-> >
-> > def test_me():
-> >    seq = [ 5, 4, 1, 3 2 ]
-> >    seq.sort()
-> >    assert seq == [1, 2, 3, 4, 5]
-> >
-> >The *only value* that unittest adds here is in the 'assertEqual'
-> >statement, which (I think) returns a richer error message than 'assert'.
-> 
-> But if you exclude the import and class definition (two lines), then the 
-> test methods themselves are identical - except with unittest you have 
-> the advantage of the 'assertEquals' error collecting and reporting.
-> 
-> The boilerplate at the end is useful for running the test file in 
-> isolation, but you don't include anything comparable in the second example.

You omit import and class definition (two lines), as well as 'self' in
every function definition.  And don't forget the multiple levels of
indentation -- that's a fair amount of gratuitous typing & boilerplate,
IMO.

With nose and py.test you always have a test runner and you can run the
tests with

	nosetests test_sort2.py

which to my mind is better than having it be the default __main__
function, anyway.

-> >If I could run the second program by doing
-> >
-> >	unittest.discover_tests('test_sort2.py')
-> >
-> >I would be a very happy man... right now it requires installing nose or
-> >py.test.
-> >
-> What about if you could run all tests in a project (of the first kind) with:
-> 
-> tests = unittest.discover_tests('path/', filter='*test.py')
-> unittest.run_tests(tests)
-> 
-> (or even just the first line).

I would very much like that, although I think some sensible defaults
would let you omit the filter and path in obvious cases (test_*.py and
cwd, basically).

I think the exact test discovery behavior is solved in similar ways by the
py.test and nose folks, and the GCD of these solutions would provide a
good baseline for unittest.  Having *one* Python-general way to name
your test files and test functions/classes that is also compatible
across nose, py.test, and unittest would be a real gain for Python, IMO.

You could even set the default unittest __main__ to run the
discover_tests function, e.g.

	python -m unittest [<directory> [<filter>]]

or some such...

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From collinw at gmail.com  Thu Jul 17 00:16:38 2008
From: collinw at gmail.com (Collin Winter)
Date: Wed, 16 Jul 2008 15:16:38 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
Message-ID: <43aa6ff70807161516k547749ddn9e92520aa9316428@mail.gmail.com>

On Wed, Jul 16, 2008 at 2:03 PM, Raymond Hettinger <python at rcn.com> wrote:
>>> If some people want to proceed down the path of "useful additions",
>>> I challenge them to think bigger.  Give me some test methods that
>>> improve my life.  Don't give me thirty ways to spell something I can
>>> already do.
>
> From: "Michael Foord" <fuzzyman at voidspace.org.uk>
>>
>> I assert that... the following changes do meet those conditions:
>>
>> assertRaisesWithMessage
>
> . . .
>>
>> Changes to assertEquals so that the failure messages are more useful
>
> ...
>>
>> assertIn / assertNotIn I use very regularly for collection membership
>
> - self.assert_(func(x) in result_set)
> + self.assertIn(func(x), result_set)
>
> Yawn.  The gain is zero.  Actually, it's negative because the second
> doesn't read as nicely as the pure python expression.

It's only negative if the method doesn't do anything special. For
example, an assertListEqual() method can tell you *how* the lists
differ, which the pure Python expression can't -- all the Python
expression can say is "yes" or "no". We have methods like this at work
and they're very useful.

That said, I see no reason why these things have to be methods. The
self. method boilerplate is cluttering line-noise in this case.  I can
easily imagine a module of nothing but comparison functions.

Collin Winter

> Think bigger!  No fat APIs.  Do something cool!  Checkout the
> dynamic test creation in test_decimal to see if it can be generalized.
> Give me some cool test runners.  Maybe find a way to automatically
> launch pdb or to dump the locals variables at the time of failure.
> Maybe move the "test_*.py" search into the unittest module.
>
> We want *small* and powerful.  The api for TestCase instances is
> already way too fat.  See an old discussion on the subject at:
>      http://bugs.python.org/issue2578
>
>
>> The run_tests function for running collections of tests. Almost every
>> project I've worked on has had an ad-hoc imnplementation of this, collecting
>> test modules and turning them into a suitable collection for use with
>> unittest.
>
> Now, that's more like it.    Propose more cool stuff like this and
> the module really will be improved.
>
>
>> assertIs / assertIsNot also sounds good, but is not something I would miss
>> if they weren't added.
>
> Doh!  We're back to replacing clean expressions using pure python syntax
> with a method name equivalent.  That's a step backwards.
>
>
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/collinw%40gmail.com
>

From tjreedy at udel.edu  Thu Jul 17 00:46:59 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Jul 2008 18:46:59 -0400
Subject: [Python-Dev] PEP: Consolidating names and classes in the
 `unittest` module (updated 2008-07-15)
In-Reply-To: <487E4DC0.7030602@palladion.com>
References: <87vdz8a983.fsf@benfinney.id.au>	<8763r89go5.fsf@benfinney.id.au>	<873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp>	<87fxqb8mlp.fsf@benfinney.id.au>	<87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp>	<g5iljh$cmt$1@ger.gmane.org>	<87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp>	<878ww27p13.fsf@benfinney.id.au>
	<487E4DC0.7030602@palladion.com>
Message-ID: <g5ltp3$5ju$1@ger.gmane.org>



Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----

> If camelCase / duplicated names are such a pain, write a *new* module,
> 'unittest2', and port Python's tests to use it, thereby leaving the much
> larger volume of not-in-Python tests still working.  One might then
> remove the 'unittest' module in a later release, packaging it as a
> standalone distibution for the projects which still need it.

There was, at one time at least, some degree of consensus for dropping 
the Fail names and keeping the Assert names, which appear to comprise 
75%-80% of usage 'in the wild'.  There seems to less consensus on also 
changing the Assert names from CamelCase to under_score versions.  So I 
was also thinking about a 'unittest2', recommended for new projects and 
gradual changeover of the Python test suite.  'Unittest' could be left 
unchanged but gradually disrecommended, deprecated, and removed (perhaps 
in 4.0 if not before).


From ben+python at benfinney.id.au  Thu Jul 17 00:49:13 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 08:49:13 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
Message-ID: <874p6p1m3a.fsf@benfinney.id.au>

"Guido van Rossum" <guido at python.org> writes:

> Having skimmed much material about proposed changes to the venerable
> unitest module, I'd like to set some boundaries. PEPs that don't
> follow the following rules are very unlikely to be accepted.

Thanks for giving the attention to this topic and producing a
pronouncement.

> 1. The API is not going to be renamed to PEP-8 conformance.
[...]

> 3. I like assertEqual better than failUnlessEqual (and similar for
> all assert* versions in favor of their fail* alias), and I don't
> like that there is both assertEqual and assertEquals. But rule #1
> means we have to live with the aliases. At best we can discourage
> the undesirables by documenting them out of existence.

These two together kill any interest I have in being PEP champion for
unittest changes, or on putting effort into the changes.

Thanks, everyone, for the lively discussion.

-- 
 \      ?The way to build large Python applications is to componentize |
  `\             and loosely-couple the hell out of everything.? ?Aahz |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Thu Jul 17 00:52:41 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 08:52:41 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<20080716211529.GA30109@caltech.edu>
	<487E659C.2000505@voidspace.org.uk>
Message-ID: <87zlohzbk6.fsf@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> Collecting testcases from the filesystem is a pain. But actually
> writing tests (including custom TestCases) using the unittest API is
> fine. I find unittest straightforward and readable, I like it.
> 
> I don't understand a lot of the criticism comes in for.

For my part, I wanted the redundancies removed and the PEP 8
conformance fixed as a precondition too *any* addition to the unittest
API.

Without those two (and the BDFL's pronouncement at the head of this
thread means they're not an option), I can't see the unittest API
getting anything except increasingly hideous and painful to use.

-- 
 \     ?Never do anything against conscience even if the state demands |
  `\                                             it.? ?Albert Einstein |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Thu Jul 17 01:01:08 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 09:01:08 +1000
Subject: [Python-Dev] PEP: Consolidating names and in the `unittest` module
	(version 0.4) - REJECTED
References: <87vdz8a983.fsf@benfinney.id.au>
Message-ID: <87vdz5zb63.fsf@benfinney.id.au>

Significant changes: now targets only Python 3.1, and recording the
new status (and rationale) as rejected by BDFL pronouncement.

Feel free to mine for ideas.


:PEP:               XXX
:Title:             Consolidating names in the `unittest` module
:Version:           0.4
:Last-Modified:     2008-07-17
:Author:            Ben Finney <ben+python at benfinney.id.au>
:Status:            Rejected
:Type:              Standards Track
:Content-Type:      test/x-rst
:Created:           2008-07-14
:Python-Version:    3.1
:Post-History:


..  contents::


Abstract
========

This PEP proposes to consolidate the names that constitute the API of
the standard library `unittest` module, with the goal of removing
redundant names, and conforming with PEP 8.


Motivation
==========

The normal use case for the `unittest` module is to subclass its
classes, overriding and re-using its functions and methods. This draws
constant attention to the fact that the existing implementation fails
several current Python standards:

* It does not conform to `PEP 8`_, requiring users to write their own
  non-PEP-8-conformant names when overriding methods, and encouraging
  extensions to further depart from PEP 8.

* It has many synonyms in its API, which goes against `PEP 20`_.


Specification
=============

Remove obsolete names
---------------------

The following module attributes are not documented as part of the API
and are marked as obsolete in the implementation. They will be
removed.

* ``_makeLoader``
* ``getTestCaseNames``
* ``makeSuite``
* ``findTestCases``

Remove redundant names
----------------------

The following attribute names exist only as synonyms for other names.
They are to be removed, leaving only one name for each attribute in
the API.

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

``assert_``
  Use ``assert_true``, or an ``assert`` statement.

``assertEquals``
  Use ``assert_equal``.

``assertNotEquals``
  Use ``assert_not_equal``.

``assertAlmostEquals``
  Use ``assert_almost_equal``.

``assertNotAlmostEquals``
  Use ``assert_not_almost_equal``.

``failIf``
  Use ``assert_false``.

``failUnless``
  Use ``assert_true``.

``failIfAlmostEqual``
  Use ``assert_not_almost_equal``.

``failIfEqual``
  Use ``assert_not_equal``.

``failUnlessAlmostEqual``
  Use ``assert_almost_equal``.
  
``failUnlessEqual``
  Use ``assert_equal``.

``failUnlessRaises``
  Use ``assert_raises``.

Conform API with PEP 8
----------------------

The following names are to be introduced, each replacing an existing
name, to make all names in the module conform with `PEP 8`_. Each name
is shown with the existing name that it replaces.

Where function parameters are to be renamed also, they are shown.
Where function parameters are not to be renamed, they are elided with
the ellipse ("?") symbol.

Module attributes
~~~~~~~~~~~~~~~~~

``default_test_loader``
  Replaces ``defaultTestLoader``

``TestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``add_error(?)``
  Replaces ``addError(?)``

``add_result(?)``
  Replaces ``addResult(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``should_stop``
  Replaces ``shouldStop``

``start_test(?)``
  Replaces ``startTest(?)``

``stop_test(?)``
  Replaces ``stopTest(?)``

``tests_run``
  Replaces ``testsRun``

``was_successful(?)``
  Replaces ``wasSuccessful(?)``

``TestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, method_name='run_test')``
  Replaces ``__init__(self, methodName='runTest')``

``_test_method_doc``
  Replaces ``_testMethodDoc``

``_test_method_name``
  Replaces ``_testMethodName``

``failure_exception``
  Replaces ``failureException``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``default_test_result(?)``
  Replaces ``defaultTestResult(?)``

``assert_true(?)``
  Replaces ``assertTrue(?)``

``assert_false(?)``
  Replaces ``assertFalse(?)``

``assert_almost_equal(?)``
  Replaces ``assertAlmostEqual(?)``

``assert_equal(?)``
  Replaces ``assertEqual(?)``

``assert_not_almost_equal(?)``
  Replaces ``assertNotAlmostEqual(?)``

``assert_not_equal(?)``
  Replaces ``assertNotEqual(?)``

``assert_raises(exc_class, callable_obj, *args, **kwargs)``
  Replaces ``assertRaises(excClass, callableObj, *args, **kwargs)``

``run_test(?)``
  Replaces ``runTest(?)``

``setup(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``teardown(?)``
  Replaces ``tearDown(?)``

``FunctionTestCase`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, test_func, set_up, tear_down, description)``
  Replaces ``__init__(self, testFunc, setUp, tearDown, description)``

``run_test(?)``
  Replaces ``runTest(?)``

``set_up(?)``
  Replaces ``setUp(?)``

``short_description(?)``
  Replaces ``shortDescription(?)``

``tear_down(?)``
  Replaces ``tearDown(?)``

``TestSuite`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~

``add_test(?)``
  Replaces ``addTest(?)``

``add_tests(?)``
  Replaces ``addTests(?)``

``count_test_cases(?)``
  Replaces ``countTestCases(?)``

``TestLoader`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~

``sort_test_methods_using``
  Replaces ``sortTestMethodsUsing``

``suite_class``
  Replaces ``suiteClass``

``test_method_prefix``
  Replaces ``testMethodPrefix``

``get_test_case_names(self, test_case_class)``
  Replaces ``getTestCaseNames(self, testCaseClass)``

``load_tests_from_module(?)``
  Replaces ``loadTestsFromModule(?)``

``load_tests_from_name(?)``
  Replaces ``loadTestsFromName(?)``

``load_tests_from_names(?)``
  Replaces ``loadTestsFromNames(?)``

``load_tests_from_test_case(self, test_case_class)``
  Replaces ``loadTestsFromTestCase(self, testCaseClass)``

``_TextTestResult`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``show_all``
  Replaces ``showAll``

``add_error(?)``
  Replaces ``addError(?)``

``add_failure(?)``
  Replaces ``addFailure(?)``

``add_success(?)``
  Replaces ``addSuccess(?)``

``get_description(?)``
  Replaces ``getDescription(?)``

``print_error_list(?)``
  Replaces ``printErrorList(?)``

``print_errors(?)``
  Replaces ``printErrors(?)``

``start_test(?)``
  Replaces ``startTest(?)``

``TextTestRunner`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``_make_result(?)``
  Replaces ``_makeResult(?)``

``TestProgram`` attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~

``__init__(self, module, default_test, argv, test_runner, test_loader)``
  Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)``

``create_tests(?)``
  Replaces ``createTests(?)``

``parse_args(?)``
  Replaces ``parseArgs(?)``

``run_tests(?)``
  Replaces ``runTests(?)``

``usage_exit(?)``
  Replaces ``usageExit(?)``


Rationale
=========

BDFL pronouncement
------------------

The BDFL has pronounced [#vanrossum-2]_ that no API renaming is to
occur in the `unittest` module. This PEP is therefore rejected.

Introduction after migration to Python 3.0
------------------------------------------

The proposal to introduce these changes along with the migration to
Python 3.0 is specifically not approved by the BDFL [#vanrossum-1]_,
has many detractors, and has little definite support.

This PEP therefore aims for introduction no earlier than Python 3.1.

Redundant names
---------------

The current API, with two or in some cases three different names
referencing exactly the same function, leads to an overbroad and
redundant API that violates `PEP 20`_ ("there should be one, and
preferably only one, obvious way to do it").

Removal of ``fail*`` names
--------------------------

While there is consensus support to `remove redundant names`_ for the
``TestCase`` test methods, the issue of which set of names should be
retained is controversial (it was several times characterised as
"bike-shedding" [#bikeshed]_ by the BDFL and others).

With good arguments made in favour of each of "retain ``assert*``" and
"retain ``fail*``", this PEP resolves in favour of stated BDFL
preference, and de facto community preference.

Arguments in favour of retaining only the ``assert*`` names:

* BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference
  for the ``assert*`` names.

* Precedent: The Python standard library currently uses the
  ``assert*`` names by a roughly 8:1 majority over the ``fail*``
  names. (Counting unit tests in the py3k tree at 2008-07-15
  [#pitrou-1]_.)

  An ad-hoc sampling of other projects that use `unittest` also
  demonstrates strong preference for use of the ``assert*`` names
  [#bennetts-1]_, [#seaver-1]_.

* Economic: The above precedent indicates a simple economic argument
  [#turnbull-1]_: the majority of tests will not need their statements
  re-phrased, so updating in favour of them will involve less
  disruption and risk of error.

* Positive admonition: The ``assert*`` names state the intent of how
  the code under test *should* behave, while the ``fail*`` names are
  phrased in terms of how the code *should not* behave.

Arguments in favour of retaining only the ``fail*`` names:

* Explicit is better than implicit: The ``fail*`` names state *what
  the function will do* explicitly: fail the test. With the
  ``assert*`` names, the action to be taken is only implicit.

* Avoid false implication: The test methods do not have any necessary
  connection with the built-in ``assert`` statement. Even the
  exception raised, while it defaults to ``AssertionException``, is
  explicitly customisable via the documented ``failure_exception``
  attribute. Choosing the ``fail*`` names avoids the false association
  with either of these.

  It has been argued [#thomas-1]_ that newcomers to `unittest` who
  already know what ``assert`` does can be confused by the behaviour
  of the ``assert*`` names. This confusion does not exist for the
  ``fail*`` names, which don't conflict with existing concepts.

* Avoid name collision: The above confusion with ``assert`` is
  exacerbated by the plain-boolean test using a name of ``assert_``
  (with a trailing underscore) to avoid a name collision with the
  built-in ``assert`` statement. The corresponding ``fail_if`` name
  has no such issue.

PEP 8 names
-----------

Although `unittest` (and its predecessor `PyUnit`) are intended to be
familiar to users of other xUnit interfaces, there is no attempt at
direct API compatibility since the only code that Python's `unittest`
interfaces with is other Python code. The module is in the standard
library and its names should all conform with `PEP 8`_.

While argument was raised [#hettinger-1]_ over the length of names
resulting from a simple conversion of names to
multi_word_with_underscores, this PEP chooses names that are
consistent in wording with the existing names.

An exception was made in the case of ``setup`` and ``teardown``. The
two-word form of these names was judged [#hettinger-1]_ too cumbersome
compared to the new single-word form.
 

Backwards Compatibility
=======================

The names to be obsoleted should be deprecated and removed according
to the schedule for modules in `PEP 4`_.

While deprecated, use of the deprecated attributes should raise a
``DeprecationWarning``, with a message stating which replacement name
should be used.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..  _PEP 4: http://www.python.org/dev/peps/pep-0004
..  _PEP 8: http://www.python.org/dev/peps/pep-0008
..  _PEP 20: http://www.python.org/dev/peps/pep-0020
..  [#bikeshed] http://www.catb.org/~esr/jargon/html/B/bikeshedding.html
..  [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-April/078485.html
..  [#vanrossum-2] http://mail.python.org/pipermail/python-dev/2008-July/081263.html
..  [#pitrou-1] http://mail.python.org/pipermail/python-dev/2008-July/081090.html
..  [#bennetts-1] http://mail.python.org/pipermail/python-dev/2008-July/081141.html
..  [#seaver-1] http://mail.python.org/pipermail/python-dev/2008-July/081166.html
..  [#thomas-1] http://mail.python.org/pipermail/python-dev/2008-July/081153.html
..  [#hettinger-1] http://mail.python.org/pipermail/python-dev/2008-July/081107.html
..  [#turnbull-1] http://mail.python.org/pipermail/python-dev/2008-July/081175.html


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From ben+python at benfinney.id.au  Thu Jul 17 01:14:48 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 09:14:48 +1000
Subject: [Python-Dev] PEP: Frequently-requested additional features for the
	`unittest` module (version 0.5)
References: <87fxqa62xb.fsf@benfinney.id.au>
Message-ID: <87r69tzajb.fsf@benfinney.id.au>

Significant changes: targeting Python 3.1, removal of separate
{lt,gt,le,ge} comparison tests, implementation of enhanced-information
failure message, reference to BDFL pronouncement.

I won't be working on this further; someone else should feel free to
champion this further if they wish.


:PEP:               XXX
:Title:             Frequently-requested additional features for the `unittest` module
:Version:           0.5
:Last-Modified:     2008-07-16
:Author:            Ben Finney <ben+python at benfinney.id.au>
:Status:            Draft
:Type:              Standards Track
:Content-Type:      test/x-rst
:Requires:          PEP XXX (Consolidating names in the `unittest` module)
:Created:           2008-07-14
:Python-Version:    3.1
:Post-History:


..  contents::


Abstract
========

This PEP proposes frequently-requested additions to the standard
library `unittest` module that are natural extensions of its existing
functionality.


Motivation
==========

The `unittest` module is functionally complete. Nevertheless, many
users request and devise extensions to perform functions that are so
common they deserve to have a standard implementation.


Specification
=============

New condition tests
-------------------

The following test methods will be added to the ``TestCase`` class.
The function signature is part of this specification. The body is to
be treated as a reference implementation only; any functionally
identical implementation also meets this specification.

::

    import operator
    import re

    class TestCase(object):
        # ?

        def assert_compare_true(op, first, second, msg=None):
            fail_detail = "%(first)r %(op)r %(second)r" % vars()
            if msg is None:
                msg = fail_detail
            else:
                msg = "%(fail_detail)s: %(msg)s" % vars()
            if not op(first, second):
                raise self.failure_exception(msg)

        def assert_in(container, member, msg=None):
            op = operator.__contains__
            self.assert_compare_true(op, container, member, msg)

        def assert_is(first, second, msg=None):
            op = operator.is_
            self.assert_compare_true(op, first, second, msg)

        def assert_members_equal(first, second, msg=None):
            op = operator.eq
            self.assert_compare_true(op, set(first), set(second), msg)

        def assert_sequence_equal(first, second, msg=None):
            op = operator.eq
            self.assert_compare_true(op, list(first), list(second), msg)

        def assert_raises_with_message_regex(
            exc_class, message_regex, callable_obj, *args, **kwargs):
            exc_name = exc_class.__name__
            message_pattern = re.compile(message_regex)
            try:
                callable_obj(*args, **kwargs)
            except exc_class, exc:
                exc_message = str(exc)
                if not message_pattern.match(exc_message):
                    msg = (
                        "%(exc_name)s raised"
                        " without message matching %(message_regex)r"
                        " (got message %(exc_message)r)"
                        ) % vars()
                    raise self.failure_exception(msg)
            else:
                msg = "%(exc_name)s not raised" % vars()
                raise self.failure_exception(msg)

The following test methods are also added. Their implementation in
each case is simply the logical inverse of a corresponding method
above.

::

        def assert_compare_false(op, first, second, msg=None):
            # Logical inverse of assert_compare_true

        def assert_not_in(container, member, msg=None):
            # Logical inverse of assert_in

        def assert_is_not(first, second, msg=None)
            # Logical inverse of assert_is

        def assert_members_not_equal(first, second, msg=None)
            # Logical inverse of assert_members_equal

        def assert_sequence_not_equal(first, second, msg=None)
            # Logical inverse of assert_sequence_equal

Enhanced failure message for comparison tests
---------------------------------------------

The comparison tests will change their behaviour such that the message
always, even if overridden with a specific message when called,
includes extra information:

* For both ``assert_equal`` and ``assert_not_equal``: the ``repr()``
  output of the objects that were compared. This matches the behaviour
  of ``assert_compare_true`` and ``assert_compare_false``, above.

* For ``assert_equal`` comparisons of ``basestring`` instances that
  are multi-line text: the output of ``diff`` comparing the two texts.

* For membership comparisons with ``assert_members_equal`` and
  ``assert_sequence_equal``: the ``repr()`` output of the members that
  were not equal in each collection. (This change is not done for the
  corresponding ``assert_*_not_equal`` tests, which only fail if the
  collection members are equal.)

Simple invocation of test collection
------------------------------------

To allow simpler loading and running of test cases from an arbitrary
collection, the following new functionality will be added to the
`unittest` module. The function signatures are part of this
specification; the implementation is to be considered a reference
implementation only.

::

    class TestLoader(object):
        # ?

        def load_tests_from_collection(self, collection):
            """ Return a suite of all test cases found in `collection`

                :param collection:
                    One of the following:
                    * a `TestSuite` instance
                    * a `TestCase` subclass
                    * a module
                    * an iterable producing any of these types
                :return:
                    A `TestSuite` instance containing all the test
                    cases found recursively within `collection`.

                """
            suite = None
            if isinstance(collection, TestSuite):
                suite = collection
            elif isinstance(collection, type):
                if issubclass(collection, TestCase):
                    suite = self.load_tests_from_test_case(collection)
            elif isinstance(collection, types.ModuleType):
                suite = self.load_tests_from_module(collection)
            elif hasattr(collection, '__iter__'):
                suite = self.suite_class()
                for item in collection:
                    suite.add_test(self.load_tests_from_collection(item))
            else:
                msg = "not a test collection: %(collection)r" % vars()
                raise TypeError(msg)
            return suite


    def run_tests(
        tests,
        loader_class=TestLoader, runner_class=TextTestRunner):
        """ Run a collection of tests with a test runner

            :param tests:
                A test collection as defined by the `loader_class`
                method `load_tests_from_collection`.
            :param loader_class:
                The type of test loader to use for collecting tests
                from the `tests` collection.
            :param runner_class:
                The type of test runner to instantiate for running the
                collected test suite.
            :return:
                None.

            """
        loader = loader_class()
        suite = loader.load_tests_from_collection(tests)
        runner = runner_class()
        runner.run(suite)


Rationale
=========

BDFL pronouncement
------------------

The BDFL pronounced [#vanrossum-1]_ a set of boundaries within which
changes to the `unittest` module need to operate. This PEP may
currently violate some of those, making it currently unacceptable.

Names for logical-inverse tests
-------------------------------

The simple pattern established by ``assert_foo`` having a logical
inverse named ``assert_not_foo`` sometimes results in gramatically
awkward names. The following names were chosen in exception of this
pattern, in the interest of the API names being more intuitive:

* ``assert_is_not``
* ``assert_members_not_equal``
* ``assert_sequence_not_equal``

Order of method parameters
--------------------------

The methods ``assert_in``, ``assert_not_in`` have the container as the
first parameter. This makes the grammatical object of the function
name come immediately after the function name: "Assert in
`container`". This matches the convention set by the existing
``assert_raises(exception, callable_obj, ?)`` "(Assert the code
raises `exception`").

Set of additional tests
-----------------------

Test methods that were considered but failed to gain significant
support include:

* ``assert_less_than``, ``assert_greater_than``,
  ``assert_less_than_or_equal``, ``assert_greater_than_or_equal``, and
  logical inverses.

  These, and other less-used boolean comparison tests, can all be
  covered adequately with the new ``assert_compare_true`` and
  ``assert_compare_false`` methods with an appropriate comparison
  operator function.


Backwards Compatibility
=======================

This PEP proposes only additional features. There are no
backward-incompatible changes.


Reference Implementation
========================

None yet.


Copyright
=========

This document is hereby placed in the public domain by its author.


..  [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-July/081263.html


..
    Local Variables:
    mode: rst
    coding: utf-8
    End:
    vim: filetype=rst :


From g.brandl at gmx.net  Thu Jul 17 01:20:46 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 17 Jul 2008 01:20:46 +0200
Subject: [Python-Dev] accepted bytearray items -- integers or numbers?
In-Reply-To: <ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com>
References: <g5lqc5$pca$2@ger.gmane.org>
	<ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com>
Message-ID: <g5lvof$bag$1@ger.gmane.org>

Guido van Rossum schrieb:
> On Wed, Jul 16, 2008 at 2:48 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Currently, most mutating bytearray methods only accept integers
>> as items (in 3k, in 2.6 they also accept single-char strings, for
>> a reason I can't remember).
>>
>> Single-index assignment accepts anything compatible with
>> operator.index(). This should be made consistent, but in which
>> direction?
> 
> I think they should all made to go through operator.index().
> 
> BTW, looking at the 3.0 code, I was initially a little confused. While
> the term 'item' generally refers to the value of a list or sequence,
> several functions in bytearrayobject.c seem to be using it for an
> argument that can be either an index or a slice. Notably
> bytes_subscript() and bytes_ass-subscript() have this confusing
> terminology. This is unrelated but you might want to fix this too.

OK, fixed in trunk and 3k. One consequence is that bytearray("xyz")
is no error in 2.6 -- I hope this is what was intended in the first place.

Georg


From g.brandl at gmx.net  Thu Jul 17 01:34:06 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 17 Jul 2008 01:34:06 +0200
Subject: [Python-Dev] PEP: Frequently-requested additional features for
 the `unittest` module (version 0.5)
In-Reply-To: <87r69tzajb.fsf@benfinney.id.au>
References: <87fxqa62xb.fsf@benfinney.id.au> <87r69tzajb.fsf@benfinney.id.au>
Message-ID: <g5m0he$dhd$1@ger.gmane.org>

Ben Finney schrieb:
> Significant changes: targeting Python 3.1, removal of separate
> {lt,gt,le,ge} comparison tests, implementation of enhanced-information
> failure message, reference to BDFL pronouncement.
> 
> I won't be working on this further; someone else should feel free to
> champion this further if they wish.

Note that even if it is abandoned, it should be submitted to peps at python.org
at some point to become an official PEP.

Georg


From guido at python.org  Thu Jul 17 01:40:57 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 16:40:57 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <87zlohzbk6.fsf@benfinney.id.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<20080716211529.GA30109@caltech.edu>
	<487E659C.2000505@voidspace.org.uk> <87zlohzbk6.fsf@benfinney.id.au>
Message-ID: <ca471dc20807161640w393db6caj45bf3caddbc5d8e4@mail.gmail.com>

On Wed, Jul 16, 2008 at 3:52 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> For my part, I wanted the redundancies removed and the PEP 8
> conformance fixed as a precondition too *any* addition to the unittest
> API.

That seems an unproductive attitude towards backwards incompatibility.
I'm glad you see the incompatibility with the way we do business here
though.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jul 17 01:42:16 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 16:42:16 -0700
Subject: [Python-Dev] accepted bytearray items -- integers or numbers?
In-Reply-To: <g5lvof$bag$1@ger.gmane.org>
References: <g5lqc5$pca$2@ger.gmane.org>
	<ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com>
	<g5lvof$bag$1@ger.gmane.org>
Message-ID: <ca471dc20807161642o17ba7988s21502ab589296ca5@mail.gmail.com>

On Wed, Jul 16, 2008 at 4:20 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Guido van Rossum schrieb:
>>
>> On Wed, Jul 16, 2008 at 2:48 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>>>
>>> Currently, most mutating bytearray methods only accept integers
>>> as items (in 3k, in 2.6 they also accept single-char strings, for
>>> a reason I can't remember).
>>>
>>> Single-index assignment accepts anything compatible with
>>> operator.index(). This should be made consistent, but in which
>>> direction?
>>
>> I think they should all made to go through operator.index().
>>
>> BTW, looking at the 3.0 code, I was initially a little confused. While
>> the term 'item' generally refers to the value of a list or sequence,
>> several functions in bytearrayobject.c seem to be using it for an
>> argument that can be either an index or a slice. Notably
>> bytes_subscript() and bytes_ass-subscript() have this confusing
>> terminology. This is unrelated but you might want to fix this too.
>
> OK, fixed in trunk and 3k. One consequence is that bytearray("xyz")
> is no error in 2.6 -- I hope this is what was intended in the first place.

Yes, since this is how you spell bytearray(b"xyz") in 2.6. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From robert.kern at gmail.com  Thu Jul 17 01:45:45 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 16 Jul 2008 18:45:45 -0500
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com>
Message-ID: <g5m179$fb3$1@ger.gmane.org>

Guido van Rossum wrote:
> On Wed, Jul 16, 2008 at 2:03 PM, Raymond Hettinger <python at rcn.com> wrote:
>> From: "Michael Foord" <fuzzyman at voidspace.org.uk>
>>> assertIn / assertNotIn I use very regularly for collection membership
>> - self.assert_(func(x) in result_set)
>> + self.assertIn(func(x), result_set)
>>
>> Yawn.  The gain is zero.  Actually, it's negative because the second
>> doesn't read as nicely as the pure python expression.
> 
> I disagree. The reason why we have assertEquals(x, y) is that the
> error message can show the values of x and y, whereas assert x == y
> can't show those. Showing the values can be tremendously useful to
> debugging the failure. (Doing an intelligent comparison, e.g. a string
> or list "diff" instead of showing the two values, can be even more
> useful, and I'd be in favor of that rather than adding new methods
> like assertListsEqual.)
> 
> (Titus asks if the assert statement could be adjusted to do better
> reporting. But that's not going to happen -- it would require a
> tremendous amount of compiler support that would have to be
> implemented in every Python implementation (last I counted there were
> at least five). In addition, python -O removes all asserts from your
> code -- that's why we use assertXxx functions in the first place.)

The assert statement itself does not have to be modified. nose and py.test are 
already capable of picking out the variables used in a failing assert expression 
and print them out. For example:


[~]$ cat test_nicefail.py
def test_nicefail():
     x = 12
     assert x == 10

[~]$ nosetests --detailed-errors test_nicefail.py
F
======================================================================
FAIL: test_nicefail.test_nicefail
----------------------------------------------------------------------
Traceback (most recent call last):
   File 
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/nose-0.10.3-py2.5.egg/nose/case.py", 
line 182, in runTest
     self.test(*self.arg)
   File "/Users/rkern/test_nicefail.py", line 3, in test_nicefail
     assert x == 10
AssertionError:
     12 = 12
 >>  assert 12 == 10


----------------------------------------------------------------------
Ran 1 test in 0.044s

FAILED (failures=1)


[~]$ py.test test_nicefail.py
====================== test process starts =======================
executable: 
/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python 
  (2.5.1-final-0)
using py lib: 
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/py 
<rev unknown>

test_nicefail.py[1] F

__________________________________________________________________
___________________ entrypoint: test_nicefail ____________________

     def test_nicefail():
         x = 12
E       assert x == 10
 >       assert 12 == 10

[/Users/rkern/test_nicefail.py:3]
__________________________________________________________________
============ tests finished: 1 failed in 0.09 seconds ============

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From kristjan at ccpgames.com  Thu Jul 17 00:46:39 2008
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 16 Jul 2008 22:46:39 +0000
Subject: [Python-Dev] defects submitted by me.
Message-ID: <4E9372E6B2234D4F859320D896059A950FFDE04E4C@exchis.ccp.ad.local>

Recently, I have submitted a number of defects (user krisvale)
I do have checkin privileges but since I have been lurking for so long, I didn't want to actualhange anything.
Is this the way to proceed?  I notic e that a couple of the defects (3367, 3368, 3369) have received attention and are presumabely being worked on, but 3327, 3377 and 3378 have not.

Not that for defect 3377 I have no patch, since I am unclear on where the logical error lies.  Also, some of these problem

Cheers,

Kristj?n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080716/fb5a8398/attachment.htm>

From andrew-pythondev at puzzling.org  Thu Jul 17 02:22:52 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu, 17 Jul 2008 10:22:52 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <487E5EB6.8070306@voidspace.org.uk>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
Message-ID: <20080717002252.GG26808@steerpike.home.puzzling.org>

Michael Foord wrote:
> Raymond Hettinger wrote:
[...]
>>
>> If some people want to proceed down the path of "useful additions",
>> I challenge them to think bigger.  Give me some test methods that
>> improve my life.  Don't give me thirty ways to spell something I can
>> already do.
>>
>
> I assert that... the following changes do meet those conditions:
>
> assertRaisesWithMessage - for testing the error messages from library  
> functions, where the error message is part of the API under test (I'm  
> less convinced with the need for a regex matching version myself.)

This one is easily solved by making assertRaises return the exception it caught.
e.g.:

    exc = self.assertRaises(AttributeError, getattr, foo, 'bar')
    self.assertEqual("'Foo' object has no attribute 'bar'", exc.message)

At least Twisted and bzr have already made this exact change.  It requires no new
methods, and is more flexible than the proposed assertRaisesWithMessage.

-Andrew.


From guido at python.org  Thu Jul 17 02:28:34 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 17:28:34 -0700
Subject: [Python-Dev] defects submitted by me.
In-Reply-To: <4E9372E6B2234D4F859320D896059A950FFDE04E4C@exchis.ccp.ad.local>
References: <4E9372E6B2234D4F859320D896059A950FFDE04E4C@exchis.ccp.ad.local>
Message-ID: <ca471dc20807161728v2b2a23e4l3ad44c09b6f27ade@mail.gmail.com>

Hi Kristjan,

You are doing the right thing. In general, we prefer to have changes
reviewed by a senior committer first. Unfortunately, there is no
guarantee that bugs will get the attention they deserve, since the
senior developers are all pretty busy. Sending email to python-dev is
good, although to really pique folks' interest it would probably be
better to add a 1-2 line summary of each of the items for which you
are hoping to get someone's attention. (And ad the Python version[s]
affected.)

--Guido

On Wed, Jul 16, 2008 at 3:46 PM, Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:
> Recently, I have submitted a number of defects (user krisvale)
>
> I do have checkin privileges but since I have been lurking for so long, I
> didn't want to actualhange anything.
>
> Is this the way to proceed?  I notic e that a couple of the defects (3367,
> 3368, 3369) have received attention and are presumabely being worked on, but
> 3327, 3377 and 3378 have not.
>
>
>
> Not that for defect 3377 I have no patch, since I am unclear on where the
> logical error lies.  Also, some of these problem
>
>
>
> Cheers,
>
>
>
> Kristj?n
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ben+python at benfinney.id.au  Thu Jul 17 02:43:30 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 10:43:30 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
Message-ID: <87ej5tz6fh.fsf@benfinney.id.au>

Andrew Bennetts <andrew-pythondev at puzzling.org> writes:

> This one is easily solved by making assertRaises return the
> exception it caught.

That breaks one simple feature of the unittest API: that all the test
methods will either raise a failure asertion, or return None.

-- 
 \           ?In case you haven't noticed, [the USA] are now almost as |
  `\     feared and hated all over the world as the Nazis were.? ?Kurt |
_o__)                                                   Vonnegut, 2004 |
Ben Finney


From andrew-pythondev at puzzling.org  Thu Jul 17 03:07:30 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu, 17 Jul 2008 11:07:30 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <87ej5tz6fh.fsf@benfinney.id.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
Message-ID: <20080717010730.GA29299@steerpike.home.puzzling.org>

Ben Finney wrote:
> Andrew Bennetts <andrew-pythondev at puzzling.org> writes:
> 
> > This one is easily solved by making assertRaises return the
> > exception it caught.
> 
> That breaks one simple feature of the unittest API: that all the test
> methods will either raise a failure asertion, or return None.

How is returning None a feature?  I've never seen code that somehow depends on
assertRaises returning None.  This ?feature? is not documented as being
significant in the unittest module documentation anywhere.  It is not mentioned
anywhere in the *eight* pages of the xUnit Test Patterns book[1] dedicated to
Assertion Methods in their general form.  Where did you get the notion that it
is a feature?

Further, I have lots of evidence that in practice returning the exception
instance from assertRaises is not a problem, and is in fact quite useful.

I'd quote ?Practicality beats purity?, but I'm not even sure if it is purity
that you have in mind.  Demanding that assertion methods return None seems like
foolish consistency and dogma.

-Andrew.

[1] _xUnit Test Patterns: Refactoring Test Code_, by Gerard Meszaros.
    <http://xunitpatterns.com/>.

From ben+python at benfinney.id.au  Thu Jul 17 03:28:17 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 11:28:17 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
Message-ID: <87abghz4cu.fsf@benfinney.id.au>

Andrew Bennetts <andrew-pythondev at puzzling.org> writes:

> Ben Finney wrote:
> > Andrew Bennetts <andrew-pythondev at puzzling.org> writes:
> > 
> > > This one is easily solved by making assertRaises return the
> > > exception it caught.
> > 
> > That breaks one simple feature of the unittest API: that all the
> > test methods will either raise a failure asertion, or return None.
> 
> How is returning None a feature?

A test method having exactly one meaning is a feature. If it's
consistent across the API, the API retains a level of simplicity.

> I've never seen code that somehow depends on assertRaises returning
> None.

I hope never to see code depending on methods named "assert*"
returning something, instead of *only* asserting a condition.

> Further, I have lots of evidence that in practice returning the
> exception instance from assertRaises is not a problem, and is in
> fact quite useful.

I'm sure it's useful. I don't think it belongs in the return value of
such a method.

> I'd quote ?Practicality beats purity?, but I'm not even sure if it
> is purity that you have in mind.

Close: I'm interested in keeping camel's noses out of tents.

-- 
 \           ?Laugh and the world laughs with you; snore and you sleep |
  `\                                                alone.? ?anonymous |
_o__)                                                                  |
Ben Finney


From andrew-pythondev at puzzling.org  Thu Jul 17 03:45:56 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu, 17 Jul 2008 11:45:56 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <87abghz4cu.fsf@benfinney.id.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
Message-ID: <20080717014556.GB29299@steerpike.home.puzzling.org>

Ben Finney wrote:
> Andrew Bennetts <andrew-pythondev at puzzling.org> writes:
[...]
> > How is returning None a feature?
> 
> A test method having exactly one meaning is a feature. If it's
> consistent across the API, the API retains a level of simplicity.

Your reply makes no sense to me.

I am proposing that it should have exactly one meaning.  Callers will be free to
ignore the return value if they don't need it, and will see zero difference in
behaviour.

It seems you have a different meaning of ?meaning? than I do, which suggests
that this conversation is doomed.  I hope I don't sound hostile, I'm just
bewildered.  Your argument isn't making any sense to me. 

I can of course keep using TestCase subclasses that override assertRaises to be
more useful.  And I will, because there are no actual downsides that I've seen
any evidence of.  I'd be even happier if the standard library adopted this
improvment, though.

-Andrew.


From fdrake at acm.org  Thu Jul 17 03:56:46 2008
From: fdrake at acm.org (Fred Drake)
Date: Wed, 16 Jul 2008 21:56:46 -0400
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <20080717014556.GB29299@steerpike.home.puzzling.org>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
Message-ID: <A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org>

On Jul 16, 2008, at 9:45 PM, Andrew Bennetts wrote:
> I am proposing that it should have exactly one meaning.  Callers  
> will be free to
> ignore the return value if they don't need it, and will see zero  
> difference in
> behaviour.


Sounds like adding a new method, catchException(...), that returns the  
exception it catches, would be a reasonable compromise.  I can't think  
of any reason that the method that catches-and-returns needs to be the  
existing API, which does something different.

OTOH, I really don't have a problem with doing a try/except/else  
myself if that's what I need.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From python at rcn.com  Thu Jul 17 04:09:24 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 16 Jul 2008 19:09:24 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com><487E5406.1000802@voidspace.org.uk><B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1><487E5EB6.8070306@voidspace.org.uk><20080717002252.GG26808@steerpike.home.puzzling.org><87ej5tz6fh.fsf@benfinney.id.au><20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
Message-ID: <FF963E56D28441879E89763BF5F07546@RaymondLaptop1>

>> I'd quote ?Practicality beats purity?, but I'm not even sure if it
>> is purity that you have in mind.

From: "Ben Finney" <ben+python at benfinney.id.au>
> Close: I'm interested in keeping camel's noses out of tents.

I have no idea what you mean or are trying to accomplish
(unless the camel's nose refers to camelcase).

You could always just fork the unittest module and see if
the masses flock to your version.  That way at least people
will have a choice about whether to accept these new design proclivities.


Raymond 


From ben+python at benfinney.id.au  Thu Jul 17 04:18:26 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 12:18:26 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
Message-ID: <871w1tz219.fsf@benfinney.id.au>

Andrew Bennetts <andrew-pythondev at puzzling.org> writes:

> Ben Finney wrote:
> > Andrew Bennetts <andrew-pythondev at puzzling.org> writes:
> [...]
> > > How is returning None a feature?
> > 
> > A test method having exactly one meaning is a feature. If it's
> > consistent across the API, the API retains a level of simplicity.
> 
> Your reply makes no sense to me.
> 
> I am proposing that it should have exactly one meaning.

You're proposing to give "assertRaises" a *new* meaning, without
changing its name to "assertRaisesAndReturnExceptionIfRaises".

The function name should say *all* that the function does, from the
perspective of the caller. If you're proposing to have the function do
extra things that aren't part of what the name says it will do, then
that deserves either a rename or a new function.

> It seems you have a different meaning of ?meaning? than I do,
> which suggests that this conversation is doomed. I hope I don't
> sound hostile, I'm just bewildered. Your argument isn't making any
> sense to me.

I hope that clarifies it. The name of a thing, in Python especially,
is very important; in an API, even more so. If the behaviour of the
function isn't matched by the name, it's a poorly chosen name, a
poorly designed function, or both.

> I can of course keep using TestCase subclasses that override
> assertRaises to be more useful.

Why would you override that function to do something not described by
the name, instead of simply adding a new method that performs this
extra task you want performed?

-- 
 \         ?Pinky, are you pondering what I'm pondering?? ?I think so, |
  `\         Brain, but isn't a cucumber that small called a gherkin?? |
_o__)                                           ?_Pinky and The Brain_ |
Ben Finney


From steve at holdenweb.com  Thu Jul 17 04:22:06 2008
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 16 Jul 2008 22:22:06 -0400
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <87zlohzbk6.fsf@benfinney.id.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>	<20080716211529.GA30109@caltech.edu>	<487E659C.2000505@voidspace.org.uk>
	<87zlohzbk6.fsf@benfinney.id.au>
Message-ID: <g5macs$11o$1@ger.gmane.org>

Ben Finney wrote:
> Michael Foord <fuzzyman at voidspace.org.uk> writes:
> 
>> Collecting testcases from the filesystem is a pain. But actually
>> writing tests (including custom TestCases) using the unittest API is
>> fine. I find unittest straightforward and readable, I like it.
>>
>> I don't understand a lot of the criticism comes in for.
> 
> For my part, I wanted the redundancies removed and the PEP 8
> conformance fixed as a precondition too *any* addition to the unittest
> API.
> 
> Without those two (and the BDFL's pronouncement at the head of this
> thread means they're not an option), I can't see the unittest API
> getting anything except increasingly hideous and painful to use.
> 
Yes, but unless I misunderstand you, you don't regard a mass renaming of 
the module's functionality and removal of existing aliases as a change 
to the API. As far as I'm concerned, if I have to alter my code to use 
the updated module you have changed the API. Test code is particularly 
sensitive to such changes.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From andrew-pythondev at puzzling.org  Thu Jul 17 04:23:49 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu, 17 Jul 2008 12:23:49 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <871w1tz219.fsf@benfinney.id.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<871w1tz219.fsf@benfinney.id.au>
Message-ID: <20080717022349.GB28341@steerpike.home.puzzling.org>

Ben Finney wrote:
[...]
> 
> I hope that clarifies it. The name of a thing, in Python especially,
> is very important; in an API, even more so. If the behaviour of the
> function isn't matched by the name, it's a poorly chosen name, a
> poorly designed function, or both.

It doesn't really clarify it for me.  We're both just repeating ourselves,
though.

-Andrew.


From ben+python at benfinney.id.au  Thu Jul 17 04:36:13 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 12:36:13 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<308761DACA7843C7B5132CF61F34C334@RaymondLaptop1>
	<20080716211529.GA30109@caltech.edu>
	<487E659C.2000505@voidspace.org.uk>
	<87zlohzbk6.fsf@benfinney.id.au> <g5macs$11o$1@ger.gmane.org>
Message-ID: <87wsjlxmn6.fsf@benfinney.id.au>

Steve Holden <steve at holdenweb.com> writes:

> Yes, but unless I misunderstand you, you don't regard a mass
> renaming of the module's functionality and removal of existing
> aliases as a change to the API.

You slightly misunderstand me. The above changes *are* a change to the
API, by definition. My position is that those changes, which *only*
(and deliberately) change names without changing behaviour, are far
below the threshold that might justify a new module.

> As far as I'm concerned, if I have to alter my code to use the
> updated module you have changed the API. Test code is particularly
> sensitive to such changes.

I agree entirely with this.

-- 
 \         ?Smoking cures weight problems. Eventually.? ?Steven Wright |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Thu Jul 17 04:42:28 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 12:42:28 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<871w1tz219.fsf@benfinney.id.au>
Message-ID: <87sku9xmcr.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> You're proposing to give "assertRaises" a *new* meaning, without
> changing its name to "assertRaisesAndReturnExceptionIfRaises".

This might be misunderstood, so I'll make it clearer.

The name "assert raises" has a strong (and, per Guido, deliberate)
association with the existing 'assert' statement. So the name makes a
strong promise of what the function will do: raise an exception if the
condition is false, and *have no effect* otherwise.

If the behaviour desired is, instead, to return a value, then that is
an important part of what the function will do and needs to be
reflected in the name. "catch exception" was one suggestion for a
name, another might be "get exception raised".

If the idea is to have a *single* function that does both disparate
things, then the name should definitely state that. That it would
result in an overly long name is a strong indicator that the function
is doing too much and should be split.

The result I'm trying to avoid by this is that of having the
externally visible behaviour of functions drift from the promise made
by their names. Either rename the function, or create a new one for
the new functionality.

-- 
 \        ?Consider the daffodil. And while you're doing that, I'll be |
  `\              over here, looking through your stuff.? ?Jack Handey |
_o__)                                                                  |
Ben Finney


From eric+python-dev at trueblade.com  Thu Jul 17 04:52:54 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Wed, 16 Jul 2008 22:52:54 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>	
	<5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>	
	<487E109C.7050806@trueblade.com>
	<5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com>
Message-ID: <487EB406.1000904@trueblade.com>

Mark Dickinson wrote:
> On Wed, Jul 16, 2008 at 4:15 PM, Eric Smith
> <eric+python-dev at trueblade.com> wrote:
>> There's no exponent until the number gets large.  I haven't looked up how
>> big the number has to get.  On my Mac, it's somewhere between 1e50 and 1e60.
> 
> I think it's around 1e50, courtesy of the rather oddly-phrased line in
> unicodeobject.c
> (this is in py3k) that looks like:
> 
>     if (type == 'f' && (fabs(x) / 1e25) >= 1e25)

I don't know why that test exists.  It comes from Guido in r3417 
(1993-03-16!).  It's from the very first incarnation of formatfloat().

I'd like to remove it, but there's no telling what it would break. 
Surely something written in the last 15+ years depends on it.  But if it 
were removed, then (as Tim points out) only INF and NAN would be in 
uppercase for 'F'.

regrtest.py still works with it removed, except for string_tests.py, 
which is explicitly looking for this behavior, and has this comment:
             # The formatfloat() code in stringobject.c and
             # unicodeobject.c uses a 120 byte buffer and switches from
             # 'f' formatting to 'g' at precision 50, so we expect
             # OverflowErrors for the ranges x < 50 and prec >= 67.

The fixed size buffer could be dealt with, but it doesn't seem worth it 
given the potential breakage.

For now, I'll just make 'F' convert to uppercase, and leave the 1e50 
issue for another release.

Eric.

From guido at python.org  Thu Jul 17 05:20:04 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 20:20:04 -0700
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <487EB406.1000904@trueblade.com>
References: <487E0724.6020002@trueblade.com>
	<5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com>
	<487E109C.7050806@trueblade.com>
	<5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com>
	<487EB406.1000904@trueblade.com>
Message-ID: <ca471dc20807162020p7b3c4bces2a579d804530f59f@mail.gmail.com>

On Wed, Jul 16, 2008 at 7:52 PM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Mark Dickinson wrote:
>>
>> On Wed, Jul 16, 2008 at 4:15 PM, Eric Smith
>> <eric+python-dev at trueblade.com> wrote:
>>>
>>> There's no exponent until the number gets large.  I haven't looked up how
>>> big the number has to get.  On my Mac, it's somewhere between 1e50 and
>>> 1e60.
>>
>> I think it's around 1e50, courtesy of the rather oddly-phrased line in
>> unicodeobject.c
>> (this is in py3k) that looks like:
>>
>>    if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
>
> I don't know why that test exists.  It comes from Guido in r3417
> (1993-03-16!).  It's from the very first incarnation of formatfloat().
>
> I'd like to remove it, but there's no telling what it would break. Surely
> something written in the last 15+ years depends on it.  But if it were
> removed, then (as Tim points out) only INF and NAN would be in uppercase for
> 'F'.
>
> regrtest.py still works with it removed, except for string_tests.py, which
> is explicitly looking for this behavior, and has this comment:
>            # The formatfloat() code in stringobject.c and
>            # unicodeobject.c uses a 120 byte buffer and switches from
>            # 'f' formatting to 'g' at precision 50, so we expect
>            # OverflowErrors for the ranges x < 50 and prec >= 67.
>
> The fixed size buffer could be dealt with, but it doesn't seem worth it
> given the potential breakage.
>
> For now, I'll just make 'F' convert to uppercase, and leave the 1e50 issue
> for another release.

It was an attempt to prevent ridiculously long output, with lots of
nonsensical digits suggesting precision that isn't there. Since nobody
in their right mind should use %f for such large numbers anyway,
removing it shouldn't hurt anything, but it might startle users who
try things at the interactive prompt, or who simply have a bug in
their code causing oversized numbers to be produced. So my
recommendation would be to leave it in.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jul 17 05:23:03 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Jul 2008 20:23:03 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <87sku9xmcr.fsf@benfinney.id.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<871w1tz219.fsf@benfinney.id.au> <87sku9xmcr.fsf@benfinney.id.au>
Message-ID: <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com>

On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> The result I'm trying to avoid by this is that of having the
> externally visible behaviour of functions drift from the promise made
> by their names. Either rename the function, or create a new one for
> the new functionality.

Oh get over it. This is getting ridiculous. Where did you *get* these
design "principles" you are so fond of? Read the zen of Python and
come back after you've memorized it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ben+python at benfinney.id.au  Thu Jul 17 05:30:26 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 13:30:26 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<871w1tz219.fsf@benfinney.id.au> <87sku9xmcr.fsf@benfinney.id.au>
	<ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com>
Message-ID: <87y741yyp9.fsf@benfinney.id.au>

"Guido van Rossum" <guido at python.org> writes:

> On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > The result I'm trying to avoid by this is that of having the
> > externally visible behaviour of functions drift from the promise
> > made by their names. Either rename the function, or create a new
> > one for the new functionality.
> 
> Oh get over it. This is getting ridiculous.

Indeed.

> Where did you *get* these design "principles" you are so fond of?

The Zen of Python.

> Read the zen of Python and come back after you've memorized it.

The wonderful thing about the Zen of Python is that I can find support
in it for *all* the contradictory views expresed in these threads.

The Zen needs interpretation by mortals with human judgement in order
to apply its principles.

-- 
 \       "Those who will not reason, are bigots, those who cannot, are |
  `\        fools, and those who dare not, are slaves." ??Lord? George |
_o__)                                                Gordon Noel Byron |
Ben Finney


From steve at pearwood.info  Thu Jul 17 06:09:29 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 17 Jul 2008 14:09:29 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
References: <20080713093936.GA3623@benfinney.id.au>
	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
Message-ID: <200807171409.30085.steve@pearwood.info>

On Tue, 15 Jul 2008 09:26:45 am Raymond Hettinger wrote:
> From: "Ben Finney" <ben+python at benfinney.id.au>
>
> > Right, so I'm putting up a separate PEP just for the renaming.
> > Should be arriving on this list soon.
>
> I would like to work with you or someone else who is interested
> on an alternative PEP for a separate, simpler test module
> using the py.test syntax.

I am interested in this suggestion. I didn't know about py.test.

I admit to dissatisfaction with unittest (too Java-ish and heavyweight 
for my tastes). I would love a test suite midway in weight between 
doctests and unittest, so I will check it out.


-- 
Steven

From barry at python.org  Thu Jul 17 06:30:51 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Jul 2008 00:30:51 -0400
Subject: [Python-Dev] No beta2 tonight
Message-ID: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

We have green buildbots, yay!  Thanks everyone for that.

However, we still have three release blocker issues that I am not  
comfortable deferring.

3088 test_multiprocessing hangs intermittently on POSIX platforms
3375 _multiprocessing.so build problems
874900 threading module can deadlock after fork

My biggest concern is 3375 and 3088, the latter has a dependency on  
874900.

3375 is very strange since a subsequent 'make' completes just fine.   
This issue happens for me on both OS X 10.5 and Ubuntu 8.04 for py3k.

3088 does not appear to be happening now on OS X or Ubuntu, but the  
issue (and its dependent issue) are not closed, so I'm not sure  
exactly what's going on here.

So we'll give these releases another 24 hours.  Please, if you have  
time see what you can do about resolving these three blockers.  Chat  
with me on #python-dev if you think we should release anyway.  I will  
try to look into 3375, but I'd like to know if anybody else is seeing  
these build failures.

I'll note that I plan to hold the beta3 releases until all release  
blocker and deferred blockers are resolved.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSH7K+3EjvBPtnXfVAQJPXgQAo+uHlgEEpOqlSYINJuXhTHNRWIQEByAu
RCJZw4jNABKR4Pyero9z2LR31bEWS0Osg8a9DdoeD7bvVmPYyIJCG9bfA3BUwpD8
LtZ1tx9ou/XVGkDD7vQLuTt3hJXSaUvovx4ieeA7OQMiI0Uf3klIny+s2mBFH9IA
COd48C7B/S4=
=tcGt
-----END PGP SIGNATURE-----

From Scott.Daniels at Acm.Org  Thu Jul 17 06:48:50 2008
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 16 Jul 2008 21:48:50 -0700
Subject: [Python-Dev] PEP: Frequently-requested additional features for
 the `unittest` module
In-Reply-To: <87d4le0x3z.fsf@benfinney.id.au>
References: <87fxqa62xb.fsf@benfinney.id.au>
	<87abgi4bxi.fsf@benfinney.id.au>	<g5krih$4f8$1@ger.gmane.org>
	<87d4le0x3z.fsf@benfinney.id.au>
Message-ID: <g5mibh$kbe$1@ger.gmane.org>

Ben Finney wrote:
> Scott David Daniels <Scott.Daniels at Acm.Org> writes:> 
>> I would rather something more like:
>>
>>       def assert_compare_true(op, first, second, msg=None):
>>           if op(first, second):
>>               return
>>           raise self.failure_exception(msg)
>>           if msg is None:
>>               self.failure_exception("%(first)r %(op)r %(second)"
>>                                          % vars())
>>           self.failure_exception("%(first)r %(op)r %(second): %(msg)"
>>                                  % vars())
> I'm confused. It appears to me that your version never gets past the
> first 'raise' statement, which is unconditional; and the rest seems to
> do nothing but instantiate exceptions without using them.

Sorry, I was too hasty last time (had to jet out of the house) and sent
out the unfinished version.  This is what I meant:

      def assert_compare_true(op, first, second, msg=None):
          if op(first, second):
              return
          if msg is None:
              raise self.failure_exception(
                     "%(first)r %(op)r %(second)" % vars())
          raise self.failure_exception(
                     "%(first)r %(op)r %(second): %(msg)" % vars())

(1) Displaying args is the whole point, otherwise just use assert_.
     This form fosters tests that say what is wrong, and not simply
     _that_ something has gone wrong.
     The point is a readable test, reducing boilerplate at the
     call location.  Something like:
          ...
          self.assert_le(sum(source) // len(source), 5, "Mean OK")

(2) No point to doing string conversion except on failure; slow
     __repr__ methods are fine to use if the result is not discarded.

--Scott David Daniels
Scott.Daniels at Acm.Org


From ben+python at benfinney.id.au  Thu Jul 17 07:10:38 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 15:10:38 +1000
Subject: [Python-Dev] Proposed unittest changes
References: <20080713093936.GA3623@benfinney.id.au>
	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<200807171409.30085.steve@pearwood.info>
Message-ID: <87tzepyu29.fsf@benfinney.id.au>

"Steven D'Aprano" <steve at pearwood.info> writes:

> On Tue, 15 Jul 2008 09:26:45 am Raymond Hettinger wrote:
> > From: "Ben Finney" <ben+python at benfinney.id.au>
> >
> > > Right, so I'm putting up a separate PEP just for the renaming.
> > > Should be arriving on this list soon.
> >
> > I would like to work with you or someone else who is interested
> > on an alternative PEP for a separate, simpler test module
> > using the py.test syntax.
> 
> I am interested in this suggestion. I didn't know about py.test.
> 
> I admit to dissatisfaction with unittest (too Java-ish and heavyweight 
> for my tastes). I would love a test suite midway in weight between 
> doctests and unittest, so I will check it out.

I still think 'nose' is a better candidate for this: it appears to
offer what people say they want from 'py.test', yet (unlike 'py.test')
is integrated well with 'unittest'.

-- 
 \         ?Pinky, are you pondering what I'm pondering?? ?I think so, |
  `\         Brain, but what kind of rides do they have in Fabioland?? |
_o__)                                           ?_Pinky and The Brain_ |
Ben Finney


From steve at holdenweb.com  Thu Jul 17 07:50:48 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 17 Jul 2008 01:50:48 -0400
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>	<871w1tz219.fsf@benfinney.id.au>
	<87sku9xmcr.fsf@benfinney.id.au>
	<ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com>
Message-ID: <487EDDB8.1030803@holdenweb.com>

Guido van Rossum wrote:
> On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> The result I'm trying to avoid by this is that of having the
>> externally visible behaviour of functions drift from the promise made
>> by their names. Either rename the function, or create a new one for
>> the new functionality.
> 
> Oh get over it. This is getting ridiculous. Where did you *get* these
> design "principles" you are so fond of? Read the zen of Python and
> come back after you've memorized it.
> 
Thank you. I was about to suggest that len() should be renamed 
length_or_raise_exception_if_not_a_container().

But then I did remark (before I stopped adding to this thread's 
interminable length, some aeons ago) that this was getting silly. 
Someone would surely have replied that the proposal had negative 
connotations and the correct name was really 
length_as_long_as_a_container. Or something equally trivial.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From grubert at users.sourceforge.net  Thu Jul 17 08:52:14 2008
From: grubert at users.sourceforge.net (grubert at users.sourceforge.net)
Date: Thu, 17 Jul 2008 08:52:14 +0200 (CEST)
Subject: [Python-Dev] Running Py2.6 with the -3 option
In-Reply-To: <ac12bf3a0807161118v252bca49ic62a14332d5132f7@mail.gmail.com>
References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1>
	<g57ll0$kqn$1@ger.gmane.org>
	<aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com>
	<bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com>
	<ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com>
	<1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com>
	<g5j80e$g4d$1@ger.gmane.org>
	<bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com>
	<ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com>
	<ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com>
	<ac12bf3a0807161118v252bca49ic62a14332d5132f7@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0807170846030.8174@lx5.local>

On Wed, 16 Jul 2008, engelbert gruber wrote:

> On Wed, Jul 16, 2008 at 7:02 PM, Guido van Rossum <guido at python.org> wrote:
>
>> On Tue, Jul 15, 2008 at 11:50 PM, engelbert gruber
>> <grubert at users.sourceforge.net> wrote:
>>> I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes
>>> ``DeprecationWarning: callable() not supported in 3.x;`` .
>>
>> Good catch, Engelbert.

isnt it that i throw and need someone who catches ?

>> But why would has_key() need a warning when 2to3 will fix it just fine?
>
> for example, if has_key is the only problem in my module, it is
> possible to support py22 to py3 with one source which i think would be
> quite handy.

rather naive assumption of me.

anyway i'll file patches on case base.

all the best

-- 

From solipsis at pitrou.net  Thu Jul 17 11:10:33 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Jul 2008 09:10:33 +0000 (UTC)
Subject: [Python-Dev] light-weight testing
References: <20080713093936.GA3623@benfinney.id.au>
	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<200807171409.30085.steve@pearwood.info>
Message-ID: <loom.20080717T090441-407@post.gmane.org>

Steven D'Aprano <steve <at> pearwood.info> writes:
> 
> I am interested in this suggestion. I didn't know about py.test.
> 
> I admit to dissatisfaction with unittest (too Java-ish and heavyweight 
> for my tastes). I would love a test suite midway in weight between 
> doctests and unittest, so I will check it out.
> 

For what it's worth, I've been using nose for quite a long time and the first 
reason I did so is, like you, because I wanted to write tests in a light way 
(without having to declare classes).

Then after writing some dozens of tests I switched back to wrapping tests in 
classes, just because it makes tests more readable and better organized 
(especially when you come to have setup/teardown functions shared by several 
tests).

(but nose is still very nice)


From solipsis at pitrou.net  Thu Jul 17 11:12:59 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Jul 2008 09:12:59 +0000 (UTC)
Subject: [Python-Dev] assertRaises
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org>
Message-ID: <loom.20080717T091112-655@post.gmane.org>

Fred Drake <fdrake <at> acm.org> writes:
> 
> Sounds like adding a new method, catchException(...), that returns the  
> exception it catches, would be a reasonable compromise.  I can't think  
> of any reason that the method that catches-and-returns needs to be the  
> existing API, which does something different.

So you'd have a method that just catches (assertRaises), and another one that 
catches-and-returns (catchException)?
It doesn't seem very practical to have two different methods based on such a 
small and trivial difference.
Let's just make assertRaises return the exception instance, it seems like it 
feels the need correctly.



From solipsis at pitrou.net  Thu Jul 17 11:54:46 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Jul 2008 09:54:46 +0000 (UTC)
Subject: [Python-Dev] assertRaises
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5406.1000802@voidspace.org.uk>
	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org>
	<loom.20080717T091112-655@post.gmane.org>
Message-ID: <loom.20080717T095323-385@post.gmane.org>


I said:
> Let's just make assertRaises return the exception instance, it seems like it 
> feels the need correctly.

and I meant "fills", not "feels", obviously...



From ncoghlan at gmail.com  Thu Jul 17 12:18:55 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Jul 2008 20:18:55 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <871w1tz219.fsf@benfinney.id.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<871w1tz219.fsf@benfinney.id.au>
Message-ID: <487F1C8F.7080405@gmail.com>

Ben Finney wrote:
> The function name should say *all* that the function does, from the
> perspective of the caller.

I have to disagree with that (and I think you'll find plenty of other 
folks here will disagree as well). A good function names needs to have a 
few characters:
- serve as a mnemonic reminder as to what the function does for a 
programmer already familiar with the API
- allow a less familiar user to easily look it up in the API documentation
- be easy to remember to make the transition to familiarity with the API 
as rapid as possible

Wanting to say *everything* a function does in its name is an utterly 
unreasonable goal that leads to ridiculously long function names that 
nobody can remember.

Taking an existing function such as assertRaises and going "hey, we 
aren't using the return value from this, wouldn't it be really 
convenient if it told us the exact exception it actually caught?" 
doesn't cause any problems for existing code, and makes it much easier 
to write tests that need to check additional details about the exception 
that is raised (e.g. to check that the correct error code is captured 
and reported for things like OSError).

The essence of the function remains unchanged - you're still asserting 
that a particular exception is raised. Returning the actual exception 
object that was caught is merely a convenience that makes a lot of sense.

Try to think of it as being like adding a new optional argument to a 
function - just because the function now provides more flexibility is no 
reason to change it's name (e.g. when the key and reversed parameters 
were added to list.sort, the method name stayed the same). Taking an 
unused return value and making it meaningful is a similar kind of change.

Cheers,
Nick.




-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jul 17 12:34:34 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Jul 2008 20:34:34 +1000
Subject: [Python-Dev] light-weight testing
In-Reply-To: <loom.20080717T090441-407@post.gmane.org>
References: <20080713093936.GA3623@benfinney.id.au>	<87iqv89kj5.fsf@benfinney.id.au>	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>	<200807171409.30085.steve@pearwood.info>
	<loom.20080717T090441-407@post.gmane.org>
Message-ID: <487F203A.6010107@gmail.com>

Antoine Pitrou wrote:
> (especially when you come to have setup/teardown functions shared by several 
> tests).

These days, I tend to just write a context manager for common 
setup/teardown code rather than using the setUp/tearDown hooks (at least 
for Python's own test suite, where I have the luxury of assuming 2.5+ as 
the Python version).

Where I find unittest.TestCase quite convenient is when I want to run 
the same set of tests with a few different settings - for those, putting 
the settings into class attributes and then inheriting from the basic 
test case and using different values is very convenient. This trick is 
particularly useful for testing that a class supports inheritance properly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ben+python at benfinney.id.au  Thu Jul 17 14:00:00 2008
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 17 Jul 2008 22:00:00 +1000
Subject: [Python-Dev] light-weight testing
References: <20080713093936.GA3623@benfinney.id.au>
	<87iqv89kj5.fsf@benfinney.id.au>
	<93F397279D1144DAB5767950F3A07868@RaymondLaptop1>
	<200807171409.30085.steve@pearwood.info>
	<loom.20080717T090441-407@post.gmane.org>
Message-ID: <87prpczpof.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> For what it's worth, I've been using nose for quite a long time and
> the first reason I did so is, like you, because I wanted to write
> tests in a light way (without having to declare classes).
> 
> Then after writing some dozens of tests I switched back to wrapping
> tests in classes, just because it makes tests more readable and
> better organized (especially when you come to have setup/teardown
> functions shared by several tests).
> 
> (but nose is still very nice)

It's also entirely compatible with wrapping one's tests in classes.
The test discovery and collection in 'nose' is one of the attractions:
it discovers them at package, module, class, and plain-function level,
whether doctest or not, whether unittest or not, and collects them all
to run.

-- 
 \       ?Well, my brother says Hello. So, hooray for speech therapy.? |
  `\                                                      ?Emo Philips |
_o__)                                                                  |
Ben Finney


From jnoller at gmail.com  Thu Jul 17 15:16:37 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Jul 2008 09:16:37 -0400
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
Message-ID: <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com>

On Thu, Jul 17, 2008 at 12:30 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> We have green buildbots, yay!  Thanks everyone for that.
>
> However, we still have three release blocker issues that I am not
> comfortable deferring.
>
> 3088 test_multiprocessing hangs intermittently on POSIX platforms
> 3375 _multiprocessing.so build problems
> 874900 threading module can deadlock after fork
>
> My biggest concern is 3375 and 3088, the latter has a dependency on 874900.
>
> 3375 is very strange since a subsequent 'make' completes just fine.  This
> issue happens for me on both OS X 10.5 and Ubuntu 8.04 for py3k.
>
> 3088 does not appear to be happening now on OS X or Ubuntu, but the issue
> (and its dependent issue) are not closed, so I'm not sure exactly what's
> going on here.
>
> So we'll give these releases another 24 hours.  Please, if you have time see
> what you can do about resolving these three blockers.  Chat with me on
> #python-dev if you think we should release anyway.  I will try to look into
> 3375, but I'd like to know if anybody else is seeing these build failures.
>
> I'll note that I plan to hold the beta3 releases until all release blocker
> and deferred blockers are resolved.
>
> - -Barry
>

Quick status update, since I'm the gating factor.

3088: I didn't want to close this until 874900's patch was submitted,
ported to py3k and I saw green build bots. By the time I went to sleep
last night, we were still seeing 2to3 test timeouts on the build bots.
I will port the patch to 3k today and lower the priority of 874900 and
close 3088 shortly.

3375: Guido (thanks guido) looked into this, and while I banged my
head on it a lot yesterday - guido's identified the issue, and now I
need to figure out a fix - help is welcome on this one.

I apologize for not getting these resolved last night

-jesse

From steve at holdenweb.com  Thu Jul 17 15:57:02 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 17 Jul 2008 09:57:02 -0400
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
Message-ID: <487F4FAE.3050600@holdenweb.com>

Barry Warsaw wrote:
[...]
> 
> I'll note that I plan to hold the beta3 releases until all release 
> blocker and deferred blockers are resolved.
> 
Ian Ozsvald of ShowMeDo.com noticed a blog post of mine about this beta, 
and responded as follows:

> Re. http://holdenweb.blogspot.com/2008/07/cpython-getting-serious-about-quality.html
> If someone on the dev side wanted to make a 5-10 minute 'cast on how to check out the python base, build it, test it, check stuff back in, we'd be honoured to host it.  Obviously the entire process deserves more attention but a short overview pointing out the key steps would be:
> 1) simple for the author to create
> 2) easy for possible contributors to watch and receive inspiration
> 3) powerful enough to attract another set of developers into the fold
> so that might be a useful, low-cost, quick win to help with the current effort?

I'd do this myself but a) I'm not the obvious candidate and b) I'm too 
busy to give it my attention now.

If someone wanted to tackle this, it might help getting people testing 
for the next beta, and also to recruit more testers in the long-term.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Thu Jul 17 15:57:02 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 17 Jul 2008 09:57:02 -0400
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
Message-ID: <487F4FAE.3050600@holdenweb.com>

Barry Warsaw wrote:
[...]
> 
> I'll note that I plan to hold the beta3 releases until all release 
> blocker and deferred blockers are resolved.
> 
Ian Ozsvald of ShowMeDo.com noticed a blog post of mine about this beta, 
and responded as follows:

> Re. http://holdenweb.blogspot.com/2008/07/cpython-getting-serious-about-quality.html
> If someone on the dev side wanted to make a 5-10 minute 'cast on how to check out the python base, build it, test it, check stuff back in, we'd be honoured to host it.  Obviously the entire process deserves more attention but a short overview pointing out the key steps would be:
> 1) simple for the author to create
> 2) easy for possible contributors to watch and receive inspiration
> 3) powerful enough to attract another set of developers into the fold
> so that might be a useful, low-cost, quick win to help with the current effort?

I'd do this myself but a) I'm not the obvious candidate and b) I'm too 
busy to give it my attention now.

If someone wanted to tackle this, it might help getting people testing 
for the next beta, and also to recruit more testers in the long-term.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From guido at python.org  Thu Jul 17 16:07:09 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 07:07:09 -0700
Subject: [Python-Dev] [Python-3000]  No beta2 tonight
In-Reply-To: <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com>
Message-ID: <ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com>

On Thu, Jul 17, 2008 at 6:16 AM, Jesse Noller <jnoller at gmail.com> wrote:
> 3375: Guido (thanks guido) looked into this, and while I banged my
> head on it a lot yesterday - guido's identified the issue, and now I
> need to figure out a fix - help is welcome on this one.

You're welcome. I would have never found this if I hadn't had a
heightened awareness of the sys.path_importer_cache variable recently,
due to implementing a zipimport.py for Google App Engine. :-)

I can try looking for a fix later today (in a few hours). A very crude
fix would be to just remove all keys that have NullImporter values
from that variable just before attempting to import the module that
was just built. I'm hoping for something subtler though; I wonder if
there's an identifyable point where the lib directory got created.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aleaxit at gmail.com  Thu Jul 17 17:07:09 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Thu, 17 Jul 2008 08:07:09 -0700
Subject: [Python-Dev] assertRaises
In-Reply-To: <loom.20080717T095323-385@post.gmane.org>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<487E5EB6.8070306@voidspace.org.uk>
	<20080717002252.GG26808@steerpike.home.puzzling.org>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org>
	<loom.20080717T091112-655@post.gmane.org>
	<loom.20080717T095323-385@post.gmane.org>
Message-ID: <e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com>

On Thu, Jul 17, 2008 at 2:54 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> I said:
>> Let's just make assertRaises return the exception instance, it seems like it
>> feels the need correctly.
>
> and I meant "fills", not "feels", obviously...

+1 : enriching the existing method in a way that's perfectly
transparent and innocuous to all existing uses _feels_ right, because
it's more practical.


Alex

From jnoller at gmail.com  Thu Jul 17 17:36:34 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Jul 2008 11:36:34 -0400
Subject: [Python-Dev] [Python-3000]  No beta2 tonight
In-Reply-To: <ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com>
	<ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com>
Message-ID: <4222a8490807170836o1875cef4u8b42d3e7edb1160b@mail.gmail.com>

On Thu, Jul 17, 2008 at 10:07 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jul 17, 2008 at 6:16 AM, Jesse Noller <jnoller at gmail.com> wrote:
>> 3375: Guido (thanks guido) looked into this, and while I banged my
>> head on it a lot yesterday - guido's identified the issue, and now I
>> need to figure out a fix - help is welcome on this one.
>
> You're welcome. I would have never found this if I hadn't had a
> heightened awareness of the sys.path_importer_cache variable recently,
> due to implementing a zipimport.py for Google App Engine. :-)
>
> I can try looking for a fix later today (in a few hours). A very crude
> fix would be to just remove all keys that have NullImporter values
> from that variable just before attempting to import the module that
> was just built. I'm hoping for something subtler though; I wonder if
> there's an identifyable point where the lib directory got created.
>

An additional note for anyone else - this is only under py3k - trunk
is perfectly fine.

From musiccomposition at gmail.com  Thu Jul 17 17:41:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 17 Jul 2008 10:41:37 -0500
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
Message-ID: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>

On Wed, Jul 16, 2008 at 11:30 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> We have green buildbots, yay!  Thanks everyone for that.

The Windows buildbots are not very happy, though. test_ssl and
test_bsddb and constantly failing on both the trunk and py3k. I don't
know much about either of these items (or Windows for that matter), so
any help would be greatly appreciated.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From eric+python-dev at trueblade.com  Thu Jul 17 17:38:41 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Thu, 17 Jul 2008 11:38:41 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <487E3E5B.5070700@trueblade.com>
References: <487E0724.6020002@trueblade.com>		<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>		<487E3030.5010203@trueblade.com>	<ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com>
	<487E3E5B.5070700@trueblade.com>
Message-ID: <487F6781.1000206@trueblade.com>

Eric Smith wrote:
> Guido van Rossum wrote:
> 
>>> It shares code with %-formatting.  Change that, too?  I couldn't find 
>>> any
>>> occurrences of %F in the stdlib.  Not that that's the entire 
>>> universe, of
>>> course.
>>>
>>> The change is slightly less elegant if I don't change %-formatting, but
>>> still doable, especially if the betas don't get cut today.
>>
>> Fine with me to change %F as well -- after all that's where the
>> misunderstanding comes from. (More and more I'm beginning to think it
>> was my mistake. :-)
> 
> I added this as 
> http://mail.python.org/pipermail/python-dev/2008-July/081242.html
> 
> I should be able to get it checked in today.

I have this ready for checkin (with docs and tests).  I'd like to get it
in for this beta, since it does involved changed behavior, no matter how
small ('1e+100' becomes '1E+100' with '%F').  But it relies on the
platform's vsnprintf to do the right thing with 'F', so I'm worried
about breakages on platforms I don't have access to.  Resolving those
issues might take a few days.

Any advice on checking this in now, or waiting until after this beta is
released?


On a related note, I've wanted to clean up float formatting for ages.
The test:
   if ((type == 'f' || type == 'F') && (fabs(x) / 1e25) >= 1e25)
is in the code in 3 places, for example.  I'm hoping I can get to that
project, but since it won't change functionality I can do it later.

Eric.


From rrr at ronadam.com  Thu Jul 17 18:19:57 2008
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 17 Jul 2008 11:19:57 -0500
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <487F1C8F.7080405@gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>	<871w1tz219.fsf@benfinney.id.au>
	<487F1C8F.7080405@gmail.com>
Message-ID: <487F712D.6050908@ronadam.com>



Nick Coghlan wrote:


> Taking an existing function such as assertRaises and going "hey, we 
> aren't using the return value from this, wouldn't it be really 
> convenient if it told us the exact exception it actually caught?" 
> doesn't cause any problems for existing code, and makes it much easier 
> to write tests that need to check additional details about the exception 
> that is raised (e.g. to check that the correct error code is captured 
> and reported for things like OSError).
> 
> The essence of the function remains unchanged - you're still asserting 
> that a particular exception is raised. Returning the actual exception 
> object that was caught is merely a convenience that makes a lot of sense.

I'm not sure I understand...

If "a particular exception is raised", every thing is good and there is no 
error to report.  ie... the code being tested did the right thing.

If it does not raise the particular desired exception, isn't either a 
failureException raised or someother exception, which is caught by the 
unittest test runner.  In that case the assertRaises method never gets a 
chance to return anything.

If this is correct, then the exception needs to be caught and passed out 
via the failureException, and not the returned value.

Ron

From rrr at ronadam.com  Thu Jul 17 18:19:57 2008
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 17 Jul 2008 11:19:57 -0500
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <487F1C8F.7080405@gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>	<871w1tz219.fsf@benfinney.id.au>
	<487F1C8F.7080405@gmail.com>
Message-ID: <487F712D.6050908@ronadam.com>



Nick Coghlan wrote:


> Taking an existing function such as assertRaises and going "hey, we 
> aren't using the return value from this, wouldn't it be really 
> convenient if it told us the exact exception it actually caught?" 
> doesn't cause any problems for existing code, and makes it much easier 
> to write tests that need to check additional details about the exception 
> that is raised (e.g. to check that the correct error code is captured 
> and reported for things like OSError).
> 
> The essence of the function remains unchanged - you're still asserting 
> that a particular exception is raised. Returning the actual exception 
> object that was caught is merely a convenience that makes a lot of sense.

I'm not sure I understand...

If "a particular exception is raised", every thing is good and there is no 
error to report.  ie... the code being tested did the right thing.

If it does not raise the particular desired exception, isn't either a 
failureException raised or someother exception, which is caught by the 
unittest test runner.  In that case the assertRaises method never gets a 
chance to return anything.

If this is correct, then the exception needs to be caught and passed out 
via the failureException, and not the returned value.

Ron


From python at rcn.com  Thu Jul 17 18:25:37 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 17 Jul 2008 09:25:37 -0700
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
References: <487E0724.6020002@trueblade.com>		<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>		<487E3030.5010203@trueblade.com>	<ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com><487E3E5B.5070700@trueblade.com>
	<487F6781.1000206@trueblade.com>
Message-ID: <E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1>

From: "Eric Smith" <eric+python-dev at trueblade.com>
> I have this ready for checkin (with docs and tests).  I'd like to get it
> in for this beta, since it does involved changed behavior, no matter how
> small ('1e+100' becomes '1E+100' with '%F').  But it relies on the
> platform's vsnprintf to do the right thing with 'F', so I'm worried
> about breakages on platforms I don't have access to.  Resolving those
> issues might take a few days.
> 
> Any advice on checking this in now, or waiting until after this beta is
> released?

I recommend doing it after the release.  It's unlikely to be exercised
much by the beta users so there's no real advantage.  If you wait
until afterwards, then there is time to let the buildbots have a go
at it and reveal any cross-platform issues.

Besides, Barry said something about getting meaner.
Would hate to find out what he meant the hard way ;-)


Raymond

From guido at python.org  Thu Jul 17 18:27:58 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 09:27:58 -0700
Subject: [Python-Dev] [Python-3000]  No beta2 tonight
In-Reply-To: <4222a8490807170836o1875cef4u8b42d3e7edb1160b@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com>
	<ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com>
	<4222a8490807170836o1875cef4u8b42d3e7edb1160b@mail.gmail.com>
Message-ID: <ca471dc20807170927g1b7c810r1cad09bf1e1a93dd@mail.gmail.com>

On Thu, Jul 17, 2008 at 8:36 AM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Jul 17, 2008 at 10:07 AM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Jul 17, 2008 at 6:16 AM, Jesse Noller <jnoller at gmail.com> wrote:
>>> 3375: Guido (thanks guido) looked into this, and while I banged my
>>> head on it a lot yesterday - guido's identified the issue, and now I
>>> need to figure out a fix - help is welcome on this one.
>>
>> You're welcome. I would have never found this if I hadn't had a
>> heightened awareness of the sys.path_importer_cache variable recently,
>> due to implementing a zipimport.py for Google App Engine. :-)
>>
>> I can try looking for a fix later today (in a few hours). A very crude
>> fix would be to just remove all keys that have NullImporter values
>> from that variable just before attempting to import the module that
>> was just built. I'm hoping for something subtler though; I wonder if
>> there's an identifyable point where the lib directory got created.

I submitted a fix along these lines -- other things I tried did not
work out. The issue is fixed.

> An additional note for anyone else - this is only under py3k - trunk
> is perfectly fine.

Right. I'm thinking that something changed in the massaging of the
default search path, but I can't be bothered to investigate further. I
do know that right after startup there are more non-trivial entries in
sys.path_importer_cache in 3.0 than there are in 2.6.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jul 17 18:29:56 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 09:29:56 -0700
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1>
References: <487E0724.6020002@trueblade.com>
	<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>
	<487E3030.5010203@trueblade.com>
	<ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com>
	<487E3E5B.5070700@trueblade.com> <487F6781.1000206@trueblade.com>
	<E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1>
Message-ID: <ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com>

On Thu, Jul 17, 2008 at 9:25 AM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Eric Smith" <eric+python-dev at trueblade.com>
>>
>> I have this ready for checkin (with docs and tests).  I'd like to get it
>> in for this beta, since it does involved changed behavior, no matter how
>> small ('1e+100' becomes '1E+100' with '%F').  But it relies on the
>> platform's vsnprintf to do the right thing with 'F', so I'm worried
>> about breakages on platforms I don't have access to.  Resolving those
>> issues might take a few days.
>>
>> Any advice on checking this in now, or waiting until after this beta is
>> released?
>
> I recommend doing it after the release.  It's unlikely to be exercised
> much by the beta users so there's no real advantage.  If you wait
> until afterwards, then there is time to let the buildbots have a go
> at it and reveal any cross-platform issues.
>
> Besides, Barry said something about getting meaner.
> Would hate to find out what he meant the hard way ;-)

I'd advise the opposite -- check it in now. It's not going to break
anything, and if it is, let's find out sooner rather than later.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Thu Jul 17 18:57:21 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 17 Jul 2008 09:57:21 PDT
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> 
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
Message-ID: <08Jul17.095729pdt."58698"@synergy1.parc.xerox.com>

> test_ssl ... constantly failing on both the trunk and py3k.

I'll take a closer look at this.  It's the new test added in lately.
Seems to be working on non-Windows platforms, so I'm guessing it's
some Windows oddity, which I'm not very good at diagnosing.  Worst
comes to worst, we can take out that test.

Bill

From tseaver at palladion.com  Thu Jul 17 19:21:24 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 17 Jul 2008 13:21:24 -0400
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <487F712D.6050908@ronadam.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>	<871w1tz219.fsf@benfinney.id.au>	<487F1C8F.7080405@gmail.com>
	<487F712D.6050908@ronadam.com>
Message-ID: <487F7F94.9080201@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ron Adam wrote:
> 
> Nick Coghlan wrote:

>> The essence of the function remains unchanged - you're still asserting 
>> that a particular exception is raised. Returning the actual exception 
>> object that was caught is merely a convenience that makes a lot of sense.
> 
> I'm not sure I understand...
> 
> If "a particular exception is raised", every thing is good and there is no 
> error to report.  ie... the code being tested did the right thing.

I *think* that in this case the propoent wants to be able to make
further assertions about the raised exception (e.g., to test its message).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIf3+T+gerLs4ltQ4RAk/VAKCK8hnX2C7DE57RSxA88THttT5tSwCg2taQ
zzRkJ/SEPwZvgfcluo2DTvw=
=KTGx
-----END PGP SIGNATURE-----


From guido at python.org  Thu Jul 17 19:22:27 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 10:22:27 -0700
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
Message-ID: <ca471dc20807171022u70b76960vf6e47b3d22efbce8@mail.gmail.com>

Please move all discussions of unittest frameworks to
python-ideas at python.org. It is an interesting topic -- so interesting,
in fact, that exploring all the different ideas under discussion is
overwhelming the primary purpose of python-dev, which at this point is
to get the 2.6 and 3.0 releases into shape.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Thu Jul 17 19:41:32 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 17 Jul 2008 10:41:32 PDT
Subject: [Python-Dev] [Python-3000]  No beta2 tonight
In-Reply-To: <08Jul17.095729pdt."58698"@synergy1.parc.xerox.com> 
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<08Jul17.095729pdt."58698"@synergy1.parc.xerox.com>
Message-ID: <08Jul17.104140pdt."58698"@synergy1.parc.xerox.com>

> > test_ssl ... constantly failing on both the trunk and py3k.
> 
> I'll take a closer look at this.  It's the new test added in lately.
> Seems to be working on non-Windows platforms, so I'm guessing it's
> some Windows oddity, which I'm not very good at diagnosing.  Worst
> comes to worst, we can take out that test.

It looks like the handshake code in _ssl.c is returning this Windows
error (I'm looking at the trunk):

ERROR: testWrongCert (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\test\test_ssl.py", line 807, in testWrongCert
    "wrongcert.pem"))
  File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\test\test_ssl.py", line 597, in badCertTest
    s.connect((HOST, server.port))
  File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\ssl.py", line 272, in connect
    self.do_handshake()
  File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\ssl.py", line 256, in do_handshake
    self._sslobj.do_handshake()
error: [Errno 10054] An existing connection was forcibly closed by the remote host

Error 10054 is WSAECONNRESET, according to Google.

Arguably, it's an error in this scrap of _ssl.c:

	} else if (ret == -1) {
	  /* underlying BIO reported an I/O error */
          return obj->Socket->errorhandler();
        }

I should be checking for EOF error returns like this one from Socket,
and returning an PY_SSL_ERROR_EOF exception.  Is there a list somewhere
of what EOF errors are signalled by Socket?

Bill

From jnoller at gmail.com  Thu Jul 17 19:45:44 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Jul 2008 13:45:44 -0400
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com>
Message-ID: <4222a8490807171045y17b7252sfc99e1fe67e4f1d6@mail.gmail.com>

On Thu, Jul 17, 2008 at 9:16 AM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Jul 17, 2008 at 12:30 AM, Barry Warsaw <barry at python.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> We have green buildbots, yay!  Thanks everyone for that.
>>
>> However, we still have three release blocker issues that I am not
>> comfortable deferring.
>>
>> 3088 test_multiprocessing hangs intermittently on POSIX platforms
>> 3375 _multiprocessing.so build problems
>> 874900 threading module can deadlock after fork
>>
>> My biggest concern is 3375 and 3088, the latter has a dependency on 874900.
>>
>> 3375 is very strange since a subsequent 'make' completes just fine.  This
>> issue happens for me on both OS X 10.5 and Ubuntu 8.04 for py3k.
>>
>> 3088 does not appear to be happening now on OS X or Ubuntu, but the issue
>> (and its dependent issue) are not closed, so I'm not sure exactly what's
>> going on here.
>>
>> So we'll give these releases another 24 hours.  Please, if you have time see
>> what you can do about resolving these three blockers.  Chat with me on
>> #python-dev if you think we should release anyway.  I will try to look into
>> 3375, but I'd like to know if anybody else is seeing these build failures.
>>
>> I'll note that I plan to hold the beta3 releases until all release blocker
>> and deferred blockers are resolved.
>>
>> - -Barry
>>
>
> Quick status update, since I'm the gating factor.
>
> 3088: I didn't want to close this until 874900's patch was submitted,
> ported to py3k and I saw green build bots. By the time I went to sleep
> last night, we were still seeing 2to3 test timeouts on the build bots.
> I will port the patch to 3k today and lower the priority of 874900 and
> close 3088 shortly.
>
> 3375: Guido (thanks guido) looked into this, and while I banged my
> head on it a lot yesterday - guido's identified the issue, and now I
> need to figure out a fix - help is welcome on this one.
>
> I apologize for not getting these resolved last night
>
> -jesse
>

Just to close the loop - 3088 is closed now, and 874900 is on py3k,
therefore lowered from a release blocker for now (it needs to remain
open to finish resolving a few issues)

Guido fixed 3375 (thanks again)

-jesse

From ranjithkannikara at gmail.com  Thu Jul 17 19:54:38 2008
From: ranjithkannikara at gmail.com (ranjith kannikara)
Date: Thu, 17 Jul 2008 23:24:38 +0530
Subject: [Python-Dev] Implementing restricted Python in Zope2
Message-ID: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>

I have taken the gsoc 08  project of porting zope2 to python2.5.
Through my way to the successful completion of the project I have to
implement Restricted python in Zope2. I could only get the information
that the python AST has not changed on moving from python2.4 to 2.5
but Restricted Python is not well documented enough for a stident to
test the Zope2 's Restricted Python implentation.

As a student I am not familiar with Restricted Python and python AST
implementation.And in need of help to start the Restricted Python
implementation.

Ranjith.

From janssen at parc.com  Thu Jul 17 20:18:34 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 17 Jul 2008 11:18:34 PDT
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> 
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
Message-ID: <08Jul17.111840pdt."58698"@synergy1.parc.xerox.com>

> The Windows buildbots are not very happy, though. test_ssl ...
> constantly failing on both the trunk and py3k.

I've checked in patches for test_ssl on both branches.  Let's see how
the Windows buildbots do.

Bill

From brett at python.org  Thu Jul 17 20:27:42 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Jul 2008 11:27:42 -0700
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
Message-ID: <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>

On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara
<ranjithkannikara at gmail.com> wrote:
> I have taken the gsoc 08  project of porting zope2 to python2.5.
> Through my way to the successful completion of the project I have to
> implement Restricted python in Zope2. I could only get the information
> that the python AST has not changed on moving from python2.4 to 2.5
> but Restricted Python is not well documented enough for a stident to
> test the Zope2 's Restricted Python implentation.
>
> As a student I am not familiar with Restricted Python and python AST
> implementation.And in need of help to start the Restricted Python
> implementation.
>

What do you mean, "Restricted Python"? If you mean rexec and Bastion,
they are no longer supported, and that began before 2.5.

-Brett

From guido at python.org  Thu Jul 17 20:47:26 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 11:47:26 -0700
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>
Message-ID: <ca471dc20807171147u26105c05ue63e6dc227b78052@mail.gmail.com>

On Thu, Jul 17, 2008 at 11:27 AM, Brett Cannon <brett at python.org> wrote:
> On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara
> <ranjithkannikara at gmail.com> wrote:
>> I have taken the gsoc 08  project of porting zope2 to python2.5.
>> Through my way to the successful completion of the project I have to
>> implement Restricted python in Zope2. I could only get the information
>> that the python AST has not changed on moving from python2.4 to 2.5
>> but Restricted Python is not well documented enough for a stident to
>> test the Zope2 's Restricted Python implentation.
>>
>> As a student I am not familiar with Restricted Python and python AST
>> implementation.And in need of help to start the Restricted Python
>> implementation.

> What do you mean, "Restricted Python"? If you mean rexec and Bastion,
> they are no longer supported, and that began before 2.5.

Maybe he's talking about PyPy's RPython, which is notoriously (and it
seems intentionally) underspecified? In that case, please contact the
PyPy team.

Or maybe there is something in Zope 2 called Restricted Python -- in
that case please contact the Zope lists.

There's nothing in the collective memory of python-dev called
Restricted Python that we can help you with.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at ctypes.org  Thu Jul 17 21:02:21 2008
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 17 Jul 2008 21:02:21 +0200
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com>
References: <487E0724.6020002@trueblade.com>	<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>	<487E3030.5010203@trueblade.com>	<ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com>	<487E3E5B.5070700@trueblade.com>
	<487F6781.1000206@trueblade.com>	<E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1>
	<ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com>
Message-ID: <g5o4uu$tpf$1@ger.gmane.org>

Guido van Rossum schrieb:
> On Thu, Jul 17, 2008 at 9:25 AM, Raymond Hettinger <python at rcn.com> wrote:
>> From: "Eric Smith" <eric+python-dev at trueblade.com>
>>>
>>> I have this ready for checkin (with docs and tests).  I'd like to get it
>>> in for this beta, since it does involved changed behavior, no matter how
>>> small ('1e+100' becomes '1E+100' with '%F').  But it relies on the
>>> platform's vsnprintf to do the right thing with 'F', so I'm worried
>>> about breakages on platforms I don't have access to.  Resolving those
>>> issues might take a few days.
>>>
>>> Any advice on checking this in now, or waiting until after this beta is
>>> released?
>>
>> I recommend doing it after the release.  It's unlikely to be exercised
>> much by the beta users so there's no real advantage.  If you wait
>> until afterwards, then there is time to let the buildbots have a go
>> at it and reveal any cross-platform issues.
>>
>> Besides, Barry said something about getting meaner.
>> Would hate to find out what he meant the hard way ;-)
> 
> I'd advise the opposite -- check it in now. It's not going to break
> anything, and if it is, let's find out sooner rather than later.
> 

Before we get old waiting for the windows buildbots, here how test_format
fails now (on XP SP2 x86, trunk rev 65072:

c:\svn\trunk\PCbuild>.\\python_d  -E -tt ../lib/test/regrtest.py test_format
test_format
test test_format produced unexpected output:
**********************************************************************
'%F' % (1.0,) == '' != '1.000000'
u'%F' % (1.0,) == u'' != '1.000000'
'%f' % (nan,) == '-1.#IND00' != 'nan'
u'%f' % (nan,) == u'-1.#IND00' != 'nan'
'%F' % (nan,) == '' != 'NAN'
u'%F' % (nan,) == u'' != 'NAN'
'%f' % (inf,) == '1.#INF' != 'inf'
u'%f' % (inf,) == u'1.#INF' != 'inf'
'%F' % (inf,) == '1.#INF' != 'INF'
u'%F' % (inf,) == u'1.#INF' != 'INF'

**********************************************************************
1 test failed:
    test_format
[23521 refs]

c:\svn\trunk\PCbuild>

Windows vsnprintf does not support '%F' at all, and '%f' does not
behave as expected with nan and inf.

-- 
Thanks,
Thomas


From eric+python-dev at trueblade.com  Thu Jul 17 21:10:56 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Thu, 17 Jul 2008 15:10:56 -0400
Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F'
In-Reply-To: <g5o4uu$tpf$1@ger.gmane.org>
References: <487E0724.6020002@trueblade.com>	<ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com>	<487E3030.5010203@trueblade.com>	<ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com>	<487E3E5B.5070700@trueblade.com>	<487F6781.1000206@trueblade.com>	<E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1>	<ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com>
	<g5o4uu$tpf$1@ger.gmane.org>
Message-ID: <487F9940.3020208@trueblade.com>

Thomas Heller wrote:
> Guido van Rossum schrieb:
>> On Thu, Jul 17, 2008 at 9:25 AM, Raymond Hettinger <python at rcn.com> wrote:
>>> From: "Eric Smith" <eric+python-dev at trueblade.com>
>>>> I have this ready for checkin (with docs and tests).  I'd like to get it
>>>> in for this beta, since it does involved changed behavior, no matter how
>>>> small ('1e+100' becomes '1E+100' with '%F').  But it relies on the
>>>> platform's vsnprintf to do the right thing with 'F', so I'm worried
>>>> about breakages on platforms I don't have access to.  Resolving those
>>>> issues might take a few days.
>>>>
>>>> Any advice on checking this in now, or waiting until after this beta is
>>>> released?
>>> I recommend doing it after the release.  It's unlikely to be exercised
>>> much by the beta users so there's no real advantage.  If you wait
>>> until afterwards, then there is time to let the buildbots have a go
>>> at it and reveal any cross-platform issues.
>>>
>>> Besides, Barry said something about getting meaner.
>>> Would hate to find out what he meant the hard way ;-)
>> I'd advise the opposite -- check it in now. It's not going to break
>> anything, and if it is, let's find out sooner rather than later.
>>
> 
> Before we get old waiting for the windows buildbots, here how test_format
> fails now (on XP SP2 x86, trunk rev 65072:
> 
> c:\svn\trunk\PCbuild>.\\python_d  -E -tt ../lib/test/regrtest.py test_format
> test_format
> test test_format produced unexpected output:
> **********************************************************************
> '%F' % (1.0,) == '' != '1.000000'
> u'%F' % (1.0,) == u'' != '1.000000'
> '%f' % (nan,) == '-1.#IND00' != 'nan'
> u'%f' % (nan,) == u'-1.#IND00' != 'nan'
> '%F' % (nan,) == '' != 'NAN'
> u'%F' % (nan,) == u'' != 'NAN'
> '%f' % (inf,) == '1.#INF' != 'inf'
> u'%f' % (inf,) == u'1.#INF' != 'inf'
> '%F' % (inf,) == '1.#INF' != 'INF'
> u'%F' % (inf,) == u'1.#INF' != 'INF'
> 
> **********************************************************************
> 1 test failed:
>     test_format
> [23521 refs]
> 
> c:\svn\trunk\PCbuild>
> 
> Windows vsnprintf does not support '%F' at all, and '%f' does not
> behave as expected with nan and inf.
> 

I was afraid of that.  I'll back these out until I can dig up a Windows 
box to test on.

Thanks for checking this.

Eric.


From shigin at rambler-co.ru  Thu Jul 17 21:41:19 2008
From: shigin at rambler-co.ru (Alexander Shigin)
Date: Thu, 17 Jul 2008 23:41:19 +0400
Subject: [Python-Dev] Issue2944: need a review
Message-ID: <1216323679.4911.6.camel@jenner>

Can anyone look at the patch for Issue2944?

I hope the issue can be fixed before the release of python 2.6.


From pje at telecommunity.com  Thu Jul 17 22:26:12 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 17 Jul 2008 16:26:12 -0400
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.co
 m>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>
Message-ID: <20080717202531.DA16A3A4080@sparrow.telecommunity.com>

At 11:27 AM 7/17/2008 -0700, Brett Cannon wrote:
>On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara
><ranjithkannikara at gmail.com> wrote:
> > I have taken the gsoc 08  project of porting zope2 to python2.5.
> > Through my way to the successful completion of the project I have to
> > implement Restricted python in Zope2. I could only get the information
> > that the python AST has not changed on moving from python2.4 to 2.5
> > but Restricted Python is not well documented enough for a stident to
> > test the Zope2 's Restricted Python implentation.
> >
> > As a student I am not familiar with Restricted Python and python AST
> > implementation.And in need of help to start the Restricted Python
> > implementation.
> >
>
>What do you mean, "Restricted Python"? If you mean rexec and Bastion,
>they are no longer supported, and that began before 2.5.

No, he means the restricted Python compiler and capability-proxy 
system used by Zope.  You know, the one I always bring up whenever 
anybody says they want to implement capabilities in Python?  ;-)

Zope's restricted Python is basically a combination of a special 
compiler, __builtin__ replacements, and a proxy type.  Instead of 
using LOAD_ATTR opcodes, the compiler generates code that calls a 
special getattr() function instead, and most objects other than 
relatively-safe builtin types are wrapped in proxies that control 
what attributes can be accessed and what operations can be performed.

The restricted Python framework itself doesn't impose any particular 
security policy; proxies delegate checks to "checker" objects that 
are essentially capabilities.  Mostly, it focuses on creating a safe 
sandbox that can be expanded.

There are two parts to the implication; one is called 
RestrictedPython and lives at:

http://svn.zope.org/RestrictedPython/trunk

The other part is "zope.security.untrustedpython", and it's part of 
the zope.security distribution; see:

http://svn.zope.org/zope.security/trunk/src/zope/security/untrustedpython/

for its specific code and docs.

Both packages appear to have automated tests.


From guido at python.org  Thu Jul 17 23:03:44 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 14:03:44 -0700
Subject: [Python-Dev] Issue2944: need a review
In-Reply-To: <1216323679.4911.6.camel@jenner>
References: <1216323679.4911.6.camel@jenner>
Message-ID: <ca471dc20807171403t638fb207h218fa11fb97caa5f@mail.gmail.com>

Suggestion for people asking developers to look into issues: indicate
more than the issue number in the email. Show the issue summary, other
relevant metadata, and what needs to be done in some detail. This will
pique the interest of those who *can* help (and allow people who can't
help anyway to skip the message more efficiently).

On Thu, Jul 17, 2008 at 12:41 PM, Alexander Shigin <shigin at rambler-co.ru> wrote:
> Can anyone look at the patch for Issue2944?
>
> I hope the issue can be fixed before the release of python 2.6.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jul 17 23:04:34 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 14:04:34 -0700
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <20080717202531.DA16A3A4080@sparrow.telecommunity.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>
	<20080717202531.DA16A3A4080@sparrow.telecommunity.com>
Message-ID: <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com>

Thanks. Then python-dev is *definitely* the wrong forum. :-)

On Thu, Jul 17, 2008 at 1:26 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:27 AM 7/17/2008 -0700, Brett Cannon wrote:
>>
>> On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara
>> <ranjithkannikara at gmail.com> wrote:
>> > I have taken the gsoc 08  project of porting zope2 to python2.5.
>> > Through my way to the successful completion of the project I have to
>> > implement Restricted python in Zope2. I could only get the information
>> > that the python AST has not changed on moving from python2.4 to 2.5
>> > but Restricted Python is not well documented enough for a stident to
>> > test the Zope2 's Restricted Python implentation.
>> >
>> > As a student I am not familiar with Restricted Python and python AST
>> > implementation.And in need of help to start the Restricted Python
>> > implementation.
>> >
>>
>> What do you mean, "Restricted Python"? If you mean rexec and Bastion,
>> they are no longer supported, and that began before 2.5.
>
> No, he means the restricted Python compiler and capability-proxy system used
> by Zope.  You know, the one I always bring up whenever anybody says they
> want to implement capabilities in Python?  ;-)
>
> Zope's restricted Python is basically a combination of a special compiler,
> __builtin__ replacements, and a proxy type.  Instead of using LOAD_ATTR
> opcodes, the compiler generates code that calls a special getattr() function
> instead, and most objects other than relatively-safe builtin types are
> wrapped in proxies that control what attributes can be accessed and what
> operations can be performed.
>
> The restricted Python framework itself doesn't impose any particular
> security policy; proxies delegate checks to "checker" objects that are
> essentially capabilities.  Mostly, it focuses on creating a safe sandbox
> that can be expanded.
>
> There are two parts to the implication; one is called RestrictedPython and
> lives at:
>
> http://svn.zope.org/RestrictedPython/trunk
>
> The other part is "zope.security.untrustedpython", and it's part of the
> zope.security distribution; see:
>
> http://svn.zope.org/zope.security/trunk/src/zope/security/untrustedpython/
>
> for its specific code and docs.
>
> Both packages appear to have automated tests.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Thu Jul 17 23:04:40 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 17 Jul 2008 14:04:40 PDT
Subject: [Python-Dev] Issue2944: need a review
In-Reply-To: <1216323679.4911.6.camel@jenner> 
References: <1216323679.4911.6.camel@jenner>
Message-ID: <08Jul17.140446pdt."58698"@synergy1.parc.xerox.com>

These requests always have a higher probability of being addressed if
you summarize the issue in the request.

Bill

> Can anyone look at the patch for Issue2944?
> 
> I hope the issue can be fixed before the release of python 2.6.



From sidnei at enfoldsystems.com  Thu Jul 17 23:10:03 2008
From: sidnei at enfoldsystems.com (Sidnei da Silva)
Date: Thu, 17 Jul 2008 18:10:03 -0300
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>
	<20080717202531.DA16A3A4080@sparrow.telecommunity.com>
	<ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com>
Message-ID: <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com>

On Thu, Jul 17, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote:
> Thanks. Then python-dev is *definitely* the wrong forum. :-)

Definitely wrong forum for RestrictedPython-specifc questions, but not
if you consider those two paragraphs from Shane's email, which clarify
that Ranjith is seeking for information on possible AST changes that
might have occurred in Python 2.5:

"""
The safety of RestrictedPython has been validated in a somewhat formal
process with Python 2.4.  Ranjith is working to validate it with
Python 2.5.  He is first working to discover all changes between
Python 2.4 and 2.5 that might have affected the safety of a
RestrictedPython sandbox. Any changes to the AST, builtin functions,
methods of builtin types, etc., need to be evaluated for safety.

So, in general, he is looking for detailed lists of changes between
Python 2.4 and 2.5--more than the "What's New" doc.
"""

-- 
Sidnei da Silva
Enfold Systems http://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

From musiccomposition at gmail.com  Thu Jul 17 23:13:04 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 17 Jul 2008 16:13:04 -0500
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>
	<20080717202531.DA16A3A4080@sparrow.telecommunity.com>
	<ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com>
	<a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com>
Message-ID: <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com>

On Thu, Jul 17, 2008 at 4:10 PM, Sidnei da Silva
<sidnei at enfoldsystems.com> wrote:
> On Thu, Jul 17, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote:
>> Thanks. Then python-dev is *definitely* the wrong forum. :-)
>
> Definitely wrong forum for RestrictedPython-specifc questions, but not
> if you consider those two paragraphs from Shane's email, which clarify
> that Ranjith is seeking for information on possible AST changes that
> might have occurred in Python 2.5:
>
> """
> The safety of RestrictedPython has been validated in a somewhat formal
> process with Python 2.4.  Ranjith is working to validate it with
> Python 2.5.  He is first working to discover all changes between
> Python 2.4 and 2.5 that might have affected the safety of a
> RestrictedPython sandbox. Any changes to the AST, builtin functions,
> methods of builtin types, etc., need to be evaluated for safety.
>
> So, in general, he is looking for detailed lists of changes between
> Python 2.4 and 2.5--more than the "What's New" doc.
> """

Then I would recommend he look at Misc/NEWS. If he needs more
information, *specific* questions will bring the best results.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From tjreedy at udel.edu  Thu Jul 17 23:38:44 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 17 Jul 2008 17:38:44 -0400
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>	<20080717202531.DA16A3A4080@sparrow.telecommunity.com>	<ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com>	<a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com>
	<1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com>
Message-ID: <g5oe52$u64$2@ger.gmane.org>



Benjamin Peterson wrote:
> On Thu, Jul 17, 2008 at 4:10 PM, Sidnei da Silva
> <sidnei at enfoldsystems.com> wrote:
>
>> """
>> The safety of RestrictedPython has been validated in a somewhat formal
>> process with Python 2.4.  Ranjith is working to validate it with
>> Python 2.5.  He is first working to discover all changes between
>> Python 2.4 and 2.5 that might have affected the safety of a
>> RestrictedPython sandbox. Any changes to the AST, builtin functions,
>> methods of builtin types, etc., need to be evaluated for safety.
>>
>> So, in general, he is looking for detailed lists of changes between
>> Python 2.4 and 2.5--more than the "What's New" doc.
>> """
> 
> Then I would recommend he look at Misc/NEWS. If he needs more
> information, *specific* questions will bring the best results.

Ultimately, he can do a diff of the corresponding C files, and read the 
checkin messages that hopefully explain some of the changes.


From ncoghlan at gmail.com  Fri Jul 18 00:41:12 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Jul 2008 08:41:12 +1000
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <487F7F94.9080201@palladion.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5406.1000802@voidspace.org.uk>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>	<871w1tz219.fsf@benfinney.id.au>	<487F1C8F.7080405@gmail.com>
	<487F712D.6050908@ronadam.com> <487F7F94.9080201@palladion.com>
Message-ID: <487FCA88.4000405@gmail.com>

Tres Seaver wrote:
> Ron Adam wrote:
>> Nick Coghlan wrote:
> 
>>> The essence of the function remains unchanged - you're still asserting 
>>> that a particular exception is raised. Returning the actual exception 
>>> object that was caught is merely a convenience that makes a lot of sense.
>> I'm not sure I understand...
> 
>> If "a particular exception is raised", every thing is good and there is no 
>> error to report.  ie... the code being tested did the right thing.
> 
> I *think* that in this case the propoent wants to be able to make
> further assertions about the raised exception (e.g., to test its message).

Exactly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Fri Jul 18 00:51:11 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Jul 2008 08:51:11 +1000
Subject: [Python-Dev] Issue2944: need a review
In-Reply-To: <ca471dc20807171403t638fb207h218fa11fb97caa5f@mail.gmail.com>
References: <1216323679.4911.6.camel@jenner>
	<ca471dc20807171403t638fb207h218fa11fb97caa5f@mail.gmail.com>
Message-ID: <487FCCDF.1070605@gmail.com>

Guido van Rossum wrote:
> Suggestion for people asking developers to look into issues: indicate
> more than the issue number in the email. Show the issue summary, other
> relevant metadata, and what needs to be done in some detail. This will
> pique the interest of those who *can* help (and allow people who can't
> help anyway to skip the message more efficiently).

Links of the form http://bugs.python.org/issue2944 don't hurt either :)

That particular issue is an asyncore problem, so I cc'ed Josiah directly 
on this message.

Cheers,
Nick.

> 
> On Thu, Jul 17, 2008 at 12:41 PM, Alexander Shigin <shigin at rambler-co.ru> wrote:
>> Can anyone look at the patch for Issue2944?
>>
>> I hope the issue can be fixed before the release of python 2.6.
> 


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mhammond at skippinet.com.au  Fri Jul 18 00:51:45 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Fri, 18 Jul 2008 08:51:45 +1000
Subject: [Python-Dev] assertRaises
In-Reply-To: <e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>	<A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org>	<loom.20080717T091112-655@post.gmane.org>	<loom.20080717T095323-385@post.gmane.org>
	<e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com>
Message-ID: <03e501c8e85f$b07b8000$11728000$@com.au>

> >> Let's just make assertRaises return the exception instance, it seems
> >> like it feels the need correctly.
> >
> > and I meant "fills", not "feels", obviously...
> 
> +1 : enriching the existing method in a way that's perfectly
> transparent and innocuous to all existing uses _feels_ right, because
> it's more practical.

For the record, and primarily to give the champion of this proposal a little
more resolve the run the python-dev gauntlet on this issue given the recent
- err - spirited discussions, I'm also strongly +1 on this idea.

Cheers,

Mark


From martin at v.loewis.de  Fri Jul 18 01:27:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 18 Jul 2008 01:27:08 +0200
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
Message-ID: <487FD54C.2040308@v.loewis.de>

> The Windows buildbots are not very happy, though. test_ssl and
> test_bsddb and constantly failing on both the trunk and py3k. I don't
> know much about either of these items (or Windows for that matter), so
> any help would be greatly appreciated.

bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
3k. I somewhat doubt that this gets resolved before the release, so
bsddb users might need to skip 3.0.

Regards,
Martin

From musiccomposition at gmail.com  Fri Jul 18 02:06:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 17 Jul 2008 19:06:37 -0500
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <487FD54C.2040308@v.loewis.de>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
Message-ID: <1afaf6160807171706p8c2b226wc7bd8d0a072840fa@mail.gmail.com>

On Thu, Jul 17, 2008 at 6:27 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> The Windows buildbots are not very happy, though. test_ssl and
>> test_bsddb and constantly failing on both the trunk and py3k. I don't
>> know much about either of these items (or Windows for that matter), so
>> any help would be greatly appreciated.
>
> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
> 3k. I somewhat doubt that this gets resolved before the release, so
> bsddb users might need to skip 3.0.

Ok. ssl has stopped failing now (Thanks Bill). test_wsgiref is failing
on the trunk, but as it is an externally maintained module, there
isn't much I can do about it. (see issue #3401). Overall, though, I
think we've done what we can here, so these shouldn't hold up the
release.

>
> Regards,
> Martin
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From barry at python.org  Fri Jul 18 04:13:33 2008
From: barry at python.org (barry at python.org)
Date: Thu, 17 Jul 2008 22:13:33 -0400
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <487FD54C.2040308@v.loewis.de>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
Message-ID: <20080717221333.1d61bfe1@mission.wooz.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 18, 2008, at 01:27 AM, Martin v. L?wis wrote:

>> The Windows buildbots are not very happy, though. test_ssl and
>> test_bsddb and constantly failing on both the trunk and py3k. I don't
>> know much about either of these items (or Windows for that matter), so
>> any help would be greatly appreciated.
>
>bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
>3k. I somewhat doubt that this gets resolved before the release, so
>bsddb users might need to skip 3.0.

We have no showstoppers and several green buildbots, so I'm going to make the
releases tonight.  Please, NO CHECKINS until I say so, or ping me on
#python-dev.

As for bsddb, we'll make a determination after beta3.  If it's terminally
busted for Python 3.0, so be it.

Thanks everyone for working so hard at getting beta2 ready.
- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIf/xT2YZpQepbvXERAtmoAKCyUkejAFxFG10Bsn/CJjZfGy0B9QCeMO0z
momJmXRLCdmxs84j2hXGrTY=
=Y3wS
-----END PGP SIGNATURE-----

From barry at python.org  Fri Jul 18 04:16:38 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Jul 2008 22:16:38 -0400
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <487F4FAE.3050600@holdenweb.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<487F4FAE.3050600@holdenweb.com>
Message-ID: <20080717221638.226ae4ee@mission.wooz.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 17, 2008, at 09:57 AM, Steve Holden wrote:

>Barry Warsaw wrote:
>[...]
>> 
>> I'll note that I plan to hold the beta3 releases until all release 
>> blocker and deferred blockers are resolved.
>> 
>Ian Ozsvald of ShowMeDo.com noticed a blog post of mine about this beta, 
>and responded as follows:
>
>> Re. http://holdenweb.blogspot.com/2008/07/cpython-getting-serious-about-quality.html
>> If someone on the dev side wanted to make a 5-10 minute 'cast on how to check out the python base, build it, test it, check stuff back in, we'd be honoured to host it.  Obviously the entire process deserves more attention but a short overview pointing out the key steps would be:
>> 1) simple for the author to create
>> 2) easy for possible contributors to watch and receive inspiration
>> 3) powerful enough to attract another set of developers into the fold
>> so that might be a useful, low-cost, quick win to help with the current effort?
>
>I'd do this myself but a) I'm not the obvious candidate and b) I'm too 
>busy to give it my attention now.
>
>If someone wanted to tackle this, it might help getting people testing 
>for the next beta, and also to recruit more testers in the long-term.

It's a great idea, but unfortunately I also don't have time to do this right
now.

- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIf/0G2YZpQepbvXERAtNzAJ9Os117wAD8KCkhkt6x+jhKuJ4snwCfbjXS
VtkFGzUUTzg+ersqU3+TObg=
=jdUf
-----END PGP SIGNATURE-----

From fdrake at acm.org  Fri Jul 18 04:30:38 2008
From: fdrake at acm.org (Fred Drake)
Date: Thu, 17 Jul 2008 22:30:38 -0400
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <487FD54C.2040308@v.loewis.de>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
Message-ID: <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>

On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
> 3k. I somewhat doubt that this gets resolved before the release, so
> bsddb users might need to skip 3.0.


In fact, bsddb as packages in core Python has rarely been in good shape.

Has anyone actually considered that bsddb might do better if  
maintained strictly as an external package?  That would make it easier  
for the any maintainers to release updates, and removes a source of  
frustration for users who either don't need it or would rather wait  
for a happier version.

I think this is worth considering.  I vaguely recall that the bsddb  
module was maintained before it was incorporated into the core Python  
release.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From guido at python.org  Fri Jul 18 04:37:32 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 19:37:32 -0700
Subject: [Python-Dev] [Python-3000]  No beta2 tonight
In-Reply-To: <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
Message-ID: <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>

On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
>>
>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
>> 3k. I somewhat doubt that this gets resolved before the release, so
>> bsddb users might need to skip 3.0.
>
>
> In fact, bsddb as packages in core Python has rarely been in good shape.
>
> Has anyone actually considered that bsddb might do better if maintained
> strictly as an external package?  That would make it easier for the any
> maintainers to release updates, and removes a source of frustration for
> users who either don't need it or would rather wait for a happier version.
>
> I think this is worth considering.  I vaguely recall that the bsddb module
> was maintained before it was incorporated into the core Python release.

+1. In my recollection maintaining bsddb has been nothing but trouble
right from the start when we were all sitting together at "Zope Corp
North" in a rented office in McLean... We can remove it from 3.0. We
can't really remove it from 2.6, but we can certainly start
end-of-lifing it in 2.6.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ranjithkannikara at gmail.com  Fri Jul 18 04:40:58 2008
From: ranjithkannikara at gmail.com (ranjith kannikara)
Date: Fri, 18 Jul 2008 08:10:58 +0530
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>
	<20080717202531.DA16A3A4080@sparrow.telecommunity.com>
	<ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com>
	<a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com>
	<1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com>
Message-ID: <20aa8c370807171940w1710ae44ga986f6f5f8a8bd85@mail.gmail.com>

Hi ,

I am really sorry that I couldn,t make clear in the first mail that
What I actually need. But Shane and my Gsoc mentor Sidnei have already
made it clear I think.

I would like to know how much the AST have been changed on moving from
python2.4 to python 2.5 so that I can bring the corresponding changes
in Zope2 to get it adapted to those changes in the AST. Also the
changes that have come to the builtin function and so which should
have affected the safety of a RestrictedPython sandbox.

As Shane have told a detailed list of such changes can help me than
the "Whats New" doc...

Ranjith.

On Fri, Jul 18, 2008 at 2:43 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Thu, Jul 17, 2008 at 4:10 PM, Sidnei da Silva
> <sidnei at enfoldsystems.com> wrote:
>> On Thu, Jul 17, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote:
>>> Thanks. Then python-dev is *definitely* the wrong forum. :-)
>>
>> Definitely wrong forum for RestrictedPython-specifc questions, but not
>> if you consider those two paragraphs from Shane's email, which clarify
>> that Ranjith is seeking for information on possible AST changes that
>> might have occurred in Python 2.5:
>>
>> """
>> The safety of RestrictedPython has been validated in a somewhat formal
>> process with Python 2.4.  Ranjith is working to validate it with
>> Python 2.5.  He is first working to discover all changes between
>> Python 2.4 and 2.5 that might have affected the safety of a
>> RestrictedPython sandbox. Any changes to the AST, builtin functions,
>> methods of builtin types, etc., need to be evaluated for safety.
>>
>> So, in general, he is looking for detailed lists of changes between
>> Python 2.4 and 2.5--more than the "What's New" doc.
>> """
>
> Then I would recommend he look at Misc/NEWS. If he needs more
> information, *specific* questions will bring the best results.
>
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
>

From barry at python.org  Fri Jul 18 04:51:06 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Jul 2008 22:51:06 -0400
Subject: [Python-Dev] NO CHECKINS - BETA 2 COMING
Message-ID: <8914AB44-1B5D-45B9-9EB9-DDC728B1B032@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Please, no checkins on the 3.0 or 2.6 branches until further notice.   
We're a go with the releases tonight.  Email is not the quickest way  
to get my attention.  For that, use irc on freenode, #python-dev.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSIAFGnEjvBPtnXfVAQIBmwP/VzIYu58DzSND0CNM0NKfsWVS9yTnBVi8
EijaxenUv0rRVloCawJbK8cMOyI0JmEa4GE5bMrCf+o9niRgFeJxrYEaw1jN4fWW
dsXH8Fuw/nHpoUnFDbu5ePaBdvcOSuWKTS/iGq2i9DzfYXdx+FaXqZ/mhx43bY3o
eCF7I+y/fPg=
=iTgT
-----END PGP SIGNATURE-----

From barry at python.org  Fri Jul 18 04:52:40 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Jul 2008 22:52:40 -0400
Subject: [Python-Dev] [Python-3000]  No beta2 tonight
In-Reply-To: <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
Message-ID: <EA2CA734-E0D1-49AB-8E14-1AE30082F971@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 17, 2008, at 10:37 PM, Guido van Rossum wrote:

> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
>>>
>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged  
>>> into
>>> 3k. I somewhat doubt that this gets resolved before the release, so
>>> bsddb users might need to skip 3.0.
>>
>>
>> In fact, bsddb as packages in core Python has rarely been in good  
>> shape.
>>
>> Has anyone actually considered that bsddb might do better if  
>> maintained
>> strictly as an external package?  That would make it easier for the  
>> any
>> maintainers to release updates, and removes a source of frustration  
>> for
>> users who either don't need it or would rather wait for a happier  
>> version.
>>
>> I think this is worth considering.  I vaguely recall that the bsddb  
>> module
>> was maintained before it was incorporated into the core Python  
>> release.
>
> +1. In my recollection maintaining bsddb has been nothing but trouble
> right from the start when we were all sitting together at "Zope Corp
> North" in a rented office in McLean... We can remove it from 3.0. We
> can't really remove it from 2.6, but we can certainly start
> end-of-lifing it in 2.6.

+1, but please, after I make tonight's releases. :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSIAFenEjvBPtnXfVAQKEewP+OWCBAH437X4+EptdcuIFFYrCCVXqbrV4
F2dZMyv/RU0jYgd6YTLspklEIzuEcH1sxYsnh2q4aWNfFlfL50LPf1/TKurZpO2C
9CrjEZpTcK0k5z5FCxAOLdVFLQDC4w7FGYop3uR27g5u9KCLdQvTrPs/mZllv263
/g2YdwhZFEE=
=NAOe
-----END PGP SIGNATURE-----

From barry at python.org  Fri Jul 18 05:42:31 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Jul 2008 23:42:31 -0400
Subject: [Python-Dev] RELEASED Python 2.6b2 and 3.0b2
Message-ID: <E9E66405-B49F-479C-A9E4-47F48FBF1AA5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I  
am happy to announce the second beta releases of Python 2.6 and Python  
3.0.

Please note that these are beta releases, and as such are not suitable  
for production environments.  We continue to strive for a high degree  
of quality, and these releases are intended to freeze the feature set  
for Python 2.6 and 3.0.

 From now until the planned final releases in October 2008, we will be  
fixing known problems and stabilizing these new Python versions.  You  
can help by downloading and testing them, providing feedback and  
hopefully helping to fix bugs.  You can also use these releases to  
determine how changes in 2.6 and 3.0 might impact you.

ONLY ONE MORE BETA RELEASE IS PLANNED, so now is a great time to  
download the releases and try them with your code.  If you find things  
broken or incorrect, please submit bug reports at

     http://bugs.python.org

For more information and downloadable distributions, see the Python  
2.6 website:

     http://www.python.org/download/releases/2.6/

and the Python 3.0 web site:

     http://www.python.org/download/releases/3.0/

See PEP 361 for release schedule details:

     http://www.python.org/dev/peps/pep-0361/

Enjoy,
- -Barry

Barry Warsaw
barry at python.org
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSIARKHEjvBPtnXfVAQL7AQQAjPSbfKz2Oh/au/hPzS4x2IR5/R6FVe+g
o9UYrONNRKJ14UHpbZRzvIvw/4G3PdpzzGxjYFIhVGEesEGJnMzT3YdkMHt4NW9d
HOZxL3hseGbTdpUJPCsIkNG+4hX7iuY3NSV81Z75LGAL4tqbooGqwwUslXMT5f8s
lRrZUcBRKZ0=
=ju6s
-----END PGP SIGNATURE-----

From barry at python.org  Fri Jul 18 05:50:24 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Jul 2008 23:50:24 -0400
Subject: [Python-Dev] SVN IS OPEN
Message-ID: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The releases have been made, so both the 3.0 branch and the trunk  
(2.6) are now open for commits.  Remember, there's only one more  
planned beta, and we /really/ want to try to hit the October 1st  
deadline.  Let's do everything we can to stabilize these releases,  
address the blockers, and turn those buildbots green.

Thanks everyone.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSIATAHEjvBPtnXfVAQIWLAP+OVUXh3H5j8WN4cm3fHcLkgd+E4FuWWwb
zKr7JPYUHhgvwpYB5iZjQuJe/62qKXh70wNWyDcxLi9/TEbx8NvhQ+nMbHAJkfE2
1/HkP9bbgiwE/JYJsN0a0DK/MSpwvxfxxakQrQV9H8SiL5aDqlVCFdzLu31SkOhx
iQ+iTCqwIdY=
=UOPo
-----END PGP SIGNATURE-----

From guido at python.org  Fri Jul 18 06:05:35 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Jul 2008 21:05:35 -0700
Subject: [Python-Dev] SVN IS OPEN
In-Reply-To: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org>
References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org>
Message-ID: <ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com>

On Thu, Jul 17, 2008 at 8:50 PM, Barry Warsaw <barry at python.org> wrote:
> The releases have been made, so both the 3.0 branch and the trunk (2.6) are
> now open for commits.  Remember, there's only one more planned beta, and we
> /really/ want to try to hit the October 1st deadline.  Let's do everything
> we can to stabilize these releases, address the blockers, and turn those
> buildbots green.

Thanks Barry! Indeed, I want everyone to focus on stabilization and
release blockers (and the occasional speed-up). Be extra careful with
what you submit between now and October, ask for a code review unless
you're really sure. Also, remember, NO NEW FEATURES!

> Thanks everyone.
> - -Barry

And thanks Barry for doing the release!!!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Fri Jul 18 06:24:12 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 18 Jul 2008 06:24:12 +0200
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <20aa8c370807171940w1710ae44ga986f6f5f8a8bd85@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>	<bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com>	<20080717202531.DA16A3A4080@sparrow.telecommunity.com>	<ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com>	<a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com>	<1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com>
	<20aa8c370807171940w1710ae44ga986f6f5f8a8bd85@mail.gmail.com>
Message-ID: <48801AEC.3090606@v.loewis.de>

> I would like to know how much the AST have been changed on moving from
> python2.4 to python 2.5 so that I can bring the corresponding changes
> in Zope2 to get it adapted to those changes in the AST. 

Notice that Parser/Python.asdl is new in Python 2.5, so (by today's
terminology) Python 2.4 did not really have any AST. The relevant NEWS
entry is

- A new AST parser implementation was completed. The abstract
  syntax tree is available for read-only (non-compile) access
  to Python code; an _ast module was added.

Python's prior notion of abstract syntax (which is rather a CST)
directly reflects from the grammar; do

  svn log Grammar/Grammar

to find out what exactly was changed.

Regards,
Martin

From brett at python.org  Fri Jul 18 07:43:15 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Jul 2008 22:43:15 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
Message-ID: <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>

On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
>>>
>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
>>> 3k. I somewhat doubt that this gets resolved before the release, so
>>> bsddb users might need to skip 3.0.
>>
>>
>> In fact, bsddb as packages in core Python has rarely been in good shape.
>>
>> Has anyone actually considered that bsddb might do better if maintained
>> strictly as an external package?  That would make it easier for the any
>> maintainers to release updates, and removes a source of frustration for
>> users who either don't need it or would rather wait for a happier version.
>>
>> I think this is worth considering.  I vaguely recall that the bsddb module
>> was maintained before it was incorporated into the core Python release.
>
> +1. In my recollection maintaining bsddb has been nothing but trouble
> right from the start when we were all sitting together at "Zope Corp
> North" in a rented office in McLean... We can remove it from 3.0. We
> can't really remove it from 2.6, but we can certainly start
> end-of-lifing it in 2.6.
>

Unless I hear otherwise, I will add it to PEP 3108.

-Brett

From jml at mumak.net  Fri Jul 18 07:50:06 2008
From: jml at mumak.net (Jonathan Lange)
Date: Fri, 18 Jul 2008 15:50:06 +1000
Subject: [Python-Dev] assertRaises
In-Reply-To: <03e501c8e85f$b07b8000$11728000$@com.au>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>
	<87ej5tz6fh.fsf@benfinney.id.au>
	<20080717010730.GA29299@steerpike.home.puzzling.org>
	<87abghz4cu.fsf@benfinney.id.au>
	<20080717014556.GB29299@steerpike.home.puzzling.org>
	<A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org>
	<loom.20080717T091112-655@post.gmane.org>
	<loom.20080717T095323-385@post.gmane.org>
	<e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com>
	<03e501c8e85f$b07b8000$11728000$@com.au>
Message-ID: <d06a5cd30807172250s20b94221tf4fc2cec2a98e27f@mail.gmail.com>

On Fri, Jul 18, 2008 at 8:51 AM, Mark Hammond <mhammond at skippinet.com.au> wrote:
>> >> Let's just make assertRaises return the exception instance, it seems
>> >> like it feels the need correctly.
>> >
>> > and I meant "fills", not "feels", obviously...
>>
>> +1 : enriching the existing method in a way that's perfectly
>> transparent and innocuous to all existing uses _feels_ right, because
>> it's more practical.
>
> For the record, and primarily to give the champion of this proposal a little
> more resolve the run the python-dev gauntlet on this issue given the recent
> - err - spirited discussions, I'm also strongly +1 on this idea.
>

I've got code and tests for this. I'll file a patch on the tracker as
soon as I get the chance -- Internet is dicey at the moment.

jml

From brett at python.org  Fri Jul 18 07:52:06 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Jul 2008 22:52:06 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <g5j4v3$6hi$1@ger.gmane.org>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
Message-ID: <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>

On Tue, Jul 15, 2008 at 2:31 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> Benjamin Peterson <musiccomposition at gmail.com> wrote:
>> Can we push branches?
>
> The git-daemon is setup as read-only.  If you have write access to
> the SVN repository then you can push back changes using git-svn.
> That's quite a nice way to work, IMHO and provides an easy path for
> people who are used to Subversion and want to dip their toe in the
> dvcs waters.
>
> While there is no support on code.python.org for publishing your own
> git branches, there's no reason why you couldn't publish a branch
> using some other host (it's a dvcs after all).  In the short term,
> perhaps using private branches in combination with "git
> format-patch" and email is the easiest process.
>

So I have a change that I have in git. I would like to just push it to
svn through git. I ran ``git svn dcommit``, but then a ton of stuff
came streaming by on my terminal. Is that normal?

Also, how does one refer to revisions for things like ``git diff``? Is
it really the commit commit number (which I think is something like a
SHA-1)?

-Brett

From nas at arctrix.com  Fri Jul 18 08:54:30 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 18 Jul 2008 00:54:30 -0600
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
Message-ID: <20080718065430.GA6991@arctrix.com>

[back on the list]

On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote:
> Turned out to be a rebuild::
> 
> ....
> r65077 = 82d954e8c20c91562c4c660859d17756cba10992
> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a
> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536
> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771
>
> How do I know what is going to be sent? ``git log`` seems to suggest
> something by not listing a git-svn-id for my last commit, but is that
> really the best I got?

The command

    git log git-svn..

will show you changes in your HEAD (by default "master") tree that
are not in the remote tree (git-svn).

> Is there some other way to see what will be pushed?

I like running "gitk" before I push something.

> And how do I diff easily between commits?

It depends on what you want, exactly.  Maybe you can describe some
use cases.  A DVCS can't use identify revisions like SVN does.
Generally I find myself using heads or tags to identify versions in
combination with the ^ operator.  For example,

    git diff HEAD^

would show the difference between the current working tree and the
commit before the head of the stored tree.  If you want the patch
for a single commit, use "git show <object>".  For example, "git
show" will display the last commit.  To see amk's typo fix:

    git show 6cadb9c1b7e30a8b66cdba01cd79aa6397a07080

You can also abbreviate the commit id, eg.

    git show 6cadb9

As I say in my guide, "git format-patch" and "git am" are very handy
when slinging patches around (e.g. to and from a bug tracker or
mailing list).

HTH,

  Neil

From theller at ctypes.org  Fri Jul 18 09:14:35 2008
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 18 Jul 2008 09:14:35 +0200
Subject: [Python-Dev] Windows buildbot trick
Message-ID: <g5pfrp$d9e$1@ger.gmane.org>

The most annoying thing on the windows buildbots is when Python crashes hard
(with a general protection fault or stack overflow, for example).
Usually the system pops up a dialog in this case which allows to
attach a debugger to the process.  This dialog will stay open until
the maintainer manually closes it, and will prevent the next builds.

On my boxes I have inserted these two lines into the PythonXY/Scripts/buildbot
script, right at the top:

  import win32api; win32api.SetErrorMode(7)
  print "SetErrorMode(7) called."

These lines prevent the dialog box to be displayed for critical errors.

For example, on Windows AMD64 the test_cpickle test in the trunk currently fails with
a stack overflow; instead of hanging with the mentioned dialog box open the test now terminates:

http://www.python.org/dev/buildbot/trunk/amd64%20XP%20trunk/builds/40/step-test/0

-- 
Thanks,
Thomas


From solipsis at pitrou.net  Fri Jul 18 11:46:03 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Jul 2008 09:46:03 +0000 (UTC)
Subject: [Python-Dev] SVN IS OPEN
References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org>
	<ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com>
Message-ID: <loom.20080718T094343-311@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> Thanks Barry! Indeed, I want everyone to focus on stabilization and
> release blockers (and the occasional speed-up). Be extra careful with
> what you submit between now and October, ask for a code review unless
> you're really sure. Also, remember, NO NEW FEATURES!

What should be done about http://bugs.python.org/issue2834?
Should it make it into 3.0 or be delayed until 3.1?

(code-wise there isn't left to be done, take a decision about a potential "(?
a)" modifier and then implement it)



From ncoghlan at gmail.com  Fri Jul 18 12:24:56 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Jul 2008 20:24:56 +1000
Subject: [Python-Dev] SVN IS OPEN
In-Reply-To: <loom.20080718T094343-311@post.gmane.org>
References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org>	<ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com>
	<loom.20080718T094343-311@post.gmane.org>
Message-ID: <48806F78.4060209@gmail.com>

Antoine Pitrou wrote:
> Guido van Rossum <guido <at> python.org> writes:
>> Thanks Barry! Indeed, I want everyone to focus on stabilization and
>> release blockers (and the occasional speed-up). Be extra careful with
>> what you submit between now and October, ask for a code review unless
>> you're really sure. Also, remember, NO NEW FEATURES!
> 
> What should be done about http://bugs.python.org/issue2834?
> Should it make it into 3.0 or be delayed until 3.1?
> 
> (code-wise there isn't left to be done, take a decision about a potential "(?
> a)" modifier and then implement it)

Before beta 3, I think if we need minor features (such as re.ASCII) to 
fix fairly major bugs (re.IGNORECASE not working properly by default in 
Py3k) then they should probably go in (case by case permission still 
needed from Barry or Guido though, as with any post-beta1 feature addition).

Once we're past beta 3 and heading for rc1 then the bar for approving 
feature additions to fix bugs will move even higher though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From solipsis at pitrou.net  Fri Jul 18 12:39:57 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Jul 2008 10:39:57 +0000 (UTC)
Subject: [Python-Dev] SVN IS OPEN
References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org>	<ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com>
	<loom.20080718T094343-311@post.gmane.org>
	<48806F78.4060209@gmail.com>
Message-ID: <loom.20080718T103811-83@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> Before beta 3, I think if we need minor features (such as re.ASCII) to 
> fix fairly major bugs (re.IGNORECASE not working properly by default in 
> Py3k) then they should probably go in (case by case permission still 
> needed from Barry or Guido though, as with any post-beta1 feature addition).

Ok I'll tackle the remaining of it before beta 3 then.

Thanks

Antoine.



From guido at python.org  Fri Jul 18 16:13:14 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Jul 2008 07:13:14 -0700
Subject: [Python-Dev] SVN IS OPEN
In-Reply-To: <48806F78.4060209@gmail.com>
References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org>
	<ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com>
	<loom.20080718T094343-311@post.gmane.org> <48806F78.4060209@gmail.com>
Message-ID: <ca471dc20807180713h4177eacese3b8b9821164ce7a@mail.gmail.com>

On Fri, Jul 18, 2008 at 3:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Antoine Pitrou wrote:
>>
>> Guido van Rossum <guido <at> python.org> writes:
>>>
>>> Thanks Barry! Indeed, I want everyone to focus on stabilization and
>>> release blockers (and the occasional speed-up). Be extra careful with
>>> what you submit between now and October, ask for a code review unless
>>> you're really sure. Also, remember, NO NEW FEATURES!
>>
>> What should be done about http://bugs.python.org/issue2834?
>> Should it make it into 3.0 or be delayed until 3.1?
>>
>> (code-wise there isn't left to be done, take a decision about a potential
>> "(?
>> a)" modifier and then implement it)
>
> Before beta 3, I think if we need minor features (such as re.ASCII) to fix
> fairly major bugs (re.IGNORECASE not working properly by default in Py3k)
> then they should probably go in (case by case permission still needed from
> Barry or Guido though, as with any post-beta1 feature addition).
>
> Once we're past beta 3 and heading for rc1 then the bar for approving
> feature additions to fix bugs will move even higher though.

Right. Once b3 is out the bar will be *really* high. Once rc1 is out
the bar will be raised even for bug fixes! So do this now.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jul 18 16:21:45 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Jul 2008 07:21:45 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
Message-ID: <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>

On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote:
> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
>>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
>>>>
>>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
>>>> 3k. I somewhat doubt that this gets resolved before the release, so
>>>> bsddb users might need to skip 3.0.
>>>
>>>
>>> In fact, bsddb as packages in core Python has rarely been in good shape.
>>>
>>> Has anyone actually considered that bsddb might do better if maintained
>>> strictly as an external package?  That would make it easier for the any
>>> maintainers to release updates, and removes a source of frustration for
>>> users who either don't need it or would rather wait for a happier version.
>>>
>>> I think this is worth considering.  I vaguely recall that the bsddb module
>>> was maintained before it was incorporated into the core Python release.
>>
>> +1. In my recollection maintaining bsddb has been nothing but trouble
>> right from the start when we were all sitting together at "Zope Corp
>> North" in a rented office in McLean... We can remove it from 3.0. We
>> can't really remove it from 2.6, but we can certainly start
>> end-of-lifing it in 2.6.

> Unless I hear otherwise, I will add it to PEP 3108.

Please do!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From josiah.carlson at gmail.com  Fri Jul 18 16:57:01 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Fri, 18 Jul 2008 07:57:01 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
Message-ID: <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>

On Fri, Jul 18, 2008 at 7:21 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote:
>> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote:
>>> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
>>>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
>>>>>
>>>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
>>>>> 3k. I somewhat doubt that this gets resolved before the release, so
>>>>> bsddb users might need to skip 3.0.
>>>>
>>>>
>>>> In fact, bsddb as packages in core Python has rarely been in good shape.
>>>>
>>>> Has anyone actually considered that bsddb might do better if maintained
>>>> strictly as an external package?  That would make it easier for the any
>>>> maintainers to release updates, and removes a source of frustration for
>>>> users who either don't need it or would rather wait for a happier version.
>>>>
>>>> I think this is worth considering.  I vaguely recall that the bsddb module
>>>> was maintained before it was incorporated into the core Python release.
>>>
>>> +1. In my recollection maintaining bsddb has been nothing but trouble
>>> right from the start when we were all sitting together at "Zope Corp
>>> North" in a rented office in McLean... We can remove it from 3.0. We
>>> can't really remove it from 2.6, but we can certainly start
>>> end-of-lifing it in 2.6.
>
>> Unless I hear otherwise, I will add it to PEP 3108.
>
> Please do!

Invariably, when someone goes and removes a module, someone else is
going to complain, "but I used feature X, not having feature X will
break my code."  We, as maintainers can then say, "if you cared,
maintain it."  But I'm not sure that is the greatest thing to tell
people.  I suspect that we may have to include some sort of
"work-alike" for 2.7 and if not 3.0, 3.1 .  If I were to vote for a
work-alike, it would be based on sqlite.  For one of the most common
use-cases (bsddb.btree), simple sqlite code can be written to do the
right thing.  Recno is a little more tricky, but can also be done.
The bsddb hash may not be possible, because sqlite doesn't support
hashed indices :/.

Just an idea.

 - Josiah

From guido at python.org  Fri Jul 18 17:11:06 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Jul 2008 08:11:06 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
Message-ID: <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>

On Fri, Jul 18, 2008 at 7:57 AM, Josiah Carlson
<josiah.carlson at gmail.com> wrote:
> Invariably, when someone goes and removes a module, someone else is
> going to complain, "but I used feature X, not having feature X will
> break my code."  We, as maintainers can then say, "if you cared,
> maintain it."  But I'm not sure that is the greatest thing to tell
> people.  I suspect that we may have to include some sort of
> "work-alike" for 2.7 and if not 3.0, 3.1 .  If I were to vote for a
> work-alike, it would be based on sqlite.  For one of the most common
> use-cases (bsddb.btree), simple sqlite code can be written to do the
> right thing.  Recno is a little more tricky, but can also be done.
> The bsddb hash may not be possible, because sqlite doesn't support
> hashed indices :/.

In my mind, BSDDB is pretty much the most heavy-weight extension we're
maintaining. I think it's an illusion that a sqlite-based look-alike
is going to fool anyone. The correct solution is to take support for
bsddb to a separate project where those who care about it can maintain
it together. That also makes it a lot easier to track the versions of
Berkeley DB as they come out.

Of course, you're free to try writing the work-alike you're proposing. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From sidnei at enfoldsystems.com  Fri Jul 18 17:42:11 2008
From: sidnei at enfoldsystems.com (Sidnei da Silva)
Date: Fri, 18 Jul 2008 12:42:11 -0300
Subject: [Python-Dev] Windows buildbot trick
In-Reply-To: <g5pfrp$d9e$1@ger.gmane.org>
References: <g5pfrp$d9e$1@ger.gmane.org>
Message-ID: <a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com>

On Fri, Jul 18, 2008 at 4:14 AM, Thomas Heller <theller at ctypes.org> wrote:
> The most annoying thing on the windows buildbots is when Python crashes hard
> (with a general protection fault or stack overflow, for example).
> Usually the system pops up a dialog in this case which allows to
> attach a debugger to the process.  This dialog will stay open until
> the maintainer manually closes it, and will prevent the next builds.
>
> On my boxes I have inserted these two lines into the PythonXY/Scripts/buildbot
> script, right at the top:
>
>  import win32api; win32api.SetErrorMode(7)
>  print "SetErrorMode(7) called."
>
> These lines prevent the dialog box to be displayed for critical errors.
>
> For example, on Windows AMD64 the test_cpickle test in the trunk currently fails with
> a stack overflow; instead of hanging with the mentioned dialog box open the test now terminates:
>
> http://www.python.org/dev/buildbot/trunk/amd64%20XP%20trunk/builds/40/step-test/0

That's a great trick! I seem to remember that there is a way to turn
that off globally though, but not sure where. Maybe running
drwtsn32.exe and un-checking 'Visual Notification' does the trick?

-- 
Sidnei da Silva
Enfold Systems http://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

From sidnei at enfoldsystems.com  Fri Jul 18 17:53:08 2008
From: sidnei at enfoldsystems.com (Sidnei da Silva)
Date: Fri, 18 Jul 2008 12:53:08 -0300
Subject: [Python-Dev] Windows buildbot trick
In-Reply-To: <a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com>
References: <g5pfrp$d9e$1@ger.gmane.org>
	<a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com>
Message-ID: <a7a2b76b0807180853g524677cbk578f322b049010cc@mail.gmail.com>

On Fri, Jul 18, 2008 at 12:42 PM, Sidnei da Silva
<sidnei at enfoldsystems.com> wrote:
> That's a great trick! I seem to remember that there is a way to turn
> that off globally though, but not sure where. Maybe running
> drwtsn32.exe and un-checking 'Visual Notification' does the trick?

FWIW,  here's a kb article that describes this. Although it refers to
32-bit apps on 64-bit XP, I'm pretty sure you can do the same for
32-bit XP?

http://support.microsoft.com/kb/283150/EN-US/

-- 
Sidnei da Silva
Enfold Systems http://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

From status at bugs.python.org  Fri Jul 18 18:07:19 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 18 Jul 2008 18:07:19 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080718160719.ECE9778282@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (07/11/08 - 07/18/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1967 open (+38) / 13262 closed (+25) / 15229 total (+63)

Open issues with patches:   619

Average duration of open issues: 705 days.
Median duration of open issues: 1617 days.

Open Issues Breakdown
   open  1939 (+38)
pending    28 ( +0)

Issues Created Or Reopened (63)
_______________________________

IDLE - use enumerate instead of zip(count(), ...)                07/11/08
       http://bugs.python.org/issue3344    created  taleinat                  
       patch                                                                   

Patch for CGIHTTPServer.is_cgi function documentation            07/11/08
CLOSED http://bugs.python.org/issue3345    created  tebeka                    
       patch                                                                   

test_ossaudiodev fails                                           07/12/08
       http://bugs.python.org/issue3346    created  cartman                   
                                                                               

urllib.robotparser doesn't work in Py3k                          07/12/08
       http://bugs.python.org/issue3347    created  mgiuca                    
       patch                                                                   

Cannot start wsgiref simple server in Py3k                       07/12/08
       http://bugs.python.org/issue3348    created  mgiuca                    
       patch                                                                   

search for 'patch' produces roundup error                        07/12/08
CLOSED http://bugs.python.org/issue3349    created  techtonik                 
                                                                               

multiprocessing adds built-in types to the global copyreg.dispat 07/13/08
       http://bugs.python.org/issue3350    created  alexandre.vassalotti      
                                                                               

Python crashed                                                   07/14/08
CLOSED http://bugs.python.org/issue3351    created  yiyuan                    
                                                                               

Deficiencies in multiprocessing/threading API                    07/14/08
       http://bugs.python.org/issue3352    created  ncoghlan                  
                                                                               

make built-in tokenizer available via Python C API               07/14/08
       http://bugs.python.org/issue3353    created  effbot                    
                                                                               

Improve error reporting for the argument parsing API             07/14/08
       http://bugs.python.org/issue3354    reopened benjamin.peterson         
                                                                               

Display bug in :show-inheritance: for class with standard docstr 07/14/08
       http://bugs.python.org/issue3355    created  kumar303                  
                                                                               

some tests fail in debug mode (test_distutils, test_set)         07/14/08
       http://bugs.python.org/issue3356    created  pitrou                    
                                                                               

A bug in the __doc__ string of the sys module                    07/14/08
CLOSED http://bugs.python.org/issue3357    created  cheDu                     
                                                                               

2to3 Iterative Wildcard Matching                                 07/15/08
       http://bugs.python.org/issue3358    created  nedds                     
       patch                                                                   

add 'rbU' mode to open()                                         07/15/08
CLOSED http://bugs.python.org/issue3359    created  techtonik                 
                                                                               

Inconsistent type-deduction of decimal floating-point            07/15/08
CLOSED http://bugs.python.org/issue3360    created  richyk                    
                                                                               

pypi issue tracker link (python.org)                             07/15/08
CLOSED http://bugs.python.org/issue3361    created  tds333                    
                                                                               

locale.getpreferredencoding() gives bus error on Mac OS X 10.4.1 07/15/08
       http://bugs.python.org/issue3362    created  cfr                       
                                                                               

python version incorrectly reported in crash reports on Mac OS X 07/15/08
       http://bugs.python.org/issue3363    created  cfr                       
                                                                               

An ortographical typo in Zen of Python text                      07/15/08
CLOSED http://bugs.python.org/issue3364    created  cheDu                     
                                                                               

in module math, PI value seems to be wrong after digit 17th      07/15/08
CLOSED http://bugs.python.org/issue3365    created  TanaT                     
                                                                               

Add gamma and error functions to math module                     07/15/08
       http://bugs.python.org/issue3366    created  tjreedy                   
                                                                               

Uninitialized value read in parsetok.c                           07/15/08
       http://bugs.python.org/issue3367    created  krisvale                  
       patch, patch, easy                                                      

Memory leak in import.c                                          07/15/08
       http://bugs.python.org/issue3368    created  krisvale                  
       patch, patch, easy                                                      

memory leak in floatobject.c                                     07/15/08
       http://bugs.python.org/issue3369    created  krisvale                  
       patch, patch, easy                                                      

importing with_statement causes exec to raise syntax error on bl 07/15/08
       http://bugs.python.org/issue3370    created  mccredie                  
                                                                               

2to3 fails in Python 2.6                                         07/16/08
CLOSED http://bugs.python.org/issue3371    created  benjamin.peterson         
                                                                               

socket.setsockopt() is broken for multicast TTL and Loop options 07/16/08
CLOSED http://bugs.python.org/issue3372    created  niallo                    
                                                                               

sys recursion limit a lot shorter on trunk?                      07/16/08
       http://bugs.python.org/issue3373    created  esrever_otua              
                                                                               

Bisect upgrades: key/cmp/reverse, parameterized handedness       07/16/08
CLOSED http://bugs.python.org/issue3374    created  dan.uznanski              
       patch                                                                   

_multiprocessing.so build problems                               07/16/08
CLOSED http://bugs.python.org/issue3375    created  barry                     
                                                                               

Use Python 3 lexer for 3.0 docs                                  07/16/08
       http://bugs.python.org/issue3376    created  georg.brandl              
                                                                               

Invalid child node access in ast.c                               07/16/08
CLOSED http://bugs.python.org/issue3377    created  krisvale                  
                                                                               

Memory leak in pythonrun.c                                       07/16/08
       http://bugs.python.org/issue3378    created  krisvale                  
       patch, patch                                                            

Option to not-exit on test                                       07/16/08
       http://bugs.python.org/issue3379    created  pupeno                    
       patch                                                                   

documentation for ElementTree is unusable                        07/16/08
CLOSED http://bugs.python.org/issue3380    created  jrw at pobox.com             
                                                                               

`./configure --enable-framework --enable-universalsdk` fails bec 07/16/08
CLOSED http://bugs.python.org/issue3381    created  trentm                    
                                                                               

Make '%F' and float.__format__('F') convert results to upper cas 07/17/08
       http://bugs.python.org/issue3382    reopened eric.smith                
       easy                                                                    

ctypes.util fails to find libc in some environments              07/16/08
       http://bugs.python.org/issue3383    created  exarkun                   
       patch                                                                   

Documentation for re.findall and re.finditer lacks "ordering" in 07/16/08
       http://bugs.python.org/issue3384    created  jkugler                   
                                                                               

cPickle to pickle conversion in py3k missing methods             07/16/08
       http://bugs.python.org/issue3385    created  jnoller                   
                                                                               

[PATCH] distutils.sysconfig.get_python_lib prefix argument broke 07/16/08
       http://bugs.python.org/issue3386    created  pjenvey                   
       patch                                                                   

undefined array passed to CryptGenRandomBytes                    07/16/08
       http://bugs.python.org/issue3387    created  krisvale                  
       patch, patch, easy                                                      

With keyword not mentioned in Input Output tutorial              07/16/08
CLOSED http://bugs.python.org/issue3388    created  segfaulthunter            
       patch                                                                   

[PATCH] Allow custom logging Handlers in logging config files    07/16/08
CLOSED http://bugs.python.org/issue3389    created  pjenvey                   
       patch                                                                   

[PATCH] replace last has_key in unittest by in operator          07/17/08
       http://bugs.python.org/issue3390    created  grubert                   
       patch                                                                   

Idle uses old showwarning signature                              07/17/08
       http://bugs.python.org/issue3391    created  schuppenies               
       patch                                                                   

subprocess fails in select when descriptors are large            07/17/08
       http://bugs.python.org/issue3392    created  yorick                    
                                                                               

`cd Mac && make installmacsubtree` fails on Mac OS X < 10.5 beca 07/17/08
CLOSED http://bugs.python.org/issue3393    created  trentm                    
                                                                               

zipfile.writestr doesn't set external attributes, so files are e 07/17/08
       http://bugs.python.org/issue3394    created  swarren                   
       patch                                                                   

typo in test_multiprocessing.py:  should _debugInfo be _debug_in 07/17/08
CLOSED http://bugs.python.org/issue3395    created  marketdickinson           
       easy                                                                    

rlcompleter can't autocomplete members of callable objects       07/17/08
       http://bugs.python.org/issue3396    created  pitrou                    
                                                                               

Mac 3.0 build cannot find cachersrc.py                           07/17/08
CLOSED http://bugs.python.org/issue3397    created  barry-scott               
                                                                               

mac build 3.0 no MacOS module                                    07/17/08
CLOSED http://bugs.python.org/issue3398    created  barry-scott               
                                                                               

Memory corruption in multiprocessing module, OS X 10.5.4         07/17/08
       http://bugs.python.org/issue3399    created  marketdickinson           
                                                                               

dis module: undocumented new bytecodes                           07/17/08
       http://bugs.python.org/issue3400    created  tjreedy                   
                                                                               

wsgiref can't handle unicode environments                        07/17/08
       http://bugs.python.org/issue3401    created  benjamin.peterson         
                                                                               

test_nis is hanging on Solaris                                   07/18/08
       http://bugs.python.org/issue3402    created  benjamin.peterson         
                                                                               

Unexpected default arguments behaviour                           07/18/08
CLOSED http://bugs.python.org/issue3403    created  SukkoPera                 
                                                                               

wrong precision in float formatting or doc error                 07/18/08
CLOSED http://bugs.python.org/issue3404    created  hagen                     
                                                                               

Add support for the new data option supported by event generate  07/18/08
       http://bugs.python.org/issue3405    created  gpolo                     
       patch                                                                   

LocaleTextCalendar and LocaleHTMLCalendar break without a locale 07/18/08
CLOSED http://bugs.python.org/issue3406    created  WoLpH                     
                                                                               



Issues Now Closed (67)
______________________

Add a factorial function                                          121 days
       http://bugs.python.org/issue2138    georg.brandl              
                                                                               

parser module chokes on unusual characters                        122 days
       http://bugs.python.org/issue2280    benjamin.peterson         
       patch                                                                   

isinstance is 4x as slow as in 2.5 because the common case raise  119 days
       http://bugs.python.org/issue2303    gregory.p.smith           
       patch                                                                   

decide what to do with gettext API                                107 days
       http://bugs.python.org/issue2512    benjamin.peterson         
       patch                                                                   

pickling of large recursive structures crashes cPickle             21 days
       http://bugs.python.org/issue2702    facundobatista            
       patch                                                                   

Clean up undoc.rst                                                 62 days
       http://bugs.python.org/issue2828    brett.cannon              
       easy                                                                    

Remove plat-mac from 3.0                                           55 days
       http://bugs.python.org/issue2910    benjamin.peterson         
                                                                               

Test_imports fails in 2.6                                          52 days
       http://bugs.python.org/issue2969    benjamin.peterson         
                                                                               

Let bin/oct/hex show floats                                        21 days
       http://bugs.python.org/issue3008    marketdickinson           
       patch                                                                   

Windows online help broken when spaces in TEMP environ             41 days
       http://bugs.python.org/issue3045    georg.brandl              
       patch                                                                   

Add alternate (#) formatting for bin, oct, hex output for str.fo   34 days
       http://bugs.python.org/issue3083    eric.smith                
                                                                               

test_multiprocessing hangs intermittently on POSIX platforms       16 days
       http://bugs.python.org/issue3088    tebeka                    
       patch                                                                   

ARCHFLAGS parsing/concatenation in unixccompiler.py breaks when    34 days
       http://bugs.python.org/issue3090    jnoller                   
       patch                                                                   

implement PEP 3134 exception reporting                             31 days
       http://bugs.python.org/issue3112    benjamin.peterson         
       patch                                                                   

os.listdir randomly fails on occasions when it shouldn't           32 days
       http://bugs.python.org/issue3115    georg.brandl              
       patch                                                                   

sys.getsizeof() gives an AttributeError for _sre objects.          30 days
       http://bugs.python.org/issue3122    schuppenies               
       patch, patch                                                            

sqlite leaks on error                                              23 days
       http://bugs.python.org/issue3153    alexandre.vassalotti      
                                                                               

bytes type has inconsistent methods (append, insert)               26 days
       http://bugs.python.org/issue3156    georg.brandl              
       patch                                                                   

function annotation for builtin and C function                     19 days
       http://bugs.python.org/issue3208    bhy                       
                                                                               

SystemError: Parent module 'foo' not loaded on import statement    16 days
       http://bugs.python.org/issue3221    schmir                    
                                                                               

can't install on OSX 10.4                                          18 days
       http://bugs.python.org/issue3226    benjamin.peterson         
                                                                               

ctypes assertion failure in trunk                                  13 days
       http://bugs.python.org/issue3258    theller                   
                                                                               

Py_CLEAR(tmp) seg faults                                           10 days
       http://bugs.python.org/issue3274    alexandre.vassalotti      
                                                                               

Use Py_XDECREF() instead of Py_DECREF() in MultibyteCodec and Mu   10 days
       http://bugs.python.org/issue3305    georg.brandl              
       patch                                                                   

Out-of-date example 3.0b1 Tutorial Classes page, 'issubclass'      10 days
       http://bugs.python.org/issue3310    georg.brandl              
       patch                                                                   

bugs in _sqlite module                                              9 days
       http://bugs.python.org/issue3312    georg.brandl              
       patch                                                                   

dlopen() error with no error message from dlerror()                 8 days
       http://bugs.python.org/issue3313    theller                   
       patch                                                                   

Proposal for fix_urllib                                             8 days
       http://bugs.python.org/issue3316    brett.cannon              
       patch                                                                   

duplicate lines in zipfile.py                                       5 days
       http://bugs.python.org/issue3317    alanmcintyre              
       patch                                                                   

Documentation: timeit: "lower bound" should read "upper bound"      9 days
       http://bugs.python.org/issue3318    georg.brandl              
                                                                               

various doc typos                                                   4 days
       http://bugs.python.org/issue3320    benjamin.peterson         
       patch                                                                   

Broken link in online doc                                           8 days
       http://bugs.python.org/issue3324    georg.brandl              
                                                                               

subprocess lib - opening same command fails                         4 days
       http://bugs.python.org/issue3335    benjamin.peterson         
                                                                               

datetime weekday() function                                         5 days
       http://bugs.python.org/issue3336    georg.brandl              
                                                                               

dummy_thread LockType.acquire() always returns None, should be T    2 days
       http://bugs.python.org/issue3339    brett.cannon              
       patch                                                                   

Tracebacks are not properly indented                                0 days
       http://bugs.python.org/issue3342    brett.cannon              
       patch                                                                   

Py_DisplaySourceLine is not documented                              0 days
       http://bugs.python.org/issue3343    amaury.forgeotdarc        
                                                                               

Patch for CGIHTTPServer.is_cgi function documentation               5 days
       http://bugs.python.org/issue3345    georg.brandl              
       patch                                                                   

search for 'patch' produces roundup error                           0 days
       http://bugs.python.org/issue3349    techtonik                 
                                                                               

Python crashed                                                      0 days
       http://bugs.python.org/issue3351    loewis                    
                                                                               

A bug in the __doc__ string of the sys module                       0 days
       http://bugs.python.org/issue3357    benjamin.peterson         
                                                                               

add 'rbU' mode to open()                                            2 days
       http://bugs.python.org/issue3359    techtonik                 
                                                                               

Inconsistent type-deduction of decimal floating-point               1 days
       http://bugs.python.org/issue3360    marketdickinson           
                                                                               

pypi issue tracker link (python.org)                                0 days
       http://bugs.python.org/issue3361    loewis                    
                                                                               

An ortographical typo in Zen of Python text                         2 days
       http://bugs.python.org/issue3364    goodger                   
                                                                               

in module math, PI value seems to be wrong after digit 17th         0 days
       http://bugs.python.org/issue3365    marketdickinson           
                                                                               

2to3 fails in Python 2.6                                            0 days
       http://bugs.python.org/issue3371    benjamin.peterson         
                                                                               

socket.setsockopt() is broken for multicast TTL and Loop options    2 days
       http://bugs.python.org/issue3372    gpolo                     
                                                                               

Bisect upgrades: key/cmp/reverse, parameterized handedness          0 days
       http://bugs.python.org/issue3374    rhettinger                
       patch                                                                   

_multiprocessing.so build problems                                  1 days
       http://bugs.python.org/issue3375    gvanrossum                
                                                                               

Invalid child node access in ast.c                                  1 days
       http://bugs.python.org/issue3377    jhylton                   
                                                                               

documentation for ElementTree is unusable                           0 days
       http://bugs.python.org/issue3380    facundobatista            
                                                                               

`./configure --enable-framework --enable-universalsdk` fails bec    2 days
       http://bugs.python.org/issue3381    trentm                    
                                                                               

With keyword not mentioned in Input Output tutorial                 0 days
       http://bugs.python.org/issue3388    georg.brandl              
       patch                                                                   

[PATCH] Allow custom logging Handlers in logging config files       1 days
       http://bugs.python.org/issue3389    vsajip                    
       patch                                                                   

`cd Mac && make installmacsubtree` fails on Mac OS X < 10.5 beca    1 days
       http://bugs.python.org/issue3393    trentm                    
                                                                               

typo in test_multiprocessing.py:  should _debugInfo be _debug_in    0 days
       http://bugs.python.org/issue3395    jnoller                   
       easy                                                                    

Mac 3.0 build cannot find cachersrc.py                              0 days
       http://bugs.python.org/issue3397    benjamin.peterson         
                                                                               

mac build 3.0 no MacOS module                                       0 days
       http://bugs.python.org/issue3398    benjamin.peterson         
                                                                               

Unexpected default arguments behaviour                              0 days
       http://bugs.python.org/issue3403    georg.brandl              
                                                                               

wrong precision in float formatting or doc error                    0 days
       http://bugs.python.org/issue3404    georg.brandl              
                                                                               

LocaleTextCalendar and LocaleHTMLCalendar break without a locale    0 days
       http://bugs.python.org/issue3406    gpolo                     
                                                                               

rlcompleter add "(" to callables feature                         2535 days
       http://bugs.python.org/issue449227  pitrou                    
       patch, easy                                                             

re.split emptyok flag (fix for #852532)                          1467 days
       http://bugs.python.org/issue988761  georg.brandl              
       patch                                                                   

Move firewall warning to "about" menu                             716 days
       http://bugs.python.org/issue1529018 taleinat                  
       patch                                                                   

Sloppy error checking in listdir() for Posix                      590 days
       http://bugs.python.org/issue1608818 georg.brandl              
       patch, easy                                                             

robotparser.py fixes                                              327 days
       http://bugs.python.org/issue1778443 benjamin.peterson         
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 18 threading module can deadlock after fork                        1650 days
open    http://bugs.python.org/issue874900 

 13 _multiprocessing.so build problems                                 1 days
closed  http://bugs.python.org/issue3375   

 11 `./configure --enable-framework --enable-universalsdk` fails be    2 days
closed  http://bugs.python.org/issue3381   

 10 sys recursion limit a lot shorter on trunk?                        3 days
open    http://bugs.python.org/issue3373   

  9 locale.getpreferredencoding() gives bus error on Mac OS X 10.4.    3 days
open    http://bugs.python.org/issue3362   

  9 Deficiencies in multiprocessing/threading API                      4 days
open    http://bugs.python.org/issue3352   

  9 Proposal for fix_urllib                                            8 days
closed  http://bugs.python.org/issue3316   

  9 Let bin/oct/hex show floats                                       21 days
closed  http://bugs.python.org/issue3008   

  8 Memory corruption in multiprocessing module, OS X 10.5.4           1 days
open    http://bugs.python.org/issue3399   

  8 2to3 Fix_imports optimization                                     21 days
open    http://bugs.python.org/issue3218   




From db3l.net at gmail.com  Fri Jul 18 18:18:34 2008
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 18 Jul 2008 12:18:34 -0400
Subject: [Python-Dev] Windows buildbot trick
References: <g5pfrp$d9e$1@ger.gmane.org>
Message-ID: <87iqv3tbc5.fsf@valheru.db3l.homeip.net>

Thomas Heller <theller at ctypes.org> writes:

> The most annoying thing on the windows buildbots is when Python crashes hard
> (with a general protection fault or stack overflow, for example).
> Usually the system pops up a dialog in this case which allows to
> attach a debugger to the process.  This dialog will stay open until
> the maintainer manually closes it, and will prevent the next builds.
>
> On my boxes I have inserted these two lines into the PythonXY/Scripts/buildbot
> script, right at the top:
>
>   import win32api; win32api.SetErrorMode(7)
>   print "SetErrorMode(7) called."
>
> These lines prevent the dialog box to be displayed for critical errors.

For my buildbot, I've been running similarly modified buildbot code
since late 2007 (in commands.py, surrounding the spawning of a process
to execute a step) so it covers any use of the buildbot (mine was also
building MSIs).  I had to modify buildbot to fix a file upload bug
anyway, so didn't mind running a patched version.

Technically using 0x8007 rather than just 7 would also cover a
possible missing file dialog, but not crucial.

It hasn't been 100% foolproof, as I seem to recall once or twice still
having to clear a dialog, but I think that was from the CRT within
Python itself, which there was an old discussion about changing the
Python code to initialize differently.  It definitely catches the
Windows hard failures though.

-- David


From eric+python-dev at trueblade.com  Fri Jul 18 19:10:28 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Fri, 18 Jul 2008 13:10:28 -0400
Subject: [Python-Dev] [Python-checkins] r65099 -
	python/trunk/Doc/library/string.rst
In-Reply-To: <20080718111506.867BF1E4007@bag.python.org>
References: <20080718111506.867BF1E4007@bag.python.org>
Message-ID: <4880CE84.7050003@trueblade.com>

georg.brandl wrote:
> Author: georg.brandl
> Date: Fri Jul 18 13:15:06 2008
> New Revision: 65099
> 
> Log:
> Document the different meaning of precision for {:f} and {:g}.
> Also document how inf and nan are formatted. #3404.

Thanks for doing this.  But see this output:
http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/builds/255/step-test/0
which shows that on Solaris with gcc it's 'NaN', not 'nan'.  This is one 
of the reasons I didn't get into documenting it.  And on Windows, it's 
even worse (no Windows box handy, sorry).

Do we want to document the actual behavior, or do we want to normalize 
all platforms so that they all return 'nan' and 'inf'?

I'm still hoping to fix "F" formatting (issue 3382) on Windows, assuming 
that it's acceptable to do that before the last beta.  It mostly comes 
down to deleting the tests, since I can't match on 'nan' and 'inf'. 
There is some additional work mapping 'F' to 'f', and then uppercasing 
the result, but that part is easy.

Eric.

From josiah.carlson at gmail.com  Fri Jul 18 19:45:06 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Fri, 18 Jul 2008 10:45:06 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
Message-ID: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>

On Fri, Jul 18, 2008 at 8:11 AM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Jul 18, 2008 at 7:57 AM, Josiah Carlson
> <josiah.carlson at gmail.com> wrote:
>> Invariably, when someone goes and removes a module, someone else is
>> going to complain, "but I used feature X, not having feature X will
>> break my code."  We, as maintainers can then say, "if you cared,
>> maintain it."  But I'm not sure that is the greatest thing to tell
>> people.  I suspect that we may have to include some sort of
>> "work-alike" for 2.7 and if not 3.0, 3.1 .  If I were to vote for a
>> work-alike, it would be based on sqlite.  For one of the most common
>> use-cases (bsddb.btree), simple sqlite code can be written to do the
>> right thing.  Recno is a little more tricky, but can also be done.
>> The bsddb hash may not be possible, because sqlite doesn't support
>> hashed indices :/.
>
> In my mind, BSDDB is pretty much the most heavy-weight extension we're
> maintaining. I think it's an illusion that a sqlite-based look-alike
> is going to fool anyone. The correct solution is to take support for
> bsddb to a separate project where those who care about it can maintain
> it together. That also makes it a lot easier to track the versions of
> Berkeley DB as they come out.
>
> Of course, you're free to try writing the work-alike you're proposing. :-)

It's entirely possible that I know very little about what was being
made available via the bsddb module, but to match the API of what is
included in the documentation (plus the dictionary interface that it
supports) shouldn't be terribly difficult.

Now, if there were *other* things that were undocumented, well,
there's not much I can do about those. ;)

 - Josiah

From fdrake at acm.org  Fri Jul 18 20:03:27 2008
From: fdrake at acm.org (Fred Drake)
Date: Fri, 18 Jul 2008 14:03:27 -0400
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
Message-ID: <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>

On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote:
> It's entirely possible that I know very little about what was being
> made available via the bsddb module, but to match the API of what is
> included in the documentation (plus the dictionary interface that it
> supports) shouldn't be terribly difficult.


It's also entirely possible that the API isn't interesting if you  
don't support existing databases, for many applications.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>


From brett at python.org  Fri Jul 18 20:04:17 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 11:04:17 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
Message-ID: <bbaeab100807181104ra7e7cdcl98183746e0016daa@mail.gmail.com>

On Fri, Jul 18, 2008 at 7:21 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote:
>> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote:
>>> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
>>>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
>>>>>
>>>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
>>>>> 3k. I somewhat doubt that this gets resolved before the release, so
>>>>> bsddb users might need to skip 3.0.
>>>>
>>>>
>>>> In fact, bsddb as packages in core Python has rarely been in good shape.
>>>>
>>>> Has anyone actually considered that bsddb might do better if maintained
>>>> strictly as an external package?  That would make it easier for the any
>>>> maintainers to release updates, and removes a source of frustration for
>>>> users who either don't need it or would rather wait for a happier version.
>>>>
>>>> I think this is worth considering.  I vaguely recall that the bsddb module
>>>> was maintained before it was incorporated into the core Python release.
>>>
>>> +1. In my recollection maintaining bsddb has been nothing but trouble
>>> right from the start when we were all sitting together at "Zope Corp
>>> North" in a rented office in McLean... We can remove it from 3.0. We
>>> can't really remove it from 2.6, but we can certainly start
>>> end-of-lifing it in 2.6.
>
>> Unless I hear otherwise, I will add it to PEP 3108.
>
> Please do!
>

Done.

-Brett

From brett at python.org  Fri Jul 18 20:12:41 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 11:12:41 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <20080718065430.GA6991@arctrix.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
Message-ID: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>

On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> [back on the list]
>
> On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote:
>> Turned out to be a rebuild::
>>
>> ....
>> r65077 = 82d954e8c20c91562c4c660859d17756cba10992
>> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a
>> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536
>> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771
>>
>> How do I know what is going to be sent? ``git log`` seems to suggest
>> something by not listing a git-svn-id for my last commit, but is that
>> really the best I got?
>
> The command
>
>    git log git-svn..
>

And those two periods are significant for people who think they are
line noise. Damn is Git quirky.

> will show you changes in your HEAD (by default "master") tree that
> are not in the remote tree (git-svn).
>
>> Is there some other way to see what will be pushed?
>
> I like running "gitk" before I push something.
>

Sure, but I don't know what the heck I am looking at. There is some
green stuff for both master and my branch, but then there is some more
green stuff just for my branch. Now if it was just the one commit I
did on my branch I would assume that is just what I have not pushed,
but considering there is also a marker on the master branch which is
pristine I am not sure what I am looking at.

>> And how do I diff easily between commits?
>
> It depends on what you want, exactly.  Maybe you can describe some
> use cases.  A DVCS can't use identify revisions like SVN does.
> Generally I find myself using heads or tags to identify versions in
> combination with the ^ operator.  For example,
>

I assume the ^ operator means "just before this commit".


>    git diff HEAD^
>
> would show the difference between the current working tree and the
> commit before the head of the stored tree.  If you want the patch
> for a single commit, use "git show <object>".  For example, "git
> show" will display the last commit.  To see amk's typo fix:
>
>    git show 6cadb9c1b7e30a8b66cdba01cd79aa6397a07080
>
> You can also abbreviate the commit id, eg.
>
>    git show 6cadb9
>

Does the abbreviation have to be exactly six characters?

> As I say in my guide, "git format-patch" and "git am" are very handy
> when slinging patches around (e.g. to and from a bug tracker or
> mailing list).

I tried that, but but format-patch didn't show me anything since I had
just committed. And when I run ``git format-patch HEAD^`` it spits out
what looks like a file name, but I don't see it anywhere.

-Brett

From ncoghlan at gmail.com  Fri Jul 18 20:16:57 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Jul 2008 04:16:57 +1000
Subject: [Python-Dev] [Python-3000]   No beta2 tonight
In-Reply-To: <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>	<487FD54C.2040308@v.loewis.de>	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
Message-ID: <4880DE19.7050704@gmail.com>

Fred Drake wrote:
> On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote:
>> It's entirely possible that I know very little about what was being
>> made available via the bsddb module, but to match the API of what is
>> included in the documentation (plus the dictionary interface that it
>> supports) shouldn't be terribly difficult.
> 
> 
> It's also entirely possible that the API isn't interesting if you don't 
> support existing databases, for many applications.

And downloading pybsddb and installing really shouldn't be all that 
difficult :)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From amk at amk.ca  Fri Jul 18 20:32:15 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 18 Jul 2008 14:32:15 -0400
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
Message-ID: <20080718183215.GA7748@amk-desktop.matrixgroup.net>

yOn Fri, Jul 18, 2008 at 10:45:06AM -0700, Josiah Carlson wrote:
>> It's entirely possible that I know very little about what was being
> made available via the bsddb module, but to match the API of what is
> included in the documentation (plus the dictionary interface that it
> supports) shouldn't be terribly difficult.

It would pose a problem for the external maintainer if Python came
with a bsddb module or package; the external package would also want
to provide a 'bsddb' package containing the 'bsddb.db' subpackage.
This would be a replay of the 'xml'/PyXML situation, and lots of
people think that was a big mistake in the first place.  If we do want
to let bsddb go, we should just let the new external package own the
package name.

We can obviously drop the module for 3.0.  For 2.x, should we just
shrug and disable most of the BerkeleyDB tests (maybe just on Windows)
by adding a new resource to enable them?  If we're stuck with it for
the remaining 2.x, at least we can stop the buildbots from going red
for bugs that we can't debug and probably won't fix.

--amk


From brett at python.org  Fri Jul 18 20:37:09 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 11:37:09 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
Message-ID: <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>

On Fri, Jul 18, 2008 at 11:12 AM, Brett Cannon <brett at python.org> wrote:
> On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote:
>> [back on the list]
>>
>> On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote:
>>> Turned out to be a rebuild::
>>>
>>> ....
>>> r65077 = 82d954e8c20c91562c4c660859d17756cba10992
>>> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a
>>> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536
>>> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771
>>>
>>> How do I know what is going to be sent? ``git log`` seems to suggest
>>> something by not listing a git-svn-id for my last commit, but is that
>>> really the best I got?
>>
>> The command
>>
>>    git log git-svn..
>>
>
> And those two periods are significant for people who think they are
> line noise. Damn is Git quirky.
>

OK, so I decided to trying committing through Git by doing ``git svn
dcommit``. But it told me that Misc/NEWS was out of date. So I then
did ``git fetch git-svn`` with a ``git merge git-svn``, which I
realize now is a mistake since I am used to other DVCSs using "merge"
for when there are changes and "update" when there are none.

So Git tells me there is a conflict; fine. I go in, fix the file, and
then try to commit Misc/NEWS. It says I can't do a partial commit, I
have to commit everything; fine. So I do a full commit, but now I get:

Misc/NEWS: needs merge
Misc/NEWS: unmerged (3dc56ad4e5918676358c4e20be738b37ce8d50d0)
Misc/NEWS: unmerged (ad4c19cecfb584be37d8b9c138791daa83ad9285)
Misc/NEWS: unmerged (853df55fc6ac71bcea0eb340a8ab52b348db9e8c)
error: Error building trees

This is not winning me over on the usability side of things. =) So
what do I do now to get this tree back in a usable state so I can
commit (although, thanks to ``git show``, I might patch a svn checkout
so I don't have to wait on this)?

-Brett

From amk at amk.ca  Fri Jul 18 20:42:06 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 18 Jul 2008 14:42:06 -0400
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
Message-ID: <20080718184206.GB7748@amk-desktop.matrixgroup.net>

On Fri, Jul 18, 2008 at 11:12:41AM -0700, Brett Cannon wrote:
> And those two periods are significant for people who think they are
> line noise. Damn is Git quirky.

Oh my, yes.  We use git at work; there's a reason I now use Bazaar for
personal projects.

> I assume the ^ operator means "just before this commit".

Correct; HEAD^^ is two commits ago, and you can do HEAD~35 for 35
commits ago.  There's a git-rev-parse man page describing these.

> Does the abbreviation have to be exactly six characters?

No, it has to be long enough to be unambiguous.  6cadb9c1b7 would also
work, and 6cadb might, depending on the other commits in your
repository.

> I tried that, but but format-patch didn't show me anything since I had
> just committed. And when I run ``git format-patch HEAD^`` it spits out
> what looks like a file name, but I don't see it anywhere.

I think it writes the file to the current working directory, so I
don't know why you're not seeing it.  (The file is something like
0001-<commit-message>.patch.)

--amk

From brett at python.org  Fri Jul 18 20:57:21 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 11:57:21 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
References: <20080714224343.GA23048@arctrix.com>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
Message-ID: <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>

On Fri, Jul 18, 2008 at 11:37 AM, Brett Cannon <brett at python.org> wrote:
> On Fri, Jul 18, 2008 at 11:12 AM, Brett Cannon <brett at python.org> wrote:
>> On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote:
>>> [back on the list]
>>>
>>> On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote:
>>>> Turned out to be a rebuild::
>>>>
>>>> ....
>>>> r65077 = 82d954e8c20c91562c4c660859d17756cba10992
>>>> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a
>>>> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536
>>>> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771
>>>>
>>>> How do I know what is going to be sent? ``git log`` seems to suggest
>>>> something by not listing a git-svn-id for my last commit, but is that
>>>> really the best I got?
>>>
>>> The command
>>>
>>>    git log git-svn..
>>>
>>
>> And those two periods are significant for people who think they are
>> line noise. Damn is Git quirky.
>>
>
> OK, so I decided to trying committing through Git by doing ``git svn
> dcommit``. But it told me that Misc/NEWS was out of date. So I then
> did ``git fetch git-svn`` with a ``git merge git-svn``, which I
> realize now is a mistake since I am used to other DVCSs using "merge"
> for when there are changes and "update" when there are none.
>
> So Git tells me there is a conflict; fine. I go in, fix the file, and
> then try to commit Misc/NEWS. It says I can't do a partial commit, I
> have to commit everything; fine. So I do a full commit, but now I get:
>
> Misc/NEWS: needs merge
> Misc/NEWS: unmerged (3dc56ad4e5918676358c4e20be738b37ce8d50d0)
> Misc/NEWS: unmerged (ad4c19cecfb584be37d8b9c138791daa83ad9285)
> Misc/NEWS: unmerged (853df55fc6ac71bcea0eb340a8ab52b348db9e8c)
> error: Error building trees
>
> This is not winning me over on the usability side of things. =) So
> what do I do now to get this tree back in a usable state so I can
> commit (although, thanks to ``git show``, I might patch a svn checkout
> so I don't have to wait on this)?
>

I figured this out. I just did ``git reset --hard``, did the proper
"fetch;rebase" dance, resolved the conflict, did ``git add`` and then
continued with the rebase. It all looks fine now.

-Brett

From nas at arctrix.com  Fri Jul 18 20:57:42 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 18 Jul 2008 12:57:42 -0600
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
Message-ID: <20080718185741.GA9433@arctrix.com>

On Fri, Jul 18, 2008 at 11:12:41AM -0700, Brett Cannon wrote:
> >    git log git-svn..
> 
> And those two periods are significant for people who think they are
> line noise. Damn is Git quirky.

I guess it would have been clearer if I had used "git-svn..HEAD".
The ".." is similar to SVN's ":" so I don't see that as much of a
quirk.  However, if you look at the git-rev-parse man page you will
see that git supports a very rich set of revision specifications and
that can be overwhelming to a new user.  You can probably mostly get
by with "<rev>" and "<rev1>..<rev2>" combined with ^.

> Sure, but I don't know what the heck I am looking at.

The top left window shows a DAG of the commits.  Each commit is a
little ball.  Parents are connected with a colored line.  Tags and
heads are shown as labels on specific commits.  You can click on any
commit to see the details (shown in the lower panels).

> I assume the ^ operator means "just before this commit".

Yes and you can use it more than once (e.g. ^^^).  This is all
documented in the git-rev-parse man page.

> Does the abbreviation have to be exactly six characters?

No.

> I tried that, but but format-patch didn't show me anything since I had
> just committed. And when I run ``git format-patch HEAD^`` it spits out
> what looks like a file name, but I don't see it anywhere.

By default it creates a file for each commit, prefixed by 0001,
0002, etc.  Use "git format-patch --stdout" to have it spit the
patches out as a mbox to stdout.

From brett at python.org  Fri Jul 18 21:07:19 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 12:07:19 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>
References: <20080714224343.GA23048@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
	<bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>
Message-ID: <bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com>

On Fri, Jul 18, 2008 at 11:57 AM, Brett Cannon <brett at python.org> wrote:
> On Fri, Jul 18, 2008 at 11:37 AM, Brett Cannon <brett at python.org> wrote:
>> On Fri, Jul 18, 2008 at 11:12 AM, Brett Cannon <brett at python.org> wrote:
>>> On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote:
>>>> [back on the list]
>>>>
>>>> On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote:
>>>>> Turned out to be a rebuild::
>>>>>
>>>>> ....
>>>>> r65077 = 82d954e8c20c91562c4c660859d17756cba10992
>>>>> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a
>>>>> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536
>>>>> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771
>>>>>
>>>>> How do I know what is going to be sent? ``git log`` seems to suggest
>>>>> something by not listing a git-svn-id for my last commit, but is that
>>>>> really the best I got?
>>>>
>>>> The command
>>>>
>>>>    git log git-svn..
>>>>
>>>
>>> And those two periods are significant for people who think they are
>>> line noise. Damn is Git quirky.
>>>
>>
>> OK, so I decided to trying committing through Git by doing ``git svn
>> dcommit``. But it told me that Misc/NEWS was out of date. So I then
>> did ``git fetch git-svn`` with a ``git merge git-svn``, which I
>> realize now is a mistake since I am used to other DVCSs using "merge"
>> for when there are changes and "update" when there are none.
>>
>> So Git tells me there is a conflict; fine. I go in, fix the file, and
>> then try to commit Misc/NEWS. It says I can't do a partial commit, I
>> have to commit everything; fine. So I do a full commit, but now I get:
>>
>> Misc/NEWS: needs merge
>> Misc/NEWS: unmerged (3dc56ad4e5918676358c4e20be738b37ce8d50d0)
>> Misc/NEWS: unmerged (ad4c19cecfb584be37d8b9c138791daa83ad9285)
>> Misc/NEWS: unmerged (853df55fc6ac71bcea0eb340a8ab52b348db9e8c)
>> error: Error building trees
>>
>> This is not winning me over on the usability side of things. =) So
>> what do I do now to get this tree back in a usable state so I can
>> commit (although, thanks to ``git show``, I might patch a svn checkout
>> so I don't have to wait on this)?
>>
>
> I figured this out. I just did ``git reset --hard``, did the proper
> "fetch;rebase" dance, resolved the conflict, did ``git add`` and then
> continued with the rebase. It all looks fine now.
>

I lied. Trying again complained about Mac/IDLE/Makefile.in which I
have not touched, nor is it listed as changed. OK, so I deleted it in
hopes that "fetch; rebase" would fix it; nope. So I did another ``git
reset --hard``, but I am still stuck with being told that
Mac/IDLE/Makefile.in is out of date::

Committing to svn+ssh://svn.python.org/python/trunk ...
	M	Mac/IDLE/Makefile.in
Transaction is out of date: Out of date:
'/python/trunk/Mac/IDLE/Makefile.in' in transaction '65108-1' at
/unix/bin/git-svn line 461

So I don't know how to deal with this since Git does not even list the
file as being modified.

-Brett

From josiah.carlson at gmail.com  Fri Jul 18 21:15:42 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Fri, 18 Jul 2008 12:15:42 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
Message-ID: <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>

On Fri, Jul 18, 2008 at 11:03 AM, Fred Drake <fdrake at acm.org> wrote:
> On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote:
>>
>> It's entirely possible that I know very little about what was being
>> made available via the bsddb module, but to match the API of what is
>> included in the documentation (plus the dictionary interface that it
>> supports) shouldn't be terribly difficult.
>
>
> It's also entirely possible that the API isn't interesting if you don't
> support existing databases, for many applications.

I see where the confusion was.  I'm not suggesting that someone write
a new bsddb module, I'm suggesting that we can provide something
called, perhaps, on_disk_dictionary, which offers the behavior of
bsddb, without using bsddb anywhere, or supporting bsddb files.

 - Josiah

From barry at python.org  Fri Jul 18 21:19:32 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 18 Jul 2008 15:19:32 -0400
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <20080718183215.GA7748@amk-desktop.matrixgroup.net>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<20080718183215.GA7748@amk-desktop.matrixgroup.net>
Message-ID: <2750F8D4-898A-4176-AE36-9E37CA17ABDB@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 18, 2008, at 2:32 PM, A.M. Kuchling wrote:

> We can obviously drop the module for 3.0.  For 2.x, should we just
> shrug and disable most of the BerkeleyDB tests (maybe just on Windows)
> by adding a new resource to enable them?  If we're stuck with it for
> the remaining 2.x, at least we can stop the buildbots from going red
> for bugs that we can't debug and probably won't fix.

+1
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSIDsxXEjvBPtnXfVAQJ6egP+O0HQPj9X7te1X3CbPBRN8RhadJucaDmF
13a5bf13D/5D+gQ4iKlYMovd9l66J6lpFpSX+cHLgU3g/BZvFUXJFl4gZegWyD2p
idQvGVFUrjGDL5TFIL4Dvg5D/b8pYSPfr5xWgJNSPHMeM5sEM6hDt/WhuvYSzuv+
lMROoc5c0NI=
=isdz
-----END PGP SIGNATURE-----

From stephen at xemacs.org  Fri Jul 18 21:06:01 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 19 Jul 2008 04:06:01 +0900
Subject: [Python-Dev] [Python-3000]   No beta2 tonight
In-Reply-To: <4880DE19.7050704@gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
	<4880DE19.7050704@gmail.com>
Message-ID: <87ljzzovvq.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:

 > And downloading pybsddb and installing really shouldn't be all that 
 > difficult :)

It shouldn't be, but lots of "enterprise"[1] environments will require
qualifying the "new" package according to corporate standards.

I won't argue that this is a sufficient reason to keep a package with
bsddb's history in stdlib, but let's not joke about the difficulties.


Footnotes: 
[1]  Which apparently means that the company can be enterprising, but
the people who do the work mustn't be!


From nas at arctrix.com  Fri Jul 18 21:31:00 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 18 Jul 2008 13:31:00 -0600
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>
References: <20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
	<bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>
Message-ID: <20080718193100.GB9433@arctrix.com>

On Fri, Jul 18, 2008 at 11:57:21AM -0700, Brett Cannon wrote:
> I figured this out. I just did ``git reset --hard``, did the proper
> "fetch;rebase" dance, resolved the conflict, did ``git add`` and then
> continued with the rebase. It all looks fine now.

Doing a fetch followed by a rebase is similar to what "svn up" does
and is what you want in this case.  Rebase rewrites patches
(commits) so that they apply to a different version of a tree (they
will get new SHA ids).  If you use merge then your patches (commits)
stay unchanged and a new commit object is created that contains the
info required to integrate them with the new tree.

Using merge is also useful in certain cases (e.g. in a distributed
environment, if you have published your changes already and people
have them in their repositories) but the downside is the extra merge
commit object.  However, since in this specific case you are always
pushing back to a central repo you should avoid merge.

BTW, the rebase command is very handy if you are maintaing some
change over a longer term.  Here's an example workflow:

    <start with git checkout>
    git checkout -b my_feature # create a private branch
    <edit code, commit changes>
    git checkout master # back to main branch
    <time passes>
    git fetch git-svn && git rebase git-svn # update master to latest code
    git checkout my_feature # back to private branch
    git rebase master # make my changes apply to latest code

To generate a series of patches to send somewhere:

    git format-patch --stdout master..my_feature > my_feature.txt

Cheers,

  Neil

From brett at python.org  Fri Jul 18 21:34:24 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 12:34:24 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <20080718193100.GB9433@arctrix.com>
References: <20080715200411.GA26998@arctrix.com> <g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
	<bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>
	<20080718193100.GB9433@arctrix.com>
Message-ID: <bbaeab100807181234j6ea71da9i550eb29cfef5b46b@mail.gmail.com>

On Fri, Jul 18, 2008 at 12:31 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> On Fri, Jul 18, 2008 at 11:57:21AM -0700, Brett Cannon wrote:
>> I figured this out. I just did ``git reset --hard``, did the proper
>> "fetch;rebase" dance, resolved the conflict, did ``git add`` and then
>> continued with the rebase. It all looks fine now.
>
> Doing a fetch followed by a rebase is similar to what "svn up" does
> and is what you want in this case.  Rebase rewrites patches
> (commits) so that they apply to a different version of a tree (they
> will get new SHA ids).  If you use merge then your patches (commits)
> stay unchanged and a new commit object is created that contains the
> info required to integrate them with the new tree.
>
> Using merge is also useful in certain cases (e.g. in a distributed
> environment, if you have published your changes already and people
> have them in their repositories) but the downside is the extra merge
> commit object.  However, since in this specific case you are always
> pushing back to a central repo you should avoid merge.
>
> BTW, the rebase command is very handy if you are maintaing some
> change over a longer term.  Here's an example workflow:
>
>    <start with git checkout>
>    git checkout -b my_feature # create a private branch
>    <edit code, commit changes>
>    git checkout master # back to main branch
>    <time passes>
>    git fetch git-svn && git rebase git-svn # update master to latest code
>    git checkout my_feature # back to private branch
>    git rebase master # make my changes apply to latest code
>

That's what I have been doing, except using "merge" in the master branch.

> To generate a series of patches to send somewhere:
>
>    git format-patch --stdout master..my_feature > my_feature.txt

OK, so that's how you use format-patch.

-Brett

From nas at arctrix.com  Fri Jul 18 21:35:31 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 18 Jul 2008 13:35:31 -0600
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com>
References: <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
	<bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>
	<bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com>
Message-ID: <20080718193531.GC9433@arctrix.com>

On Fri, Jul 18, 2008 at 12:07:19PM -0700, Brett Cannon wrote:
> I lied. Trying again complained about Mac/IDLE/Makefile.in which I
> have not touched, nor is it listed as changed.

I think you are running into the fact that the git tree on
code.python.org is only updated every 30 minutes.  Using the git://
protocol is fast but can't get changes that are in SVN and not yet
in the git tree.  If your repo is fairly up-to-date, you can just
use "git svn rebase".  That will grab the latest changes from the
SVN server and rebase your local changes, very similar to what "svn
up" does.

BTW, I'm on #python-dev at the moment if you run into more trouble.

  Neil

From charleshixsn at earthlink.net  Fri Jul 18 21:35:46 2008
From: charleshixsn at earthlink.net (Charles Hixson)
Date: Fri, 18 Jul 2008 12:35:46 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
Message-ID: <200807181235.46259.charleshixsn@earthlink.net>

On Friday 18 July 2008 07:57:01 am Josiah Carlson wrote:
> On Fri, Jul 18, 2008 at 7:21 AM, Guido van Rossum <guido at python.org> wrote:
> > On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote:
> >> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> 
wrote:
> >>> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
> >>>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
> >>>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
> >>>>> 3k. I somewhat doubt that this gets resolved before the release, so
> >>>>> bsddb users might need to skip 3.0.
> >>>>
> >>>> In fact, bsddb as packages in core Python has rarely been in good
> >>>> shape.
> >>>>
> >>>> Has anyone actually considered that bsddb might do better if
> >>>> maintained strictly as an external package?  That would make it easier
> >>>> for the any maintainers to release updates, and removes a source of
> >>>> frustration for users who either don't need it or would rather wait
> >>>> for a happier version.
> >>>>
> >>>> I think this is worth considering.  I vaguely recall that the bsddb
> >>>> module was maintained before it was incorporated into the core Python
> >>>> release.
> >>>
> >>> +1. In my recollection maintaining bsddb has been nothing but trouble
> >>> right from the start when we were all sitting together at "Zope Corp
> >>> North" in a rented office in McLean... We can remove it from 3.0. We
> >>> can't really remove it from 2.6, but we can certainly start
> >>> end-of-lifing it in 2.6.
> >>
> >> Unless I hear otherwise, I will add it to PEP 3108.
> >
> > Please do!
>
> Invariably, when someone goes and removes a module, someone else is
> going to complain, "but I used feature X, not having feature X will
> break my code."  We, as maintainers can then say, "if you cared,
> maintain it."  But I'm not sure that is the greatest thing to tell
> people.  I suspect that we may have to include some sort of
> "work-alike" for 2.7 and if not 3.0, 3.1 .  If I were to vote for a
> work-alike, it would be based on sqlite.  For one of the most common
> use-cases (bsddb.btree), simple sqlite code can be written to do the
> right thing.  Recno is a little more tricky, but can also be done.
> The bsddb hash may not be possible, because sqlite doesn't support
> hashed indices :/.
>
> Just an idea.
>
>  - Josiah
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/charleshixsn%40earthlink.
>net

Were I to vote for "something" it would be a B+Tree in collections.  One that 
didn't impose a requirement that the key be a string (and not, e.g., an 
integer or a float).

OTOH, I don't care enough to build it.  (I've proven this to myself 
repeatedly, as I've started to create such a thing, and then kludged a 
different solution.)


From brett at python.org  Fri Jul 18 21:38:13 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 12:38:13 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <20080718193531.GC9433@arctrix.com>
References: <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
	<bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com>
	<bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com>
	<20080718193531.GC9433@arctrix.com>
Message-ID: <bbaeab100807181238r75f9ef43pb75d07a3ca4a8c0b@mail.gmail.com>

On Fri, Jul 18, 2008 at 12:35 PM, Neil Schemenauer <nas at arctrix.com> wrote:
> On Fri, Jul 18, 2008 at 12:07:19PM -0700, Brett Cannon wrote:
>> I lied. Trying again complained about Mac/IDLE/Makefile.in which I
>> have not touched, nor is it listed as changed.
>
> I think you are running into the fact that the git tree on
> code.python.org is only updated every 30 minutes.  Using the git://
> protocol is fast but can't get changes that are in SVN and not yet
> in the git tree.  If your repo is fairly up-to-date, you can just
> use "git svn rebase".  That will grab the latest changes from the
> SVN server and rebase your local changes, very similar to what "svn
> up" does.
>

Did that, still complained.

> BTW, I'm on #python-dev at the moment if you run into more trouble.

If I get daring again I will try you in the IRC channel, but I just
gave up and used a patch against svn so the change could get
committed.

-Brett

From g.brandl at gmx.net  Fri Jul 18 21:38:46 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 18 Jul 2008 21:38:46 +0200
Subject: [Python-Dev] [Python-checkins] r65099 -
	python/trunk/Doc/library/string.rst
In-Reply-To: <4880CE84.7050003@trueblade.com>
References: <20080718111506.867BF1E4007@bag.python.org>
	<4880CE84.7050003@trueblade.com>
Message-ID: <g5qrgh$2iu$1@ger.gmane.org>

Eric Smith schrieb:
> georg.brandl wrote:
>> Author: georg.brandl
>> Date: Fri Jul 18 13:15:06 2008
>> New Revision: 65099
>> 
>> Log:
>> Document the different meaning of precision for {:f} and {:g}.
>> Also document how inf and nan are formatted. #3404.
> 
> Thanks for doing this.  But see this output:
> http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/builds/255/step-test/0
> which shows that on Solaris with gcc it's 'NaN', not 'nan'.  This is one 
> of the reasons I didn't get into documenting it.  And on Windows, it's 
> even worse (no Windows box handy, sorry).

I see.

> Do we want to document the actual behavior, or do we want to normalize 
> all platforms so that they all return 'nan' and 'inf'?

I'd find a consistent behavior nice, if it isn't too much work to
get one.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From janssen at parc.com  Fri Jul 18 21:54:13 2008
From: janssen at parc.com (Bill Janssen)
Date: Fri, 18 Jul 2008 12:54:13 PDT
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> 
References: <20080714224343.GA23048@arctrix.com>
	<1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org>
	<20080715200411.GA26998@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
Message-ID: <08Jul18.125419pdt."58698"@synergy1.parc.xerox.com>

Why is it you're trying to use "git"?

Bill

From theller at ctypes.org  Fri Jul 18 22:27:53 2008
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 18 Jul 2008 22:27:53 +0200
Subject: [Python-Dev] Windows buildbot trick
In-Reply-To: <a7a2b76b0807180853g524677cbk578f322b049010cc@mail.gmail.com>
References: <g5pfrp$d9e$1@ger.gmane.org>	<a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com>
	<a7a2b76b0807180853g524677cbk578f322b049010cc@mail.gmail.com>
Message-ID: <g5qub5$be9$1@ger.gmane.org>

Sidnei da Silva schrieb:
> On Fri, Jul 18, 2008 at 12:42 PM, Sidnei da Silva
> <sidnei at enfoldsystems.com> wrote:
>> That's a great trick! I seem to remember that there is a way to turn
>> that off globally though, but not sure where. Maybe running
>> drwtsn32.exe and un-checking 'Visual Notification' does the trick?
> 
> FWIW,  here's a kb article that describes this. Although it refers to
> 32-bit apps on 64-bit XP, I'm pretty sure you can do the same for
> 32-bit XP?
> 
> http://support.microsoft.com/kb/283150/EN-US/
> 

Well, generally it is a good idea on a developer machine to automatically
start the debugger when a problem in the tested application happens, so
I prefer to not change this setting globally.

However, some time ago I have tried to turn the notifications off in
every way I could find but did not succeed.  Although I'm not sure I found
the article you mentioned.

-- 
Thanks,
Thomas


From dickinsm at gmail.com  Fri Jul 18 23:08:47 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 18 Jul 2008 22:08:47 +0100
Subject: [Python-Dev] [Python-checkins] r65099 -
	python/trunk/Doc/library/string.rst
In-Reply-To: <4880CE84.7050003@trueblade.com>
References: <20080718111506.867BF1E4007@bag.python.org>
	<4880CE84.7050003@trueblade.com>
Message-ID: <5c6f2a5d0807181408g595cde7fv8a9888914291c4bd@mail.gmail.com>

On Fri, Jul 18, 2008 at 6:10 PM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Do we want to document the actual behavior, or do we want to normalize all
> platforms so that they all return 'nan' and 'inf'?

I seem to recall that Christian Heimes recently put some work into
making sure that repr() or str() of an infinity or nan is 'inf' or 'nan'
(or '-inf'), regardless of platform.

+1 for normalizing '%f' and '%F' behaviour across platforms.

Mark

From mal at egenix.com  Fri Jul 18 23:42:34 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 18 Jul 2008 23:42:34 +0200
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <200807181235.46259.charleshixsn@earthlink.net>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<200807181235.46259.charleshixsn@earthlink.net>
Message-ID: <48810E4A.6070304@egenix.com>

On 2008-07-18 21:35, Charles Hixson wrote:
>> Invariably, when someone goes and removes a module, someone else is
>> going to complain, "but I used feature X, not having feature X will
>> break my code."  We, as maintainers can then say, "if you cared,
>> maintain it."  But I'm not sure that is the greatest thing to tell
>> people.  I suspect that we may have to include some sort of
>> "work-alike" for 2.7 and if not 3.0, 3.1 .  If I were to vote for a
>> work-alike, it would be based on sqlite.  For one of the most common
>> use-cases (bsddb.btree), simple sqlite code can be written to do the
>> right thing.  Recno is a little more tricky, but can also be done.
>> The bsddb hash may not be possible, because sqlite doesn't support
>> hashed indices :/.
>>
>> Just an idea.
>>
>>  - Josiah
> 
> Were I to vote for "something" it would be a B+Tree in collections.  One that 
> didn't impose a requirement that the key be a string (and not, e.g., an 
> integer or a float).
> 
> OTOH, I don't care enough to build it.  (I've proven this to myself 
> repeatedly, as I've started to create such a thing, and then kludged a 
> different solution.)

If pybsddb is sufficient as work-around for the stdlib bsddb module
(or perhaps even better), then I don't see much of a problem removing
the module from the stdlib using a PEP 4 process.

Can't really say, since I've never used any of these myself...
for on-disk dictionaries, we use mxBeeBase:

http://www.egenix.com/products/python/mxBase/mxBeeBase/

For anything more complex, a SQL database.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 18 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From brett at python.org  Fri Jul 18 23:57:41 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 14:57:41 -0700
Subject: [Python-Dev] git repositories for trunk and py3k
In-Reply-To: <7815067257654475794@unknownmsgid>
References: <20080714224343.GA23048@arctrix.com>
	<1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com>
	<g5j4v3$6hi$1@ger.gmane.org>
	<bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com>
	<20080718061113.GA6848@arctrix.com>
	<bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com>
	<20080718065430.GA6991@arctrix.com>
	<bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com>
	<bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com>
	<7815067257654475794@unknownmsgid>
Message-ID: <bbaeab100807181457m6d8c380bt9a7a11fb9ab184e0@mail.gmail.com>

On Fri, Jul 18, 2008 at 12:54 PM, Bill Janssen <janssen at parc.com> wrote:
> Why is it you're trying to use "git"?

Neil set up a mirror and I was curious. Same with the bzr mirror.

-Brett

From shane at hathawaymix.org  Thu Jul 17 20:42:21 2008
From: shane at hathawaymix.org (Shane Hathaway)
Date: Thu, 17 Jul 2008 12:42:21 -0600
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
Message-ID: <487F928D.50501@hathawaymix.org>

ranjith kannikara wrote:
> As a student I am not familiar with Restricted Python and python AST
> implementation.And in need of help to start the Restricted Python
> implementation.

Here is some context for Python-Dev.

RestrictedPython is a custom Python compiler that, when combined with a 
restricted environment, provides a sandbox safe enough to allow 
partly-trusted people to write and execute scripts on a Zope server.  It 
has been used in Zope 2 for a long time and will have a future in Zope 
3.  The sandbox is more extensive than what the rexec module provides.

The safety of RestrictedPython has been validated in a somewhat formal 
process with Python 2.4.  Ranjith is working to validate it with Python 
2.5.  He is first working to discover all changes between Python 2.4 and 
2.5 that might have affected the safety of a RestrictedPython sandbox. 
Any changes to the AST, builtin functions, methods of builtin types, 
etc., need to be evaluated for safety.

So, in general, he is looking for detailed lists of changes between 
Python 2.4 and 2.5--more than the "What's New" doc.

Shane

From shane at hathawaymix.org  Fri Jul 18 16:36:45 2008
From: shane at hathawaymix.org (Shane Hathaway)
Date: Fri, 18 Jul 2008 08:36:45 -0600
Subject: [Python-Dev] Test, please ignore
Message-ID: <4880AA7D.6090302@hathawaymix.org>

...

From brett at python.org  Sat Jul 19 01:36:33 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 18 Jul 2008 16:36:33 -0700
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <487F928D.50501@hathawaymix.org>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<487F928D.50501@hathawaymix.org>
Message-ID: <bbaeab100807181636s286d6ab2u618c2b823ae33197@mail.gmail.com>

On Thu, Jul 17, 2008 at 11:42 AM, Shane Hathaway <shane at hathawaymix.org> wrote:
> ranjith kannikara wrote:
>>
>> As a student I am not familiar with Restricted Python and python AST
>> implementation.And in need of help to start the Restricted Python
>> implementation.
>
> Here is some context for Python-Dev.
>
> RestrictedPython is a custom Python compiler that, when combined with a
> restricted environment, provides a sandbox safe enough to allow
> partly-trusted people to write and execute scripts on a Zope server.  It has
> been used in Zope 2 for a long time and will have a future in Zope 3.  The
> sandbox is more extensive than what the rexec module provides.
>
> The safety of RestrictedPython has been validated in a somewhat formal
> process with Python 2.4.  Ranjith is working to validate it with Python 2.5.
>  He is first working to discover all changes between Python 2.4 and 2.5 that
> might have affected the safety of a RestrictedPython sandbox. Any changes to
> the AST, builtin functions, methods of builtin types, etc., need to be
> evaluated for safety.
>
> So, in general, he is looking for detailed lists of changes between Python
> 2.4 and 2.5--more than the "What's New" doc.
>

As it has been said, his best chance is to either diff between
versions or look at the log output from svn on the files he cares
about.

-Brett

From jcea at jcea.es  Sat Jul 19 03:41:32 2008
From: jcea at jcea.es (Jesus Cea)
Date: Sat, 19 Jul 2008 03:41:32 +0200
Subject: [Python-Dev] No beta2 tonight
In-Reply-To: <487FD54C.2040308@v.loewis.de>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
Message-ID: <4881464C.3000908@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
| bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
| 3k. I somewhat doubt that this gets resolved before the release, so
| bsddb users might need to skip 3.0.

Working on the 3.0 port just now. 03:40 in the morning, in Spain.

bsddb will be ready for python 3.0.

(writting this while compiling new 2.6 and 3.0 beta2 :-) )

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIFGTJlgi5GaxT1NAQKTBAP/f2AiHwEmMCSE6xhYVKj/0Rkh3A9/7XWG
f8CrJ8Dn//QoG7+FZ63mF1BoT9aJ3RGUIrkIePe7+XJUpa8Fq264sQMoz94Estkc
KyzSN69j+O9fTBGdI3PByFuCkfrrcl3+i0sEg0WQylOzzTpEfwKcRT5hGG2qb8c1
RZKV8BxQVXA=
=4fO2
-----END PGP SIGNATURE-----

From jcea at jcea.es  Sat Jul 19 03:36:27 2008
From: jcea at jcea.es (Jesus Cea)
Date: Sat, 19 Jul 2008 03:36:27 +0200
Subject: [Python-Dev] [Python-3000]   No beta2 tonight
In-Reply-To: <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>	<487FD54C.2040308@v.loewis.de>	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
Message-ID: <4881451B.9010304@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brett Cannon wrote:
| On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org>
wrote:
|> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote:
|>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote:
|>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into
|>>> 3k. I somewhat doubt that this gets resolved before the release, so
|>>> bsddb users might need to skip 3.0.
|>>
|>> In fact, bsddb as packages in core Python has rarely been in good shape.
|>>
|>> Has anyone actually considered that bsddb might do better if maintained
|>> strictly as an external package?  That would make it easier for the any
|>> maintainers to release updates, and removes a source of frustration for
|>> users who either don't need it or would rather wait for a happier
version.
|>>
|>> I think this is worth considering.  I vaguely recall that the bsddb
module
|>> was maintained before it was incorporated into the core Python release.
|> +1. In my recollection maintaining bsddb has been nothing but trouble
|> right from the start when we were all sitting together at "Zope Corp
|> North" in a rented office in McLean... We can remove it from 3.0. We
|> can't really remove it from 2.6, but we can certainly start
|> end-of-lifing it in 2.6.
|>
|
| Unless I hear otherwise, I will add it to PEP 3108.

03:31 AM in the morning in Spain. Just working in porting bsddb to
python3000. I have a lot of issues compiling the C code and with the
2to3 automated conversion, but I'm pretty sure can get it ready for
october release.

I'm still loooking for a *GOOD* python2->python3 conversion guide for C
language modules.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIFFFJlgi5GaxT1NAQLHlQP/cXmobkxlEs5gL9MRVNOAhcCtmATJqkQd
lpAkzExa+75cQSoP28iIa+GhxRuJJCAz9NxAw22VIkwab/93R1eFYVjdC9jqYprK
gwZsZDHKs12RlRRHFk0562ZDMgtdFB4Ne9iJXsFOKNz6ZvOhMFHfoj/blZyyocsq
zdS6SoxTtXM=
=aIrU
-----END PGP SIGNATURE-----

From jcea at jcea.es  Sat Jul 19 04:25:41 2008
From: jcea at jcea.es (Jesus Cea)
Date: Sat, 19 Jul 2008 04:25:41 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
Message-ID: <488150A5.8050804@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lennart Regebro wrote:
| On Sun, May 25, 2008 at 6:25 AM, Jesus Cea <jcea at jcea.es> wrote:
|> Since I need to port bsddb3 to py3k, what I need to know?. Is any
|> *updated* document out there?.
|
| No, but there is a not yet complete, but quite updated set of examples.
|
| http://code.google.com/p/python-incompatibility/
|
| This is a collection of code snippets that will show code that does
| work under 2.x but not under 3.x, and the 3.x way of doing it (as well
| as a way that works under both 2.6 and 3.0, in applicable cases).
|
| There is no tests for changes in the C-API, if you (or somebody else)
| would like to add that (or any other tests) that would be very
| appreciated!

I can't checkout the code. What username/password must I use?.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIFQnZlgi5GaxT1NAQL3LQP/ZxblxpJwZAQ9qIUXHAZpFlmK86y7UPfT
DX707wR1YMOujiNazE2NuJSVCVkbbOlc1G2dxTAqDm7FAThkjuIeO74ijueUeFdN
p6LP2dS5pHH8h8G9bUfDynGCjRQ4t1TUblksQgPDtrYiXlzIBqYL1qkHRf72w2c3
fyuZWBpaH0w=
=nnN4
-----END PGP SIGNATURE-----

From greg.ewing at canterbury.ac.nz  Sat Jul 19 04:43:42 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Jul 2008 14:43:42 +1200
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
Message-ID: <488154DE.8050405@canterbury.ac.nz>

Josiah Carlson wrote:
> It's entirely possible that I know very little about what was being
> made available via the bsddb module, but to match the API of what is
> included in the documentation (plus the dictionary interface that it
> supports) shouldn't be terribly difficult.

Maybe for new databases, but what about people with
existing bsddb files that they need to read?

-- 
Greg

From kmtracey at gmail.com  Sat Jul 19 05:05:11 2008
From: kmtracey at gmail.com (Karen Tracey)
Date: Fri, 18 Jul 2008 23:05:11 -0400
Subject: [Python-Dev] Change in repr of Decimal in 2.6
Message-ID: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com>

[Originally posted to python-list but on further reflection and some
feedback I think it might be more appropriate here.]

I noticed when trying out Python's 2.6b2 release that the repr of Decimal
has changed since 2.5.  On 2.5:

Python 2.5.1 (r251:54863, Mar  7 2008, 04:10:12)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import decimal
>>> decimal.Decimal(7)
Decimal("7")
>>>

double quotes were used whereas on 2.6b2:

Python 2.6b2 (r26b2:65082, Jul 18 2008, 13:36:54)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import decimal
>>> decimal.Decimal(7)
Decimal('7')
>>>

single quotes are used.  Searching around I see this was done in r60773 with
the log message:

Fix decimal repr which should have used single quotes like other reprs.

but I can't find any discussion other than that.

My problem is this breaks a bunch of doctests that were written assuming the
prior repr.  I can't just update the tests to assume the new single quotes
because they are for code that is supposed to run on everything back to
Python 2.3.

So my questions:

Is this backwards-incompatible change really necessary and could it be
reconsidered?

If it's here to stay, is there some straightforward way that I am unaware of
to construct tests that use Decimal repr but will work correctly on Python
2.3-2.6?

Thanks for any feedback,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080718/efa07deb/attachment.htm>

From python at rcn.com  Sat Jul 19 05:19:05 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 18 Jul 2008 20:19:05 -0700
Subject: [Python-Dev] Change in repr of Decimal in 2.6
References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com>
Message-ID: <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1>

From: Karen Tracey 
> I noticed when trying out Python's 2.6b2 release that the repr of Decimal has changed since 2.5.  On 2.5:
 ...
>  quotes were used whereas on 2.6b2:
...
> single quotes are used.  Searching around I see this was done in r60773 with the log message:
> Fix decimal repr which should have used single quotes like other reprs.
>
> but I can't find any discussion other than that.  

Guido requested this change so that the Decimal repr would match the style used elsewhere (i.e. str(7) --> '7').


> If it's here to stay, is there some straightforward way that I am unaware 
> of to construct tests that use Decimal repr but will work correctly on 
> Python 2.3-2.6?

A quick hack is to monkey patch the decimal module:

    >>> import decimal
    >>> decimal.Decimal.__repr__ = lambda s: 'Decimal("%s")' % str(s)

A better fix is to amend to doctests to not rely on the __repr__ format.
Instead of:

   >>> Decimal('7.1')
   Decimal("7.1")

use:

   >>> print Decimal('7.1')
   7.1


Raymond




_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com

From guido at python.org  Sat Jul 19 05:55:51 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Jul 2008 20:55:51 -0700
Subject: [Python-Dev] Change in repr of Decimal in 2.6
In-Reply-To: <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1>
References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com>
	<7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1>
Message-ID: <ca471dc20807182055k2e3b52c6h25b6478320a9a4b4@mail.gmail.com>

This is an example of the problem with doctest -- it's easy to
overspecify the tests. I don't think that whether the repr() of a
Decimal uses single or double quotes should be considered a spec cast
in stone by doctests.

On Fri, Jul 18, 2008 at 8:19 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: Karen Tracey
>>
>> I noticed when trying out Python's 2.6b2 release that the repr of Decimal
>> has changed since 2.5.  On 2.5:
>
> ...
>>
>>  quotes were used whereas on 2.6b2:
>
> ...
>>
>> single quotes are used.  Searching around I see this was done in r60773
>> with the log message:
>> Fix decimal repr which should have used single quotes like other reprs.
>>
>> but I can't find any discussion other than that.
>
> Guido requested this change so that the Decimal repr would match the style
> used elsewhere (i.e. str(7) --> '7').
>
>
>> If it's here to stay, is there some straightforward way that I am unaware
>> of to construct tests that use Decimal repr but will work correctly on
>> Python 2.3-2.6?
>
> A quick hack is to monkey patch the decimal module:
>
>   >>> import decimal
>   >>> decimal.Decimal.__repr__ = lambda s: 'Decimal("%s")' % str(s)
>
> A better fix is to amend to doctests to not rely on the __repr__ format.
> Instead of:
>
>  >>> Decimal('7.1')
>  Decimal("7.1")
>
> use:
>
>  >>> print Decimal('7.1')
>  7.1
>
>
> Raymond
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python%40rcn.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From kmtracey at gmail.com  Sat Jul 19 05:59:49 2008
From: kmtracey at gmail.com (Karen Tracey)
Date: Fri, 18 Jul 2008 23:59:49 -0400
Subject: [Python-Dev] Fwd:  Change in repr of Decimal in 2.6
In-Reply-To: <af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com>
References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com>
	<7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1>
	<af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com>
Message-ID: <af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com>

Meant to copy list on reply, sorry.

---------- Forwarded message ----------
From: Karen Tracey <kmtracey at gmail.com>
Date: Fri, Jul 18, 2008 at 11:48 PM
Subject: Re: [Python-Dev] Change in repr of Decimal in 2.6
To: Raymond Hettinger <python at rcn.com>


On Fri, Jul 18, 2008 at 11:19 PM, Raymond Hettinger <python at rcn.com> wrote:

> From: Karen Tracey
>
>> I noticed when trying out Python's 2.6b2 release that the repr of Decimal
>> has changed since 2.5.  On 2.5:
>>
> ...
>
>>  quotes were used whereas on 2.6b2:
>>
> ...
>
>> single quotes are used.  Searching around I see this was done in r60773
>> with the log message:
>> Fix decimal repr which should have used single quotes like other reprs.
>>
>> but I can't find any discussion other than that.
>>
>
> Guido requested this change so that the Decimal repr would match the style
> used elsewhere (i.e. str(7) --> '7').
>

Thanks for the background.  Can't say I ever noticed the inconsistency
myself.


>
>  If it's here to stay, is there some straightforward way that I am unaware
>> of to construct tests that use Decimal repr but will work correctly on
>> Python 2.3-2.6?
>>
>
> A quick hack is to monkey patch the decimal module:
>
>   >>> import decimal
>   >>> decimal.Decimal.__repr__ = lambda s: 'Decimal("%s")' % str(s)
>

That is quick, but does seem hackish.


> A better fix is to amend to doctests to not rely on the __repr__ format.
> Instead of:
>
>  >>> Decimal('7.1')
>  Decimal("7.1")
>
> use:
>
>  >>> print Decimal('7.1')
>  7.1
>

Yeah, but the testcases are not quite that simple.  They're often testing
return values from functions and as much verifying that the type is correct
as the value, so I think I'd have to change stuff like:

>>> f.clean('1')
Decimal("1")

to:

>>> x = f.clean('1')
>>> print type(x), x
<class 'decimal.Decimal'> 1

right?

Thanks for the answer,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080718/0fbbfe7e/attachment-0001.htm>

From python at rcn.com  Sat Jul 19 06:06:54 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 18 Jul 2008 21:06:54 -0700
Subject: [Python-Dev] Issue 3008:  Binary repr of floats
Message-ID: <D2EA66CFD5F84761AB5E2D634E99426D@RaymondLaptop1>

The new float.hex() is really nice.  Would like to augment it with a matching float.bin() method using the same notation and 
normalization and leaving all the rightmost bits as Guido suggested. I think this would help demystify floats and make it 
straightforward to show exactly what is happening during a floating point calculation that is losing precision.

def float_as_bin(x):
    '3.125 --> -0b1.1101000000000000000000000000000000000000000000000000p+1'
    hex2bin =  {'0' : '0000', '1' : '0001', '2' : '0010', '3' : '0011',
                '4' : '0100', '5' : '0101', '6' : '0110', '7' : '0111',
                '8' : '1000', '9' : '1001', 'a' : '1010', 'b' : '1011',
                'c' : '1100', 'd' : '1101', 'e' : '1110', 'f' : '1111'}
    hex_pattern = '(\-)?0x([0-9a-f]+)\.([0-9a-f]*)(.*)'
    sign, intpart, fracpart, exp = re.search(hex_pattern, x.hex().lower()).groups()
    return ((sign or '') + '0b' + intpart + '.' +
            ''.join(hex2bin[d] for d in fracpart)[:53] + exp)

The implementation would re-use Mark's code, substituting binary output for hex in the fractional part.

Raymond 


From kmtracey at gmail.com  Sat Jul 19 06:18:38 2008
From: kmtracey at gmail.com (Karen Tracey)
Date: Sat, 19 Jul 2008 00:18:38 -0400
Subject: [Python-Dev] Change in repr of Decimal in 2.6
In-Reply-To: <ca471dc20807182055k2e3b52c6h25b6478320a9a4b4@mail.gmail.com>
References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com>
	<7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1>
	<ca471dc20807182055k2e3b52c6h25b6478320a9a4b4@mail.gmail.com>
Message-ID: <af3536270807182118o4ea1a59eh1647830bcc5d1e98@mail.gmail.com>

On Fri, Jul 18, 2008 at 11:55 PM, Guido van Rossum <guido at python.org> wrote:

> This is an example of the problem with doctest -- it's easy to
> overspecify the tests. I don't think that whether the repr() of a
> Decimal uses single or double quotes should be considered a spec cast
> in stone by doctests.
>

OK, but I'm just thinking of all the places in tests we have stuff like:

>>> var.method(params)
expected_repr_of_return_value

We should really be doing something else to guard against a potential future
change in repr of whatever it is?

Thanks,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080719/1deba7d9/attachment.htm>

From jcea at jcea.es  Sat Jul 19 10:54:38 2008
From: jcea at jcea.es (Jesus Cea)
Date: Sat, 19 Jul 2008 10:54:38 +0200
Subject: [Python-Dev] Progress on switching Windows build to Berkeley DB
	4.7.25...
In-Reply-To: <6167796BFEB5D0438720AC212E89A6B00787275B@exchange.onresolve.com>
References: <6167796BFEB5D0438720AC212E89A6B00787275B@exchange.onresolve.com>
Message-ID: <4881ABCE.3000303@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Trent Nelson wrote:
| Hi all,
|
| Jesus, apologies that this has taken so long for me to get back
| to, I've been completely and utterly swamped with client work the
| past few weeks.  However, thanks to a couple of hours spare at
| Detroit airport yesterday, I was finally able to make some
| progress on updating the Windows Berkeley DB build to 4.7.25.
| I've checked in the work I've done so far to
| branches/tnelson-trunk-bsddb-47-upgrade.  One thing I wanted
| to double check with you is the following change:
|
| Modified:
python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py
|
==============================================================================
| ---
python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py
   (original)
| +++
python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py
   Wed Jun 18 06:13:44 2008
| @@ -94,7 +94,7 @@
|          # The timeout is necessary in BDB 4.5, since
DB_EVENT_REP_STARTUPDONE
|          # is not generated if the master has no new transactions.
|          # This is solved in BDB 4.6 (#15542).
| -        timeout = time.time()+2
| +        timeout = time.time()+10
|          while (time.time()<timeout) and not (self.confirmed_master
and self.client_startupdone) :
|              time.sleep(0.02)
|          if db.version() >= (4,6) :
|
| Basically, when using +2, the test failed every so often when running
the entire test_bsddb3 suite.  I picked 10 arbitrarily; it improves
things, but it's still not 100%, I still encounter the following failure
every so often:
[...]
| Can you comment on this?

Yes. The problem is a race condition between the client and the server
in the replication. If you have bad luck, the starting order will be
reversed and the sync will fail... temporally. BDB Replication Manager
has some configurable timeouts for reconnection, master election, etc.
The right thing to do is to configure them in the right way.

This work is already done in my SVN version. It will be in the 4.7.2
bsddb release. In the meanwhile, your workaround would be enough,
although the test checking will be slower.

My intent is to publish 4.7.2 as soon as I have a Python 3.0 compatible
version. Hope 2-4 weeks from now. Less if I have help solving some ugly
issues.

| Apart from this small issue, the other 311 tests pass on x86 and
| x64 with flying colours, so nice work, whatever you've been doing ;-)

Nice to see the tests passing on MS Windows :-p. I'm a bit surprised,
actually O:-).

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIGrx5lgi5GaxT1NAQI6wQP/f3oDcbPfjHQa6YcSy8KH5DYk971eNMrl
Phg0mDmy5XGgzMZArVEexVZTq1ykTBaqJ3S9hjM1RMcnNkTlB+pYS8zJxQ09psYA
YDJDnSHzPDDhLsElTzfHs/nIKaAGyEnSpsm3RCdONCJgu0LGnsjgfTmyh2bBZNol
IcAd/+dCMVU=
=Mh4Y
-----END PGP SIGNATURE-----

From ncoghlan at gmail.com  Sat Jul 19 12:22:06 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Jul 2008 20:22:06 +1000
Subject: [Python-Dev] Removing bsddb module from py3k (was Re: [Python-3000]
 No beta2 tonight)
In-Reply-To: <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>	<487FD54C.2040308@v.loewis.de>	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
	<e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>
Message-ID: <4881C04E.5060208@gmail.com>

Josiah Carlson wrote:
> On Fri, Jul 18, 2008 at 11:03 AM, Fred Drake <fdrake at acm.org> wrote:
>> On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote:
>>> It's entirely possible that I know very little about what was being
>>> made available via the bsddb module, but to match the API of what is
>>> included in the documentation (plus the dictionary interface that it
>>> supports) shouldn't be terribly difficult.
>>
>> It's also entirely possible that the API isn't interesting if you don't
>> support existing databases, for many applications.
> 
> I see where the confusion was.  I'm not suggesting that someone write
> a new bsddb module, I'm suggesting that we can provide something
> called, perhaps, on_disk_dictionary, which offers the behavior of
> bsddb, without using bsddb anywhere, or supporting bsddb files.

No, I knew what you were suggesting, I just don't see the point in doing 
it. If an app depends on bsddb specifically, they can either stick with 
the 2.x series, or they can move to 3.0 and download the externally 
maintained pybsddb (modulo any additional licensing checks required by a 
company's contracts department) or they can switch to a simpler 
file-based database format like sqlite3.

I'm not clear on what problem you are attempting to solve with the idea 
of a module with the bsddb API but without an actual bsddb backend.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Jul 19 12:43:05 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Jul 2008 20:43:05 +1000
Subject: [Python-Dev] Implementing restricted Python in Zope2
In-Reply-To: <487F928D.50501@hathawaymix.org>
References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com>
	<487F928D.50501@hathawaymix.org>
Message-ID: <4881C539.5030500@gmail.com>

Shane Hathaway wrote:
> ranjith kannikara wrote:
>> As a student I am not familiar with Restricted Python and python AST
>> implementation.And in need of help to start the Restricted Python
>> implementation.
> 
> Here is some context for Python-Dev.
> 
> RestrictedPython is a custom Python compiler that, when combined with a 
> restricted environment, provides a sandbox safe enough to allow 
> partly-trusted people to write and execute scripts on a Zope server.  It 
> has been used in Zope 2 for a long time and will have a future in Zope 
> 3.  The sandbox is more extensive than what the rexec module provides.
> 
> The safety of RestrictedPython has been validated in a somewhat formal 
> process with Python 2.4.  Ranjith is working to validate it with Python 
> 2.5.  He is first working to discover all changes between Python 2.4 and 
> 2.5 that might have affected the safety of a RestrictedPython sandbox. 
> Any changes to the AST, builtin functions, methods of builtin types, 
> etc., need to be evaluated for safety.

As others have noted, Python 2.4 didn't really have an AST - it had a 
concrete syntax tree that it called an AST.

Python 2.5 introduced an actual AST written in ASDL and the parsing and 
compilation process was rewritten on that basis.

The most relevant areas of the source tree to compare are the respective 
Parser subdirectories in 2.4 and 2.5:
http://svn.python.org/projects/python/branches/release24-maint/Parser/
http://svn.python.org/projects/python/branches/release25-maint/Parser/

The changes to symtable.c and compile.c in the Python subdirectory 
between the two versions are also highly relevant.

There may be other changes of relevance, but even going over just the 
changes I mentioned should keep you busy for quite a while (I don't 
think there was too much of the old compiler left once the AST compiler 
went into the tree).

It's easy to get a diff between files in the two versions using the 
read-only access to the SVN server:

   svn diff --old <Python 2.4 URL> --new <Python 2.5 URL>

(e.g. using the two parser directory URLs given above).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From victor.stinner at haypocalc.com  Sat Jul 19 13:23:12 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 19 Jul 2008 13:23:12 +0200
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
Message-ID: <200807191323.13124.victor.stinner@haypocalc.com>

Hi,

I filled 14 issues about bugs found by fuzzing (see my other email "Play with 
fuzzing" for more informations). Most bugs are now closed, cool :-) Last 
bugs:


== Trivial open bugs ==

segfault on locale.gettext(None)
- http://bugs.python.org/issue3302
- attached patch is trivial: fix the PyArg_ParseTuple() to block None value,
  and reject empty domain string for bindtextdomain() (to avoid strange 
  error "OSError(0): success")

invalid ref count on locale.strcoll() error
- http://bugs.python.org/issue3303
- attached patch is trivial: add "if (rel1)"

_multiprocessing.Connection() doesn't check handle
- http://bugs.python.org/issue3321
- _multiprocessing.Connection(fd) doesn't check that fd is a valid file handle
  and so may crash on poll (the "evil" FD_SET() call)
- my patch add "|| fstat(handle, &statbuf)" to make sure that the 
  file descriptor is valid


== Complex open bugs ==

block operation on closed socket/pipe for multiprocessing
- http://bugs.python.org/issue3311
- close() method sets the file handle to -1 but most methods don't check 
  the handle and so may fail or crash. Especially poll() calls
  FD_SET((SOCKET)conn->handle, &rfds); with handle=-1 => crash.
- my patch creates a new MP error: "return MP_CLOSED_FILE;", used if handle 
  is INVALID_HANDLE_VALUE to block operations (send, receive, poll) on 
  closed files for socket and pipe.

bugs in scanstring_str() and scanstring_unicode() of _json module
- http://bugs.python.org/issue3322
- scanstring() function crashs if second argument is a big negative 
  integer. There is no attached patch because I don't understand this 
  function enough to fix it correctly, but I suggest to raise a ValueError
  if end is too small/big

invalid object destruction in re.finditer()
- or "PyObject_DEL inconsistency if pydebug option is used"
- http://bugs.python.org/issue3299
- It's the most complex bug, I prefer to write a new email :-)


== Need backport / port to python 3.0 ==

invalid call to PyMem_Free() in fileio_init()
- http://bugs.python.org/issue3304
- patch applied in Python 2.6 (trunk) but not in Python 3000:
  "i'm assuming that'll be merged into py3k automagically."
  wrote Gregory P. Smith

missing lock release in BZ2File_iternext()
- http://bugs.python.org/issue3309
- patch applied in Python 2.6 but "Needs backporting to release25-maint."
  wrote Gregory P. Smith


When all bugs will be closed, I will restart a fuzzing Python ;-) But I also 
tried with my patches and I was unable to find new bugs, great!

Victor

From victor.stinner at haypocalc.com  Sat Jul 19 13:40:34 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 19 Jul 2008 13:40:34 +0200
Subject: [Python-Dev] Problem with PyObject_DEL in pydebug mode
Message-ID: <200807191340.34939.victor.stinner@haypocalc.com>

Hi,

I filled an issue about the crash: "import re; re.finditer("a", {})"
   http://bugs.python.org/issue3299

It appears quickly that the bug is specific to Python compiled in pydebug 
mode, or to be exact: when Py_TRACE_REFS is defined.

== The Py_TRACE_REFS option ==

The problem is that PyObject_Del(obj) and PyObject_DEL(obj) don't remove obj 
from the "object list" (_ob_prev and _ob_next attributes of all objects when 
Py_TRACE_REFS is defined). And so obj will be reused later (maybe removed by 
garbage collector? or something like that) whereas it's invalid (memory 
freed).

PyObject_NEW(obj) and PyObject_NEW_VAR(obj) call PyObject_Init() which 
registers the obj to the object list. So a developer can expect that 
PyObject_DEL(obj) will remove the object from this list.

PyObject_Del(obj) and PyObject_DEL(obj) are used on object initialization 
failure, but also in "dealloc" callbacks.

Classic object destruction is done by Py_DECREF(): when reference count is 
zero, dealloc() is called. PyObject_Del/DEL are different because they just 
free memory but don't call dealloc() whereas some attributes may be set (and 
some other may be uninitialized or NULL).


== Solutions ==

(a) Replace PyObject_Del/PyObject_DEL by Py_DECREF to call dealloc callback 
which will call PyObject_Del/PyObject_DEL. I prefer this solution because 
dealloc is called and so we make that all attributes are deinitialized. 

New problem: dealloc expects that the object is fully initialized (all 
attributes are set and are not NULL), which is wrong is initialization fails. 
Eg. with re module,scanner_dealloc() calls Py_DECREF(self->pattern); whereas 
pattern attribute is NULL. Fix: replace Py_DECREF by Py_XDECREF. But all 
dealloc have to be reviewed.

(b) Fix PyObject_Del/PyObject_DEL to remove object from the object list

(c) Create new macro which is PyObject_Del/PyObject_DEL + remove the object 
from the list

(d) Never use Py_TRACE_REFS :-)


I wrote many informations in http://bugs.python.org/issue3299.

-- 
Victor Stinner aka haypo
http://fusil.hachoir.org/

From jnoller at gmail.com  Sat Jul 19 15:14:44 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Sat, 19 Jul 2008 09:14:44 -0400
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <200807191323.13124.victor.stinner@haypocalc.com>
References: <200807191323.13124.victor.stinner@haypocalc.com>
Message-ID: <4222a8490807190614m370447d9s30d4af12de3e3152@mail.gmail.com>

On Sat, Jul 19, 2008 at 7:23 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> I filled 14 issues about bugs found by fuzzing (see my other email "Play with
> fuzzing" for more informations). Most bugs are now closed, cool :-) Last
> bugs:
>
>
> == Trivial open bugs ==
>
> segfault on locale.gettext(None)
> - http://bugs.python.org/issue3302
> - attached patch is trivial: fix the PyArg_ParseTuple() to block None value,
>  and reject empty domain string for bindtextdomain() (to avoid strange
>  error "OSError(0): success")
>
> invalid ref count on locale.strcoll() error
> - http://bugs.python.org/issue3303
> - attached patch is trivial: add "if (rel1)"
>
> _multiprocessing.Connection() doesn't check handle
> - http://bugs.python.org/issue3321
> - _multiprocessing.Connection(fd) doesn't check that fd is a valid file handle
>  and so may crash on poll (the "evil" FD_SET() call)
> - my patch add "|| fstat(handle, &statbuf)" to make sure that the
>  file descriptor is valid
>
>
> == Complex open bugs ==
>
> block operation on closed socket/pipe for multiprocessing
> - http://bugs.python.org/issue3311
> - close() method sets the file handle to -1 but most methods don't check
>  the handle and so may fail or crash. Especially poll() calls
>  FD_SET((SOCKET)conn->handle, &rfds); with handle=-1 => crash.
> - my patch creates a new MP error: "return MP_CLOSED_FILE;", used if handle
>  is INVALID_HANDLE_VALUE to block operations (send, receive, poll) on
>  closed files for socket and pipe.
>
> bugs in scanstring_str() and scanstring_unicode() of _json module
> - http://bugs.python.org/issue3322
> - scanstring() function crashs if second argument is a big negative
>  integer. There is no attached patch because I don't understand this
>  function enough to fix it correctly, but I suggest to raise a ValueError
>  if end is too small/big
>
> invalid object destruction in re.finditer()
> - or "PyObject_DEL inconsistency if pydebug option is used"
> - http://bugs.python.org/issue3299
> - It's the most complex bug, I prefer to write a new email :-)
>
>
> == Need backport / port to python 3.0 ==
>
> invalid call to PyMem_Free() in fileio_init()
> - http://bugs.python.org/issue3304
> - patch applied in Python 2.6 (trunk) but not in Python 3000:
>  "i'm assuming that'll be merged into py3k automagically."
>  wrote Gregory P. Smith
>
> missing lock release in BZ2File_iternext()
> - http://bugs.python.org/issue3309
> - patch applied in Python 2.6 but "Needs backporting to release25-maint."
>  wrote Gregory P. Smith
>
>
> When all bugs will be closed, I will restart a fuzzing Python ;-) But I also
> tried with my patches and I was unable to find new bugs, great!
>
> Victor

Thank you Victor - I didn't want to change any underlying
multiprocessing code until we had the test suite in a better state
(which we do now). Now that beta 2 is out, I will address the
multiprocessing ones asap.

One suggestion would be to include tests to prove the bugs is fixed if
possible (to the patch), so we can add them to the suite. I know that
that is not always possible, but it does help. I worry about making
code changes without appropriate tests. If anything, a snippet of code
"exploiting" the flaw may help generate a test case on my end. Thanks
again for doing this.

-jesse

From josiah.carlson at gmail.com  Sat Jul 19 16:54:39 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Sat, 19 Jul 2008 07:54:39 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <488154DE.8050405@canterbury.ac.nz>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<487FD54C.2040308@v.loewis.de>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<488154DE.8050405@canterbury.ac.nz>
Message-ID: <e6511dbf0807190754p6c311945l3dabcc8168754f46@mail.gmail.com>

On Fri, Jul 18, 2008 at 7:43 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Josiah Carlson wrote:
>>
>> It's entirely possible that I know very little about what was being
>> made available via the bsddb module, but to match the API of what is
>> included in the documentation (plus the dictionary interface that it
>> supports) shouldn't be terribly difficult.
>
> Maybe for new databases, but what about people with
> existing bsddb files that they need to read?

Classic data transition issue.  Dump the data with the old software,
reload the data with the new software.  Both bsddb and sqlite are
pretty fast, the latter of which has nice bulk inserts, after-the-fact
index creation, ..., which can reduce the time it takes to do the
transition.

 - Josiah

From josiah.carlson at gmail.com  Sat Jul 19 17:43:28 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Sat, 19 Jul 2008 08:43:28 -0700
Subject: [Python-Dev] [Python-3000] No beta2 tonight
In-Reply-To: <e6511dbf0807190754p6c311945l3dabcc8168754f46@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<488154DE.8050405@canterbury.ac.nz>
	<e6511dbf0807190754p6c311945l3dabcc8168754f46@mail.gmail.com>
Message-ID: <e6511dbf0807190843y16a7a26dwb54d29130918592c@mail.gmail.com>

On Sat, Jul 19, 2008 at 7:54 AM, Josiah Carlson
<josiah.carlson at gmail.com> wrote:
> On Fri, Jul 18, 2008 at 7:43 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Josiah Carlson wrote:
>>>
>>> It's entirely possible that I know very little about what was being
>>> made available via the bsddb module, but to match the API of what is
>>> included in the documentation (plus the dictionary interface that it
>>> supports) shouldn't be terribly difficult.
>>
>> Maybe for new databases, but what about people with
>> existing bsddb files that they need to read?
>
> Classic data transition issue.  Dump the data with the old software,
> reload the data with the new software.  Both bsddb and sqlite are
> pretty fast, the latter of which has nice bulk inserts, after-the-fact
> index creation, ..., which can reduce the time it takes to do the
> transition.

After discussing with Guido off-list about how some companies have
come to rely on deep features in bsddb, concerns that these companies
wouldn't be able to transition away from bsddb, plus Jesus' picking up
the code for 3.0 maintenance, I'm taking the offer to write a wrapper
for sqlite off the table.  Not because I don't think it could be done,
but because no one seems to want what I'm offering (a 95% solution
that doesn't require weeks/months of maintenance every release).

 - Josiah

From josiah.carlson at gmail.com  Sat Jul 19 18:27:15 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Sat, 19 Jul 2008 09:27:15 -0700
Subject: [Python-Dev] Removing bsddb module from py3k (was Re:
	[Python-3000] No beta2 tonight)
In-Reply-To: <4881C04E.5060208@gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>
	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
	<e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>
	<4881C04E.5060208@gmail.com>
Message-ID: <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com>

On Sat, Jul 19, 2008 at 3:22 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Josiah Carlson wrote:
>>
>> On Fri, Jul 18, 2008 at 11:03 AM, Fred Drake <fdrake at acm.org> wrote:
>>>
>>> On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote:
>>>>
>>>> It's entirely possible that I know very little about what was being
>>>> made available via the bsddb module, but to match the API of what is
>>>> included in the documentation (plus the dictionary interface that it
>>>> supports) shouldn't be terribly difficult.
>>>
>>> It's also entirely possible that the API isn't interesting if you don't
>>> support existing databases, for many applications.
>>
>> I see where the confusion was.  I'm not suggesting that someone write
>> a new bsddb module, I'm suggesting that we can provide something
>> called, perhaps, on_disk_dictionary, which offers the behavior of
>> bsddb, without using bsddb anywhere, or supporting bsddb files.
>
> No, I knew what you were suggesting, I just don't see the point in doing it.
> If an app depends on bsddb specifically, they can either stick with the 2.x
> series, or they can move to 3.0 and download the externally maintained
> pybsddb (modulo any additional licensing checks required by a company's
> contracts department) or they can switch to a simpler file-based database
> format like sqlite3.
>
> I'm not clear on what problem you are attempting to solve with the idea of a
> module with the bsddb API but without an actual bsddb backend.

On-disk key -> value dictionary.  In every use of bsddb that I've seen
(or done myself), that's been the extent of it's use.  That's what I
*was* offering.  But it seems that everyone has had experience with
bsddb on a far deeper level (beyond dictionary + cursor access), and
would find (like you) absolutely no use in an on-disk dictionary with
a sqlite backend (which hasn't had the same maintenance issues as
bsddb), which is why I withdrew the offer.

 - Josiah

From steve at holdenweb.com  Thu Jul 17 07:50:48 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 17 Jul 2008 01:50:48 -0400
Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement)
In-Reply-To: <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com>
References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com>	<B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1>	<487E5EB6.8070306@voidspace.org.uk>	<20080717002252.GG26808@steerpike.home.puzzling.org>	<87ej5tz6fh.fsf@benfinney.id.au>	<20080717010730.GA29299@steerpike.home.puzzling.org>	<87abghz4cu.fsf@benfinney.id.au>	<20080717014556.GB29299@steerpike.home.puzzling.org>	<871w1tz219.fsf@benfinney.id.au>
	<87sku9xmcr.fsf@benfinney.id.au>
	<ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com>
Message-ID: <487EDDB8.1030803@holdenweb.com>

Guido van Rossum wrote:
> On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> The result I'm trying to avoid by this is that of having the
>> externally visible behaviour of functions drift from the promise made
>> by their names. Either rename the function, or create a new one for
>> the new functionality.
> 
> Oh get over it. This is getting ridiculous. Where did you *get* these
> design "principles" you are so fond of? Read the zen of Python and
> come back after you've memorized it.
> 
Thank you. I was about to suggest that len() should be renamed 
length_or_raise_exception_if_not_a_container().

But then I did remark (before I stopped adding to this thread's 
interminable length, some aeons ago) that this was getting silly. 
Someone would surely have replied that the proposal had negative 
connotations and the correct name was really 
length_as_long_as_a_container. Or something equally trivial.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From jcea at jcea.es  Sun Jul 20 03:44:10 2008
From: jcea at jcea.es (Jesus Cea)
Date: Sun, 20 Jul 2008 03:44:10 +0200
Subject: [Python-Dev] Removing bsddb module from py3k (was
 Re:	[Python-3000] No beta2 tonight)
In-Reply-To: <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>	<ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com>	<bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com>	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>	<e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>	<4881C04E.5060208@gmail.com>
	<e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com>
Message-ID: <4882986A.3060509@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Josiah Carlson wrote:
| On-disk key -> value dictionary.  In every use of bsddb that I've seen
| (or done myself), that's been the extent of it's use.  That's what I
| *was* offering.  But it seems that everyone has had experience with
| bsddb on a far deeper level (beyond dictionary + cursor access), and
| would find (like you) absolutely no use in an on-disk dictionary with
| a sqlite backend (which hasn't had the same maintenance issues as
| bsddb), which is why I withdrew the offer.

For such a simple application, you can use the already in stdlib "gdbm"
module. Just remember it is not transactional, so beware diskfulls,
application crashes, etc.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIKXKJlgi5GaxT1NAQLmSQP+PEDrOy5AM6IRhIDFriyks0nTpA5a3vYi
bgrVRNJYQXTkV43AFD0bUkDFT1VBsROSlSO0jEsSfHXG2FG51TJa7Hp1LCX62ryV
H6NtztvqlA5OpTxEcWGPdha0XRueRwjYe/a+cVFXjAo+fXirlIac2W6ZhNJiAehg
VP9lxrffQSM=
=KV8P
-----END PGP SIGNATURE-----

From tjreedy at udel.edu  Sun Jul 20 04:52:21 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 19 Jul 2008 22:52:21 -0400
Subject: [Python-Dev] Fwd:  Change in repr of Decimal in 2.6
In-Reply-To: <af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com>
References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com>	<7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1>	<af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com>
	<af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com>
Message-ID: <g5u993$3p2$1@ger.gmane.org>



Karen Tracey wrote:

> Yeah, but the testcases are not quite that simple.  They're often 
> testing return values from functions and as much verifying that the type 
> is correct as the value, so I think I'd have to change stuff like:
> 
>  >>> f.clean('1')
> Decimal("1")
> 
> to:
> 
>  >>> x = f.clean('1')
>  >>> print type(x), x
> <class 'decimal.Decimal'> 1
>  
> right? 

 >>> f.clean('1') == Decimal('1')
True

Since 'True' is a keyword, and Guido is *very* reluctant to even add 
keywords, let alone change their spelling, I think you can depend on 
that working indefinitely.  Similarly, if you now have

 >>> type(a)
<type 'int'>

you test will fail in 3.0 which instead prints '<class 'int'>.  But

 >>>type(a) is int
True

will continue working.

Doctest has two quite different uses.  One is to check text with 
interactive examples to make sure there are no mistakes in the examples. 
  For that, printing representations is usually the natural style.  Then 
one must accept that representations change faster that object behavior.

The other use is to check code.  For that, the more stilted  style is 
more future-proof.  For text doing double duty as a formal reference and 
code test, I would go for cross-version dependability.

Terry Jan Reedy


From charleshixsn at earthlink.net  Sun Jul 20 05:01:15 2008
From: charleshixsn at earthlink.net (Charles Hixson)
Date: Sat, 19 Jul 2008 20:01:15 -0700
Subject: [Python-Dev] Python-2.6b2.tar.bz missing sig file on web site
Message-ID: <200807192001.16011.charleshixsn@earthlink.net>

Python-2.6b2.tar.bz missing sig file on web site.  That's about all the info I 
have, except that the tgz is also missing the sig file, and that 3.0b2 has 
it's sig file.

Hope this is the correct place to report this.

From barry at python.org  Sun Jul 20 06:20:55 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 20 Jul 2008 00:20:55 -0400
Subject: [Python-Dev] Python-2.6b2.tar.bz missing sig file on web site
In-Reply-To: <200807192001.16011.charleshixsn@earthlink.net>
References: <200807192001.16011.charleshixsn@earthlink.net>
Message-ID: <64701199-2C4F-4BAD-9E7D-57D8F74855F6@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 19, 2008, at 11:01 PM, Charles Hixson wrote:

> Python-2.6b2.tar.bz missing sig file on web site.  That's about all  
> the info I
> have, except that the tgz is also missing the sig file, and that  
> 3.0b2 has
> it's sig file.
>
> Hope this is the correct place to report this.

It is, thanks.  Looks like I forgot to svn add those files.  Please  
try again now.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSIK9KHEjvBPtnXfVAQLBdwQAok5D6yqAkfOEHjvq5YEVuOwnLLWz1UbV
Y/bH6Q7lxnh4aIKJn2Mty82fLM7MpJQhF7zM+wLwLp2ilqgvRhZ1B9H8ykSErT23
Xpa7NAUzlxlL3Ca5r1jF7G10L+O46IpTz9+F1j5uvY1wXCjK9EP+HctpLzBRf6Hz
ng6EyO/cfto=
=nPmM
-----END PGP SIGNATURE-----

From kmtracey at gmail.com  Sun Jul 20 06:55:25 2008
From: kmtracey at gmail.com (Karen Tracey)
Date: Sun, 20 Jul 2008 00:55:25 -0400
Subject: [Python-Dev] Fwd: Change in repr of Decimal in 2.6
In-Reply-To: <g5u993$3p2$1@ger.gmane.org>
References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com>
	<7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1>
	<af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com>
	<af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com>
	<g5u993$3p2$1@ger.gmane.org>
Message-ID: <af3536270807192155q3381a21cl3f94ca72794acf88@mail.gmail.com>

On Sat, Jul 19, 2008 at 10:52 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> >>> f.clean('1') == Decimal('1')
> True
>

Ah, yes, why didn't I think of that?


>
> Since 'True' is a keyword, and Guido is *very* reluctant to even add
> keywords, let alone change their spelling, I think you can depend on that
> working indefinitely.  Similarly, if you now have
>
> >>> type(a)
> <type 'int'>
>
> you test will fail in 3.0 which instead prints '<class 'int'>.  But
>
> >>>type(a) is int
> True
>
> will continue working.
>
> Doctest has two quite different uses.  One is to check text with
> interactive examples to make sure there are no mistakes in the examples.
>  For that, printing representations is usually the natural style.  Then one
> must accept that representations change faster that object behavior.
>
> The other use is to check code.  For that, the more stilted  style is more
> future-proof.  For text doing double duty as a formal reference and code
> test, I would go for cross-version dependability.
>
>
Thanks, I see your point about the different styles.   I think we have a
fair number of tests that use the print-style (probably because that's
easiest, and then people see that style everywhere and do the same when
adding new tests) when to be future proof they really should be using the
alternative.

Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080720/0853ceb9/attachment.htm>

From barry at barrys-emacs.org  Sun Jul 20 09:44:05 2008
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 20 Jul 2008 08:44:05 +0100
Subject: [Python-Dev] Web site type: Python 2.6b2 Released: 18-Jun-2008
Message-ID: <AA73256D-736B-4923-9102-6A8D6CDB05A0@barrys-emacs.org>

I think you mean july.

Barry


From barry at barrys-emacs.org  Sun Jul 20 10:57:21 2008
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 20 Jul 2008 09:57:21 +0100
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <488150A5.8050804@jcea.es>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
Message-ID: <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>

See http://code.google.com/p/python-incompatibility/source/checkout

Barry



On Jul 19, 2008, at 03:25, Jesus Cea wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Lennart Regebro wrote:
> | On Sun, May 25, 2008 at 6:25 AM, Jesus Cea <jcea at jcea.es> wrote:
> |> Since I need to port bsddb3 to py3k, what I need to know?. Is any
> |> *updated* document out there?.
> |
> | No, but there is a not yet complete, but quite updated set of  
> examples.
> |
> | http://code.google.com/p/python-incompatibility/
> |
> | This is a collection of code snippets that will show code that does
> | work under 2.x but not under 3.x, and the 3.x way of doing it (as  
> well
> | as a way that works under both 2.6 and 3.0, in applicable cases).
> |
> | There is no tests for changes in the C-API, if you (or somebody  
> else)
> | would like to add that (or any other tests) that would be very
> | appreciated!
>
> I can't checkout the code. What username/password must I use?.
>
> - --
> Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
> jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/   
> _/_/
> jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
> .                              _/_/  _/_/    _/_/          _/_/  _/_/
> "Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
> "My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iQCVAwUBSIFQnZlgi5GaxT1NAQL3LQP/ZxblxpJwZAQ9qIUXHAZpFlmK86y7UPfT
> DX707wR1YMOujiNazE2NuJSVCVkbbOlc1G2dxTAqDm7FAThkjuIeO74ijueUeFdN
> p6LP2dS5pHH8h8G9bUfDynGCjRQ4t1TUblksQgPDtrYiXlzIBqYL1qkHRf72w2c3
> fyuZWBpaH0w=
> =nnN4
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/barry 
> %40barrys-emacs.org
>


From barry at python.org  Sun Jul 20 14:23:28 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 20 Jul 2008 08:23:28 -0400
Subject: [Python-Dev] Web site type: Python 2.6b2 Released: 18-Jun-2008
In-Reply-To: <AA73256D-736B-4923-9102-6A8D6CDB05A0@barrys-emacs.org>
References: <AA73256D-736B-4923-9102-6A8D6CDB05A0@barrys-emacs.org>
Message-ID: <11B5E8BA-1BEB-4D9F-A2FE-6AA4D49B020C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 20, 2008, at 3:44 AM, Barry Scott wrote:

> I think you mean july.

Thanks, I'll fix that.
- -B

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSIMuQXEjvBPtnXfVAQLFVwP/SYQkNNHnReOOuPxnnJQkNqKTnDYLpZqT
9J0y/fExHPDNxrawPnxQfxKO9ARFv+DrJSO0aBaQrQP3F164xVfudSM/jN1eq9Th
w5im9VxdQTDp/QueK2JZA14LkGK+tj0owY7W599GFavX/OE1nUAWA16YwQAXI7EL
mJO4KyFciOk=
=v/Pl
-----END PGP SIGNATURE-----

From josiah.carlson at gmail.com  Sun Jul 20 19:24:56 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Sun, 20 Jul 2008 10:24:56 -0700
Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was
	Re: No beta2 tonight)
In-Reply-To: <4882986A.3060509@jcea.es>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
	<e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>
	<4881C04E.5060208@gmail.com>
	<e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com>
	<4882986A.3060509@jcea.es>
Message-ID: <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com>

On Sat, Jul 19, 2008 at 6:44 PM, Jesus Cea <jcea at jcea.es> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Josiah Carlson wrote:
> | On-disk key -> value dictionary.  In every use of bsddb that I've seen
> | (or done myself), that's been the extent of it's use.  That's what I
> | *was* offering.  But it seems that everyone has had experience with
> | bsddb on a far deeper level (beyond dictionary + cursor access), and
> | would find (like you) absolutely no use in an on-disk dictionary with
> | a sqlite backend (which hasn't had the same maintenance issues as
> | bsddb), which is why I withdrew the offer.
>
> For such a simple application, you can use the already in stdlib "gdbm"
> module. Just remember it is not transactional, so beware diskfulls,
> application crashes, etc.

But sqlite is transactional, can offer cursors, getrange, etc., etc.

I'm still curious as to what deep features people are using in bsddb.
Anyone have any pointers to open source software?

 - Josiah

From greg at krypto.org  Sun Jul 20 20:19:13 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 20 Jul 2008 11:19:13 -0700
Subject: [Python-Dev] Search broken on the python dev docs?
Message-ID: <52dc1c820807201119k5b3589cfwb0b378c8da2c008e@mail.gmail.com>

http://docs.python.org/dev/

the search box worked for earlier releases but has been broken and returns
nothing useful of late.

If I enter simple terms like 'time' or 'os' or 'os.walk' what is returned is
pathetic.

how does this work?  is an index corrupt or not being regenerated?

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080720/e473c036/attachment.htm>

From victor.stinner at haypocalc.com  Sun Jul 20 22:42:05 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 20 Jul 2008 22:42:05 +0200
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <4222a8490807190614m370447d9s30d4af12de3e3152@mail.gmail.com>
References: <200807191323.13124.victor.stinner@haypocalc.com>
	<4222a8490807190614m370447d9s30d4af12de3e3152@mail.gmail.com>
Message-ID: <200807202242.05347.victor.stinner@haypocalc.com>

Hi,

Le Saturday 19 July 2008 15:14:44, vous avez ?crit :
> Thank you Victor - I didn't want to change any underlying
> multiprocessing code until we had the test suite in a better state
> (which we do now) (...)
>
> One suggestion would be to include tests to prove the bugs is fixed if
> possible (to the patch), so we can add them to the suite. 

Ok, here is a patch for test_multiprocessing.py.
- TestClosedFile.test_open() verify that Connection() rejects closed file 
descriptor
- TestClosedFile.test_operations() verify that Connection() raises IOError for 
operations on closed file

I don't know if Connection() is a socket or a pipe. Both should be tested.

Victor
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_multiprocessing.patch
Type: text/x-diff
Size: 1883 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080720/87610750/attachment.patch>

From victor.stinner at haypocalc.com  Sun Jul 20 22:45:39 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 20 Jul 2008 22:45:39 +0200
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <20080719195209.GA18271@amk.local>
References: <200807191323.13124.victor.stinner@haypocalc.com>
	<20080719195209.GA18271@amk.local>
Message-ID: <200807202245.39874.victor.stinner@haypocalc.com>

Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit?:
> Excellent work!  Another fruitful area for fuzzing might be the
> miniature virtual machine used by the re module.  It's possible to
> import _sre and call the compile() function directly (see the end of
> Lib/sre_compile.py for how it's invoked); I wonder how the regex VM
> copes with random strings of bytecode.

Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted 
_sre.compile() in my fuzzer.

For information, it's also very easy to crash CPython with fuzzed .pyc file.

It's hard to check bytecode without execute it. It's maybe better to add 
checks directly in the VM.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From g.brandl at gmx.net  Sun Jul 20 23:39:54 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 20 Jul 2008 23:39:54 +0200
Subject: [Python-Dev] Search broken on the python dev docs?
In-Reply-To: <52dc1c820807201119k5b3589cfwb0b378c8da2c008e@mail.gmail.com>
References: <52dc1c820807201119k5b3589cfwb0b378c8da2c008e@mail.gmail.com>
Message-ID: <g60bbq$nv3$1@ger.gmane.org>

Gregory P. Smith schrieb:
> http://docs.python.org/dev/
> 
> the search box worked for earlier releases but has been broken and 
> returns nothing useful of late.
> 
> If I enter simple terms like 'time' or 'os' or 'os.walk' what is 
> returned is pathetic.
> 
> how does this work?  is an index corrupt or not being regenerated?

Something is wrong; I'll check this and try to fix it.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From andrewm at object-craft.com.au  Mon Jul 21 02:13:02 2008
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Mon, 21 Jul 2008 10:13:02 +1000
Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was
	Re: No beta2 tonight)
In-Reply-To: <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com> 
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>
	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
	<e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>
	<4881C04E.5060208@gmail.com>
	<e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com>
	<4882986A.3060509@jcea.es>
	<e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com>
Message-ID: <20080721001302.EAED5600808@longblack.object-craft.com.au>

>But sqlite is transactional, can offer cursors, getrange, etc., etc.
>
>I'm still curious as to what deep features people are using in bsddb.

It's not using "deep features", unless you define their on-disk layout
as deep, but it does get used for things such as interactions with other
systems - for example, using it to maintain Radius user databases for a
(proprietary/commercial) Radius auth daemon. But dropping it from the
core won't stop this.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From steve at holdenweb.com  Mon Jul 21 03:37:47 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 20 Jul 2008 21:37:47 -0400
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com>
References: <200807191323.13124.victor.stinner@haypocalc.com>	<20080719195209.GA18271@amk.local>
	<200807202245.39874.victor.stinner@haypocalc.com>
Message-ID: <4883E86B.1000901@holdenweb.com>

Victor Stinner wrote:
> Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit :
>> Excellent work!  Another fruitful area for fuzzing might be the
>> miniature virtual machine used by the re module.  It's possible to
>> import _sre and call the compile() function directly (see the end of
>> Lib/sre_compile.py for how it's invoked); I wonder how the regex VM
>> copes with random strings of bytecode.
> 
> Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted 
> _sre.compile() in my fuzzer.
> 
> For information, it's also very easy to crash CPython with fuzzed .pyc file.
> 
> It's hard to check bytecode without execute it. It's maybe better to add 
> checks directly in the VM.
> 

I think you'll find most developers (and many users too, come to that) 
reluctant to add any checking that would slow down eval.c, the heart of 
the virtual machine.

So unless you can find a way to add the checks without slowing it down, 
an external checker might be better.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Mon Jul 21 03:37:47 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 20 Jul 2008 21:37:47 -0400
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com>
References: <200807191323.13124.victor.stinner@haypocalc.com>	<20080719195209.GA18271@amk.local>
	<200807202245.39874.victor.stinner@haypocalc.com>
Message-ID: <4883E86B.1000901@holdenweb.com>

Victor Stinner wrote:
> Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit :
>> Excellent work!  Another fruitful area for fuzzing might be the
>> miniature virtual machine used by the re module.  It's possible to
>> import _sre and call the compile() function directly (see the end of
>> Lib/sre_compile.py for how it's invoked); I wonder how the regex VM
>> copes with random strings of bytecode.
> 
> Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted 
> _sre.compile() in my fuzzer.
> 
> For information, it's also very easy to crash CPython with fuzzed .pyc file.
> 
> It's hard to check bytecode without execute it. It's maybe better to add 
> checks directly in the VM.
> 

I think you'll find most developers (and many users too, come to that) 
reluctant to add any checking that would slow down eval.c, the heart of 
the virtual machine.

So unless you can find a way to add the checks without slowing it down, 
an external checker might be better.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From jcea at jcea.es  Mon Jul 21 11:06:51 2008
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 21 Jul 2008 11:06:51 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
References: <4838EA2D.7060402@jcea.es>	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
Message-ID: <488451AB.50406@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Scott wrote:
| See http://code.google.com/p/python-incompatibility/source/checkout

Thanks.

I'm *VERY* interested in 2.6->3.0 migration guide for C module
extensions. 3.0 is around the corner and the API is changing almost
daily :-p.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRRpZlgi5GaxT1NAQLUlwP9GDwp2zKFfHQWmRnhOyGfTwa0GqWGW1rZ
IrHrPXgFweh1mnbhGIjVkvlMJxHnhP2yGKStkN4641zYJ5KgdRJBaIdfrqoOkPTb
xzzHJ7DqaAGmRSKRoCZ8jzjCuC4BKeuLalii3V7ofYBimDgsHL42lb0MWxZBVNK5
SnafyX+rqGU=
=re/E
-----END PGP SIGNATURE-----

From g.brandl at gmx.net  Mon Jul 21 11:11:50 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 21 Jul 2008 11:11:50 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <488451AB.50406@jcea.es>
References: <4838EA2D.7060402@jcea.es>	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>	<488150A5.8050804@jcea.es>	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es>
Message-ID: <g61jt8$hag$2@ger.gmane.org>

Jesus Cea schrieb:
> Barry Scott wrote:
> | See http://code.google.com/p/python-incompatibility/source/checkout
> 
> Thanks.
> 
> I'm *VERY* interested in 2.6->3.0 migration guide for C module
> extensions. 3.0 is around the corner and the API is changing almost
> daily :-p.

So it's good that nobody has written a migration guide yet; he'd have
to rewrite it daily.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From regebro at gmail.com  Mon Jul 21 11:26:07 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 21 Jul 2008 11:26:07 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <488451AB.50406@jcea.es>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es>
Message-ID: <319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com>

On Mon, Jul 21, 2008 at 11:06, Jesus Cea <jcea at jcea.es> wrote:
> I'm *VERY* interested in 2.6->3.0 migration guide for C module
> extensions. 3.0 is around the corner and the API is changing almost
> daily :-p.

It would be great if python-incompatibility would have examples of the
C-api changes as well. That would really help in migrating and writing
a migration guide. It would be great if you could help with this!

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From jcea at jcea.es  Mon Jul 21 11:26:43 2008
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 21 Jul 2008 11:26:43 +0200
Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was
 Re: No beta2 tonight)
In-Reply-To: <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>	<ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com>	<e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com>	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>	<e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>	<4881C04E.5060208@gmail.com>	<e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com>	<4882986A.3060509@jcea.es>
	<e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com>
Message-ID: <48845653.5000100@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Josiah Carlson wrote:
| I'm still curious as to what deep features people are using in bsddb.
| Anyone have any pointers to open source software?

I'm using replication and distributed transactions. Database encryption
and page integrity checks. Abusing the master election BDB
infrastructure for some evil but fun purposes. Managing databases in the
200 terabyte range. Locking & logging infrastructure to implement
application logic integrated with database operation. MVCC everywhere.

Nothing I can show you without killing you after, though.

Independently of current bsddb usage profile, current 2.6 code support
for distributed transactions and replication enables new application
uses that sqlite can't match.

Since I'm taking full responsability over bsddb both in 2.6 and 3.0, I
don't really see what the issue is. Past mistakes and maintenance
nightmares WILL NOT be repeated.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRWSplgi5GaxT1NAQL2CgQAnbv7hrxb4I5KzdZ1T05A/pGzzuX4NATm
SN5TBeKsesl1LlEwI1+5xaBv9uxuJFfnPYMFLlW7NZdGumIdNCo7QXMhPmEdvuNx
2BxcmlS2Zuag8piAbmgq31Ov2RnnskOCf5ZxmuK4smk5Y0H2L+hUGG96Rk8dz9BU
s+AVkiaorKY=
=szyU
-----END PGP SIGNATURE-----

From jcea at jcea.es  Mon Jul 21 11:34:23 2008
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 21 Jul 2008 11:34:23 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <g61jt8$hag$2@ger.gmane.org>
References: <4838EA2D.7060402@jcea.es>	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>	<488150A5.8050804@jcea.es>	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>	<488451AB.50406@jcea.es>
	<g61jt8$hag$2@ger.gmane.org>
Message-ID: <4884581F.8010201@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Brandl wrote:
| So it's good that nobody has written a migration guide yet; he'd have
| to rewrite it daily.

Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid
redoing the same work 15 times XDDDDDDDDDD

I'm not lazy, but it is so sunny outside...

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRYHplgi5GaxT1NAQL4hwP+LUSeAZxP/6q2K8GBs9ptqMnZUPh6ct0j
mhJqnAGDrI3XgcyHa3DeWwAltIfbLj9KC+UcgBNSvVQvqqMaPxKhc+tsbXqmsude
+o2e9IhZCgWg91FyJv64LPJr/tgMG4g7QBxHR1G9GF76YYrjCQCzi1f06lNkplQh
o/KIbRBmwRc=
=Q6o6
-----END PGP SIGNATURE-----

From jcea at jcea.es  Mon Jul 21 11:48:05 2008
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 21 Jul 2008 11:48:05 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com>
References: <4838EA2D.7060402@jcea.es>	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>	<488150A5.8050804@jcea.es>	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>	<488451AB.50406@jcea.es>
	<319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com>
Message-ID: <48845B55.3090306@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lennart Regebro wrote:
| On Mon, Jul 21, 2008 at 11:06, Jesus Cea <jcea at jcea.es> wrote:
|> I'm *VERY* interested in 2.6->3.0 migration guide for C module
|> extensions. 3.0 is around the corner and the API is changing almost
|> daily :-p.
|
| It would be great if python-incompatibility would have examples of the
| C-api changes as well. That would really help in migrating and writing
| a migration guide. It would be great if you could help with this!

I can comment about some issues I had this weekend.

Remember that my intention is to keep a single C codebase for 2.6 and 3.0.

* Int/Long integration. In Python 3.0 "PyInt_*" has vanished. But using
"PyLong_*" in Python 2.x surfaces so many issues that I have decided to
define "NUMBER_*" macros to be conditionally expanded to
"PyInt"/"PyLong" when compiling to 2.x/3.0. Not nice, but I can't see a
better way.

* The module initialization has changed completely. In fact I am
fighting this, still. Some help needed.

* "Py_FindMember" was eliminated ?last week?. I workaround this, but I
don't know really how to write a "smart" getattr function now. Any pointer?

At least my current code compiles under 3.0, but it still bombs when
importing it.

Life sucks.

PS: I'm learning the hard way, doing "diff" between 2.6 and 3.0 module
sourcecode. It must be a better way!.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRbTplgi5GaxT1NAQJTBAP/cHndbYfnpNF+UoJP0vswoJrMtrNRu6F4
TAAMGbv4sfE5a308sfjoXxFuvUA8kXAyqdJCMXlNWDfX6CEsozQwoS06d6HNkkZn
k9iGEbjwiWVdcw0y6gXvt9rL+klK004Rg4pUHMYtO3yUbaFSqvCI1kox6Rf1Q4dB
3CywJ2/fAu4=
=V42Q
-----END PGP SIGNATURE-----

From regebro at gmail.com  Mon Jul 21 11:59:42 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 21 Jul 2008 11:59:42 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <48845B55.3090306@jcea.es>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es>
	<319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com>
	<48845B55.3090306@jcea.es>
Message-ID: <319e029f0807210259q36250024vc1147c47ff3953ee@mail.gmail.com>

On Mon, Jul 21, 2008 at 11:48, Jesus Cea <jcea at jcea.es> wrote:
> I can comment about some issues I had this weekend.

I don't do C development myself, so comments aren't that useful for
me, but code examples are, so we can stick them into
python-incompatibility.

> Remember that my intention is to keep a single C codebase for 2.6 and 3.0.

Cool.

> * Int/Long integration. In Python 3.0 "PyInt_*" has vanished. But using
> "PyLong_*" in Python 2.x surfaces so many issues that I have decided to
> define "NUMBER_*" macros to be conditionally expanded to
> "PyInt"/"PyLong" when compiling to 2.x/3.0. Not nice, but I can't see a
> better way.

Seems resonable.

> PS: I'm learning the hard way, doing "diff" between 2.6 and 3.0 module
> sourcecode. It must be a better way!.

Yeah, these changes should be properly documented in the CHANGES.txt.
I've seen some C-API chnges mentiones at least.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From jcea at jcea.es  Mon Jul 21 12:07:04 2008
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 21 Jul 2008 12:07:04 +0200
Subject: [Python-Dev] Subversion 1.5 and better merge support
Message-ID: <48845FC8.1030506@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Subversion 1.5 solves the "repeated merge" problem. At last!!.

Somebody is considering upgrading python svn to 1.5, and allowing only
commits coming from 1.5 clients?.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRfxZlgi5GaxT1NAQLvggQAj8+RlRxL6Vtntr9GhMIcQ4WnyY7FezKn
XyoMwN7mMzOPOxcyiKhakk/bQFLwezBW2tYa5Z40tI5oD8t/xJx6+1gJ2e3aMQAm
NttgAke0ix8Q79kOI8iak4Kp60AeO/eD/80VIY6yOkaEAygOPT2WqzPofMO5bkCK
E/MwQaqwwK4=
=hyn0
-----END PGP SIGNATURE-----

From bristow.thankachan at gmail.com  Mon Jul 21 13:13:21 2008
From: bristow.thankachan at gmail.com (Bristow Thankachan)
Date: Mon, 21 Jul 2008 16:43:21 +0530
Subject: [Python-Dev] Syntax error in python2.6
Message-ID: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>

Hi everybody,

During the porting of Zope2 to Python2.6, I am stuck with a syntax error in
the module AccessControl, which is given below.

def reorder(s, with=None, without=()):
                           ^
SyntaxError: invalid syntax

in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
The same code when run in python2.4 and python2.5 didn't give any syntax
errors. Can anybody suggest  the reason for this syntax error in python2.6.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/88126a0e/attachment.htm>

From jnoller at gmail.com  Mon Jul 21 13:25:55 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 21 Jul 2008 07:25:55 -0400
Subject: [Python-Dev] Syntax error in python2.6
In-Reply-To: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
Message-ID: <5F6472C2-10A8-4227-8F2D-D78B6FABD3EE@gmail.com>

Because as of 2.6 the 'with' word is now part of the with statement,  
see pep 343.


On Jul 21, 2008, at 7:13 AM, "Bristow Thankachan" <bristow.thankachan at gmail.com 
 > wrote:

> Hi everybody,
>
> During the porting of Zope2 to Python2.6, I am stuck with a syntax  
> error in the module AccessControl, which is given below.
>
> def reorder(s, with=None, without=()):
>                            ^
> SyntaxError: invalid syntax
>
> in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/ 
> Utilities.py.
> The same code when run in python2.4 and python2.5 didn't give any  
> syntax errors. Can anybody suggest  the reason for this syntax error  
> in python2.6.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com

From bristow.thankachan at gmail.com  Mon Jul 21 13:45:09 2008
From: bristow.thankachan at gmail.com (Bristow Thankachan)
Date: Mon, 21 Jul 2008 17:15:09 +0530
Subject: [Python-Dev] [Zope-dev] Syntax error in python2.6
In-Reply-To: <488472BB.7070608@wildenhain.de>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
	<488472BB.7070608@wildenhain.de>
Message-ID: <d886b2060807210445t1e4fa661k4a49f5a2b18aa023@mail.gmail.com>

Thank you for the quick response. I made the change and got the syntax error
corrected.

with regards

Bristow.

On Mon, Jul 21, 2008 at 4:57 PM, Tino Wildenhain <tino at wildenhain.de> wrote:

> Bristow Thankachan wrote:
>
>> Hi everybody,
>>
>> During the porting of Zope2 to Python2.6, I am stuck with a syntax error
>> in the module AccessControl, which is given below.
>>
>> def reorder(s, with=None, without=()):
>>                           ^
>> SyntaxError: invalid syntax
>>
>> in line 56 of
>> /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
>> The same code when run in python2.4 and python2.5 didn't give any syntax
>> errors. Can anybody suggest  the reason for this syntax error in python2.6.
>>
>
> I'd say, "with" is now a keyword.
>
> http://docs.python.org/ref/keywords.html
>
> btw, shouldn't this already give a warning in 2.5?
>
> Cheers
> Tino
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/b98cf97c/attachment.htm>

From mal at egenix.com  Mon Jul 21 14:03:08 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 21 Jul 2008 14:03:08 +0200
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com>
References: <200807191323.13124.victor.stinner@haypocalc.com>	<20080719195209.GA18271@amk.local>
	<200807202245.39874.victor.stinner@haypocalc.com>
Message-ID: <48847AFC.8000206@egenix.com>

On 2008-07-20 22:45, Victor Stinner wrote:
> Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit :
>> Excellent work!  Another fruitful area for fuzzing might be the
>> miniature virtual machine used by the re module.  It's possible to
>> import _sre and call the compile() function directly (see the end of
>> Lib/sre_compile.py for how it's invoked); I wonder how the regex VM
>> copes with random strings of bytecode.
> 
> Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted 
> _sre.compile() in my fuzzer.
> 
> For information, it's also very easy to crash CPython with fuzzed .pyc file.
> 
> It's hard to check bytecode without execute it. It's maybe better to add 
> checks directly in the VM.

I don't see that as a big problem: if you execute untrusted byte code,
you are on your own anyway... whether that's byte code for the re
engine or ceval.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 21 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From tino at wildenhain.de  Mon Jul 21 13:27:55 2008
From: tino at wildenhain.de (Tino Wildenhain)
Date: Mon, 21 Jul 2008 13:27:55 +0200
Subject: [Python-Dev] [Zope-dev] Syntax error in python2.6
In-Reply-To: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
Message-ID: <488472BB.7070608@wildenhain.de>

Bristow Thankachan wrote:
> Hi everybody,
> 
> During the porting of Zope2 to Python2.6, I am stuck with a syntax error 
> in the module AccessControl, which is given below.
> 
> def reorder(s, with=None, without=()):
>                            ^
> SyntaxError: invalid syntax
> 
> in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
> The same code when run in python2.4 and python2.5 didn't give any syntax 
> errors. Can anybody suggest  the reason for this syntax error in python2.6.

I'd say, "with" is now a keyword.

http://docs.python.org/ref/keywords.html

btw, shouldn't this already give a warning in 2.5?

Cheers
Tino
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3241 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/bff9aa2c/attachment.bin>

From bristow.thankachan at gmail.com  Mon Jul 21 14:09:08 2008
From: bristow.thankachan at gmail.com (Bristow Thankachan)
Date: Mon, 21 Jul 2008 17:39:08 +0530
Subject: [Python-Dev] Syntax error in python2.6
In-Reply-To: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
Message-ID: <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com>

hi all,

 I changed the attribute 'with' in
/home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py
to 'wit' and I ran the tests in python2.4 and I got the following result.


Running tests at level 1
Running unit tests:
  Running:

  Ran 285 tests with 0 failures and 0 errors in 6.587 seconds.
Running Testing.ZopeTestCase.layer.ZopeLite tests:
  Set up Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
  Running:

  Ran 1 tests with 0 failures and 0 errors in 0.064 seconds.
Tearing down left over layers:
  Tear down Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
Total: 286 tests, 0 failures, 0 errors in 1.044 seconds.

Can anybody tell me whether this implies it is backward compatible in
python2.4?

with regards

Bristow

On Mon, Jul 21, 2008 at 4:43 PM, Bristow Thankachan <
bristow.thankachan at gmail.com> wrote:

> Hi everybody,
>
>
> During the porting of Zope2 to Python2.6, I am stuck with a syntax error in
> the module AccessControl, which is given below.
>
> def reorder(s, with=None, without=()):
>                            ^
> SyntaxError: invalid syntax
>
> in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
> The same code when run in python2.4 and python2.5 didn't give any syntax
> errors. Can anybody suggest  the reason for this syntax error in python2.6.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/dce77047/attachment-0001.htm>

From bristow.thankachan at gmail.com  Mon Jul 21 14:12:38 2008
From: bristow.thankachan at gmail.com (Bristow Thankachan)
Date: Mon, 21 Jul 2008 17:42:38 +0530
Subject: [Python-Dev] import error in python2.6
Message-ID: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com>

Hi all,

During the porting of Zope2, I am stuck with import errors in many modules.
The same code works well in python2.5 and 2.4. Can anybody give the details
of this import error in python2.6 and how to get the error corrected?

with regards

Bristow
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/c996c210/attachment.htm>

From qgallet at gmail.com  Mon Jul 21 14:20:36 2008
From: qgallet at gmail.com (Quentin Gallet-Gilles)
Date: Mon, 21 Jul 2008 14:20:36 +0200
Subject: [Python-Dev] import error in python2.6
In-Reply-To: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com>
References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com>
Message-ID: <8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com>

Hi Bristow,

You didn't provide any broken code that could help us give an explanation.
Also this kind of question is best answered on the python-users mailing
list. Python-dev is reserved for discussion about the evolution of Python,
not its use.

Cheers,
Quentin

On Mon, Jul 21, 2008 at 2:12 PM, Bristow Thankachan <
bristow.thankachan at gmail.com> wrote:

> Hi all,
>
> During the porting of Zope2, I am stuck with import errors in many modules.
> The same code works well in python2.5 and 2.4. Can anybody give the details
> of this import error in python2.6 and how to get the error corrected?
>
> with regards
>
> Bristow
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/2f47ed17/attachment.htm>

From regebro at gmail.com  Mon Jul 21 14:25:57 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 21 Jul 2008 14:25:57 +0200
Subject: [Python-Dev] Syntax error in python2.6
In-Reply-To: <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
	<d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com>
Message-ID: <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com>

On Mon, Jul 21, 2008 at 14:09, Bristow Thankachan
<bristow.thankachan at gmail.com> wrote:
> Can anybody tell me whether this implies it is backward compatible in
> python2.4?

If you change the API it isn't backwards compatible. The question is
if this is a problem or not, if anything outside Zope itself is using
this call, which is hard to say.

Options here are
1. Deprecating the "with" parameter for "with_" and supporting both in
2.12 but not supporting Python2.6.
2. Using **kw in the argument and looking for noth "with" and "with_",
that way, which will be backwards compatible.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From mborch at gmail.com  Mon Jul 21 14:30:23 2008
From: mborch at gmail.com (Malthe Borch)
Date: Mon, 21 Jul 2008 14:30:23 +0200
Subject: [Python-Dev] Syntax error in python2.6
In-Reply-To: <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>	<d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com>
	<319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com>
Message-ID: <4884815F.5080201@gmail.com>

Lennart Regebro wrote:
> 2. Using **kw in the argument and looking for noth "with" and "with_",
> that way, which will be backwards compatible.

+1

\malthe


From mborch at gmail.com  Mon Jul 21 14:30:23 2008
From: mborch at gmail.com (Malthe Borch)
Date: Mon, 21 Jul 2008 14:30:23 +0200
Subject: [Python-Dev] Syntax error in python2.6
In-Reply-To: <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>	<d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com>
	<319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com>
Message-ID: <4884815F.5080201@gmail.com>

Lennart Regebro wrote:
> 2. Using **kw in the argument and looking for noth "with" and "with_",
> that way, which will be backwards compatible.

+1

\malthe


From musiccomposition at gmail.com  Mon Jul 21 15:16:31 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 21 Jul 2008 08:16:31 -0500
Subject: [Python-Dev] Subversion 1.5 and better merge support
In-Reply-To: <48845FC8.1030506@jcea.es>
References: <48845FC8.1030506@jcea.es>
Message-ID: <1afaf6160807210616j57f637a2s4fedb1a9396f1e02@mail.gmail.com>

On Mon, Jul 21, 2008 at 5:07 AM, Jesus Cea <jcea at jcea.es> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Subversion 1.5 solves the "repeated merge" problem. At last!!.
>
> Somebody is considering upgrading python svn to 1.5, and allowing only
> commits coming from 1.5 clients?.

While Subversion 1.5 has merge tracking, it doesn't support blocking
revisions to be merged as well as svnmerge.py. (This is important for
merging trunk -> py3k). Basically, I think we can get more from
svnmerge.py than builtin merge tracking.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From musiccomposition at gmail.com  Mon Jul 21 15:20:20 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 21 Jul 2008 08:20:20 -0500
Subject: [Python-Dev] import error in python2.6
In-Reply-To: <8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com>
References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com>
	<8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com>
Message-ID: <1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com>

On Mon, Jul 21, 2008 at 7:20 AM, Quentin Gallet-Gilles
<qgallet at gmail.com> wrote:
> Hi Bristow,
>
> You didn't provide any broken code that could help us give an explanation.
> Also this kind of question is best answered on the python-users mailing
> list. Python-dev is reserved for discussion about the evolution of Python,
> not its use.

Since this is prelease software, it's probably ok to talk about issues
with it. You could file a bug next time. However, AFAIK, Zope hasn't
even been ported to 2.5.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From amk at amk.ca  Mon Jul 21 15:33:19 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 21 Jul 2008 09:33:19 -0400
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com>
References: <200807191323.13124.victor.stinner@haypocalc.com>
	<20080719195209.GA18271@amk.local>
	<200807202245.39874.victor.stinner@haypocalc.com>
Message-ID: <20080721133319.GA15912@amk-desktop.matrixgroup.net>

On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote:
> Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted 
> _sre.compile() in my fuzzer.

We should certainly try to fix those issues, then; people usually
assume the re module is safe for use inside a sandbox and probably
aren't careful enough to block importing of the _sre module.

--amk

From fdrake at acm.org  Mon Jul 21 15:33:36 2008
From: fdrake at acm.org (Fred Drake)
Date: Mon, 21 Jul 2008 09:33:36 -0400
Subject: [Python-Dev] import error in python2.6
In-Reply-To: <1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com>
References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com>
	<8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com>
	<1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com>
Message-ID: <A31A031A-B209-45A6-9AC9-6A6986AC173E@acm.org>

On Jul 21, 2008, at 9:20 AM, Benjamin Peterson wrote:
> Since this is prelease software, it's probably ok to talk about issues
> with it. You could file a bug next time. However, AFAIK, Zope hasn't
> even been ported to 2.5.


Many people are using Zope 3 with Python 2.5 without problems, though  
Python 2.5 isn't "officially" supported (whatever that means).

I think current versions of Zope 2 work with Python 2.5 as well.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>


From victor.stinner at haypocalc.com  Mon Jul 21 15:54:11 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 21 Jul 2008 15:54:11 +0200
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <20080721133319.GA15912@amk-desktop.matrixgroup.net>
References: <200807191323.13124.victor.stinner@haypocalc.com>
	<200807202245.39874.victor.stinner@haypocalc.com>
	<20080721133319.GA15912@amk-desktop.matrixgroup.net>
Message-ID: <200807211554.11468.victor.stinner@haypocalc.com>

Le Monday 21 July 2008 15:33:19 A.M. Kuchling, vous avez ?crit?:
> On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote:
> > Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted
> > _sre.compile() in my fuzzer.
>
> We should certainly try to fix those issues, then; people usually
> assume the re module is safe for use inside a sandbox and probably
> aren't careful enough to block importing of the _sre module.

Why is this function public? Is it used by re module? Only _sre module should 
be allowed to generated "regex bytecode".

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From bristow.thankachan at gmail.com  Mon Jul 21 15:56:05 2008
From: bristow.thankachan at gmail.com (Bristow Thankachan)
Date: Mon, 21 Jul 2008 19:26:05 +0530
Subject: [Python-Dev] import error in python2.6
Message-ID: <d886b2060807210656i57c99a3fgc844c1de50b06deb@mail.gmail.com>

Hi all,

I asked about the import error in python2.6 while trying to port zope2. The
error message is given below.

 File "/home/zope/ztrunk26/lib/python/AccessControl/ImplC.py", line 30, in
<module>
from ImplPython import RestrictedDTML, SecurityManager, ZopeSecurityPolicy
ImportError: No module named ImplPython

It works well in python2.4 and 2.5. Please give your suggestions.

with regards

Bristow
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/a97dd32d/attachment.htm>

From regebro at gmail.com  Mon Jul 21 16:15:57 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 21 Jul 2008 16:15:57 +0200
Subject: [Python-Dev] [Zope-dev] import error in python2.6
In-Reply-To: <d886b2060807210656i57c99a3fgc844c1de50b06deb@mail.gmail.com>
References: <d886b2060807210656i57c99a3fgc844c1de50b06deb@mail.gmail.com>
Message-ID: <319e029f0807210715i4776bd67p57dcdcde2f4bfce0@mail.gmail.com>

On Mon, Jul 21, 2008 at 15:56, Bristow Thankachan
<bristow.thankachan at gmail.com> wrote:
> Hi all,
>
> I asked about the import error in python2.6 while trying to port zope2. The
> error message is given below.
>
>  File "/home/zope/ztrunk26/lib/python/AccessControl/ImplC.py", line 30, in
> <module>
> from ImplPython import RestrictedDTML, SecurityManager, ZopeSecurityPolicy
> ImportError: No module named ImplPython
>
> It works well in python2.4 and 2.5. Please give your suggestions.

ImplPython is a python module in Zope. It likely has some sort of
syntax error under 2.6 that is not a syntaxerror under 2.5, so the
import fails, or the C-modules that it uses fail it's compile during
setup, so import of them failed.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From tseaver at palladion.com  Mon Jul 21 16:35:44 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 21 Jul 2008 10:35:44 -0400
Subject: [Python-Dev] Syntax error in python2.6
In-Reply-To: <4884815F.5080201@gmail.com>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>	<d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com>	<319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com>
	<4884815F.5080201@gmail.com>
Message-ID: <48849EC0.5020502@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Malthe Borch wrote:
> Lennart Regebro wrote:
>> 2. Using **kw in the argument and looking for noth "with" and "with_",
>> that way, which will be backwards compatible.
> 
> +1

The implementation of this function is already so obscure that using
keywords should not be a problem.

OTOH, the reorder function in RestrictedPython was ported from a version
originally designed for doing ordered select lists in DTML:

 $ cvs log lib/python/DocumentTemplate/DT_Util.py
 ...
 revision 1.52
 date: 1999/03/22 17:28:37;  author: jim;  state: Exp;  lines: +31 -2
 Added a new "builtin" function reorder:

   reorder(s, [with, without]]) --

      Reorder the items in s according to the order given in with
      and with items mentioned in without removed.  Items from s
      not mentioned in with are removed.

      s, with, and without are all either sequences if strings
      or sequences of key-value tuples, with ordering done on the
      keys.

 This function is useful for constructing ordered select lists.
 ...

The only use of the method in Zope's own DTML is in the addZClass.dtml
template.  I'm willing to bet that *nobody* who is still using it would
be surprised if they had to recode it when upgrading to a recent Zope,
so maybe we can just change the trunk without worrying about it.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIhJ7A+gerLs4ltQ4RAk6lAJ0T6JJC/4nr6kA7jzSQqKSNZUGP7ACeIO2R
H7k2D46BKQ+DL0girE8EGOk=
=HLxo
-----END PGP SIGNATURE-----


From musiccomposition at gmail.com  Mon Jul 21 16:47:08 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 21 Jul 2008 09:47:08 -0500
Subject: [Python-Dev] Deprecate parser's "ast" functions?
In-Reply-To: <g2ej5h$tnl$1@ger.gmane.org>
References: <g2ej5h$tnl$1@ger.gmane.org>
Message-ID: <1afaf6160807210747v7756589bnac5d07c6d3bbb6f8@mail.gmail.com>

On Sat, Jun 7, 2008 at 3:13 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> The parser module exports each function and type twice, once with "AST" in
> the name, once with "ST".  Since "AST" now has a different meaning for
> Python code compilation, I propose to deprecate the "ast" variants in 2.6
> and remove them in Python 3000.

+1 This personally has confused me!

>
> (Also, all keyword arguments are called "ast", that cannot change in 2.6
> but should in 3.0.)
>
> I'm at the moment changing the documentation of parser to only refer to
> the "st" variants anymore.
>
> Georg

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From solipsis at pitrou.net  Mon Jul 21 17:53:18 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 21 Jul 2008 15:53:18 +0000 (UTC)
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
References: <200807191323.13124.victor.stinner@haypocalc.com>
	<200807202245.39874.victor.stinner@haypocalc.com>
	<20080721133319.GA15912@amk-desktop.matrixgroup.net>
	<200807211554.11468.victor.stinner@haypocalc.com>
Message-ID: <loom.20080721T154744-225@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> Le Monday 21 July 2008 15:33:19 A.M. Kuchling, vous avez ?crit?:
> > On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote:
> > > Hum... how can I say it? It's trivial to crash _sre  So I blacklisted
> > > _sre.compile() in my fuzzer.
> >
> > We should certainly try to fix those issues, then; people usually
> > assume the re module is safe for use inside a sandbox and probably
> > aren't careful enough to block importing of the _sre module.
> 
> Why is this function public? Is it used by re module? Only _sre module should 
> be allowed to generated "regex bytecode".

The underscore at the beginning of _sre clearly indicates that the module is 
not recommended for direct consumption, IMO. Even the functions that don't 
themselves start with an underscore...



From cadr4u at gmail.com  Mon Jul 21 17:58:20 2008
From: cadr4u at gmail.com (Jakob Sievers)
Date: Mon, 21 Jul 2008 17:58:20 +0200
Subject: [Python-Dev] Python VM
Message-ID: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com>

Hi,
I've been reading the Python VM sources over the last few afternoons and
I took some notes, which I thought I'd share (and if anyone familiar
with the VM internals could have a quick look at them, I'd really
appreciate it).

Cheers,
-jakob


Unless otherwise noted, the source file in question is Python/ceval.c.

Control Flow
============
The calling sequence is:
main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) ->
run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx()
(ceval.c) -> PyEval_EvalFrameEx() (ceval.c).

EvalCodeEx() does some initialization (creating a new execution frame,
argument processing, and some generator-specific stuff I haven't looked at yet)
before calling EvalFrameEx() which contains the main interpreter loop.


Threads
=======
PyEval_InitThreads() initializes the GIL (interpreter_lock) and sets
main_thread to the (threading package dependent) ID of the current thread.
Thread switching is done using PyThreadState_Swap(), which sets
_PyThreadState_Current (both defined in pystate.c) and PyThreadState_GET()
(an alias for _PyThreadState_Current) (pystate.h).


Async Callbacks
===============
Asynchronous callbacks can be registered by adding the function to be called
to pendingcalls[] (see Py_AddPendingCall()). The state of this queue is
communicated to the main loop via things_to_do.


State
=====
The global state is recorded in a (per-process?) PyInterpreterState struct and
a per-thread PyThreadState struct.
Each execution frame's state is contained in that frame's PyFrameObject
(which includes the instruction stream, the environment (globals, locals,
builtins, etc.), the value stack and so forth).
EvalFrameEx()'s local variables are initialized from this frame object.


Instruction Stream
==================
The instruction stream looks as follows (c.f. assemble_emit() in compile.c):
A byte stream where each instruction consists of either
1) a single byte opcode: OP
2) a single byte opcode plus a two-byte immediate argument: OP LO HI
3) a special opcode followed by the real opcode followed by a four byte
   argument: EXTENDED_ARG OP BYTE2 BYTE1 BYTE4 BYTE3


Opcode Prediction
=================
One nice trick used to speed up opcode dispatch is the following:
Using the macros PREDICT() and PREDICTED() it is sometimes possible
to jump directly to the code implementing the next instruction
rather than having to go through the whole loop preamble, e.g.

case FOO:
      // ...
      PREDICT(BAR);
      continue;

PREDICTED(BAR);
case BAR:
     // ...

expands to

case FOO:
      // ...
      if (*next_instr == BAR) goto PRED_BAR;
      continue;

PRED_BAR: next_instr++;
case BAR:
      // ...


Main Loop
=========

Variables and macros used in EvalFrameEx()
------------------------------------------
The value stack:
  PyObject **stack_pointer;
The instruction stream:
  unsigned char *next_instr;
NEXTOP(), NEXTARG(), PEEKARG(), JUMPTO(), and JUMPBY() simply fiddle
with next_instr. Likewise for TOP(), SET_SECOND(), PUSH(), POP(),
etc. and stack_pointer.

Current opcode plus argument:
  int opcode;
  int oparg;

Error status:
  enum why_code why; // no, exn, exn re-raised, return, break, continue, yield
  int err;           // non-zero is error

The environment:
  PyObject *names;
  PyObject *consts;
and
  PyObject **fastlocals;
which is accessed via GETLOCAL() and SETLOCAL().

Finally, there are some more PyObject *'s (v, w, u, and so forth, used
as temporary variables) as well as
  PyObject *retval;


Basic structure
---------------
EvalFrameEx() {
    why = WHY_NOT;
    err = 0;

    for (;;) {    <------------------+---+
        // do periodic tasks         |   |
                                     |   |
    fast_next_opcode:                |   |
        opcode = NEXTOP();           |   |
        if (HAS_ARG(opcode))         |   |
            oparg = NEXTARG();       |   |
                                     |   |
    dispatch_opcode:                 |   |
        switch(opcode) {             |   |
                                     |   |
        continue; -------------------+   |
                                         |
        break; ----------------------+   |
                                     |   |
        // Also, opcode prediction   |   |
        // jumps around inside the   |   |
        // switch statement          |   |
                                     |   |
        }    <-----------------------+   |
                                         |
    on_error:                            |
        // no error: continue -----------+
        // otherwise why == WHY_EXCEPTION after this

    fast_block_end:
        // unwind stacks if there was an error
    }

    // more unwinding

fast_yield:
    // reset current thread's exception info
exit_eval_frame:
    // set thread's execution frame to previous execution frame
    return retval;
}

Periodic Tasks
--------------
By checking and decrementing _Py_Ticker, the main loop executes certain tasks
once every _Py_CheckInterval iterations (in fact Py_AddPendingCall() sets
_Py_Ticker to zero, ensuring that pending calls are executed right after the
next instruction which doesn't jump to fast_next_opcode):
  - If there are things_to_do, Py_MakePendingCalls() is called.
  - The GIL is releases and re-acquired, giving other threads a chance to run.

Instruction implementation
--------------------------
Some general notes:
  - Successful instructions either goto fast_next_opcode or continue.
  - Unsuccessful instructions break out of the switch.
  - The value stack holds only PyObject *'s.
  - Instructions must take care to Py_INCREF() and Py_DECREF() the reference
    counts of PyObject's whose addresses have been pushed onto/popped off the
    value stack.
  - Objects are transferred onto the value stack by GETITEM()'ing them from
    consts or names, or by GETLOCAL()'ing them using oparg as an offset into
    fastlocals (c.f. LOAD_* instructions).
  - err is used as a general `error occurred' flag, both inside the code
    implementing an opcode and `globally' for the entire loop.

Nested blocks
-------------
Nested loop and try blocks are handled as follows:
Each frame maintains a block stack; when entering a nested block, a SETUP_*
instruction adds a PyTryBlock to the PyFrameObject's f_blockstack and
registers that block's type (instruction which created the block), handler
(offset into the instruction stream) and level (value stack level before the
nested block was entered).
When such blocks are exited normally (e.g. last iteration of a loop), the final
POP_BLOCK instruction restores the value stack to the state it was in before
the block.
If a block is exited abnormally (e.g. a break instruction), the code following
fast_block_end unwinds the value stack and jumps to the block's handler.
Certain instructions, e.g. RETURN_VALUE, cause the entire block stack to be
unwound (leading to multiple unwinds of the value stack).
See also the comments in compile.c (compiler_try_finally()) which include nice
ASCII art diagrams (incidentally, there's a typo on line 2382: s/en/an/).

Error handling
--------------
Internal errors (bad oparg, say) generally result in
why being set to WHY_EXCEPTION and breaking out of the switch
(if the code implementing the instruction doesn't set why, the code
after on_error will).
The code following fast_block_end will jump to the correct exception
handler and set the global exception related variables
(exception information is stored in the current execution frame and
the thread state. See set_exc_info() and reset_exc_info()).

// eof

From amk at amk.ca  Mon Jul 21 19:41:41 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 21 Jul 2008 13:41:41 -0400
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <loom.20080721T154744-225@post.gmane.org>
References: <200807191323.13124.victor.stinner@haypocalc.com>
	<200807202245.39874.victor.stinner@haypocalc.com>
	<20080721133319.GA15912@amk-desktop.matrixgroup.net>
	<200807211554.11468.victor.stinner@haypocalc.com>
	<loom.20080721T154744-225@post.gmane.org>
Message-ID: <20080721174141.GA16598@amk-desktop.matrixgroup.net>

yOn Mon, Jul 21, 2008 at 03:53:18PM +0000, Antoine Pitrou wrote:
> The underscore at the beginning of _sre clearly indicates that the module is 
> not recommended for direct consumption, IMO. Even the functions that don't 
> themselves start with an underscore...

Sure, but if someone is trying to break in or DoS your application
server, they don't care if the module starts with an underscore or
not.

To answer Victor's original question: the parser & compiler that turn
a regex into bytecode is written in Python.  I can't think of a way to
prevent other Python modules from importing _sre or accessing the
compile() function; if nothing else, code could always do 'import re ;
re.sre_compile._sre.compile(...)'.

--amk

From brett at python.org  Mon Jul 21 20:16:22 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 21 Jul 2008 11:16:22 -0700
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <4884581F.8010201@jcea.es>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org>
	<4884581F.8010201@jcea.es>
Message-ID: <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>

On Mon, Jul 21, 2008 at 2:34 AM, Jesus Cea <jcea at jcea.es> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Georg Brandl wrote:
> | So it's good that nobody has written a migration guide yet; he'd have
> | to rewrite it daily.
>
> Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid
> redoing the same work 15 times XDDDDDDDDDD
>

But waiting until all the betas have gone out totally defeats the
purpose of the betas! It has already been stated that new code changes
that are even remotely shaky or anything not small needs a code review
at this point since we only have one beta left. Barry can correct me
if I am wrong, but dropping a major rewrite of bsddb into 3.0 between
b3 and rc1 is not acceptable. I wouldn't be surprised if all code
after b3 requires a code review since at that point we should just be
squashing bugs, not rewriting code. And if you are migrating all of
bsddb3 at once then that is going to be a massive code review that
should be happening in prep for rc1.

I really hate feeling like I am coming off as a jerk harping on this,
Jesus, as I don't want to discourage your contributions to pybsddb,
but it just doesn't feel reasonable to me to be doing this so late in
the release cycle.

-Brett

From brett at python.org  Mon Jul 21 20:18:38 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 21 Jul 2008 11:18:38 -0700
Subject: [Python-Dev] Subversion 1.5 and better merge support
In-Reply-To: <1afaf6160807210616j57f637a2s4fedb1a9396f1e02@mail.gmail.com>
References: <48845FC8.1030506@jcea.es>
	<1afaf6160807210616j57f637a2s4fedb1a9396f1e02@mail.gmail.com>
Message-ID: <bbaeab100807211118i6d6d1c35if2d6699b061da423@mail.gmail.com>

On Mon, Jul 21, 2008 at 6:16 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Mon, Jul 21, 2008 at 5:07 AM, Jesus Cea <jcea at jcea.es> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Subversion 1.5 solves the "repeated merge" problem. At last!!.
>>
>> Somebody is considering upgrading python svn to 1.5, and allowing only
>> commits coming from 1.5 clients?.
>
> While Subversion 1.5 has merge tracking, it doesn't support blocking
> revisions to be merged as well as svnmerge.py. (This is important for
> merging trunk -> py3k). Basically, I think we can get more from
> svnmerge.py than builtin merge tracking.
>

This has also been discussed by the infrastructure committee and the
person in charge of maintaining the svn install does not want to
upgrade until 1.5 lands in debian-stable. So as of right now there are
no immediate plans to upgrade.

-Brett

From musiccomposition at gmail.com  Mon Jul 21 20:50:44 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 21 Jul 2008 13:50:44 -0500
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org>
	<4884581F.8010201@jcea.es>
	<bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>
Message-ID: <1afaf6160807211150m64ec7b46ha5d71c5492520f03@mail.gmail.com>

On Mon, Jul 21, 2008 at 1:16 PM, Brett Cannon <brett at python.org> wrote:

> On Mon, Jul 21, 2008 at 2:34 AM, Jesus Cea <jcea at jcea.es> wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Georg Brandl wrote:
> > | So it's good that nobody has written a migration guide yet; he'd have
> > | to rewrite it daily.
> >
> > Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid
> > redoing the same work 15 times XDDDDDDDDDD
> >
>
> But waiting until all the betas have gone out totally defeats the
> purpose of the betas! It has already been stated that new code changes
> that are even remotely shaky or anything not small needs a code review
> at this point since we only have one beta left. Barry can correct me
> if I am wrong, but dropping a major rewrite of bsddb into 3.0 between
> b3 and rc1 is not acceptable. I wouldn't be surprised if all code
> after b3 requires a code review since at that point we should just be
> squashing bugs, not rewriting code. And if you are migrating all of
> bsddb3 at once then that is going to be a massive code review that
> should be happening in prep for rc1.


I have a sickening feeling that this isn't the only major change that may
hit the betas. PEP 3118 is only about 2/3's implemented (with hardly any
tests, I might add), and Travis has said the earliest he can get to it is
August...



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/a4326ae9/attachment.htm>

From victor.stinner at haypocalc.com  Mon Jul 21 21:17:14 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 21 Jul 2008 21:17:14 +0200
Subject: [Python-Dev] fileobj.read(float): warning or error?
Message-ID: <200807212117.14485.victor.stinner@haypocalc.com>

Hi,

Since Python 2.4 (maybe 2.2 or older), fileobj.read(4.2) displays an error and 
works as fileobj.read(4).

>>> i=open('/etc/issue')
>>> i.read(4.2)
__main__:1: DeprecationWarning: integer argument expected, got float

It should raises an error instead of a warning, it has no sense to read a 
partial byte :-) But that should breaks some applications?

Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:

  while len(bytes) < n:
      bytes += read(_urandomfd, n - len(bytes))

because read(0.2) works as read(0) :-/

Victor

From musiccomposition at gmail.com  Mon Jul 21 21:23:21 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 21 Jul 2008 14:23:21 -0500
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <200807212117.14485.victor.stinner@haypocalc.com>
References: <200807212117.14485.victor.stinner@haypocalc.com>
Message-ID: <1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com>

On Mon, Jul 21, 2008 at 2:17 PM, Victor Stinner <
victor.stinner at haypocalc.com> wrote:

> Hi,
>
> Since Python 2.4 (maybe 2.2 or older), fileobj.read(4.2) displays an error
> and
> works as fileobj.read(4).
>
> >>> i=open('/etc/issue')
> >>> i.read(4.2)
> __main__:1: DeprecationWarning: integer argument expected, got float


This warning is actually given by the argument parser when "i" gets a Python
non-integer.

>
>
> It should raises an error instead of a warning, it has no sense to read a
> partial byte :-) But that should breaks some applications?


This doesn't come into effect until 3.0.

>
>
> Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
>
>  while len(bytes) < n:
>      bytes += read(_urandomfd, n - len(bytes))
>
> because read(0.2) works as read(0) :-/
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/de60ff54/attachment.htm>

From srichter at cosmos.phy.tufts.edu  Mon Jul 21 16:42:16 2008
From: srichter at cosmos.phy.tufts.edu (Stephan Richter)
Date: Mon, 21 Jul 2008 07:42:16 -0700
Subject: [Python-Dev] [Zope-dev] Syntax error in python2.6
In-Reply-To: <488472BB.7070608@wildenhain.de>
References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com>
	<488472BB.7070608@wildenhain.de>
Message-ID: <200807210742.16411.srichter@cosmos.phy.tufts.edu>

On Monday 21 July 2008, Tino Wildenhain wrote:
> btw, shouldn't this already give a warning in 2.5?

It does. I see it.

Regards,
Stephan
-- 
Stephan Richter
Web Software Design, Development and Training
Google me. "Zope Stephan Richter"

From optilude at gmx.net  Mon Jul 21 22:59:28 2008
From: optilude at gmx.net (Martin Aspeli)
Date: Mon, 21 Jul 2008 21:59:28 +0100
Subject: [Python-Dev] import error in python2.6
In-Reply-To: <A31A031A-B209-45A6-9AC9-6A6986AC173E@acm.org>
References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com>	<8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com>	<1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com>
	<A31A031A-B209-45A6-9AC9-6A6986AC173E@acm.org>
Message-ID: <g62tbi$3p9$2@ger.gmane.org>

Fred Drake wrote:
> On Jul 21, 2008, at 9:20 AM, Benjamin Peterson wrote:
>> Since this is prelease software, it's probably ok to talk about issues
>> with it. You could file a bug next time. However, AFAIK, Zope hasn't
>> even been ported to 2.5.
> 
> 
> Many people are using Zope 3 with Python 2.5 without problems, though  
> Python 2.5 isn't "officially" supported (whatever that means).
> 
> I think current versions of Zope 2 work with Python 2.5 as well.

No, they don't. Work is ongoing to port it, though.

Martin

-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


From tjreedy at udel.edu  Mon Jul 21 23:15:40 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 21 Jul 2008 17:15:40 -0400
Subject: [Python-Dev] Python VM
In-Reply-To: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com>
References: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com>
Message-ID: <g62u9p$7r4$1@ger.gmane.org>



Jakob Sievers wrote:
> Hi,
> I've been reading the Python VM sources over the last few afternoons and
> I took some notes, which I thought I'd share (and if anyone familiar
> with the VM internals could have a quick look at them, I'd really
> appreciate it).

This is the CPython VM.  Other implementations do what they do.  Perhaps 
similar, but certainly different in details.

There have been various requests over the years on 
Python-list/comp.lang.python for documents describing the C-VM.  The 
usual answer has been 'read the source' as you did.  So I encourage you 
to announce it there and perhaps consider adding a PyWiki page (I assume 
there is not one now).

tjr


From regebro at gmail.com  Mon Jul 21 23:37:04 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 21 Jul 2008 23:37:04 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org>
	<4884581F.8010201@jcea.es>
	<bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>
Message-ID: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com>

On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote:
> But waiting until all the betas have gone out totally defeats the
> purpose of the betas!

I agree. Writing an actual *guide* can wait, but documenting the
differences with code examples is a work that can start now, and I
agree that it would be great if this would start now.

But writing a guide might not be a good idea until we know what the
changes are, and if the API is changing quickly now we don't. :-)

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From martin at v.loewis.de  Tue Jul 22 00:17:56 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 22 Jul 2008 00:17:56 +0200
Subject: [Python-Dev] Python VM
In-Reply-To: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com>
References: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com>
Message-ID: <48850B14.5020806@v.loewis.de>

Jakob,

This looks fairly correct. A few comments below.

> Control Flow
> ============
> The calling sequence is:
> main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) ->
> run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx()
> (ceval.c) -> PyEval_EvalFrameEx() (ceval.c).

What this misses is the compiler stuff, i.e. PyParser_ASTFromFoo and
PyAST_Compile, which precedes the call to PyEval_ (atleast, no byte code
file is available).

> Threads
> =======
> PyEval_InitThreads() initializes the GIL (interpreter_lock) and sets
> main_thread to the (threading package dependent) ID of the current thread.
> Thread switching is done using PyThreadState_Swap(), which sets
> _PyThreadState_Current (both defined in pystate.c) and PyThreadState_GET()
> (an alias for _PyThreadState_Current) (pystate.h).

True, however, in most cases, this is triggered through
Py_BEGIN_ALLOW_THREADS, which passes NULL for the new thread. The actual
*switching* occurs by releasing the GIL, not by ThreadState_Swap.

Actually, Python doesn't dispatch threads at all. It just releases the
GIL, giving the operating system permission to wake up a different
thread - which the operating system may or may not chose to do. After
some time, the original thread will try to reacquire the GIL. Assuming
the OS applies fairness, it will not get it back if a different thread
was also waiting for it, so our thread will block - and *then* the OS
will dispatch (at latest).

> State
> =====
> The global state is recorded in a (per-process?) PyInterpreterState struct and
> a per-thread PyThreadState struct.

Yes and no. In principle, multiple interpreter states are supported per
process (and the current interpreter is identified by thread). However,
there are many limitations and quirks in the multiple-interpreter code.

> Each execution frame's state is contained in that frame's PyFrameObject
> (which includes the instruction stream, the environment (globals, locals,
> builtins, etc.), the value stack and so forth).
> EvalFrameEx()'s local variables are initialized from this frame object.

Not only. A lot of stuff also lives on the regular C stack, which exists
in parallel to the frame object stack (the latter being a spaghetti
stack).

> The instruction stream looks as follows (c.f. assemble_emit() in compile.c):

See also dis.py for the inverse operation.

> Basic structure
> ---------------
> EvalFrameEx() {

Somewhere you need to merge the thread-switching for threads that
have been executing a lot of instructions.

>   - Objects are transferred onto the value stack by GETITEM()'ing them from
>     consts or names, or by GETLOCAL()'ing them using oparg as an offset into
>     fastlocals (c.f. LOAD_* instructions).

Or, of course, as the result from some operation or function call, or
load from a global variable, or import, or ...

Regards,
Martin

From tom at vector-seven.com  Tue Jul 22 02:28:08 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Tue, 22 Jul 2008 10:28:08 +1000
Subject: [Python-Dev] Python VM
In-Reply-To: <48850B14.5020806@v.loewis.de>
References: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com>
	<48850B14.5020806@v.loewis.de>
Message-ID: <48852998.6050405@vector-seven.com>

Martin v. L?wis wrote:
> Jakob,
>
> This looks fairly correct. A few comments below.
>
>   
>> Control Flow
>> ============
>> The calling sequence is:
>> main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) ->
>> run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx()
>> (ceval.c) -> PyEval_EvalFrameEx() (ceval.c).
>>     
>
> What this misses is the compiler stuff, i.e. PyParser_ASTFromFoo and
> PyAST_Compile, which precedes the call to PyEval_ (atleast, no byte code
> file is available).
>
>   
Further, if I have my way with the AST optimization code, the symtable 
construction will be an explicit step in between these >:)

In any case, this is awesome work Jakob. It'd be great for this stuff to 
be documented in such detail -- I sure wish I had something like this to 
go by when I first started hacking on the source -- but the details seem 
to change quite often. Still, seeing the detail distilled once in a 
while is sort of nice, and great for anybody looking to get their teeth 
into the code.

Thanks for doing the hard yards. :)

Cheers,
T


From martin at v.loewis.de  Tue Jul 22 07:01:42 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 22 Jul 2008 07:01:42 +0200
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <200807212117.14485.victor.stinner@haypocalc.com>
References: <200807212117.14485.victor.stinner@haypocalc.com>
Message-ID: <488569B6.8050108@v.loewis.de>

> Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
> 
>   while len(bytes) < n:
>       bytes += read(_urandomfd, n - len(bytes))
> 
> because read(0.2) works as read(0) :-/

I can't quite accept that as a bug in the library. If you give invalid
parameters, Python should not crash, but it may start to behave in a
nonsensical way.

Of course, it would be possible to move the conversion warning one layer
up, into os.urandom; if the argument is float, raise a warning, and then
truncate.

Regards,
Martin

From aleaxit at gmail.com  Tue Jul 22 07:22:38 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 21 Jul 2008 22:22:38 -0700
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <488569B6.8050108@v.loewis.de>
References: <200807212117.14485.victor.stinner@haypocalc.com>
	<488569B6.8050108@v.loewis.de>
Message-ID: <e8a0972d0807212222v9e0b289h312bf33bee344170@mail.gmail.com>

I thought that's what we had __index__ for -- reject arguments that
don't SMOOTHLY turn into integers when an integer is actually
required!

Alex


On Mon, Jul 21, 2008 at 10:01 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
>>
>>   while len(bytes) < n:
>>       bytes += read(_urandomfd, n - len(bytes))
>>
>> because read(0.2) works as read(0) :-/
>
> I can't quite accept that as a bug in the library. If you give invalid
> parameters, Python should not crash, but it may start to behave in a
> nonsensical way.
>
> Of course, it would be possible to move the conversion warning one layer
> up, into os.urandom; if the argument is float, raise a warning, and then
> truncate.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>

From martin at v.loewis.de  Tue Jul 22 07:44:00 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 22 Jul 2008 07:44:00 +0200
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <e8a0972d0807212222v9e0b289h312bf33bee344170@mail.gmail.com>
References: <200807212117.14485.victor.stinner@haypocalc.com>	
	<488569B6.8050108@v.loewis.de>
	<e8a0972d0807212222v9e0b289h312bf33bee344170@mail.gmail.com>
Message-ID: <488573A0.5020408@v.loewis.de>

Alex Martelli wrote:
> I thought that's what we had __index__ for -- reject arguments that
> don't SMOOTHLY turn into integers when an integer is actually
> required!

Sure. However, using that would create an incompatibility, that's why
you only get a warning when it falls back to not using __index__.

Regards,
Martin

From cs at zip.com.au  Tue Jul 22 07:37:44 2008
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 22 Jul 2008 15:37:44 +1000
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <200807212117.14485.victor.stinner@haypocalc.com>
Message-ID: <20080722053744.GA11290@cskk.homeip.net>

On 21Jul2008 21:17, Victor Stinner <victor.stinner at haypocalc.com> wrote:
| Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
|   while len(bytes) < n:
|       bytes += read(_urandomfd, n - len(bytes))
| because read(0.2) works as read(0) :-/

Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
exception if asked for < 1 bytes? Or is there a legitimate use for read(0)
with which I was not previously aware?
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

From leif.walsh at gmail.com  Tue Jul 22 08:35:54 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Mon, 21 Jul 2008 23:35:54 -0700 (PDT)
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <20080722053744.GA11290@cskk.homeip.net>
References: <20080722053744.GA11290@cskk.homeip.net>
Message-ID: <Pine.LNX.4.64.0807212328340.23485@lappy>

On Tue, 22 Jul 2008, Cameron Simpson wrote:
> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
> exception if asked for < 1 bytes? Or is there a legitimate use for
> read(0) with which I was not previously aware?

I think read(0) should be a no-op, just like it is in libc.  This lets
you write 'read(bytes)' without worrying about checking bytes, and
also lets you silently stop reading when you have no more space, like
in the following:

buf = f.read(max(bytes_left, page_size))
while buf:
  process(buf)  # updates bytes_left
  buf = f.read(max(bytes_left, page_size))

-- 
Cheers,
Leif

From cadr4u at gmail.com  Tue Jul 22 15:54:27 2008
From: cadr4u at gmail.com (Jakob Sievers)
Date: Tue, 22 Jul 2008 15:54:27 +0200
Subject: [Python-Dev] Python VM
Message-ID: <e59aede0807220654x6ef57210g7f2d043099921ee6@mail.gmail.com>

I added a page to wiki.python.org:
  http://wiki.python.org/moin/CPythonVmInternals
This incorporates most of  Martin v. L?wis's additions (except for a
few bits which
I need to look into more).

In any case, thanks for the feedback!
Cheers,
-jakob

From jjl at pobox.com  Tue Jul 22 21:56:31 2008
From: jjl at pobox.com (John J Lee)
Date: Tue, 22 Jul 2008 20:56:31 +0100 (BST)
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <20080722053744.GA11290@cskk.homeip.net>
References: <20080722053744.GA11290@cskk.homeip.net>
Message-ID: <alpine.DEB.1.00.0807222053270.6381@alice>

On Tue, 22 Jul 2008, Cameron Simpson wrote:
[...]
> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
> exception if asked for < 1 bytes? Or is there a legitimate use for read(0)
> with which I was not previously aware?

http://docs.python.org/lib/bltin-file-objects.html

read([size])

... If the size argument is negative or omitted, read all data until EOF 
is reached. ...


John


From cs at zip.com.au  Wed Jul 23 00:28:44 2008
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 23 Jul 2008 08:28:44 +1000
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <alpine.DEB.1.00.0807222053270.6381@alice>
Message-ID: <20080722222844.GA23074@cskk.homeip.net>

On 22Jul2008 20:56, John J Lee <jjl at pobox.com> wrote:
> On Tue, 22 Jul 2008, Cameron Simpson wrote:
> [...]
>> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
>> exception if asked for < 1 bytes? Or is there a legitimate use for read(0)
>> with which I was not previously aware?
>
> http://docs.python.org/lib/bltin-file-objects.html
>
> read([size])
>
> ... If the size argument is negative or omitted, read all data until EOF  
> is reached. ...

Hmm, yeah, but 0 is not negative and not omitted so this does not apply.

Personally I'm not very fond of that spec; I'm good with the omitted size
provoking a "read everything" mode but I'd rather a non-numeric value like
None rather than a negative one (eg the conventional "def read(size=None)")
if an explicit size should do so. That way bad arithmetic in the caller
could have a chance of triggering an exception from read instead of a
silent (and to my taste, nasty) "slurp the file" mode.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

From jjl at pobox.com  Wed Jul 23 00:45:56 2008
From: jjl at pobox.com (John J Lee)
Date: Tue, 22 Jul 2008 23:45:56 +0100 (BST)
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <20080722222844.GA23074@cskk.homeip.net>
References: <20080722222844.GA23074@cskk.homeip.net>
Message-ID: <alpine.DEB.1.00.0807222339120.7713@alice>

On Wed, 23 Jul 2008, Cameron Simpson wrote:
> On 22Jul2008 20:56, John J Lee <jjl at pobox.com> wrote:
>> On Tue, 22 Jul 2008, Cameron Simpson wrote:
>> [...]
>>> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
>>> exception if asked for < 1 bytes? Or is there a legitimate use for read(0)
>>> with which I was not previously aware?
>>
>> http://docs.python.org/lib/bltin-file-objects.html
>>
>> read([size])
>>
>> ... If the size argument is negative or omitted, read all data until EOF
>> is reached. ...
>
> Hmm, yeah, but 0 is not negative and not omitted so this does not apply.

Well, -1 *is* < 1 (and is in the domain of the function), but yes -- 
sorry, read too quickly, took your "< 1" too literally.


John


From cs at zip.com.au  Wed Jul 23 00:46:29 2008
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 23 Jul 2008 08:46:29 +1000
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <Pine.LNX.4.64.0807212328340.23485@lappy>
Message-ID: <20080722224629.GA24798@cskk.homeip.net>

On 21Jul2008 23:35, Leif Walsh <leif.walsh at gmail.com> wrote:
| On Tue, 22 Jul 2008, Cameron Simpson wrote:
| > Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
| > exception if asked for < 1 bytes? Or is there a legitimate use for
| > read(0) with which I was not previously aware?
| 
| I think read(0) should be a no-op, just like it is in libc.  This lets
| you write 'read(bytes)' without worrying about checking bytes, and
| also lets you silently stop reading when you have no more space, like
| in the following:
| 
| buf = f.read(max(bytes_left, page_size))
| while buf:
|   process(buf)  # updates bytes_left
|   buf = f.read(max(bytes_left, page_size))

[ Don't you mean "min()"? Unimportant. ]

I see the convenience here, but doubt I'd ever do that myself.
I'd write the above like this:

  while bytes_left > 0:
    buf = f.read(max(bytes_left, page_size))
    if buf == 0:
      break
    process(buf)  # updates bytes_left

I'm kind of picky about doing things exactly as often as required and no
more. Especially things that call another facility.

read(0) itself must internally have a check for size == 0 anyway, so
it's not like the overall system is less complex. If we're unlucky it
could trickle all the way down to an OS system call to read(2) (UNIX,
substitute as suitable elsewhere) and for a no-op that would be overkill
by far. The only way the read() implementation would avoid that is by
doing the test on size anyway. But since read() is opaque IMO it is
better to avoid it at the upper level if we know it will produce
nothing.

Which leaves me unconvinced of the utility of this mode.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

From victor.stinner at haypocalc.com  Wed Jul 23 01:43:18 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 23 Jul 2008 01:43:18 +0200
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com>
References: <200807212117.14485.victor.stinner@haypocalc.com>
	<1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com>
Message-ID: <200807230143.18189.victor.stinner@haypocalc.com>

Le Monday 21 July 2008 21:23:21, vous avez ?crit?:
> > It should raises an error instead of a warning, it has no sense to read a
> > partial byte :-) But that should breaks some applications?
>
> This doesn't come into effect until 3.0.

Would it possible to create an option "strict mode" which disallow dangerous 
cast? Especially in PyArg_Parse*() functions.

--

I hate "transparent" cast, like C and PHP do everywhere. The 
worst "transparent" cast in Python (2.x) is between str (bytes) and unicode 
(characters) types.

Victor Stinner aka haypo

From musiccomposition at gmail.com  Wed Jul 23 01:47:20 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 22 Jul 2008 18:47:20 -0500
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <200807230143.18189.victor.stinner@haypocalc.com>
References: <200807212117.14485.victor.stinner@haypocalc.com>
	<1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com>
	<200807230143.18189.victor.stinner@haypocalc.com>
Message-ID: <1afaf6160807221647s3b6274b9nacc016fddea20b8e@mail.gmail.com>

On Tue, Jul 22, 2008 at 6:43 PM, Victor Stinner <
victor.stinner at haypocalc.com> wrote:

> Le Monday 21 July 2008 21:23:21, vous avez ?crit :
> > > It should raises an error instead of a warning, it has no sense to read
> a
> > > partial byte :-) But that should breaks some applications?
> >
> > This doesn't come into effect until 3.0.
>
> Would it possible to create an option "strict mode" which disallow
> dangerous
> cast? Especially in PyArg_Parse*() functions.


You could use -Wall to make the warning an error.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080722/ba03076c/attachment.htm>

From musiccomposition at gmail.com  Wed Jul 23 01:47:49 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 22 Jul 2008 18:47:49 -0500
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <1afaf6160807221647s3b6274b9nacc016fddea20b8e@mail.gmail.com>
References: <200807212117.14485.victor.stinner@haypocalc.com>
	<1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com>
	<200807230143.18189.victor.stinner@haypocalc.com>
	<1afaf6160807221647s3b6274b9nacc016fddea20b8e@mail.gmail.com>
Message-ID: <1afaf6160807221647l7d85aa36r606e0ec1aef48ee8@mail.gmail.com>

On Tue, Jul 22, 2008 at 6:47 PM, Benjamin Peterson <
musiccomposition at gmail.com> wrote:

>
>
> On Tue, Jul 22, 2008 at 6:43 PM, Victor Stinner <
> victor.stinner at haypocalc.com> wrote:
>
>> Le Monday 21 July 2008 21:23:21, vous avez ?crit :
>> > > It should raises an error instead of a warning, it has no sense to
>> read a
>> > > partial byte :-) But that should breaks some applications?
>> >
>> > This doesn't come into effect until 3.0.
>>
>> Would it possible to create an option "strict mode" which disallow
>> dangerous
>> cast? Especially in PyArg_Parse*() functions.
>
>
> You could use -Wall to make the warning an error.
>

I meant -Werr.

>
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080722/70fca1da/attachment.htm>

From leif.walsh at gmail.com  Wed Jul 23 03:19:49 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Tue, 22 Jul 2008 18:19:49 -0700
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <20080722224629.GA24798@cskk.homeip.net>
References: <Pine.LNX.4.64.0807212328340.23485@lappy>
	<20080722224629.GA24798@cskk.homeip.net>
Message-ID: <cc7430500807221819j6384105dq4c3b0114cc9d62a7@mail.gmail.com>

On Tue, Jul 22, 2008 at 3:46 PM, Cameron Simpson <cs at zip.com.au> wrote:
> [ Don't you mean "min()"? Unimportant. ]

Haha, that's what I get for not actually _running_ the code example.

> I see the convenience here, but doubt I'd ever do that myself.
> I'd write the above like this:
>
>  while bytes_left > 0:
>    buf = f.read(max(bytes_left, page_size))
>    if buf == 0:
>      break
>    process(buf)  # updates bytes_left
>
> I'm kind of picky about doing things exactly as often as required and no
> more. Especially things that call another facility.

I do the same, but I know lots of people that prefer the example I
sent earlier.  Also, if we ever adopt the "while ... as ..." syntax
(here's not hoping) currently being discussed in another thread,
having read(0) return None or an empty buffer will cause that idiom to
short circuit as well.

> read(0) itself must internally have a check for size == 0 anyway, so
> it's not like the overall system is less complex. If we're unlucky it
> could trickle all the way down to an OS system call to read(2) (UNIX,
> substitute as suitable elsewhere) and for a no-op that would be overkill
> by far. The only way the read() implementation would avoid that is by
> doing the test on size anyway. But since read() is opaque IMO it is
> better to avoid it at the upper level if we know it will produce
> nothing.

If we are going to make read(0) a no-op, we should definitely do it
before it hits the underlying implementation, for portability's sake.

On Tue, Jul 22, 2008 at 4:43 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Would it possible to create an option "strict mode" which disallow dangerous
> cast? Especially in PyArg_Parse*() functions.

Ack!  We're not writing Perl here, guys.  Can we please not start
having multiple subsets of the language that are separately valid?

-- 
Cheers,
Leif

From jcea at jcea.es  Wed Jul 23 17:33:01 2008
From: jcea at jcea.es (Jesus Cea)
Date: Wed, 23 Jul 2008 17:33:01 +0200
Subject: [Python-Dev] bsddb: Test failures on windows (HELP!)
Message-ID: <48874F2D.5070201@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Trent, I was wondering if you could look at some test failures in MS
Windows builds. I can't debug Windows issues myself :-(. This is a MS
free environment...

Details:
<http://www.python.org/dev/buildbot/all/x86%20XP-4%20trunk/builds/1364/step-test/0>

Thanks in advance for your time and attention.

Of course, any other help appreciated :-)

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIdPJJlgi5GaxT1NAQLzFwP9E0huY63jOZWx9v/H7NhDGwIqFl5OGfhO
EZ6uCdRBuyH4Y4jQBaIKTFN9jXu2sSONNMBDgOZznkBuYtFIA8Xmn9KDrkuZ5k8G
1GlcG+DcoG1aP1PANgrPMkEpptw5/TNBlA3s6p/F4oDSye1kMW0CbGcSHeECYF9R
CIhZztqZvWk=
=aZoO
-----END PGP SIGNATURE-----

From loisel at temple.edu  Wed Jul 23 23:14:22 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Wed, 23 Jul 2008 17:14:22 -0400
Subject: [Python-Dev] Infix operators
Message-ID: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>

Dear Pythonistas,

I've googled for this but I wasn't able to find anything definitive. I was
recently looking at scipy to see if I could use it in stead of MATLAB for my
class on numerical PDEs, but I noticed that there's some difficulty related
to the notation; mainly, the matrix multiplication, and the linear solver
(i.e., MATLAB's \ operator). After giving it some thought, I've decided to
hold back for now and use MATLAB.

Since scipy/numeric python have been around for a while and some of it is
even integrated in Python 2.5, I'm sure this conversation has happened
before, so please just educate me. Now I think that there's always going to
be people wanting more and more operators in Python (although the operators
I'm talking about are privileged -- the Chinese were using matrices some
2000 years ago), so I was thinking that it would be simpler to have a way
for defining new infix operators, which would simply be binary functions
whose names are punctuation marks; see any language with this feature, e.g.,
Haskell.

Now since this is such a simple idea, I'm guessing it occured to pythonistas
at some point in 1992, and got voted down and that decision has now become
scripture. Is that the case?

The one thing which I found was PEP 211
http://www.python.org/dev/peps/pep-0211/, which has an amazing quote from
James Rawlings. His comment about inv is completely absurd, and while he
claims not to know \ and /, he is quoted as an authority on these operators.
This particular PEP should probably be deleted from history. For a more
authoritative explanation, a quick search yields MathWorks' own Loren Shure:
http://blogs.mathworks.com/loren/2007/05/16/purpose-of-inv/

Essentially, in almost all applications, inv(A) is entirely wrong. You can
ask any numerical analyst who works with large problems, and they will
confirm it. One of the main reason is that, even if A is sparse, inv(A) is
full.

That absurdity aside, if I were a language designer, I would be reticent to
add one operator to Python (like in PEP 211), because there will always be
more operators that people want (how long until someone asks for an operator
for the Kronecker product or the convolution?) But why not put in this infix
possibility?

Sincerely,

-- 
S?bastien Loisel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/6d68d438/attachment.htm>

From josiah.carlson at gmail.com  Thu Jul 24 01:11:19 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Wed, 23 Jul 2008 16:11:19 -0700
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
Message-ID: <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>

On Wed, Jul 23, 2008 at 2:14 PM, Sebastien Loisel <loisel at temple.edu> wrote:
> Dear Pythonistas,
>
> I've googled for this but I wasn't able to find anything definitive. I was
> recently looking at scipy to see if I could use it in stead of MATLAB for my
> class on numerical PDEs, but I noticed that there's some difficulty related
> to the notation; mainly, the matrix multiplication, and the linear solver
> (i.e., MATLAB's \ operator). After giving it some thought, I've decided to
> hold back for now and use MATLAB.
>
> Since scipy/numeric python have been around for a while and some of it is
> even integrated in Python 2.5, I'm sure this conversation has happened
> before, so please just educate me. Now I think that there's always going to
> be people wanting more and more operators in Python (although the operators
> I'm talking about are privileged -- the Chinese were using matrices some
> 2000 years ago), so I was thinking that it would be simpler to have a way
> for defining new infix operators, which would simply be binary functions
> whose names are punctuation marks; see any language with this feature, e.g.,
> Haskell.
>
> Now since this is such a simple idea, I'm guessing it occured to pythonistas
> at some point in 1992, and got voted down and that decision has now become
> scripture. Is that the case?
>
> The one thing which I found was PEP 211
> http://www.python.org/dev/peps/pep-0211/, which has an amazing quote from
> James Rawlings. His comment about inv is completely absurd, and while he
> claims not to know \ and /, he is quoted as an authority on these operators.
> This particular PEP should probably be deleted from history. For a more
> authoritative explanation, a quick search yields MathWorks' own Loren Shure:
> http://blogs.mathworks.com/loren/2007/05/16/purpose-of-inv/
>
> Essentially, in almost all applications, inv(A) is entirely wrong. You can
> ask any numerical analyst who works with large problems, and they will
> confirm it. One of the main reason is that, even if A is sparse, inv(A) is
> full.
>
> That absurdity aside, if I were a language designer, I would be reticent to
> add one operator to Python (like in PEP 211), because there will always be
> more operators that people want (how long until someone asks for an operator
> for the Kronecker product or the convolution?) But why not put in this infix
> possibility?

Why not add the possibility for arbitrary infix operators?  Others
will most likely disagree with me, but I would claim that adding
arbitrary infix operators are a great way to destroy readability.
What the heck does 'x = 4 $ 6' mean in Python?  Oh, that's right, it's
a custom infix operator.  But where is it defined?  In the module?  In
some other module that is imported?  Can I override or do some
pre-processing to the arguments and pass it on to the previously
defined $ operator?  So many questions, none of which tell me what '$'
does.

Really though, PEP 211 was a perfect example of a k-line function that
someone attempted to make syntax that really didn't need to be syntax.
 And arguably, any infix operator will need to result in a lookup for
the new infix operator, which will be as fast (or slow) as a globals
lookup, so you may as well define your operator as a two-argument
function, which has defined semantics, and can be imported and passed
just like all functions defined for years.

I would argue current operators are *convenience* for the 99%+
majority of "mathematical" operations done in Python, with
well-defined and understood semantics in the 99%+ cases they are used
for.  Further, the addition of further infix operators are, strictly
speaking, a domain-specific language, which Python (as a language)
doesn't frown upon, but doesn't encourage either.

If you still think that X \ Y would honestly make Python a better
language, I would ask you if logix
(http://www.livelogix.net/logix/intro.html) would suit you better.  It
seems to allow you to use arbitrary punctuation for operators.

 - Josiah

From loisel at temple.edu  Thu Jul 24 01:48:06 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Wed, 23 Jul 2008 19:48:06 -0400
Subject: [Python-Dev] Infix operators
In-Reply-To: <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
Message-ID: <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>

Dear Josiah,

Thank you for your email.

On Wed, Jul 23, 2008 at 7:11 PM, Josiah Carlson <josiah.carlson at gmail.com>
wrote:

> What the heck does 'x = 4 $ 6' mean in Python?  Oh, that's right, it's
> a custom infix operator.  But where is it defined?  In the module?  In
> some other module that is imported?  Can I override or do some
> pre-processing to the arguments and pass it on to the previously
> defined $ operator?  So many questions, none of which tell me what '$'
> does.
>

My question is really what the history of this thing is and what sort of
opposition there is to this idea, in the python community. From your email,
I suspect that this conversation has already taken place.

I'm not going to become a python developer/politician and try to push this,
my job involves teaching and research in mathematics and the programming
language is just a tool. If I want to "put food on my family" (in the words
of a wise man), I can't really spend a whole lot of time on this mailing
list debating this.

However, just for posterity (and I'm not going to pursue the argument
further than this), I'll say this. The problem of determining the meaning
(or overridability or whatever) of x=4$6 is the same as the problem of
determining the meaning of x=fooz(4,6). Since it's not a good argument
against user-defined functions, I don't see it as a good argument against
user-defined operators. Of course, obviously the LISP people have decided
that fooz(4,6) (or rather, (fooz 4 6)) is okay, and 4$6 is not, so who am I
to dispute that.


> If you still think that X \ Y would honestly make Python a better


Well, I'm not arguing for adding one more special-purpose operator, because
probably the need for more operators will never end. If arbitrary operators
were defined like ordinary functions, they could be brought in with the
import numeric or whatever, and docstrings could be looked up, etc...


>
> language, I would ask you if logix
> (http://www.livelogix.net/logix/intro.html) would suit you better.  It
> seems to allow you to use arbitrary punctuation for operators.
>

Thank you for the pointer. I have taken a look and it does look interesting,
on first blush I would love to use that language. The main thing is that I
worry about being out on the fringe, using a language that nobody uses, and
which may get abandoned without warning (like Sun abandoned `self'), or be
buggier just because people don't use it... Although I love metaprogramming
and I would use it if it were in python, I don't really need to go that far
(arbitrary infix operators would suffice to me.)

Sincerely,

-- 
S?bastien Loisel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/4e31179b/attachment.htm>

From anthonybaxter at gmail.com  Thu Jul 24 01:50:45 2008
From: anthonybaxter at gmail.com (Anthony Baxter)
Date: Wed, 23 Jul 2008 16:50:45 -0700
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org>
	<4884581F.8010201@jcea.es>
	<bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>
	<319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com>
Message-ID: <e69d3ed20807231650x1cc1716ew3789b2aacc280d1d@mail.gmail.com>

As a data point, I just presented a tutorial here at OSCON on Python 3
Porting, and had a number of people asking about C API changes. I
punted for my talk (*) because things are still in flux. Plus I only
had 3 hours. I did suggest that if your extension is just glue to a C
library, you should consider using ctypes.

So yes, collecting this information, even if it's just in a wiki page,
would be a good and popular thing.

Anthony

(*) slides: http://www.interlink.com.au/anthony/tech/talks/OSCON2008/porting3.pdf

On Mon, Jul 21, 2008 at 2:37 PM, Lennart Regebro <regebro at gmail.com> wrote:
> On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote:
>> But waiting until all the betas have gone out totally defeats the
>> purpose of the betas!
>
> I agree. Writing an actual *guide* can wait, but documenting the
> differences with code examples is a work that can start now, and I
> agree that it would be great if this would start now.
>
> But writing a guide might not be a good idea until we know what the
> changes are, and if the API is changing quickly now we don't. :-)
>
> --
> Lennart Regebro: Zope and Plone consulting.
> http://www.colliberty.com/
> +33 661 58 14 64
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/anthony%40interlink.com.au
>
>

From curt at hagenlocher.org  Thu Jul 24 02:08:39 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Wed, 23 Jul 2008 17:08:39 -0700
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
Message-ID: <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>

On Wed, Jul 23, 2008 at 4:48 PM, Sebastien Loisel <loisel at temple.edu> wrote:
>
> On Wed, Jul 23, 2008 at 7:11 PM, Josiah Carlson <josiah.carlson at gmail.com>
> wrote:
>>
>> language, I would ask you if logix
>> (http://www.livelogix.net/logix/intro.html) would suit you better.  It
>> seems to allow you to use arbitrary punctuation for operators.
>
> Thank you for the pointer. I have taken a look and it does look interesting,
> on first blush I would love to use that language. The main thing is that I
> worry about being out on the fringe, using a language that nobody uses, and
> which may get abandoned without warning (like Sun abandoned `self'), or be
> buggier just because people don't use it...

Have you considered OCaml? (http://en.wikipedia.org/wiki/Ocaml) It's
in reasonably broad use and is actively maintained, and it allows
user-defined infix operators.

A programming language can't be all things to all people.  That's why
there's room in the world for more than one.

--
Curt Hagenlocher
curt at hagenlocher.org

From loisel at temple.edu  Thu Jul 24 02:21:18 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Wed, 23 Jul 2008 20:21:18 -0400
Subject: [Python-Dev] Infix operators
In-Reply-To: <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
Message-ID: <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>

Dear Curt,

Thank you for your email.

Have you considered OCaml? (http://en.wikipedia.org/wiki/Ocaml) It's
>

I have. I've considered a lot of languages, but OCaml isn't for me. My
current language is MATLAB. Python is pretty close in syntax, and it's
widely accepted, so you can teach students Python and they can go out and
use it. You can also publish a paper and write a code snippet without having
to explain the syntax. I'm not trying to go out on the blistering leading
edge of computer language sophistication; that would not be good for my
students, and it would not be good for my research.

For most of my programs, what I need is a language where you don't need to
give types or declare variables, because that just takes up space and
obscures the math. So I need a mainstream dynamic language like Python or
MATLAB.

OCaml is not dynamic, and linear algebra in OCaml is a pain in the neck
because of the explosion of the number of operators for various pairs of
operand types.

Sincerely,

-- 
S?bastien Loisel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/b4edbfa0/attachment.htm>

From kirklin.mcdonald at gmail.com  Thu Jul 24 02:24:28 2008
From: kirklin.mcdonald at gmail.com (Kirk McDonald)
Date: Wed, 23 Jul 2008 17:24:28 -0700
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org>
	<4884581F.8010201@jcea.es>
	<bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>
	<319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com>
Message-ID: <25bd58d10807231724u1d60f5e7y8cc57905a599e60c@mail.gmail.com>

On Mon, Jul 21, 2008 at 2:37 PM, Lennart Regebro <regebro at gmail.com> wrote:
> On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote:
>> But waiting until all the betas have gone out totally defeats the
>> purpose of the betas!
>
> I agree. Writing an actual *guide* can wait, but documenting the
> differences with code examples is a work that can start now, and I
> agree that it would be great if this would start now.
>
> But writing a guide might not be a good idea until we know what the
> changes are, and if the API is changing quickly now we don't. :-)
>

I am the nominal maintainer of the D programming language's bindings
to the Python/C API.[1] Keeping on top of the changes in the C API is
therefore of some interest to me. At the heart of these bindings is
3000 or so lines of D code containing the translated C headers. Adding
the changes made in 3.0 (and 2.6, for that matter) will probably have
to be done by hand. Any document detailing every change, from the
broadest refactorings to the tiniest name-changes, will therefore be
of invaluable assistance to me. Obviously, I don't plan on even
starting this upgrade until the C API is nailed down and such a
document exists. But any drafts, or even a bulleted list of changes,
would at least give me an idea of the scope of what I'm in for.

[1] http://pyd.dsource.org

-Kirk McDonald

From greg.ewing at canterbury.ac.nz  Thu Jul 24 04:05:01 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 24 Jul 2008 14:05:01 +1200
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
Message-ID: <4887E34D.2000608@canterbury.ac.nz>

Sebastien Loisel wrote:

> Essentially, in almost all applications, inv(A) is entirely wrong. You 
> can ask any numerical analyst who works with large problems, and they 
> will confirm it. One of the main reason is that, even if A is sparse, 
> inv(A) is full.

This argues for a function such as solve(A, b). It
doesn't argue for an infix operator.

> But why not put in this infix possibility?

Guido is on record as opposing any kind of macros
or other "extensible syntax", and this probably comes
under the same heading.

Besides which there are technical difficulties, such
as how the parser is supposed to know the precedence
of these user-defined operators.

To address your original problem, I believe that numpy
comes with a matrix type that defines * as matrix
multiplication. That's the usual Python solution to
these kinds of things -- rather than invent a new
operator, invent a new type that behaves the way
you want.

-- 
Greg

From loisel at temple.edu  Thu Jul 24 04:28:09 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Wed, 23 Jul 2008 22:28:09 -0400
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
Message-ID: <b7975ab80807231928p468a5e7fj636e4b80d2d88b1e@mail.gmail.com>

Dear Greg,

Thanks for your email.

Guido is on record as opposing any kind of macros
> or other "extensible syntax", and this probably comes
> under the same heading.
>

Thanks, that's exactly what I was looking for when I asked:

Now since this is such a simple idea, I'm guessing it occured to pythonistas
> at some point in 1992, and got voted down and that decision has now become
> scripture. Is that the case?
>

Case closed.

Sincerely,

-- 
S?bastien Loisel

PS:

This argues for a function such as solve(A, b). It
> doesn't argue for an infix operator.
>

I know, I was demolishing PEP 211, not trying to support my point.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/abd24192/attachment.htm>

From tjreedy at udel.edu  Thu Jul 24 05:13:35 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 23 Jul 2008 23:13:35 -0400
Subject: [Python-Dev] Close PEP 211? (was Re: Infix operators)
In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
Message-ID: <g68s10$646$1@ger.gmane.org>


Sebastien Loisel wrote:

> 
> The one thing which I found was PEP 211 
> http://www.python.org/dev/peps/pep-0211/

This now has a status of deferred.

Given that itertools.product(A,B) == ((x,y) for x in A for y in B)
== the proposed 'A @ B' and given Guido's pronounced distaste for new 
infix, should this be closed?



From greg.ewing at canterbury.ac.nz  Thu Jul 24 05:21:07 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 24 Jul 2008 15:21:07 +1200
Subject: [Python-Dev] Infix operators
In-Reply-To: <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
Message-ID: <4887F523.7060207@canterbury.ac.nz>

Josiah Carlson wrote:
> What the heck does 'x = 4 $ 6' mean in Python?  Oh, that's right, it's
> a custom infix operator.  But where is it defined?

It's not quite as bad as that -- it would be defined by
the relevant operator method on one of the operands. But
a convention would be needed for mapping arbitary non-
alphanumeric sequences to method names, and it's not
obvious how to do that.

But the main technical problem I see is that of precedence.
It would have to be declared somehow, or a declaration
imported from another module. If it's imported, then
parsing a module can't be done in isolation any more,
since it depends on the contents of other modules.
Things could get very messy.

> Really though, PEP 211 was a perfect example of a k-line function that
> someone attempted to make syntax that really didn't need to be syntax.

It looks like a case of someone wanting a matrix multiplication
operator in numpy, and then hunting around for something to
make it mean in Python.

I would actually be in favour of adding a matrix multiplication
operator, but I would make it mean matrix multiplication in
Python as well, operating on anything that can be treated as
a 2D sequence.

Why matrix multiplication in particular? Because it's the one
thing that you do all the time with matrices for which there
isn't an available operator. I think this one addition could
be justified on the grounds of very wide usage.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Jul 24 05:26:23 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 24 Jul 2008 15:26:23 +1200
Subject: [Python-Dev] Close PEP 211? (was Re: Infix operators)
In-Reply-To: <g68s10$646$1@ger.gmane.org>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<g68s10$646$1@ger.gmane.org>
Message-ID: <4887F65F.9000004@canterbury.ac.nz>

Terry Reedy wrote:

> Given that itertools.product(A,B) == ((x,y) for x in A for y in B)
> == the proposed 'A @ B' and given Guido's pronounced distaste for new 
> infix, should this be closed?

Would there likely be any support for an alternative
PEP defining @ as matrix multiplication in both Python
and numpy?

-- 
Greg

From aahz at pythoncraft.com  Thu Jul 24 06:03:17 2008
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 23 Jul 2008 21:03:17 -0700
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
Message-ID: <20080724040317.GA5358@panix.com>

On Wed, Jul 23, 2008, Sebastien Loisel wrote:
>
> [...] I was thinking that it would be simpler to have a way
> for defining new infix operators, [...]

python-dev is the wrong place for this discussion.  Please use either
comp.lang.python or python-ideas.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From mhammond at skippinet.com.au  Thu Jul 24 07:44:31 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 24 Jul 2008 15:44:31 +1000
Subject: [Python-Dev] bsddb: Test failures on windows (HELP!)
In-Reply-To: <48874F2D.5070201@jcea.es>
References: <48874F2D.5070201@jcea.es>
Message-ID: <095c01c8ed50$60bdfed0$2239fc70$@com.au>

> Trent, I was wondering if you could look at some test failures in MS
> Windows builds. I can't debug Windows issues myself :-(. This is a MS
> free environment...

In these errors I see lots of bsdbd errors, many of the form:

| DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit
(100) exceeded')
 
Maybe an old test file isn't being nuked?  Others of the form:

|    self.assertTrue(time.time()<timeout)
|  AssertionError

which also look more related to the test suite than to Windows.

There are also lots of errors due to the environment having a unicode object
in it:

  | test test_site failed -- Traceback (most recent call last):
  |  ...
  | TypeError: environment can only contain strings

  | test test_subprocess failed -- errors occurred; run in verbose mode for
details
  | [Possibly the same error as below?]

  | test test_sys failed -- Traceback (most recent call last):
  | File "...\subprocess.py", line 817, in _execute_child
  |    startupinfo)
  | TypeError: environment can only contain strings

* A couple of wsgi related errors, including one about the environment - but
this has more information:

AssertionError: "AssertionError: Environmental variable LIB is not a string:
<type 'unicode'> (value: u'C:\\\\Program Files\\\\Microsoft Visual Studio
9.0\\\\VC\\\\LIB;C:\\\\Program Files\\\\Microsoft
SDKs\\\\Windows\\\\v6.0A\\\\lib;c:\\\\program files\\\\microsoft visual
studio .NET 2003\\\\vc7\\\\atlmfc\\\\lib;c:\\\\program files\\\\microsoft
visual studio .NET 2003\\\\vc7\\\\lib;c:\\\\program files\\\\microsoft
visual studio .NET 2003\\\\vc7\\\\PlatformSDK\\\\lib;C:\\\\Program
Files\\\\Microsoft Visual Studio .NET 2003\\\\SDK\\\\v1.1\\\\Lib\\\\')" !=
"AssertionError: Headers (('Content-Type', 'text/plain')) must be of type
list: <type 'tuple'>"

So LIB has a Unicode value - evn though it has no Unicode characters, and
was presumably in the environment Python inherited (but presumably was
initially a string).

I can't reproduce the Unicode errors: 'python test_sys.py' works and I
couldn't run a full test suite (see below).  Python allows you to set
unicode objects in os.environ, so its possible part of the test suite is
modifying the environment.

I tried to run a full test, but it hangs in various places before test_sys
(bz2, multiprocessing), and I ended up giving up.  This was on for either 32
or 64bits with the current trunk, but sadly I've no idea what could be
happening there :(

So it sounds like just tracking down how a unicode object is getting into
the environment will solve the vast majority of the errors - except for the
bsddb and wsgi ones, which don't look particularly Windows specific...

Hope that helps a bit...

Mark


From stefan_ml at behnel.de  Thu Jul 24 09:27:52 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 24 Jul 2008 09:27:52 +0200
Subject: [Python-Dev] crash in 3.0b2 exception code
Message-ID: <g69atp$716$1@ger.gmane.org>

Hi,

I get a crash in one of lxml's doctests in 3.0b2, and it looks like it's
coming from plain Python code (as opposed to Cython code). Just a quick check
before I start digging into this, has anyone seen this before?

Stefan

[...]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1209997120 (LWP 15431)]
0x080f3b89 in BaseException_str (self=0x8d747d4) at Objects/exceptions.c:88
88          switch (PyTuple_GET_SIZE(self->args)) {
(gdb) bt
#0  0x080f3b89 in BaseException_str (self=0x8d747d4) at Objects/exceptions.c:88
#1  0x0805b158 in PyObject_Str (v=0x8d747d4) at Objects/object.c:414
#2  0x0807951a in unicode_new (type=0x81523a0, args=0x8de77ac, kwds=0x0) at
Objects/unicodeobject.c:9247
#3  0x0806068d in type_call (type=0x81523a0, args=0x8de77ac, kwds=0x0) at
Objects/typeobject.c:636
#4  0x080d83a9 in PyObject_Call (func=0x81523a0, arg=0x8de77ac, kw=0x0) at
Objects/abstract.c:2178
#5  0x0808de50 in PyEval_EvalFrameEx (f=0x8e2d07c, throwflag=0) at
Python/ceval.c:3606
#6  0x0808fccd in PyEval_EvalFrameEx (f=0x8e2bf14, throwflag=0) at
Python/ceval.c:3481
#7  0x0808fccd in PyEval_EvalFrameEx (f=0x8e2cef4, throwflag=0) at
Python/ceval.c:3481
#8  0x0808fccd in PyEval_EvalFrameEx (f=0x8e2d4ec, throwflag=0) at
Python/ceval.c:3481
#9  0x08090f6b in PyEval_EvalCodeEx (co=0x8268458, globals=0xb7c182d4,
locals=0x0, args=0x8e2cea4, argcount=3, kws=0x8e2ceb0, kwcount=1,
defs=0x826c678, defcount=3, kwdefs=0x0,
    closure=0x0) at Python/ceval.c:2830
[...]


From tseaver at palladion.com  Thu Jul 24 14:39:08 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 24 Jul 2008 08:39:08 -0400
Subject: [Python-Dev] bsddb: Test failures on windows (HELP!)
In-Reply-To: <095c01c8ed50$60bdfed0$2239fc70$@com.au>
References: <48874F2D.5070201@jcea.es> <095c01c8ed50$60bdfed0$2239fc70$@com.au>
Message-ID: <488877EC.600@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark Hammond wrote:

>> Trent, I was wondering if you could look at some test failures in MS
>> Windows builds. I can't debug Windows issues myself :-(. This is a MS
>> free environment...
> 
> In these errors I see lots of bsdbd errors, many of the form:
> 
> | DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit
> (100) exceeded')

Maybe this one is due to the fact that Windows, unlike POSIX, doesn't
allow unlinking an open file?

> Maybe an old test file isn't being nuked?  Others of the form:
> 
> |    self.assertTrue(time.time()<timeout)
> |  AssertionError
> 
> which also look more related to the test suite than to Windows.

Perhaps this is due to differing timer granularity on Windows?

> There are also lots of errors due to the environment having a unicode object
> in it:
> 
>   | test test_site failed -- Traceback (most recent call last):
>   |  ...
>   | TypeError: environment can only contain strings
> 
>   | test test_subprocess failed -- errors occurred; run in verbose mode for
> details
>   | [Possibly the same error as below?]
> 
>   | test test_sys failed -- Traceback (most recent call last):
>   | File "...\subprocess.py", line 817, in _execute_child
>   |    startupinfo)
>   | TypeError: environment can only contain strings

That definitely shouldn't happen on Unix either.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIiHfr+gerLs4ltQ4RAk5KAJ9It0Am1VfFNQNaE+wA8uWkkTZ6wQCgtwlx
o16eVKpEXTHED4X1/Vi0Nk0=
=zzKd
-----END PGP SIGNATURE-----


From musiccomposition at gmail.com  Thu Jul 24 14:41:11 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 24 Jul 2008 07:41:11 -0500
Subject: [Python-Dev] crash in 3.0b2 exception code
In-Reply-To: <g69atp$716$1@ger.gmane.org>
References: <g69atp$716$1@ger.gmane.org>
Message-ID: <1afaf6160807240541n2d01b0d9yb7bc3fc5ff56407b@mail.gmail.com>

On Thu, Jul 24, 2008 at 2:27 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Hi,
>
> I get a crash in one of lxml's doctests in 3.0b2, and it looks like it's
> coming from plain Python code (as opposed to Cython code). Just a quick
> check
> before I start digging into this, has anyone seen this before?


Please file a bug report.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080724/9018b196/attachment.htm>

From amauryfa at gmail.com  Thu Jul 24 16:23:21 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 24 Jul 2008 16:23:21 +0200
Subject: [Python-Dev] bsddb: Test failures on windows (HELP!)
In-Reply-To: <488877EC.600@palladion.com>
References: <48874F2D.5070201@jcea.es> <095c01c8ed50$60bdfed0$2239fc70$@com.au>
	<488877EC.600@palladion.com>
Message-ID: <e27efe130807240723v155b05a5x1bd2a9b1a64ab057@mail.gmail.com>

Tres Seaver wrote:
> Mark Hammond wrote:
>
>>> Trent, I was wondering if you could look at some test failures in MS
>>> Windows builds. I can't debug Windows issues myself :-(. This is a MS
>>> free environment...
>>
>> In these errors I see lots of bsdbd errors, many of the form:
>>
>> | DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit
>> (100) exceeded')
>
> Maybe this one is due to the fact that Windows, unlike POSIX, doesn't
> allow unlinking an open file?

I see that some tests use os.unlink. They should use
test_support.unlink() instead.

-- 
Amaury Forgeot d'Arc

From tom at vector-seven.com  Thu Jul 24 16:56:35 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Fri, 25 Jul 2008 00:56:35 +1000
Subject: [Python-Dev] lnotab and the AST optimizer
Message-ID: <48889823.20506@vector-seven.com>

I'm making some good progress with the AST optimizer, and now the main 
thing standing in my way is lnotab. Currently lnotab expects bytecode 
sequencing to be roughly in-sync with the order of the source file and a 
few things that the optimizer does (e.g. swapping the bodies of an 
if/else after removing negation such that "if not X: A; else: B" becomes 
"if X: B; else A") breaks this assumption. This will result in either an 
assertion failure or incorrect line numbers being reported.

It seems that lnotab is used in relatively few places in the source code 
at the moment, but if I'm going to make a change to how lnotab works I 
want to do so in a way that's going to allow me to move forward while 
keeping everybody happy.

I'm away for a few days so I probably won't be able to get back to 
anybody until either Sunday or Monday, but I'd appreciate it if anybody 
in the know can weigh in on this.

Cheers,
Tom

From solipsis at pitrou.net  Thu Jul 24 17:19:04 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 24 Jul 2008 15:19:04 +0000 (UTC)
Subject: [Python-Dev] lnotab and the AST optimizer
References: <48889823.20506@vector-seven.com>
Message-ID: <loom.20080724T151343-964@post.gmane.org>


Hi,

> I'm making some good progress with the AST optimizer, and now the main 
> thing standing in my way is lnotab. Currently lnotab expects bytecode 
> sequencing to be roughly in-sync with the order of the source file and a 
> few things that the optimizer does (e.g. swapping the bodies of an 
> if/else after removing negation such that "if not X: A; else: B" becomes 
> "if X: B; else A") breaks this assumption. This will result in either an 
> assertion failure or incorrect line numbers being reported.

In http://bugs.python.org/issue2459 ("speedup for / while / if with better
bytecode") I had the same problem and decided to change the lnotab format so
that line number increments are signed bytes rather than unsigned. The proposed
patch contains many other changes, but with a bit of perseverance you may be
able to extract the lnotab-related modifications... ;)

This modification will allow many more types of control flow transformations
than the current scheme does.


By the way:
> swapping the bodies of an 
> if/else after removing negation such that "if not X: A; else: B" becomes 
> "if X: B; else A")

Is this really an optimization? "if" and "if not" should use the same number of
opcodes (the former produces JUMP_IF_FALSE and the latter JUMP_IF_TRUE).

Regards

Antoine.



From solipsis at pitrou.net  Thu Jul 24 17:43:30 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 24 Jul 2008 15:43:30 +0000 (UTC)
Subject: [Python-Dev] lnotab and the AST optimizer
References: <48889823.20506@vector-seven.com>
	<loom.20080724T151343-964@post.gmane.org>
Message-ID: <loom.20080724T153939-366@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> In http://bugs.python.org/issue2459 ("speedup for / while / if with better
> bytecode") I had the same problem and decided to change the lnotab format so
> that line number increments are signed bytes rather than unsigned.

By the way, the same change could be done for relative jump offsets in the
bytecode (change them from unsigned shorts to signed shorts). Taken together,
both modifications would release a lot of constraints on the ordering of code
blocks.



From tom at vector-seven.com  Thu Jul 24 17:49:37 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Fri, 25 Jul 2008 01:49:37 +1000
Subject: [Python-Dev] lnotab and the AST optimizer
In-Reply-To: <loom.20080724T151343-964@post.gmane.org>
References: <48889823.20506@vector-seven.com>
	<loom.20080724T151343-964@post.gmane.org>
Message-ID: <4888A491.4070108@vector-seven.com>

Antoine Pitrou wrote:
> Hi,
>
>   
Hi. Thanks for getting back to me so quickly. I can even respond before 
I have to drag myself off to bed. :)

>> I'm making some good progress with the AST optimizer, and now the main 
>> thing standing in my way is lnotab. Currently lnotab expects bytecode 
>> sequencing to be roughly in-sync with the order of the source file and a 
>> few things that the optimizer does (e.g. swapping the bodies of an 
>> if/else after removing negation such that "if not X: A; else: B" becomes 
>> "if X: B; else A") breaks this assumption. This will result in either an 
>> assertion failure or incorrect line numbers being reported.
>>     
>
> In http://bugs.python.org/issue2459 ("speedup for / while / if with better
> bytecode") I had the same problem and decided to change the lnotab format so
> that line number increments are signed bytes rather than unsigned. The proposed
> patch contains many other changes, but with a bit of perseverance you may be
> able to extract the lnotab-related modifications... ;)
>
> This modification will allow many more types of control flow transformations
> than the current scheme does.
>
>   
Great, thanks! I'll check it out next week.
> By the way:
>   
>> swapping the bodies of an 
>> if/else after removing negation such that "if not X: A; else: B" becomes 
>> "if X: B; else A")
>>     
>
> Is this really an optimization? "if" and "if not" should use the same number of
> opcodes (the former produces JUMP_IF_FALSE and the latter JUMP_IF_TRUE).
>
>   

Not quite. :) Even if we were producing a JUMP_IF_FALSE, it'd still be 
nice to optimize away the UNARY_NOT in the former.

In practice, both actually produce a JUMP_IF_TRUE due to an existing 
optimization in the peephole optimizer which does just that. I'm trying 
to emulate this at the AST level because I'm part of a secret, evil 
conspiracy to be rid of the peephole optimizer. Shh. The relevant code 
in the peepholer, plus comment:

            /* Replace UNARY_NOT JUMP_IF_FALSE POP_TOP with
               with    JUMP_IF_TRUE POP_TOP */
            case UNARY_NOT:
                if (codestr[i+1] != JUMP_IF_FALSE  ||
                    codestr[i+4] != POP_TOP  ||
                    !ISBASICBLOCK(blocks,i,5))
                    continue;
                tgt = GETJUMPTGT(codestr, (i+1));
                if (codestr[tgt] != POP_TOP)
                    continue;
                j = GETARG(codestr, i+1) + 1;
                codestr[i] = JUMP_IF_TRUE;
                SETARG(codestr, i, j);
                codestr[i+3] = POP_TOP;
                codestr[i+4] = NOP;
                break;

A little hackage with the dis module seems to confirm this is the case.

Of course, if you know of a situation where this optimization doesn't 
apply and we actually wind up with a JUMP_IF_FALSE for an if/else 
post-peephole, I'm all ears.

Thanks again!

Cheers,
T


From tom at vector-seven.com  Thu Jul 24 17:59:54 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Fri, 25 Jul 2008 01:59:54 +1000
Subject: [Python-Dev] lnotab and the AST optimizer
In-Reply-To: <loom.20080724T153939-366@post.gmane.org>
References: <48889823.20506@vector-seven.com>	<loom.20080724T151343-964@post.gmane.org>
	<loom.20080724T153939-366@post.gmane.org>
Message-ID: <4888A6FA.9050100@vector-seven.com>

Antoine Pitrou wrote:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
>   
>> In http://bugs.python.org/issue2459 ("speedup for / while / if with better
>> bytecode") I had the same problem and decided to change the lnotab format so
>> that line number increments are signed bytes rather than unsigned.
>>     
>
> By the way, the same change could be done for relative jump offsets in the
> bytecode (change them from unsigned shorts to signed shorts). Taken together,
> both modifications would release a lot of constraints on the ordering of code
> blocks.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com
>   
By the way, you were right about JUMP_IF_TRUE/JUMP_IF_FALSE. It's far 
too late. Apologies.

I'm still pretty sure this is the peepholer's doing, though, and if 
that's the case then I want to try and deal with it at the AST level.

Which is what's being achieved with the AST optimization I originally 
proposed, right?

Cheers,
T

From jcea at jcea.es  Thu Jul 24 18:03:20 2008
From: jcea at jcea.es (Jesus Cea)
Date: Thu, 24 Jul 2008 18:03:20 +0200
Subject: [Python-Dev] bsddb: Test failures on windows (HELP!)
In-Reply-To: <48874F2D.5070201@jcea.es>
References: <48874F2D.5070201@jcea.es>
Message-ID: <4888A7C8.9010308@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jesus Cea wrote:
| Trent, I was wondering if you could look at some test failures in MS
| Windows builds. I can't debug Windows issues myself :-(. This is a MS
| free environment...

I will be out of the city, 100% offline, until monday/tuesday. I will
read your suggestions and do some tests as soon as possible. Please,
keep the brainstorming going :)

Feel free to (minimally };-)) touch the testcases trying to improve the
situation. If that does the trick, please let me know to integrate the
changes in my code. Remember this code must work in python 2.[3-6]
(reason, for example, because I have my own "test_support" code, in
pybsddb).

Thanks a lot for your invaluable suggestions.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIinuplgi5GaxT1NAQK6qQP+LOv1lU6G6+GaSrxUqrnFM62bTcmXCMay
S0ic3rWYUL4YTvWT/Ips/qBgYvRCPl3uHnmIDia9UAOnYh3EYjkFN+/4GDofGwM+
1UBRu86C7LsYdJl2VlHJyHGWmz6tgbbtAue306CNX01yD+pwYsCUqMSTuzjiiNCx
/q1DHdJv8Qo=
=OpNx
-----END PGP SIGNATURE-----

From solipsis at pitrou.net  Thu Jul 24 18:15:57 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 24 Jul 2008 16:15:57 +0000 (UTC)
Subject: [Python-Dev] lnotab and the AST optimizer
References: <48889823.20506@vector-seven.com>	<loom.20080724T151343-964@post.gmane.org>
	<loom.20080724T153939-366@post.gmane.org>
	<4888A6FA.9050100@vector-seven.com>
Message-ID: <loom.20080724T160958-653@post.gmane.org>

Thomas Lee <tom <at> vector-seven.com> writes:
> By the way, you were right about JUMP_IF_TRUE/JUMP_IF_FALSE. It's far 
> too late. Apologies.
> 
> I'm still pretty sure this is the peepholer's doing,

Yes indeed.

> Which is what's being achieved with the AST optimization I originally 
> proposed, right?

Well, not exactly, your optimization eliminates the UNARY_NOT by swapping the
if/else blocks, while the peepholer eliminates the UNARY_NOT by fusing it with
the subsequent jump opcode. In this case it doesn't make much of a difference,
but if there is only an "if" without an "else", the peepholer's optimization is
still possible while yours is not.

(bottom line: the peepholer is not dead!)

cheers,

Antoine.



From tom at vector-seven.com  Thu Jul 24 18:23:00 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Fri, 25 Jul 2008 02:23:00 +1000
Subject: [Python-Dev] lnotab and the AST optimizer
In-Reply-To: <loom.20080724T160958-653@post.gmane.org>
References: <48889823.20506@vector-seven.com>	<loom.20080724T151343-964@post.gmane.org>	<loom.20080724T153939-366@post.gmane.org>	<4888A6FA.9050100@vector-seven.com>
	<loom.20080724T160958-653@post.gmane.org>
Message-ID: <4888AC64.5060303@vector-seven.com>

Antoine Pitrou wrote:
> Thomas Lee <tom <at> vector-seven.com> writes:
>   
>> By the way, you were right about JUMP_IF_TRUE/JUMP_IF_FALSE. It's far 
>> too late. Apologies.
>>
>> I'm still pretty sure this is the peepholer's doing,
>>     
>
> Yes indeed.
>
>   
>> Which is what's being achieved with the AST optimization I originally 
>> proposed, right?
>>     
>
> Well, not exactly, your optimization eliminates the UNARY_NOT by swapping the
> if/else blocks, while the peepholer eliminates the UNARY_NOT by fusing it with
> the subsequent jump opcode. In this case it doesn't make much of a difference,
> but if there is only an "if" without an "else", the peepholer's optimization is
> still possible while yours is not.
>
>   

Unless a pass is injected into the if body, which will generate no 
additional bytecode and still have the same net effect.

> (bottom line: the peepholer is not dead!)
>
>   
We'll see ;)

Thanks for all your help, I'm looking forward to getting my hands on 
that patch.

Cheers,
T


From pje at telecommunity.com  Thu Jul 24 18:34:26 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 24 Jul 2008 12:34:26 -0400
Subject: [Python-Dev] lnotab and the AST optimizer
In-Reply-To: <48889823.20506@vector-seven.com>
References: <48889823.20506@vector-seven.com>
Message-ID: <20080724163339.D053F3A406B@sparrow.telecommunity.com>

At 12:56 AM 7/25/2008 +1000, Thomas Lee wrote:
>I'm making some good progress with the AST optimizer, and now the 
>main thing standing in my way is lnotab. Currently lnotab expects 
>bytecode sequencing to be roughly in-sync with the order of the 
>source file and a few things that the optimizer does (e.g. swapping 
>the bodies of an if/else after removing negation such that "if not 
>X: A; else: B" becomes "if X: B; else A") breaks this assumption. 
>This will result in either an assertion failure or incorrect line 
>numbers being reported.
>
>It seems that lnotab is used in relatively few places in the source 
>code at the moment, but if I'm going to make a change to how lnotab 
>works I want to do so in a way that's going to allow me to move 
>forward while keeping everybody happy.
>
>I'm away for a few days so I probably won't be able to get back to 
>anybody until either Sunday or Monday, but I'd appreciate it if 
>anybody in the know can weigh in on this.

I'd personally love it if the lnotab were capable of handling line 
numbers from different files as well as out-of-order lines.  (For 
function inlining, among other more esoteric things.)


From josiah.carlson at gmail.com  Thu Jul 24 19:26:39 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Thu, 24 Jul 2008 10:26:39 -0700
Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was
	Re: No beta2 tonight)
In-Reply-To: <48845653.5000100@jcea.es>
References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org>
	<ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com>
	<e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com>
	<F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org>
	<e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com>
	<4881C04E.5060208@gmail.com>
	<e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com>
	<4882986A.3060509@jcea.es>
	<e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com>
	<48845653.5000100@jcea.es>
Message-ID: <e6511dbf0807241026qc2aa105o11a3b6ca8bdd5a5a@mail.gmail.com>

On Mon, Jul 21, 2008 at 2:26 AM, Jesus Cea <jcea at jcea.es> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Josiah Carlson wrote:
> | I'm still curious as to what deep features people are using in bsddb.
> | Anyone have any pointers to open source software?
>
> I'm using replication and distributed transactions. Database encryption
> and page integrity checks. Abusing the master election BDB
> infrastructure for some evil but fun purposes. Managing databases in the
> 200 terabyte range. Locking & logging infrastructure to implement
> application logic integrated with database operation. MVCC everywhere.
>
> Nothing I can show you without killing you after, though.

That's the kind of answer I was looking for :).  Though I don't know
if you are the "typical user"; I don't have enough data (the few
others that sent messages here and privately weren't using it to your
extent).

Some of those features are possible to add without huge amounts of
work (an afternoon or two), but indeed, some of those are not possible
without substantial investment and testing.


> Independently of current bsddb usage profile, current 2.6 code support
> for distributed transactions and replication enables new application
> uses that sqlite can't match.
>
> Since I'm taking full responsability over bsddb both in 2.6 and 3.0, I
> don't really see what the issue is. Past mistakes and maintenance
> nightmares WILL NOT be repeated.

Your taking over of maintenance is part of the reason why I withdrew
my offer ;) .

 - Josiah

From loisel at temple.edu  Thu Jul 24 23:19:27 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Thu, 24 Jul 2008 17:19:27 -0400
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
Message-ID: <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>

Greg Ewing said:

> I would actually be in favour of adding a matrix multiplication
> operator

That would be helpful to me, for my students as well as my papers.

Sincerely,

--
S?bastien Loisel

From scott+python-dev at scottdial.com  Fri Jul 25 00:06:10 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Thu, 24 Jul 2008 18:06:10 -0400
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
Message-ID: <4888FCD2.6060908@scottdial.com>

Sebastien Loisel wrote:
> Greg Ewing said:
>> I would actually be in favour of adding a matrix multiplication
>> operator
> 
> That would be helpful to me, for my students as well as my papers.
> 

Perhaps I'm nobody, but I think this would be ridiculous. Matrices are
not native objects to the language. There is no type(matrix). The notion
of what makes a Python object a matrix is a convention and to have
built-in operators dedicated to such objects makes no sense. There are
multiple ways to stuff matrices into Python. Please submit a PEP for a
type(matrix) first. Until a matrix is a first-order object in Python,
there is no logic to making operators for them.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From barry at barrys-emacs.org  Fri Jul 25 00:33:46 2008
From: barry at barrys-emacs.org (Barry Scott)
Date: Thu, 24 Jul 2008 23:33:46 +0100
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org>
	<4884581F.8010201@jcea.es>
	<bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com>
	<319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com>
Message-ID: <BAF14EE0-6706-40F9-90F1-EBB97EEAB28E@barrys-emacs.org>


On Jul 21, 2008, at 22:37, Lennart Regebro wrote:

> On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote:
>> But waiting until all the betas have gone out totally defeats the
>> purpose of the betas!
>
> I agree. Writing an actual *guide* can wait, but documenting the
> differences with code examples is a work that can start now, and I
> agree that it would be great if this would start now.
>
> But writing a guide might not be a good idea until we know what the
> changes are, and if the API is changing quickly now we don't. :-)

I'm working on  getting a version of PyCXX working with Python 3.0.
The lack of any docs outside of the header files is making this a slow
process.

I think its a mistake to expect a serious beta test of extensions
when you have no guide to the changes in the C API.

If you had a guide then diff it between releases would be a guide to
what need fixing up going from beta to beta to rc.

Oh and I'm not going to try and make a version of PyCXX that works
on 2.x and 3.x as the changes are too fundamental.

Barry


From fredrik.johansson at gmail.com  Fri Jul 25 02:02:41 2008
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Fri, 25 Jul 2008 02:02:41 +0200
Subject: [Python-Dev] Infix operators
In-Reply-To: <4888FCD2.6060908@scottdial.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
	<4888FCD2.6060908@scottdial.com>
Message-ID: <3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com>

On Fri, Jul 25, 2008 at 12:06 AM, Scott Dial
<scott+python-dev at scottdial.com> wrote:

> Perhaps I'm nobody, but I think this would be ridiculous. Matrices are
> not native objects to the language. There is no type(matrix). The notion
> of what makes a Python object a matrix is a convention and to have
> built-in operators dedicated to such objects makes no sense. There are
> multiple ways to stuff matrices into Python. Please submit a PEP for a
> type(matrix) first. Until a matrix is a first-order object in Python,
> there is no logic to making operators for them.

Though I would personally find a matrix multiplication operator
useful, I have to agree with this.

Anyway, it is easy to define pseudo-operators in Python; just create
an Operator class and implement its __mul__ and __rmul__ methods
appropriately (there are recipes for this around somewhere). Then you
can define various custom multiplication operators with syntax like
this:

A *matrixmul* B
A *dot* B
A *cross* B
A *elementwise* B

Some other fun possibilities:

A +concat+ B
A /solve/ B
A **left_inverse** (-1)
A **right_inverse** (-1)
x **tetrate** y
n |choose| k

Fredrik

From greg.ewing at canterbury.ac.nz  Fri Jul 25 03:26:00 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Jul 2008 13:26:00 +1200
Subject: [Python-Dev] lnotab and the AST optimizer
In-Reply-To: <48889823.20506@vector-seven.com>
References: <48889823.20506@vector-seven.com>
Message-ID: <48892BA8.2090603@canterbury.ac.nz>

Thomas Lee wrote:
> I'm making some good progress with the AST optimizer, and now the main 
> thing standing in my way is lnotab.

My suggestion would be to drop the idea of trying to
compress the lnotab in clever ways, and just make it
a straightforward list of bytecode offset/line number
pairs. I can't imagine that the size of an uncompressed
lnotab would be a problem in this day and age.

If ordering is an issue, generate it internally as a
dict and convert it to a sorted list on output.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Jul 25 04:02:30 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Jul 2008 14:02:30 +1200
Subject: [Python-Dev] Infix operators
In-Reply-To: <4888FCD2.6060908@scottdial.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
	<4888FCD2.6060908@scottdial.com>
Message-ID: <48893436.5030805@canterbury.ac.nz>

Scott Dial wrote:

> Perhaps I'm nobody, but I think this would be ridiculous. Matrices are
> not native objects to the language.

Why should that matter? We already have things like
sum(), which operates on any sequence of numbers,
without needing a special "array of numbers" data
type. I don't see why one shouldn't be able to
perform matrix multiplication on a plausible
representation of a matrix using the existing
built-in sequence types.

> Until a matrix is a first-order object in Python,
> there is no logic to making operators for them.

If there were a dedicated matrix type, there would
actually be *less* reason to have a new operator,
since that type could just implement * as matrix
multiplication.

However, there are disadvantages to that, which
become apparent when numpy is considered. There are
good reasons for wanting * to mean elementwise
multiplication of numpy arrays. There are also
good reasons for wanting to use numpy arrays as
matrices, since you get the benefit of all the
other powerful things that numpy can do with
arrays.

You can use a matrix type that's based somehow
on a numpy array, but there are problems with that
too. E.g. if you add a matrix and a plain numpy
array, what type should the result be? If plain
numpy arrays can be used directly as matrices, that
problem goes away.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Jul 25 04:08:42 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Jul 2008 14:08:42 +1200
Subject: [Python-Dev] Infix operators
In-Reply-To: <3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
	<4888FCD2.6060908@scottdial.com>
	<3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com>
Message-ID: <488935AA.6070509@canterbury.ac.nz>

Fredrik Johansson wrote:

> Anyway, it is easy to define pseudo-operators in Python;
> 
> A *matrixmul* B
> A *dot* B
> A *cross* B
> A *elementwise* B

Urg. This is another one of those recipes that I consider
is too clever for its own good. Very nice in theory,
but I would never use it in real life.

What's more, it doesn't address the real problem at hand,
which is providing a notation for matrix multiplication
that is concise enough to be used *very* frequently --
like multiple times in every line of code that manipulates
matrices.

-- 
Greg

From scott+python-dev at scottdial.com  Fri Jul 25 04:46:24 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Thu, 24 Jul 2008 22:46:24 -0400
Subject: [Python-Dev] Infix operators
In-Reply-To: <48893436.5030805@canterbury.ac.nz>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>	<4888FCD2.6060908@scottdial.com>
	<48893436.5030805@canterbury.ac.nz>
Message-ID: <48893E80.6030907@scottdial.com>

Greg Ewing wrote:
> Scott Dial wrote:
>> Perhaps I'm nobody, but I think this would be ridiculous. Matrices are
>> not native objects to the language.
> 
> Why should that matter? We already have things like
> sum(), which operates on any sequence of numbers,
> without needing a special "array of numbers" data
> type.

I would argue that Python contains a "array of some_type" data type. 
That sum() performs a left-fold of __add__ on the array is completely 
independent of them being numbers. In all fact, they could be any type 
that supports __add__/__radd__ or even a mix of types. I do not believe 
"array of numbers" to be analogically equivalent to a "matrix of 
numbers". We have an "array of" type in Python, we do not have a "matrix 
of" type in Python. sum() is not an operator, it is a builtin; the 
suggestion was for there to be an operator, not a builtin. If you want 
to suggest there be a mmul() builtin, then perhaps there is a viable 
answer there, despite the lack of a "matrix of" type still.

> I don't see why one shouldn't be able to
> perform matrix multiplication on a plausible
> representation of a matrix using the existing
> built-in sequence types.

What is "a plausible representation of a matrix"? Is it a list of lists? 
Row-major (m[1][2]) or column-major (m[2][1])? Is it a dictionary of 
tuple'd indices (m[1,2])? Also, You went on to talk about wanting to 
using  numpy.array.

How does this not make it clear that there is not a case of TOOWTDI?

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From josiah.carlson at gmail.com  Fri Jul 25 04:47:24 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Thu, 24 Jul 2008 19:47:24 -0700
Subject: [Python-Dev] Infix operators
In-Reply-To: <488935AA.6070509@canterbury.ac.nz>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
	<4888FCD2.6060908@scottdial.com>
	<3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com>
	<488935AA.6070509@canterbury.ac.nz>
Message-ID: <e6511dbf0807241947q479e8459y7d424be4ffef2f77@mail.gmail.com>

On Thu, Jul 24, 2008 at 7:08 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Fredrik Johansson wrote:
>
>> Anyway, it is easy to define pseudo-operators in Python;
>>
>> A *matrixmul* B
>> A *dot* B
>> A *cross* B
>> A *elementwise* B
>
> Urg. This is another one of those recipes that I consider
> is too clever for its own good. Very nice in theory,
> but I would never use it in real life.

That's unfortunate ;), because it's available in Python today, and
with the common local caching of globals, can be as short as 3
characters (M for matrixmul, D for dot, X for cross, E for
elementwise).

> What's more, it doesn't address the real problem at hand,
> which is providing a notation for matrix multiplication
> that is concise enough to be used *very* frequently --
> like multiple times in every line of code that manipulates
> matrices.

This is the first time anyone has mentioned "conciseness" in this
thread.  And what's a 3 character operator between friends?  The "and"
and "not" operators don't seem to be bothered by three characters. ;)

 - Josiah

From musiccomposition at gmail.com  Fri Jul 25 04:53:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 24 Jul 2008 21:53:37 -0500
Subject: [Python-Dev] Warnings for intra-package imports
Message-ID: <1afaf6160807241953s4a4b2d5aq5e704ae852a0ad12@mail.gmail.com>

I was just reading up on PEP 328. In the "Timeline" section, it
mentions that intra-package imports should raise a DeprecationWarning
in 2.6. This doesn't seem to be implemented currently.

Is this still the plan? I would like to see Py3k warnings for these
kinds of imports at least.

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From brett at python.org  Fri Jul 25 06:17:30 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 25 Jul 2008 00:17:30 -0400
Subject: [Python-Dev] Warnings for intra-package imports
In-Reply-To: <1afaf6160807241953s4a4b2d5aq5e704ae852a0ad12@mail.gmail.com>
References: <1afaf6160807241953s4a4b2d5aq5e704ae852a0ad12@mail.gmail.com>
Message-ID: <bbaeab100807242117y19adb39g805b9bc20f60d4a5@mail.gmail.com>

On Thu, Jul 24, 2008 at 10:53 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> I was just reading up on PEP 328. In the "Timeline" section, it
> mentions that intra-package imports should raise a DeprecationWarning
> in 2.6. This doesn't seem to be implemented currently.
>
> Is this still the plan? I would like to see Py3k warnings for these
> kinds of imports at least.
>

It should still be the plan.

-Brett

From greg.ewing at canterbury.ac.nz  Fri Jul 25 07:27:06 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Jul 2008 17:27:06 +1200
Subject: [Python-Dev] Infix operators
In-Reply-To: <48893E80.6030907@scottdial.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
	<4888FCD2.6060908@scottdial.com> <48893436.5030805@canterbury.ac.nz>
	<48893E80.6030907@scottdial.com>
Message-ID: <4889642A.8040204@canterbury.ac.nz>

Scott Dial wrote:

> I would argue that Python contains a "array of some_type" data type. 
> That sum() performs a left-fold of __add__ on the array is completely 
> independent of them being numbers.

That's not strictly true -- it explicitly refuses to operate
on strings (or at least it did last time I heard). Guido has
said that he *intends* it to only be used for numbers, even
if it happens to accidentally do something with other types.

However, as I envisage it, the @ operator wouldn't be
restricted to numbers either -- it would do whatever the
* and + operations do on the elements.

 > sum() is not an operator, it is a builtin; the
> suggestion was for there to be an operator, not a builtin.

That's true, and it means that the built-in implementation
wouldn't have as wide applicability as the sum() function,
since it would be restricted to lists and tuples (and
perhaps array.array instances). But I don't think that's a
fatal flaw -- if you create your own sequence type, you
have to take steps if you want to be able e.g. to use +
to concatenate them, and nobody complains about that.

> If you want 
> to suggest there be a mmul() builtin, then perhaps there is a viable 
> answer there

No, that's not a viable answer to people who want a matrix
multiplication operator. They want an operator because a
function is nowhere near concise enough. Telling these
people they have to use a function to multiply matrices
is like telling them they have to use a function to
multiply numbers.

> What is "a plausible representation of a matrix"? Is it a list of lists?

That's one. A tuple of tuples would be another.

> Row-major (m[1][2]) or column-major (m[2][1])?

Pick one and stick to it. Probably row-major, since it's
what the numpy matrixmultiply function uses.

> Is it a dictionary of tuple'd indices (m[1,2])?

I think dictionaries would have to be excluded, because
there's no easy way of finding out the dimensions, it's
not obvious what to do about missing elements, etc.

> Also, You went on to talk about wanting to using numpy.array.

Yes -- what's wrong with that?

> How does this not make it clear that there is not a case of TOOWTDI?

I think there *is* one obvious way of representing a matrix,
or any 2D array, using built-in Python types, or rather two
ways -- a list of lists if you want mutability, or a tuple
of tuples if you want immutability.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Jul 25 07:30:45 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Jul 2008 17:30:45 +1200
Subject: [Python-Dev] Infix operators
In-Reply-To: <e6511dbf0807241947q479e8459y7d424be4ffef2f77@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com>
	<b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com>
	<b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com>
	<4888FCD2.6060908@scottdial.com>
	<3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com>
	<488935AA.6070509@canterbury.ac.nz>
	<e6511dbf0807241947q479e8459y7d424be4ffef2f77@mail.gmail.com>
Message-ID: <48896505.5080205@canterbury.ac.nz>

Josiah Carlson wrote:

> This is the first time anyone has mentioned "conciseness" in this
> thread.

I thought it more or less went without saying. After
all, if conciseness isn't a goal, there's nothing
wrong with a plain function call, which can be as short
as 3 characters as well.

The trouble is that, for the people who badly want
a matrix multiplication operator, 3 characters is
far too long!

-- 
Greg

From status at bugs.python.org  Fri Jul 25 18:06:29 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 25 Jul 2008 18:06:29 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080725160629.C01FD78350@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (07/18/08 - 07/25/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1951 open (+29) / 13318 closed (+11) / 15269 total (+40)

Open issues with patches:   615

Average duration of open issues: 710 days.
Median duration of open issues: 1621 days.

Open Issues Breakdown
   open  1937 (+28)
pending    14 ( +1)

Issues Created Or Reopened (44)
_______________________________

function annotation for builtin and C function                   07/21/08
       http://bugs.python.org/issue3208    reopened amaury.forgeotdarc        
                                                                               

`./configure --enable-framework --enable-universalsdk` fails bec 07/21/08
CLOSED http://bugs.python.org/issue3381    reopened georg.brandl              
       patch                                                                   

test_urllib2_localnet fails on MacOS X 10.4.11 (Intel)           07/18/08
       http://bugs.python.org/issue3407    created  MrJean1                   
                                                                               

urllib incomplete and urllib2 does not exist                     07/18/08
CLOSED http://bugs.python.org/issue3408    created  vizcayno                  
                                                                               

ElementPath.Path.findall problem with unicode input              07/18/08
       http://bugs.python.org/issue3409    created  qual                      
       patch                                                                   

platform.version() don't work as expected in Vista in portuguese 07/18/08
       http://bugs.python.org/issue3410    created  portella                  
                                                                               

str.format() on negative floats                                  07/18/08
CLOSED http://bugs.python.org/issue3411    created  hagen                     
                                                                               

Fraction and Decimal in the Tutorial                             07/18/08
       http://bugs.python.org/issue3412    created  segfaulthunter            
                                                                               

typo in Mac/Makefile.in breaks installing IDLE                   07/19/08
CLOSED http://bugs.python.org/issue3413    created  erickt                    
                                                                               

frameworkinstall doesn't create Python.app, which breaks python  07/19/08
CLOSED http://bugs.python.org/issue3414    created  erickt                    
                                                                               

Interpreter error when running a script under debugger control   07/19/08
CLOSED http://bugs.python.org/issue3415    created  nirai                     
                                                                               

Wrong inherit in PickleHTMLBuilder                               07/19/08
CLOSED http://bugs.python.org/issue3416    created  is                        
                                                                               

make the fix_dict fixer smarter                                  07/19/08
       http://bugs.python.org/issue3417    created  benjamin.peterson         
       patch                                                                   

heavy resource usage with string functions                       07/19/08
CLOSED http://bugs.python.org/issue3418    created  mgogoulos                 
                                                                               

multiprocessing module is racy                                   07/19/08
       http://bugs.python.org/issue3419    created  cartman                   
                                                                               

2to3 fails to run on Mac OS X 10.4 PPC 3.0b2                     07/20/08
       http://bugs.python.org/issue3420    created  barry-scott               
                                                                               

Test failure in test_math::testSum                               07/20/08
       http://bugs.python.org/issue3421    created  georg.brandl              
                                                                               

sphinx.doc.autodoc: Hook for changing argspec                    07/20/08
       http://bugs.python.org/issue3422    created  pv                        
       patch                                                                   

DeprecationWarning message applies to wrong context with exec()  07/22/08
       http://bugs.python.org/issue3423    created  ghazel                    
                                                                               

imghdr test order makes it slow                                  07/22/08
       http://bugs.python.org/issue3424    created  biny                      
                                                                               

posixmodule.c always using res = utime(path, NULL)               07/22/08
       http://bugs.python.org/issue3425    created  oskar86                   
                                                                               

os.path.abspath with unicode argument should call os.getcwdu     07/22/08
       http://bugs.python.org/issue3426    created  saturn_mimas              
                                                                               

urllib documentation: urlopen().info() return type               07/22/08
       http://bugs.python.org/issue3427    created  ThomasH                   
                                                                               

httplib.HTTPMessage undocumented                                 07/22/08
       http://bugs.python.org/issue3428    created  ThomasH                   
                                                                               

urllib.urlopen() return type                                     07/22/08
       http://bugs.python.org/issue3429    created  ThomasH                   
                                                                               

httplib.HTTPResponse documentations inconsistent                 07/22/08
       http://bugs.python.org/issue3430    created  ThomasH                   
                                                                               

multiprocessing uses Pickler.dispatch which isn't in 3.0 _pickle 07/23/08
CLOSED http://bugs.python.org/issue3431    created  georg.brandl              
                                                                               

Mac, 2.6 framework install error                                 07/24/08
       http://bugs.python.org/issue3432    created  robind                    
                                                                               

Mac, 3.0 framework install error with fink cp                    07/24/08
       http://bugs.python.org/issue3433    created  robind                    
                                                                               

Mac, 3.0 framework install, Python.app not created               07/24/08
CLOSED http://bugs.python.org/issue3434    created  robind                    
                                                                               

trace.py tries to get coverage data from non Python files        07/24/08
       http://bugs.python.org/issue3435    created  Gustavo                   
       patch                                                                   

csv.DictReader inconsistency                                     07/24/08
       http://bugs.python.org/issue3436    created  mishok13                  
       patch                                                                   

robotparser.py missing one line                                  07/24/08
       http://bugs.python.org/issue3437    created  mbloore                   
                                                                               

PyCF_DONT_IMPLY_DEDENT can be used to activate the with statemen 07/24/08
       http://bugs.python.org/issue3438    created  gpolo                     
       patch                                                                   

math.frexp and obtaining the bit size of a large integer         07/24/08
       http://bugs.python.org/issue3439    created  fredrikj                  
       patch                                                                   

Starting Python as a subprocess fails when subprocess.Popen has  07/24/08
       http://bugs.python.org/issue3440    created  kermode                   
                                                                               

Regression in "module as a script" command-line option           07/25/08
CLOSED http://bugs.python.org/issue3441    created  richard                   
                                                                               

wsgiref.validate.InputWrapper.readline does not accept optional  07/25/08
CLOSED http://bugs.python.org/issue3442    created  xaka                      
                                                                               

crash on badly initialised AttributeError                        07/25/08
       http://bugs.python.org/issue3443    created  scoder                    
                                                                               

add warnings for intra-package imports                           07/25/08
       http://bugs.python.org/issue3444    created  benjamin.peterson         
                                                                               

functools.update_wrapper bug                                     07/25/08
       http://bugs.python.org/issue3445    created  Antoine d'Otreppe         
                                                                               

center, ljust and rjust are inconsistent with unicode parameters 07/25/08
       http://bugs.python.org/issue3446    created  ignas                     
                                                                               

list(obj) can swallow KeyboardInterrupt                          07/18/08
       http://bugs.python.org/issue1242657 reopened georg.brandl              
                                                                               

long(float('nan'))!=0L                                           07/21/08
       http://bugs.python.org/issue1481296 reopened georg.brandl              
       patch                                                                   



Issues Now Closed (61)
______________________

python 2.4.4 fails on solaris (sun4u sparc SUNW,Sun-Fire-880)     264 days
       http://bugs.python.org/issue1363    georg.brandl              
                                                                               

os.system() fails for commands with multiple quoted file names    237 days
       http://bugs.python.org/issue1524    likes                     
                                                                               

socket functions that should return unsigned int return signed i  203 days
       http://bugs.python.org/issue1711    georg.brandl              
                                                                               

PythonLauncher not working correctly on OS X 10.5/Leopad          180 days
       http://bugs.python.org/issue1905    georg.brandl              
                                                                               

UnboundLocalError when trying to raise exceptions inside execfil  126 days
       http://bugs.python.org/issue2378    amaury.forgeotdarc        
       patch                                                                   

[py3k] Integer floor division (//): small int check omitted       128 days
       http://bugs.python.org/issue2417    facundobatista            
       patch                                                                   

Py3.0a4 wsgiref simple_server failed to start                     109 days
       http://bugs.python.org/issue2566    pitrou                    
                                                                               

setuptools gets site-packages wrong on Mac                         96 days
       http://bugs.python.org/issue2641    georg.brandl              
                                                                               

2to3 discards comments before import statements                    64 days
       http://bugs.python.org/issue2894    georg.brandl              
       patch                                                                   

idlelib/EditorWindow.py uses xrange()                              61 days
       http://bugs.python.org/issue2913    georg.brandl              
       patch                                                                   

urlgrabber.grabber calls setdefaulttimeout                         66 days
       http://bugs.python.org/issue2916    georg.brandl              
                                                                               

Wrong unicode size detection in pybench                            40 days
       http://bugs.python.org/issue3092    pitrou                    
       patch                                                                   

overzealous garbage collector (dict)                               37 days
       http://bugs.python.org/issue3104    georg.brandl              
                                                                               

Document exception chaining                                        35 days
       http://bugs.python.org/issue3113    georg.brandl              
                                                                               

ConfigParsers are classic classes                                  27 days
       http://bugs.python.org/issue3181    georg.brandl              
       patch                                                                   

re.compile fails with some bytes patterns                          24 days
       http://bugs.python.org/issue3231    pitrou                    
       patch                                                                   

segfault on gettext(None)                                          13 days
       http://bugs.python.org/issue3302    georg.brandl              
       patch                                                                   

invalid ref count on locale.strcoll() error                        14 days
       http://bugs.python.org/issue3303    georg.brandl              
       patch                                                                   

invalid check of _bsddb creation failure                           12 days
       http://bugs.python.org/issue3307    jcea                      
       patch                                                                   

pystone.main(10) causes ZeroDivisionError                          11 days
       http://bugs.python.org/issue3319    georg.brandl              
       patch                                                                   

bugs in scanstring_str() and scanstring_unicode() of _json modul   11 days
       http://bugs.python.org/issue3322    georg.brandl              
       patch                                                                   

Clarify __slots__ behaviour when inheriting                        10 days
       http://bugs.python.org/issue3323    georg.brandl              
                                                                               

webbrowser module doesn't correctly handle '|' character.          10 days
       http://bugs.python.org/issue3330    georg.brandl              
                                                                               

Possible inconsistency in behavior of list comprehensions vs. ge    9 days
       http://bugs.python.org/issue3331    georg.brandl              
                                                                               

2to3 looses indentation on import fix                               9 days
       http://bugs.python.org/issue3334    georg.brandl              
       patch                                                                   

test_ossaudiodev fails                                              6 days
       http://bugs.python.org/issue3346    benjamin.peterson         
                                                                               

urllib.robotparser doesn't work in Py3k                             6 days
       http://bugs.python.org/issue3347    jhylton                   
       patch                                                                   

Display bug in :show-inheritance: for class with standard docstr    4 days
       http://bugs.python.org/issue3355    georg.brandl              
                                                                               

add 'rbU' mode to open()                                           10 days
       http://bugs.python.org/issue3359    amaury.forgeotdarc        
                                                                               

Memory leak in import.c                                             4 days
       http://bugs.python.org/issue3368    georg.brandl              
       patch, patch, easy                                                      

memory leak in floatobject.c                                        6 days
       http://bugs.python.org/issue3369    marketdickinson           
       patch, patch, easy                                                      

Memory leak in pythonrun.c                                          3 days
       http://bugs.python.org/issue3378    georg.brandl              
       patch, patch                                                            

`./configure --enable-framework --enable-universalsdk` fails bec    1 days
       http://bugs.python.org/issue3381    trentm                    
       patch                                                                   

Documentation for re.findall and re.finditer lacks "ordering" in    3 days
       http://bugs.python.org/issue3384    jkugler                   
                                                                               

undefined array passed to CryptGenRandomBytes                       5 days
       http://bugs.python.org/issue3387    amaury.forgeotdarc        
       patch, patch, easy                                                      

[PATCH] replace last has_key in unittest by in operator             1 days
       http://bugs.python.org/issue3390    georg.brandl              
       patch                                                                   

rlcompleter can't autocomplete members of callable objects          4 days
       http://bugs.python.org/issue3396    facundobatista            
                                                                               

dis module: undocumented new bytecodes                              3 days
       http://bugs.python.org/issue3400    georg.brandl              
                                                                               

urllib incomplete and urllib2 does not exist                        0 days
       http://bugs.python.org/issue3408    georg.brandl              
                                                                               

str.format() on negative floats                                     0 days
       http://bugs.python.org/issue3411    eric.smith                
                                                                               

typo in Mac/Makefile.in breaks installing IDLE                      0 days
       http://bugs.python.org/issue3413    benjamin.peterson         
                                                                               

frameworkinstall doesn't create Python.app, which breaks python     0 days
       http://bugs.python.org/issue3414    georg.brandl              
                                                                               

Interpreter error when running a script under debugger control      2 days
       http://bugs.python.org/issue3415    amaury.forgeotdarc        
                                                                               

Wrong inherit in PickleHTMLBuilder                                  0 days
       http://bugs.python.org/issue3416    georg.brandl              
                                                                               

heavy resource usage with string functions                          0 days
       http://bugs.python.org/issue3418    loewis                    
                                                                               

multiprocessing uses Pickler.dispatch which isn't in 3.0 _pickle    0 days
       http://bugs.python.org/issue3431    alexandre.vassalotti      
                                                                               

Mac, 3.0 framework install, Python.app not created                  0 days
       http://bugs.python.org/issue3434    georg.brandl              
                                                                               

Regression in "module as a script" command-line option              0 days
       http://bugs.python.org/issue3441    benjamin.peterson         
                                                                               

wsgiref.validate.InputWrapper.readline does not accept optional     0 days
       http://bugs.python.org/issue3442    pje                       
                                                                               

pydoc.Helper.topics not based on docs                            2096 days
       http://bugs.python.org/issue628258  georg.brandl              
                                                                               

test_htmlparser -- more robust SCRIPT tag handling               2003 days
       http://bugs.python.org/issue674449  georg.brandl              
       patch                                                                   

copy_reg globals in cPickle                                      1804 days
       http://bugs.python.org/issue787077  georg.brandl              
                                                                               

SimpleHTTPServer reports wrong content-length for text files       37 days
       http://bugs.python.org/issue839496  georg.brandl              
       patch, easy                                                             

(ref-manual) position docstrings in source not documented        1572 days
       http://bugs.python.org/issue926501  georg.brandl              
                                                                               

tp_subclasses grow without bounds                                1485 days
       http://bugs.python.org/issue980092  georg.brandl              
                                                                               

Make Generators Pickle-able                                      1299 days
       http://bugs.python.org/issue1092962 georg.brandl              
                                                                               

Add proxies arg to urllib.urlretrieve                            1170 days
       http://bugs.python.org/issue1197207 georg.brandl              
       patch                                                                   

floating point literals don't work in non-US locale in 2.5        935 days
       http://bugs.python.org/issue1391872 georg.brandl              
                                                                               

64-bit Universal Binary build broken                              578 days
       http://bugs.python.org/issue1619130 georg.brandl              
                                                                               

terminalcommand doesn't work under Darwin                         513 days
       http://bugs.python.org/issue1666952 georg.brandl              
                                                                               

sqlite3.dll cannot be relocated                                   409 days
       http://bugs.python.org/issue1733134 georg.brandl              
                                                                               



Top Issues Most Discussed (10)
______________________________

 10 add 'rbU' mode to open()                                          10 days
closed  http://bugs.python.org/issue3359   

 10 function annotation for builtin and C function                     4 days
open    http://bugs.python.org/issue3208   

  9 Add gamma and error functions to math module                      10 days
open    http://bugs.python.org/issue3366   

  9 Clean up usage of map() in the stdlib                            759 days
open    http://bugs.python.org/issue1513299

  8 make the fix_dict fixer smarter                                    6 days
open    http://bugs.python.org/issue3417   

  7 bugs in scanstring_str() and scanstring_unicode() of _json modu   11 days
closed  http://bugs.python.org/issue3322   

  7 bsddb/test/test_replication.py bus error, segfault, assertion e   62 days
open    http://bugs.python.org/issue2960   

  7 Crash in PyObject_Malloc                                         370 days
open    http://bugs.python.org/issue1758146

  6 binary buffered reading is quadratic                             116 days
open    http://bugs.python.org/issue2523   

  5 math.frexp and obtaining the bit size of a large integer           1 days
open    http://bugs.python.org/issue3439   




From jek-gmane at kleckner.net  Sat Jul 26 03:22:24 2008
From: jek-gmane at kleckner.net (Jim Kleckner)
Date: Fri, 25 Jul 2008 18:22:24 -0700
Subject: [Python-Dev] Status of mingw and Python 2.6 ?
Message-ID: <g6du8g$cue$1@ger.gmane.org>

I gave it a try with cygwin-hosted mingw just to see
if that would work as an alternative to VS2008/VC9 to
figure out some linkage problems.

I tried:
  python setup.py build_ext --compiler mingw32

and got a version string issue noted below which
rejects the version string printed from the loader
ld as an inappropriate form.

I seem to recall that mingw was expected to work with 2.6.
Is it?

Thanks - Jim



running build_ext
Traceback (most recent call last):
   File "finsim/setup.py", line 96, in <module>
     package_dir = {'finsim': 'finsim'},
   File "c:\Python26\lib\distutils\core.py", line 152, in setup
     dist.run_commands()
   File "c:\Python26\lib\distutils\dist.py", line 972, in run_commands
     self.run_command(cmd)
   File "c:\Python26\lib\distutils\dist.py", line 992, in run_command
     cmd_obj.run()
   File "c:\Python26\lib\distutils\command\build_ext.py", line 309, in run
     force=self.force)
   File "c:\Python26\lib\distutils\ccompiler.py", line 1175, in new_compiler
     return klass (None, dry_run, force)
   File "c:\Python26\lib\distutils\cygwinccompiler.py", line 306, in 
__init__
     CygwinCCompiler.__init__ (self, verbose, dry_run, force)
   File "c:\Python26\lib\distutils\cygwinccompiler.py", line 107, in 
__init__
     get_versions()
   File "c:\Python26\lib\distutils\cygwinccompiler.py", line 430, in 
get_versions
     ld_version = StrictVersion(result.group(1))
   File "c:\Python26\lib\distutils\version.py", line 40, in __init__
     self.parse(vstring)
   File "c:\Python26\lib\distutils\version.py", line 107, in parse
     raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number '2.18.50.20080523'


From greg.ewing at canterbury.ac.nz  Sat Jul 26 03:50:58 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 26 Jul 2008 13:50:58 +1200
Subject: [Python-Dev] Matrix product
In-Reply-To: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
Message-ID: <488A8302.3050408@canterbury.ac.nz>

Sebastien Loisel wrote:

> What are the odds of this thing going in?

I don't know. Guido has said nothing about it so far this
time round, and his is the only opinion that matters in the
end.

I may write a PEP about this. However, since yesterday I've
realised that there's a rather serious problem with part
of my proposal.

The problem is that being able to multiply matrices isn't
much use without also being able to add them and multiply
them by numbers, which obviously can't work with the
built-in sequence types, since they already have other
meanings for + and *.

However, I still think that adding an @ operator for numpy
to use is a good idea. So I'm now suggesting that the
operator be added, with the intended meaning of matrix
multiplication, but that no implementation of it be
provided in core Python.

There is a precedent for this -- the ellipsis notation was
added purely for use by Numeric and its successors, and
nothing in core Python attaches any meaning to it.

> How do the PEPs work?

Someone writes a PEP. People talk about it. Eventually, Guido
either accepts it or rejects it (although in some cases it
is an infinitely long time before that happens:-).

-- 
Greg

From amauryfa at gmail.com  Sat Jul 26 09:19:32 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Sat, 26 Jul 2008 09:19:32 +0200
Subject: [Python-Dev] Status of mingw and Python 2.6 ?
In-Reply-To: <g6du8g$cue$1@ger.gmane.org>
References: <g6du8g$cue$1@ger.gmane.org>
Message-ID: <e27efe130807260019q405b31cdm1ce9c337f7c313e7@mail.gmail.com>

Jim Kleckner wrote:
> I gave it a try with cygwin-hosted mingw just to see
> if that would work as an alternative to VS2008/VC9 to
> figure out some linkage problems.
>
> I tried:
>  python setup.py build_ext --compiler mingw32
>
> and got a version string issue noted below which
> rejects the version string printed from the loader
> ld as an inappropriate form.
>
> I seem to recall that mingw was expected to work with 2.6.
> Is it?
>
[...]
>
>  File "c:\Python26\lib\distutils\version.py", line 107, in parse
>    raise ValueError, "invalid version number '%s'" % vstring
> ValueError: invalid version number '2.18.50.20080523'

There are actually two problems with MinGW. Your issue is the same as
See http://bugs.python.org/issue2234

But MinGW is also not completely compatible with the msvcr90.dll runtime:
http://bugs.python.org/issue3308
For example, if the extension module contains standard date functions
(using time_t), it won't load.

-- 
Amaury Forgeot d'Arc

From ncoghlan at gmail.com  Sat Jul 26 10:23:17 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 26 Jul 2008 18:23:17 +1000
Subject: [Python-Dev] Infix operators
In-Reply-To: <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>	<e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
Message-ID: <488ADEF5.5080002@gmail.com>

Sebastien Loisel wrote:
> However, just for posterity (and I'm not going to pursue the argument 
> further than this), I'll say this. The problem of determining the 
> meaning (or overridability or whatever) of x=4$6 is the same as the 
> problem of determining the meaning of x=fooz(4,6). Since it's not a good 
> argument against user-defined functions, I don't see it as a good 
> argument against user-defined operators.

The namespace of usefully mnemonic function names is infinitely larger 
than that of usefully mnemonic punctuation marks. User-defined functions 
are good, but once you have those there is no reason to have 
user-defined operators *as well*.

Cheers,
Nick.


From daniel.stutzbach at gmail.com  Sat Jul 26 18:23:11 2008
From: daniel.stutzbach at gmail.com (daniel.stutzbach at gmail.com)
Date: Sat, 26 Jul 2008 11:23:11 -0500
Subject: [Python-Dev] Matrix product
In-Reply-To: <488A8302.3050408@canterbury.ac.nz>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
Message-ID: <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com>

The desire for a new operator for matrix mutltiplication is because
binary prefix operators are horrible for expressin this kind of thing,
right?

Stuff like this is hard to write, read, and debug (especially when
checking it against an infix formula):

mmul(mmul(mmul(a, b), c), d)

How about just making a matrix multiply function that can take many
arguments?  I think this is pretty readable:

mmul(a, b, c, d)

Additionally, mmul could then optimize the order of the
multiplications (e.g., depending the dimensions of the matrices, it
may be much more efficient to perform a*((b*c)*d) rather than
((a*b)*c)*d).

(please forgive typos--writing this on a smartphone)

--
Daniel Stutzbach

From steve at pearwood.info  Sat Jul 26 19:11:16 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 27 Jul 2008 03:11:16 +1000
Subject: [Python-Dev] Matrix product
In-Reply-To: <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<eae285400807260923g51441d69j29e033184f408156@mail.gmail.com>
Message-ID: <200807270311.17414.steve@pearwood.info>

On Sun, 27 Jul 2008 02:23:11 am daniel.stutzbach at gmail.com wrote:

> How about just making a matrix multiply function that can take many
> arguments?  I think this is pretty readable:
>
> mmul(a, b, c, d)
>
> Additionally, mmul could then optimize the order of the
> multiplications (e.g., depending the dimensions of the matrices, it
> may be much more efficient to perform a*((b*c)*d) rather than
> ((a*b)*c)*d).

But be careful there: matrix multiplication is associative, so that 
a*b*c = (a*b)*c = a*(b*c), but that doesn't necessarily apply once the 
elements of the matrices are floats. For instance, a*(b*c) might 
underflow some elements to zero, while multiplying (a*b)*c does not. As 
a general rule, compilers should not mess with the order of floating 
point calculations.

Also, some classes that people might want to multiply may not be 
associative even in principle. E.g. the vector cross product:

(a*b)*c != a*(b*c) in general.

I think a product() function that multiplies the arguments from left to 
right would be useful. But it won't solve the problems that people want 
custom operators to solve. I'm not even sure if it will solve the 
problem of matrix multiplication.


-- 
Steven.

From regebro at gmail.com  Sat Jul 26 21:00:09 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 26 Jul 2008 21:00:09 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <488451AB.50406@jcea.es>
References: <4838EA2D.7060402@jcea.es>
	<319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>
	<488150A5.8050804@jcea.es>
	<B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org>
	<488451AB.50406@jcea.es>
Message-ID: <319e029f0807261200p373bc176ibd70d6e6ef5dee79@mail.gmail.com>

I've added a setup.py to the python-incompatibilities projects code,
so adding c-extention modules should now be much easier. I don't do
much c-development myself, so I'm not the right person to do this, but
anybody that feels like adding C-extensions to this project is more
than welcome to do so. Just mail me for write access.

http://code.google.com/p/python-incompatibility/

From skip at pobox.com  Sun Jul 27 05:21:07 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 26 Jul 2008 22:21:07 -0500
Subject: [Python-Dev] Possible small csv.DictReader change
Message-ID: <18571.59811.311511.28772@montanaro-dyndns-org.local>

A new issue in the tracker:

    http://bugs.python.org/issue3436

points out that the csv module's DictReader class doesn't know the
fieldnames defined in the file until you've fetched the first row of data.
It's fairly easy to change the behavior so that __init__ automatically grabs
the fieldnames if they aren't passed into it (I attached a patch) but
Raymond commented that __init__() shouldn't read any line(s) from the file,
instead preferring a separate method to set the fieldnames attribute.

There's the issue though of whether __init__() should implicitly read the
fieldnames from the file or if that task should be delegated to a separate
method which must be called either by the user or from the next() method.
If you have an opinion either way, I'd appreciate it if you added a comment
to that issue.

Regardless which way this issue plays out, is it something that can be put
into 2.6 & 3.0 or does it need to wait at this point?

Thanks,

Skip

From greg.ewing at canterbury.ac.nz  Sun Jul 27 09:21:39 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 27 Jul 2008 19:21:39 +1200
Subject: [Python-Dev] Matrix product
In-Reply-To: <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<eae285400807260923g51441d69j29e033184f408156@mail.gmail.com>
Message-ID: <488C2203.3080306@canterbury.ac.nz>

daniel.stutzbach at gmail.com wrote:

> How about just making a matrix multiply function that can take many
> arguments?  I think this is pretty readable:
> 
> mmul(a, b, c, d)

The multiplications aren't necessarily all together, e.g.

   a*b + c*d + e*f

would become

   mmul(a, b) + mmul(c, d) + mmul(e, f)

-- 
Greg

From charleshixsn at earthlink.net  Sun Jul 27 19:43:38 2008
From: charleshixsn at earthlink.net (Charles Hixson)
Date: Sun, 27 Jul 2008 10:43:38 -0700
Subject: [Python-Dev] Infix operators
In-Reply-To: <488ADEF5.5080002@gmail.com>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<488ADEF5.5080002@gmail.com>
Message-ID: <200807271043.38692.charleshixsn@earthlink.net>

On Saturday 26 July 2008 01:23:17 am Nick Coghlan wrote:
> Sebastien Loisel wrote:
> > However, just for posterity (and I'm not going to pursue the argument
> > further than this), I'll say this. The problem of determining the
> > meaning (or overridability or whatever) of x=4$6 is the same as the
> > problem of determining the meaning of x=fooz(4,6). Since it's not a good
> > argument against user-defined functions, I don't see it as a good
> > argument against user-defined operators.
>
> The namespace of usefully mnemonic function names is infinitely larger
> than that of usefully mnemonic punctuation marks. User-defined functions
> are good, but once you have those there is no reason to have
> user-defined operators *as well*.
>
> Cheers,
> Nick.
>
Most mathematicians would disagree with you.  I'll grant that it tends to make 
the code extremely obscure to those who don't work in the field, but it tends 
to make it much clearer to those who do so work.

OTOH, it's also true that there aren't sufficient punctuation symbols.  E.g. 
math people drafted capital sigma as sum of a series, etc.

Therefore it seems to me that the appropriate thing is to create a convention 
that bar-somethingprintable-bar should be interpreted as a user defined 
operation, e.g. |+| or |@#|.  The first one is reasonable as matrix 
multiplication, the second as some user defined operation that hasn't yet 
been specified (in this context).  And since such things don't yet have 
a "secret name" I would suggest that they be defined either via an ordinary 
def, or via a def with their name in quotes, i.e., either:
def |+|   or   def "|+|"

If an ordinary functional reference is desireable (probably) there could be a  
decorators that assigned the name, and possibly the precedence, e.g.:
@name=madd
@precedence="+"
def |+| (a, b):
   etc.

The main drawback that I see is that code that relies heavily on this approach 
would become much less readable to anyone not in the particular field.  Think 
of the first time you encountered the curl or del operators, or even the 
kronecker delta.

OTOH, it seems far too late in the development process to be inserting such a 
change in Python 2.6 or 3.0.  If this is important to you, you should 
probably propose it for 2.7/3.1.

From regebro at gmail.com  Sun Jul 27 20:18:17 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 27 Jul 2008 20:18:17 +0200
Subject: [Python-Dev] Infix operators
In-Reply-To: <200807271043.38692.charleshixsn@earthlink.net>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>
	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>
	<488ADEF5.5080002@gmail.com>
	<200807271043.38692.charleshixsn@earthlink.net>
Message-ID: <319e029f0807271118r429faaaep309b5b77be1c63d9@mail.gmail.com>

On Sun, Jul 27, 2008 at 19:43, Charles Hixson
<charleshixsn at earthlink.net> wrote:
> Therefore it seems to me that the appropriate thing is to create a convention
> that bar-somethingprintable-bar

And the "something-printable" shows the main flaw of this approach.
Mathematics indeed uses a lot of symbols to make things clear and
unambigous. Many of these are hard to print in a line, even with
unicode, and entering and editing this from a keyboard would be very
difficult. So to make this scheme useable you would have to limit
yourself to ascii-codes, and then most of the point goes away since
you can't use the proper symbols anyway, and ambiguity is
reintroduced.

It seems to me that mathematicians who need these things would be
better served by dedicated maths-software.

Just my 2 cents.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From steve at holdenweb.com  Sun Jul 27 23:42:20 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 27 Jul 2008 17:42:20 -0400
Subject: [Python-Dev] Infix operators
In-Reply-To: <200807271043.38692.charleshixsn@earthlink.net>
References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com>	<b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com>	<488ADEF5.5080002@gmail.com>
	<200807271043.38692.charleshixsn@earthlink.net>
Message-ID: <g6iq4p$knn$1@ger.gmane.org>

Charles Hixson wrote:
[...]
> OTOH, it seems far too late in the development process to be inserting such a 
> change in Python 2.6 or 3.0.  If this is important to you, you should 
> probably propose it for 2.7/3.1.

It's been too late for over three months now, and the suggestions I've 
seen so far are all way impractical anyway.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From ondrej at certik.cz  Mon Jul 28 10:28:34 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 28 Jul 2008 10:28:34 +0200
Subject: [Python-Dev] str(container) should call str(item), not repr(item)
Message-ID: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>

Hi,

as discussed before here:

http://mail.python.org/pipermail/python-3000/2008-May/013876.html

if you do:

>>> from decimal import Decimal
>>> a = Decimal(40)
>>> print a, a**2, a**3
40 1600 64000
>>> print [a, a**2, a**3]
[Decimal("40"), Decimal("1600"), Decimal("64000")]
>>> print {a: 3}
{Decimal("40"): 3}
>>> print {a: a**2}
{Decimal("40"): Decimal("1600")}


i.e. the str on list (and tuple and dict) calls repr() on the
elements, instead of str. This really seems to me like a bug. Because
if I wanted the repr() representation, I'd call repr() on the
list/tuple/dict. If I want a nice readable representation, I call
str(). That's the philosophy, no?

I understant there can be technical reasons why this cannot be made
into python 3.0, so why not 3.1? or 2.8?

If this cannot be made into python for some reason, how would you
suggest us we solve the following problem in sympy (symbolic
manipulation library):

>>> from sympy import Symbol, srepr
>>> x = Symbol("x")
>>> l = [x, x**2/2, x**3/3, x**4/4]
>>> str(l)
'[x, 1/2*x**2, 1/3*x**3, 1/4*x**4]'
>>> repr(l)
'[x, 1/2*x**2, 1/3*x**3, 1/4*x**4]'
>>> srepr(l)
"[Symbol('x'), Mul(Half(1, 2), Pow(Symbol('x'), Integer(2))),
Mul(Rational(1, 3), Pow(Symbol('x'), Integer(3))), Mul(Rational(1, 4),
Pow(Symbol('x'), Integer(4)))]"


As you can see, we currently have to hack our __repr__ to return the
same thing as __str__ so that things look nice in lists and
dictionaries. We provide the srepr() function that does what the
__repr__ should do instead. And as you can see, the result of srepr is
really not very readable, but it is easy to convert it back to a sympy
expression using eval. That's the purpose of repr(), no? However, you
cannot easily convert the str() back to an expression, because the
eval() doesn't know what "x" is.
One solution that we'll probably use is that we'll change our repr()
to the srepr output above, so str([x, x**2,...]) will not be pretty
and we'll use sys.displayhook to hook our own function into the
interactive session that overcomes this python behavior. This solves
the problem for interactive sessions, but it doesn't solve it if
people just call "print [x, x**2, ...]" inside their programs, for
example when debugging. Another option is to write a C extension
module that monkey patches the list class and changes the C pointer to
the (currently null) __str__ method to our own method. But I find it
very hackish and also it requires a C module.


I was discussing this with other people at EuroSciPy2008 and
everyone's first reaction that str(list) calls repr() on the elements
was that it's a bug. So was my reaction when I discovered that. So, I
think people find this highly unintuitive. So I just wanted to discuss
this more with core Python developers what you think about it and if
you also find this not very intuitive and if so, if there is any
chance to get this fixed.

Thanks,
Ondrej

From skip at pobox.com  Mon Jul 28 10:59:42 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 28 Jul 2008 03:59:42 -0500
Subject: [Python-Dev] str(container) should call str(item),
	not repr(item)
In-Reply-To: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
Message-ID: <18573.35454.142435.398911@montanaro-dyndns-org.local>


    Ondrej> i.e. the str on list (and tuple and dict) calls repr() on the
    Ondrej> elements, instead of str. This really seems to me like a bug.
    Ondrej> Because if I wanted the repr() representation, I'd call repr()
    Ondrej> on the list/tuple/dict. If I want a nice readable
    Ondrej> representation, I call str(). That's the philosophy, no?

I think this is the case which calls for the distinction:

    >>> str(["1", "2", "3"])
    "['1', '2', '3']"
    >>> str([1, 2, 3])
    '[1, 2, 3]'

If the first case did as you suggested you couldn't distinguish it from the
second.

Skip

From phd at phd.pp.ru  Mon Jul 28 11:36:56 2008
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Mon, 28 Jul 2008 13:36:56 +0400
Subject: [Python-Dev] str(container) should call str(item),
	not repr(item)
In-Reply-To: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
Message-ID: <20080728093656.GC25461@phd.pp.ru>

On Mon, Jul 28, 2008 at 10:28:34AM +0200, Ondrej Certik wrote:
> http://mail.python.org/pipermail/python-3000/2008-May/013876.html

   The PEP is pep-3140: http://python.org/peps/pep-3140.html and it has
been rejected. To revive it we need better, more compelling arguments. We
also need a plan for implementation that would be good enough to be
accepted. Current proposals for implementation are listed in the PEP with
their disadvantages.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From haase at msg.ucsf.edu  Mon Jul 28 12:04:21 2008
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Mon, 28 Jul 2008 12:04:21 +0200
Subject: [Python-Dev] str(container) should call str(item),
	not repr(item)
In-Reply-To: <18573.35454.142435.398911@montanaro-dyndns-org.local>
References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
	<18573.35454.142435.398911@montanaro-dyndns-org.local>
Message-ID: <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com>

On Mon, Jul 28, 2008 at 10:59 AM,  <skip at pobox.com> wrote:
>
>    Ondrej> i.e. the str on list (and tuple and dict) calls repr() on the
>    Ondrej> elements, instead of str. This really seems to me like a bug.
>    Ondrej> Because if I wanted the repr() representation, I'd call repr()
>    Ondrej> on the list/tuple/dict. If I want a nice readable
>    Ondrej> representation, I call str(). That's the philosophy, no?
>
> I think this is the case which calls for the distinction:
>
>    >>> str(["1", "2", "3"])
>    "['1', '2', '3']"
>    >>> str([1, 2, 3])
>    '[1, 2, 3]'
>
> If the first case did as you suggested you couldn't distinguish it from the
> second.
>
Look at this -- it seems to me that it should work fine....
---- Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26)
>>> str("qwer")
'qwer'
>>> repr("qwer")
"'qwer'"

- Sebastian Haase

From matt.giuca at gmail.com  Mon Jul 28 12:58:05 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Mon, 28 Jul 2008 20:58:05 +1000
Subject: [Python-Dev] str(container) should call str(item),
	not repr(item)
In-Reply-To: <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com>
References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
	<18573.35454.142435.398911@montanaro-dyndns-org.local>
	<bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com>
Message-ID: <e6268af30807280358j21cf127dw29195e1568e1726a@mail.gmail.com>

Another disadvantage of calling str recursively rather than repr is that it
places an onus on anyone writing a class to write both a repr and a str
method (or be inconsistent with the newly-accepted standard for container
types).

I personally write a repr method for most classes, which aids debugging.
This means all my classes behave like containers currently do - their str
will call repr on the items. This proposal will make all of my classes
behave inconsistently with the standard container types.

- Matt Giuca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080728/188881f0/attachment.htm>

From steve at holdenweb.com  Mon Jul 28 13:24:18 2008
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 28 Jul 2008 07:24:18 -0400
Subject: [Python-Dev] str(container) should call str(item),
	not repr(item)
In-Reply-To: <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com>
References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>	<18573.35454.142435.398911@montanaro-dyndns-org.local>
	<bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com>
Message-ID: <g6kaa9$cf5$1@ger.gmane.org>

Sebastian Haase wrote:
> On Mon, Jul 28, 2008 at 10:59 AM,  <skip at pobox.com> wrote:
>>    Ondrej> i.e. the str on list (and tuple and dict) calls repr() on the
>>    Ondrej> elements, instead of str. This really seems to me like a bug.
>>    Ondrej> Because if I wanted the repr() representation, I'd call repr()
>>    Ondrej> on the list/tuple/dict. If I want a nice readable
>>    Ondrej> representation, I call str(). That's the philosophy, no?
>>
>> I think this is the case which calls for the distinction:
>>
>>    >>> str(["1", "2", "3"])
>>    "['1', '2', '3']"
>>    >>> str([1, 2, 3])
>>    '[1, 2, 3]'
>>
>> If the first case did as you suggested you couldn't distinguish it from the
>> second.
>>
> Look at this -- it seems to me that it should work fine....
> ---- Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26)
>>>> str("qwer")
> 'qwer'
>>>> repr("qwer")
> "'qwer'"
> 
No it doesn't. What's happening in these examples is that the 
interpreter is calling repr() on the expression result - otherwise you 
wouldn't see the quotes:

 >>> str("qwer")
'qwer'
 >>> print str("qwer")
qwer
 >>>

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From aahz at pythoncraft.com  Mon Jul 28 16:58:53 2008
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 28 Jul 2008 07:58:53 -0700
Subject: [Python-Dev] str(container) should call str(item),
	not repr(item)
In-Reply-To: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com>
Message-ID: <20080728145853.GA4266@panix.com>

On Mon, Jul 28, 2008, Ondrej Certik wrote:
> 
> as discussed before here:
> 
> http://mail.python.org/pipermail/python-3000/2008-May/013876.html

Precisely because this has been discussed extensively with little to
show, I recommend that any further discussion be held on either
python-ideas or comp.lang.python
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From jcea at jcea.es  Tue Jul 29 15:51:06 2008
From: jcea at jcea.es (Jesus Cea)
Date: Tue, 29 Jul 2008 15:51:06 +0200
Subject: [Python-Dev] bsddb: Test failures on windows (HELP!)
In-Reply-To: <e27efe130807240723v155b05a5x1bd2a9b1a64ab057@mail.gmail.com>
References: <48874F2D.5070201@jcea.es>
	<095c01c8ed50$60bdfed0$2239fc70$@com.au>	<488877EC.600@palladion.com>
	<e27efe130807240723v155b05a5x1bd2a9b1a64ab057@mail.gmail.com>
Message-ID: <488F204A.40401@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Amaury Forgeot d'Arc wrote:
| I see that some tests use os.unlink. They should use
| test_support.unlink() instead.

Old stuff. Fix just committed.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSI8gQ5lgi5GaxT1NAQJyHwQAiU6izAcI5eI3tizxWkvsw4MBRmQzlGNi
Ib+U/ZxuO9bqYOXZLqIQWkH2Ry1/Li7KeepVRehdkVlSnFEkWVXPhNofxvlXoQpl
Rt0T8aGJ1GhWkdojkWE7Ab2L8mdTCunHuVyiAQBagTET1E9iRnjrf5//XsRvd09Z
SUfPgEnvqv4=
=96Vt
-----END PGP SIGNATURE-----

From razvan.a.lupusoru at intel.com  Tue Jul 29 20:56:43 2008
From: razvan.a.lupusoru at intel.com (Lupusoru, Razvan A)
Date: Tue, 29 Jul 2008 11:56:43 -0700
Subject: [Python-Dev] Relocating Python
Message-ID: <BAD036B67C739F45BE062D0A03D19286035ACFD7@orsmsx414.amr.corp.intel.com>

Hello,

 

I am trying to get Python 2.5.2 working for an IA32 system. The
compilation is done on an Ubuntu 8.04.1 dev system. I am using a custom
gcc and ld specific to the IA32 system.

 

This is my makefile:

##############################

BUILD_DEST = /i686-custom-kernel

CC = $(BUILD_DEST)/bin/i686-linux-gcc

CPP = $(CC) -E

CXX = $(BUILD_DEST)/bin/i686-linux-g++

LD = $(BUILD_DEST)/bin/i686-linux-ld

PYTHONINSTALLPATH = $(BUILD_DEST)/usr

Export

 

all:

                tar xzfv Python-2.5.2.tgz

                ./Python-2.5.2/configure -prefix=${PYTHONINSTALLPATH}
-host=i686-linux -enable-shared

                cd Python-2.5.2

                make

                make install

##############################

 

Everything compiles correctly. I then copy the contents of the
$BUILD_DEST and put them on the hard drive for my IA32 system. I
basically use the contents of $BUILD_DEST as the root directory on my
IA32 system. Python seems to run correctly when I run it, but when I do
things like "import pysqlite", it cannot find it. Is there anything
special I have to do to relocate my python (since on my IA32 system it
runs from /usr/bin/python but it originally gets created in
${BUILD_DEST}/usr/bin/python)?

 

Thank you,

 

Razvan A. Lupusoru

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080729/ad910a7e/attachment.htm>

From casey at pandora.com  Tue Jul 29 21:54:31 2008
From: casey at pandora.com (Casey Duncan)
Date: Tue, 29 Jul 2008 13:54:31 -0600
Subject: [Python-Dev] Relocating Python
In-Reply-To: <BAD036B67C739F45BE062D0A03D19286035ACFD7@orsmsx414.amr.corp.intel.com>
References: <BAD036B67C739F45BE062D0A03D19286035ACFD7@orsmsx414.amr.corp.intel.com>
Message-ID: <A8F28B58-2168-4662-B261-495CB1F29284@pandora.com>

On Jul 29, 2008, at 12:56 PM, Lupusoru, Razvan A wrote:

> Hello,
>
> I am trying to get Python 2.5.2 working for an IA32 system. The  
> compilation is done on an Ubuntu 8.04.1 dev system. I am using a  
> custom gcc and ld specific to the IA32 system.
>
> This is my makefile:
> ##############################
> BUILD_DEST = /i686-custom-kernel
> CC = $(BUILD_DEST)/bin/i686-linux-gcc
> CPP = $(CC) ?E
> CXX = $(BUILD_DEST)/bin/i686-linux-g++
> LD = $(BUILD_DEST)/bin/i686-linux-ld
> PYTHONINSTALLPATH = $(BUILD_DEST)/usr
> Export
>
> all:
>                 tar xzfv Python-2.5.2.tgz
>                 ./Python-2.5.2/configure ?prefix=$ 
> {PYTHONINSTALLPATH} ?host=i686-linux ?enable-shared
>                 cd Python-2.5.2
>                 make
>                 make install
> ##############################
>
> Everything compiles correctly. I then copy the contents of the  
> $BUILD_DEST and put them on the hard drive for my IA32 system. I  
> basically use the contents of $BUILD_DEST as the root directory on  
> my IA32 system. Python seems to run correctly when I run it, but  
> when I do things like ?import pysqlite?, it cannot find it. Is there  
> anything special I have to do to relocate my python (since on my  
> IA32 system it runs from /usr/bin/python but it originally gets  
> created in ${BUILD_DEST}/usr/bin/python)?

You'll want to pass configure the prefix where python will ultimately  
be installed, otherwise the paths used during make won't  make sense  
on the destination system. That said, pysqlite is not part of the  
stdlib, so your actual problem may have more to do with how you've  
installed it then anything. When you run python once relocated, what  
does os.path contain?

What we do for packaging, is run 'configure' normally and then 'make',  
then override some make variables for 'make install' to temporarily  
install it in a different place for package staging. It looks  
something like this:

./configure && \
make && \
make BINDIR=$(shell pwd)/path/to/tmp/bin \
      CONFINCLUDEDIR=$(shell pwd)/path/to/tmp/include \
      INCLUDEDIR=$(shell pwd)/path/to/tmp/include \
      LIBDIR=$(shell pwd)/path/to/tmp/lib \
      MANDIR=$(shell pwd)/path/to/tmp/man \
      SCRIPTDIR=$(shell pwd)/path/to/tmp/lib \
      install

Then when the package is deployed, the files are actually installed  
under the standard 'configure' prefix (/usr/local I think).

hth,

-Casey

From guido at python.org  Wed Jul 30 00:37:11 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Jul 2008 15:37:11 -0700
Subject: [Python-Dev] Deprecate parser's "ast" functions?
In-Reply-To: <1afaf6160807210747v7756589bnac5d07c6d3bbb6f8@mail.gmail.com>
References: <g2ej5h$tnl$1@ger.gmane.org>
	<1afaf6160807210747v7756589bnac5d07c6d3bbb6f8@mail.gmail.com>
Message-ID: <ca471dc20807291537xc3e03d1r6287b27142335beb@mail.gmail.com>

+1 as well.

On Mon, Jul 21, 2008 at 7:47 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Sat, Jun 7, 2008 at 3:13 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> The parser module exports each function and type twice, once with "AST" in
>> the name, once with "ST".  Since "AST" now has a different meaning for
>> Python code compilation, I propose to deprecate the "ast" variants in 2.6
>> and remove them in Python 3000.
>
> +1 This personally has confused me!
>
>>
>> (Also, all keyword arguments are called "ast", that cannot change in 2.6
>> but should in 3.0.)
>>
>> I'm at the moment changing the documentation of parser to only refer to
>> the "st" variants anymore.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jul 30 02:11:56 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Jul 2008 17:11:56 -0700
Subject: [Python-Dev] fileobj.read(float): warning or error?
In-Reply-To: <20080722053744.GA11290@cskk.homeip.net>
References: <200807212117.14485.victor.stinner@haypocalc.com>
	<20080722053744.GA11290@cskk.homeip.net>
Message-ID: <ca471dc20807291711m48d5837ap33c9b58f31eb82b5@mail.gmail.com>

On Mon, Jul 21, 2008 at 10:37 PM, Cameron Simpson <cs at zip.com.au> wrote:
> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
> exception if asked for < 1 bytes? Or is there a legitimate use for read(0)
> with which I was not previously aware?

Indeed. read(0) is quite often generated as an edge case when one is
computing buffer sizes, and returning an empty string is most
definitely the right thing to do here (otherwise some application code
becomes more complex by having to avoid calling read(0) at all).

Of course, read(), read(None), read(-1) and read(<any negative int>)
should all read all data until EOF.

On the main topic here, read(<float>) and read(<anything that supports
__int__ but not __index__>) should definitely raise an exception in
3.0. In 2.6 it should show a warning as it does in 2.5.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jul 30 02:26:59 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Jul 2008 17:26:59 -0700
Subject: [Python-Dev] Matrix product
In-Reply-To: <488A8302.3050408@canterbury.ac.nz>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
Message-ID: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>

On Fri, Jul 25, 2008 at 6:50 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Sebastien Loisel wrote:
>
>> What are the odds of this thing going in?
>
> I don't know. Guido has said nothing about it so far this
> time round, and his is the only opinion that matters in the
> end.

I'd rather stay silent until a PEP exists, but I should point out that
last time '@' was considered as a new operator, that character had no
uses in the language at all. Now it is the decorator marker. Therefore
it may not be so attractive any more.

I understand that you can't use A*B as matrix multiplication because
it should mean elementwise multiplication instead, just like A+B is
elementwise addition (for matrixes, as opposed to Python sequences).

But would it be totally outlandish to propose A**B for matrix
multiplication? I can't think of what "matrix exponentiation" would
mean...

--Guido

> I may write a PEP about this. However, since yesterday I've
> realised that there's a rather serious problem with part
> of my proposal.
>
> The problem is that being able to multiply matrices isn't
> much use without also being able to add them and multiply
> them by numbers, which obviously can't work with the
> built-in sequence types, since they already have other
> meanings for + and *.
>
> However, I still think that adding an @ operator for numpy
> to use is a good idea. So I'm now suggesting that the
> operator be added, with the intended meaning of matrix
> multiplication, but that no implementation of it be
> provided in core Python.
>
> There is a precedent for this -- the ellipsis notation was
> added purely for use by Numeric and its successors, and
> nothing in core Python attaches any meaning to it.
>
>> How do the PEPs work?
>
> Someone writes a PEP. People talk about it. Eventually, Guido
> either accepts it or rejects it (although in some cases it
> is an infinitely long time before that happens:-).
>
> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jul 30 02:30:13 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Jul 2008 17:30:13 -0700
Subject: [Python-Dev] bsddb: Test failures on windows (HELP!)
In-Reply-To: <095c01c8ed50$60bdfed0$2239fc70$@com.au>
References: <48874F2D.5070201@jcea.es> <095c01c8ed50$60bdfed0$2239fc70$@com.au>
Message-ID: <ca471dc20807291730tdb9caaewc7e87aa0c25bd88@mail.gmail.com>

On Wed, Jul 23, 2008 at 10:44 PM, Mark Hammond
<mhammond at skippinet.com.au> wrote:
>> Trent, I was wondering if you could look at some test failures in MS
>> Windows builds. I can't debug Windows issues myself :-(. This is a MS
>> free environment...
>
> In these errors I see lots of bsdbd errors, many of the form:
>
> | DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit
> (100) exceeded')

I've had this a few times in the past (Retry limit (100) exceeded)and
it has always been caused by cruft left behind by a previous run of
the test that didn't end well. Somehow these tests do not do a good
job of cleaning up old cruft before they run.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From sidnei.da.silva at gmail.com  Wed Jul 30 02:33:51 2008
From: sidnei.da.silva at gmail.com (Sidnei da Silva)
Date: Tue, 29 Jul 2008 21:33:51 -0300
Subject: [Python-Dev] Matrix product
In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
Message-ID: <a7a2b76b0807291733j29f30588n6e103457203e49d8@mail.gmail.com>

On Tue, Jul 29, 2008 at 9:26 PM, Guido van Rossum <guido at python.org> wrote:
> But would it be totally outlandish to propose A**B for matrix
> multiplication? I can't think of what "matrix exponentiation" would
> mean...

Before even reading this paragraph, A**B came to my mind, so I suspect
it would be the most intuitive option.

-- 
Sidnei da Silva
Enfold Systems http://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

From loisel at temple.edu  Wed Jul 30 02:54:15 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Tue, 29 Jul 2008 20:54:15 -0400
Subject: [Python-Dev] Matrix product
In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
Message-ID: <b7975ab80807291754x4053036fv5402951f01c7e082@mail.gmail.com>

Dear Guido,

Thank you for your email.

On Tue, Jul 29, 2008 at 8:26 PM, Guido van Rossum <guido at python.org> wrote:
> But would it be totally outlandish to propose A**B for matrix
> multiplication? I can't think of what "matrix exponentiation" would
> mean...

Right now, ** is the pointwise power:

>>> from numpy import *
>>> A=array([[1,2],[3,4]])
>>> print(A**A)
[[  1   4]
 [ 27 256]]

Since all the scalar operators have meaning as pointwise operator,
like you say it's hard to bump one off to give it to the matrix
product instead. I don't know if it's a good idea with **, it will
destroy the orthogonality of the system.

They used to say, ignore LISP at your own peril. In that spirit, let
me describe MATLAB's approach to this. It features a complete suite of
matrix operators (+-*/\^), and their pointwise variants (.+ .- ./ .*
.^), although + and .+ are synonyms, as are - and.-. Right now,
numpy's *,**,/ correspond to MATLAB .*,.^,./.

MATLAB implements scalar^matrix, matrix^scalar, but not matrix^matrix
(although since log and exp are defined, I guess you could clobber
together a meaning for matrix^matrix). Since ^ is the matrix-product
version of "power", 2^A may not be what you expect:

>> 2^A
   10.4827   14.1519
   21.2278   31.7106

Sincerely,

-- 
S?bastien Loisel

From fredrik.johansson at gmail.com  Wed Jul 30 02:59:36 2008
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Wed, 30 Jul 2008 02:59:36 +0200
Subject: [Python-Dev] Matrix product
In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
Message-ID: <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>

On Wed, Jul 30, 2008 at 2:26 AM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Jul 25, 2008 at 6:50 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Sebastien Loisel wrote:
>>
>>> What are the odds of this thing going in?
>>
>> I don't know. Guido has said nothing about it so far this
>> time round, and his is the only opinion that matters in the
>> end.
>
> I'd rather stay silent until a PEP exists, but I should point out that
> last time '@' was considered as a new operator, that character had no
> uses in the language at all. Now it is the decorator marker. Therefore
> it may not be so attractive any more.

I don't like @.

> I understand that you can't use A*B as matrix multiplication because
> it should mean elementwise multiplication instead, just like A+B is
> elementwise addition (for matrixes, as opposed to Python sequences).
>
> But would it be totally outlandish to propose A**B for matrix
> multiplication? I can't think of what "matrix exponentiation" would
> mean...

http://mathworld.wolfram.com/MatrixExponential.html :-)

In fact Mathematica uses ** to denote general noncommutative
multiplication (though . for matrix multiplication in particular).
However, this wouldn't solve the problem, because an important reason
to introduce a matrix multiplication operator is to distinguish
between matrix and elementwise operations for arrays. The ** operator
already denotes the obvious elementwise operation in numpy.

Further, while A**B is not so common, A**n is quite common (for
integral n, in the sense of repeated matrix multiplication). So a
matrix multiplication operator really should come with a power
operator cousin.

Matlab uses * for matrix and .* for elementwise multiplication.
Introducing .* for elementwise multiplication in Python would not be
compatible with existing numpy code, and introducing .* with the
reversed meaning of Matlab would be *very* confusing :-)

Maple uses &* for matrix multiplication. However, Maple's syntax is
not a good style reference for anything.

Besides those alternatives and the regular *, I don't know any other
ASCII operators used by existing mathematical software for matrix
multiplication. Well, Fortress probably has some unicode symbol for it
(I suppose that would be one desperate possibility).

Fredrik

From greg.ewing at canterbury.ac.nz  Wed Jul 30 03:30:20 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Jul 2008 13:30:20 +1200
Subject: [Python-Dev] Matrix product
In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
Message-ID: <488FC42C.80807@canterbury.ac.nz>

Guido van Rossum wrote:

> last time '@' was considered as a new operator, that character had no
> uses in the language at all. Now it is the decorator marker.

The only alternatives left would seem to be ?, ! or $,
none of which look particularly multiplicationish.

> But would it be totally outlandish to propose A**B for matrix
> multiplication? I can't think of what "matrix exponentiation" would
> mean...

But ** has the same problem -- it already represents
an elementwise operation on numpy arrays.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Jul 30 03:38:34 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Jul 2008 13:38:34 +1200
Subject: [Python-Dev] Matrix product
In-Reply-To: <b7975ab80807291754x4053036fv5402951f01c7e082@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
	<b7975ab80807291754x4053036fv5402951f01c7e082@mail.gmail.com>
Message-ID: <488FC61A.7040303@canterbury.ac.nz>

Sebastien Loisel wrote:
> let
> me describe MATLAB's approach to this. It features a complete suite of
> matrix operators (+-*/\^), and their pointwise variants (.+ .- ./ .*
> .^)

That was considered before as well, but rejected on
the grounds that the dot-prefixed operators were too
cumbersome to use heavily.

In MATLAB, the elementwise operations are probably
used fairly infrequently. But numpy arrays are often
used to vectorise what are otherwise scalar operations,
in which case elementwise operations are used almost
exclusively.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Jul 30 03:43:16 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Jul 2008 13:43:16 +1200
Subject: [Python-Dev] Matrix product
In-Reply-To: <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
	<3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>
Message-ID: <488FC734.9030503@canterbury.ac.nz>

Fredrik Johansson wrote:

> Further, while A**B is not so common, A**n is quite common (for
> integral n, in the sense of repeated matrix multiplication). So a
> matrix multiplication operator really should come with a power
> operator cousin.

Which obviously should be @@ :-)

> Well, Fortress probably has some unicode symbol for it
> (I suppose that would be one desperate possibility).

I've been carefully refraining from suggesting that.
Although now that unicode is allowed in identifiers,
it's not *quite* as heretical as it used to be.

-- 
Greg

From loisel at temple.edu  Wed Jul 30 03:53:49 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Tue, 29 Jul 2008 21:53:49 -0400
Subject: [Python-Dev] Matrix product
In-Reply-To: <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
	<3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>
Message-ID: <b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com>

Dear Greg,

Thank you for your email.

> In MATLAB, the elementwise operations are probably
> used fairly infrequently. But numpy arrays are often
> used to vectorise what are otherwise scalar operations,
> in which case elementwise operations are used almost
> exclusively.

Your assessment of pointwise operators in MATLAB is incorrect. The
pointwise operators in MATLAB are used heavily to vectorise the scalar
operations, exactly the same as what you describe in numpy. Recently,
the MATLAB JIT has become good enough that looping is fast, however,
the pointwise operators remain "the MATLAB way" of programming.

-- 
S?bastien Loisel

From python at rcn.com  Wed Jul 30 10:29:36 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 30 Jul 2008 01:29:36 -0700
Subject: [Python-Dev] Matrix product
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com><488A8302.3050408@canterbury.ac.nz><ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com><3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>
	<488FC734.9030503@canterbury.ac.nz>
Message-ID: <89992E9D6DEB460988282F7F1080E73A@RaymondLaptop1>

>> Further, while A**B is not so common, A**n is quite common (for
>> integral n, in the sense of repeated matrix multiplication). So a
>> matrix multiplication operator really should come with a power
>> operator cousin.
> 
> Which obviously should be @@ :-)

I think much of this thread is a repeat of conversations
that were held for PEP 225:
http://www.python.org/dev/peps/pep-0225/

That PEP is marked as deferred.  Maybe it's time to
bring it back to life.


Raymond

From loisel at temple.edu  Wed Jul 30 11:13:03 2008
From: loisel at temple.edu (Sebastien Loisel)
Date: Wed, 30 Jul 2008 05:13:03 -0400
Subject: [Python-Dev] Matrix product
In-Reply-To: <b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
	<3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>
	<b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com>
Message-ID: <b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com>

Dear Raymond,

Thank you for your email.

> I think much of this thread is a repeat of conversations
> that were held for PEP 225:
> http://www.python.org/dev/peps/pep-0225/
>
> That PEP is marked as deferred.  Maybe it's time to
> bring it back to life.

This is a much better PEP than the one I had found, and would solve
all of the numpy problems. The PEP is very well thought-out.

Sincerely,

-- 
S?bastien Loisel

From matt.giuca at gmail.com  Wed Jul 30 16:11:40 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Thu, 31 Jul 2008 00:11:40 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <-6533647106000374959@unknownmsgid>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
	<e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
Message-ID: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>

Hi folks,

This issue got some attention a few weeks back but it seems to have
fallen quiet, and I haven't had a good chance to sit down and reply
again till now.

As I've said before this is a serious issue which will affect a great
deal of code. However it's obviously not as clear-cut as I originally
believed, since there are lots of conflicting opinions. Let us see if
we can come to a consensus.

(For those who haven't seen the discussion, the thread starts here:
http://mail.python.org/pipermail/python-dev/2008-July/081013.html
continues here for some reason:
http://mail.python.org/pipermail/python-dev/2008-July/081066.html
and I've got a bug report with a fully tested and documented patch here:
http://bugs.python.org/issue3300)

Firstly, it looks like most of the people agree we should add an
optional "encoding" argument which lets the caller customize which
encoding to use. What we tend to disagree about is what the default
encoding should be.

Here I present the various options as I see it (and I'm trying to be
impartial), and the people who've indicated support for that option
(apologies if I've misrepresented anybody's opinion, feel free to
correct):

1. Leave it as it is. quote is Latin-1 if range(0,256), fallback to
UTF-8. unquote is Latin-1.
In favour: Anybody who doesn't reply to this thread
Pros: Already implemented; some existing code depends upon ord values
of string being the same as they were for byte strings; possible to
hack around it.
Cons: unquote is not inverse of quote; quote behaviour
internally-inconsistent; garbage when unquoting UTF-8-encoded URIs.

2. Default to UTF-8.
In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven
Pros: Fully working and tested solution is implemented; recommended by
RFC 3986 for all future schemes; recommended by W3C for use with HTML;
UTF-8 used by all major browsers; supports all characters; most
existing code compatible by default; unquote is inverse of quote.
Cons: By default, URIs may have invalid octet sequences (not possible
to reverse).

3. quote default to UTF-8, unquote default to Latin-1.
In favour: Andr? Malo
Pros: quote able to handle all characters; unquote able to handle all sequences.
Cons: unquote is not inverse of quote; totally inconsistent.

4. quote accepts either bytes or str, unquote default to outputting
bytes unless given an encoding argument.
In favour: Bill Janssen
Pros: Technically does what the spec says, which is treat it as an
octet encoding.
Cons: unquote will break most existing code; almost 100% of the time
people will want it as a string.

</impartiality>

I'll just comment on #4 since I haven't already. Let's talk about
quote and unquote separately. For quote, I'm all for letting it accept
a bytes as well as a str. That doesn't break anything or surprise
anyone.

For unquote, I think it will break a lot and surprise everyone. I
think that while this may be "purely" the best option, it's pretty
silly. I reckon the vast majority of users will be surprised when they
see it spitting out a bytes object, and all that most people will do
is decode it as UTF-8. Besides, while you're reading the RFCs as "URLs
specify a method for encoding octet sequences", I'm reading them as
"URLs specify a method for encoding strings, and leave the character
encoding unspecified." The second reading supports the idea that
unquote outputs a str.

I'm also recommending we add unquote_to_bytes to do what you suggest
unquote should do. (So either way we'll get both versions of unquote;
I'm just suggesting the one called "unquote" do the thing everybody
expects). But that's less of a priority so I want to commit these
urgent fixes first.

I'm basically saying just two things: 1. The standards are undefined;
2. Therefore we should pick the most useful and/or intuitive default.
IMHO choosing UTF-8 *is* the most useful AND intuitive, and will be
more so in the future when more technologies are hard-coded as UTF-8
(which this RFC recommends they do in the future).

I am also quite adamant that unquote be the inverse of quote.

Are there any more opinions on this matter? It would be good to reach
a consensus. If anyone seriously wants to push a different alternative
to mine, please write a working implementation and attach it to issue
3300.

On the technical side of things, does anybody have time to review my
patch for this issue?
http://bugs.python.org/issue3300
Patch 5.
It's just a patch for unquote, quote, and small related functions, as
well as numerous changes to test cases and documentation.

Cheers
Matt

From matt.giuca at gmail.com  Wed Jul 30 16:34:33 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Thu, 31 Jul 2008 00:34:33 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
	<e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
Message-ID: <e6268af30807300734h68a848a1pdcff123512b6ec20@mail.gmail.com>

Arg! Damnit, why do my replies get split off from the main thread?
Sorry about any confusion this may be causing.

From phd at phd.pp.ru  Wed Jul 30 16:53:06 2008
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 30 Jul 2008 18:53:06 +0400
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
	<e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
Message-ID: <20080730145306.GA26756@phd.pp.ru>

On Thu, Jul 31, 2008 at 12:11:40AM +1000, Matt Giuca wrote:
> 2. Default to UTF-8.
> In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven

   Count me too: +1. Most sites I use theese days use UTF-8 for URL
encoding. Examples:

Wikipedia:
http://ru.wikipedia.org/wiki/%D0%93%D0%B2%D0%B8%D0%B4%D0%BE_%D0%B2%D0%B0%D0%BD_%D0%A0%D0%BE%D1%81%D1%81%D1%83%D0%BC

LingVo (Russian-English dictionary):
http://lingvo.yandex.ru/en?text=%D0%BF%D0%B8%D1%82%D0%BE%D0%BD

>>> print urllib.quote(unicode('?????', 'koi8-r').encode('utf-8'))
%D0%BF%D0%B8%D1%82%D0%BE%D0%BD

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From facundobatista at gmail.com  Wed Jul 30 17:00:02 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 30 Jul 2008 12:00:02 -0300
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
	<e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
Message-ID: <e04bdf310807300800x3fe77520se6707ff794253d85@mail.gmail.com>

2008/7/30 Matt Giuca <matt.giuca at gmail.com>:

> 2. Default to UTF-8.
> In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven
> Pros: Fully working and tested solution is implemented; recommended by
> RFC 3986 for all future schemes; recommended by W3C for use with HTML;
> UTF-8 used by all major browsers; supports all characters; most
> existing code compatible by default; unquote is inverse of quote.
> Cons: By default, URIs may have invalid octet sequences (not possible
> to reverse).

+1, assuming that if you have a different encoding in the URI you can
pass it as a parameter.

Regards,

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From solipsis at pitrou.net  Wed Jul 30 17:08:29 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 30 Jul 2008 15:08:29 +0000 (UTC)
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
	<e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<e04bdf310807300800x3fe77520se6707ff794253d85@mail.gmail.com>
Message-ID: <loom.20080730T150601-899@post.gmane.org>

Facundo Batista <facundobatista <at> gmail.com> writes:

> 
> 2008/7/30 Matt Giuca <matt.giuca <at> gmail.com>:
> 
> > 2. Default to UTF-8.
> > In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven
> > Pros: Fully working and tested solution is implemented; recommended by
> > RFC 3986 for all future schemes; recommended by W3C for use with HTML;
> > UTF-8 used by all major browsers; supports all characters; most
> > existing code compatible by default; unquote is inverse of quote.
> > Cons: By default, URIs may have invalid octet sequences (not possible
> > to reverse).
> 
> +1, assuming that if you have a different encoding in the URI you can
> pass it as a parameter.

+1 for me as well, with an optional encoding parameter to override the default.
Also, your "con" is a "pro" to me, since it means errors are reported instead of
silently producing garbage (as would be the case with latin1).

Regards

Antoine.



From nd at perlig.de  Wed Jul 30 17:09:51 2008
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Wed, 30 Jul 2008 17:09:51 +0200
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
Message-ID: <200807301709.51528.nd@perlig.de>

[I was pretty busy these days, so sorry for jumping in late again]

* Matt Giuca wrote: 

> 1. Leave it as it is. quote is Latin-1 if range(0,256), fallback to
> UTF-8. unquote is Latin-1.
> In favour: Anybody who doesn't reply to this thread
> Pros: Already implemented; some existing code depends upon ord values
> of string being the same as they were for byte strings; possible to
> hack around it.
> Cons: unquote is not inverse of quote; quote behaviour
> internally-inconsistent; garbage when unquoting UTF-8-encoded URIs.

> 2. Default to UTF-8.
> In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven
> Pros: Fully working and tested solution is implemented; recommended by
> RFC 3986 for all future schemes; recommended by W3C for use with HTML;
> UTF-8 used by all major browsers; supports all characters; most
> existing code compatible by default; unquote is inverse of quote.
> Cons: By default, URIs may have invalid octet sequences (not possible
> to reverse).

Con: URI encoding does not encode characters.

>
> 3. quote default to UTF-8, unquote default to Latin-1.
> In favour: Andr? Malo
> Pros: quote able to handle all characters; unquote able to handle all
> sequences. Cons: unquote is not inverse of quote; totally inconsistent.

I'm not in favour of that. I merely answered a question there ;)

I'm actually in favour of encoding bytes only back and forth. A useful 
extension would be *another* function which wraps quote/unquote and encodes 
and decodes characters.


> 4. quote accepts either bytes or str, unquote default to outputting
> bytes unless given an encoding argument.
> In favour: Bill Janssen
> Pros: Technically does what the spec says, which is treat it as an
> octet encoding.
> Cons: unquote will break most existing code; almost 100% of the time
> people will want it as a string.
>
> </impartiality>
>
> I'll just comment on #4 since I haven't already. Let's talk about
> quote and unquote separately. For quote, I'm all for letting it accept
> a bytes as well as a str. That doesn't break anything or surprise
> anyone.
>
> For unquote, I think it will break a lot and surprise everyone. I
> think that while this may be "purely" the best option, it's pretty
> silly. I reckon the vast majority of users will be surprised when they
> see it spitting out a bytes object, and all that most people will do
> is decode it as UTF-8. Besides, while you're reading the RFCs as "URLs
> specify a method for encoding octet sequences", I'm reading them as
> "URLs specify a method for encoding strings, and leave the character
> encoding unspecified." The second reading supports the idea that
> unquote outputs a str.
>
> I'm also recommending we add unquote_to_bytes to do what you suggest
> unquote should do. (So either way we'll get both versions of unquote;
> I'm just suggesting the one called "unquote" do the thing everybody
> expects). But that's less of a priority so I want to commit these
> urgent fixes first.
>
> I'm basically saying just two things: 1. The standards are undefined;

That's still disputed...

> 2. Therefore we should pick the most useful and/or intuitive default.
> IMHO choosing UTF-8 *is* the most useful AND intuitive, and will be
> more so in the future when more technologies are hard-coded as UTF-8
> (which this RFC recommends they do in the future).

See my suggestion above.

nd

From guido at python.org  Wed Jul 30 18:27:02 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Jul 2008 09:27:02 -0700
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <200807301709.51528.nd@perlig.de>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
Message-ID: <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>

On Wed, Jul 30, 2008 at 8:09 AM, Andr? Malo <nd at perlig.de> wrote:
> I'm actually in favour of encoding bytes only back and forth. A useful
> extension would be *another* function which wraps quote/unquote and encodes
> and decodes characters.

I'd reverse this. By all means, add a new pair of functions that is
bytes in / bytes out. But keep the existing functions purely string in
/ string out, hardcoded to UTF-8. People wanting another encoding can
use the bytes functions and explicit encode / decode calls.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Jul 30 18:46:05 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 30 Jul 2008 09:46:05 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<20080712222801.GC27106@nexus.in-nomine.org>
	<e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com>
	<8249917582531821653@unknownmsgid>
	<e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
Message-ID: <08Jul30.094609pdt."58698"@synergy1.parc.xerox.com>

> For unquote, I think it will break a lot and surprise everyone. I
> think that while this may be "purely" the best option, it's pretty
> silly.

I don't mind being silly to do the right thing.  Happens to me a lot :-).

Bill

From janssen at parc.com  Wed Jul 30 18:52:26 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 30 Jul 2008 09:52:26 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
Message-ID: <08Jul30.095234pdt."58698"@synergy1.parc.xerox.com>

> On Wed, Jul 30, 2008 at 8:09 AM, Andr? Malo <nd at perlig.de> wrote:
> > I'm actually in favour of encoding bytes only back and forth. A useful
> > extension would be *another* function which wraps quote/unquote and encod=
> es
> > and decodes characters.
> 
> I'd reverse this. By all means, add a new pair of functions that is
> bytes in / bytes out. But keep the existing functions purely string in
> / string out, hardcoded to UTF-8. People wanting another encoding can
> use the bytes functions and explicit encode / decode calls.

Actually (as I pointed out before) the existing functions are not
string-in/string-out.  They are something-in and bytes-out.  just look
like string-in/string-out because of the confusion between byte
strings and Unicode strings in Python 1 and 2.

Look, Matt's suggestion is a degradation of the integrity of the
stdlib, because it enthrones a broken understanding, a misreading of
the RFC, in a very prominent place.  I'd prefer not to have Python
contribute to that breakage.  Keep the functions the way they are now:
bytes-in and bytes-out.

Bill

From guido at python.org  Wed Jul 30 19:12:46 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Jul 2008 10:12:46 -0700
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <6046832403127758775@unknownmsgid>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
Message-ID: <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>

On Wed, Jul 30, 2008 at 9:52 AM, Bill Janssen <janssen at parc.com> wrote:
>> On Wed, Jul 30, 2008 at 8:09 AM, Andr? Malo <nd at perlig.de> wrote:
>> > I'm actually in favour of encoding bytes only back and forth. A useful
>> > extension would be *another* function which wraps quote/unquote and encod=
>> es
>> > and decodes characters.
>>
>> I'd reverse this. By all means, add a new pair of functions that is
>> bytes in / bytes out. But keep the existing functions purely string in
>> / string out, hardcoded to UTF-8. People wanting another encoding can
>> use the bytes functions and explicit encode / decode calls.
>
> Actually (as I pointed out before) the existing functions are not
> string-in/string-out.  They are something-in and bytes-out.  just look
> like string-in/string-out because of the confusion between byte
> strings and Unicode strings in Python 1 and 2.

Actually, we'd need to look at the various other APIs in Py3k before
we can decide whether these should be considered taking or returning
bytes or text. It looks like all other APIs in the Py3k version of
urllib treat URLs as text. I don't think switching these to bytes
would be a good idea; you might as well claim that filenames should be
bytes because that's how the filesystem stores them.

> Look, Matt's suggestion is a degradation of the integrity of the
> stdlib, because it enthrones a broken understanding, a misreading of
> the RFC, in a very prominent place.  I'd prefer not to have Python
> contribute to that breakage.  Keep the functions the way they are now:
> bytes-in and bytes-out.

I think that would break too much code, without a good way to
automatically fix it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Jul 30 19:23:47 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 30 Jul 2008 10:23:47 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <08Jul30.095234pdt."58698"@synergy1.parc.xerox.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<08Jul30.095234pdt."58698"@synergy1.parc.xerox.com>
Message-ID: <08Jul30.102355pdt."58698"@synergy1.parc.xerox.com>

> Actually (as I pointed out before) the existing functions are not
> string-in/string-out.  They are something-in and bytes-out.

Sorry, this is wrong.  "quote" is clearly bytes-in and string-out.
"unquote" is clearly string-in and bytes-out.

The whole point of "quote" is to take an arbitrary sequence of bytes
and represent them as an ASCII string, while unquote reverses this
process.  Again, I urge everyone participating in this discussion to
read RFC 3986.  We're not creating in a vacuum here; we're talking
about implementation of a standard.

Bill

From janssen at parc.com  Wed Jul 30 19:33:51 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 30 Jul 2008 10:33:51 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
Message-ID: <08Jul30.103400pdt."58698"@synergy1.parc.xerox.com>

> It looks like all other APIs in the Py3k version of
> urllib treat URLs as text.

The URL is text, a string of ASCII characters.  We're just talking
about urllib.quote() and urllib.unquote(), which are there to support
the text-ization of binary values, and the de-text-ization.

> I think that would break too much code, without a good way to
> automatically fix it.

You'd rather break Python?  Somehow I don't think so.

Here's the signature I'm proposing:

  quote() -- takes string or bytes, and produces string.

     If input is a string, looks to optional "encoding" parameter to
     determine character set encoding to use to transform it to byte before
     quoting it.  If "encoding" is not specified, defaults to UTF-8.

  unquote() -- takes string, produces bytes or string

     If optional "encoding" parameter is specified, decodes bytes with
     that encoding and returns string.  Otherwise, returns bytes.

Bill

From guido at python.org  Wed Jul 30 20:00:09 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Jul 2008 11:00:09 -0700
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <3968642431361249330@unknownmsgid>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
Message-ID: <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>

On Wed, Jul 30, 2008 at 10:33 AM, Bill Janssen <janssen at parc.com> wrote:
>> It looks like all other APIs in the Py3k version of
>> urllib treat URLs as text.
>
> The URL is text, a string of ASCII characters.  We're just talking
> about urllib.quote() and urllib.unquote(), which are there to support
> the text-ization of binary values, and the de-text-ization.
>
>> I think that would break too much code, without a good way to
>> automatically fix it.
>
> You'd rather break Python?  Somehow I don't think so.

Let's stop the rhetoric, or I'll have to beat you over the head with
the Zen of Python. :-)

urllib is not meant as a reference implementation of any RFC; it is
meant as a practical tool for Python users writing web apps (servers
and clients).

> Here's the signature I'm proposing:
>
>  quote() -- takes string or bytes, and produces string.
>
>     If input is a string, looks to optional "encoding" parameter to
>     determine character set encoding to use to transform it to byte before
>     quoting it.  If "encoding" is not specified, defaults to UTF-8.

No contest here, since it supports the common string->string use case.
E.g. quote('a%b') returns 'a%25b'.

>  unquote() -- takes string, produces bytes or string
>
>     If optional "encoding" parameter is specified, decodes bytes with
>     that encoding and returns string.  Otherwise, returns bytes.

The default of returning bytes will break almost all uses. Most code
will uses the unquoted result as a text string, not as bytes -- e.g. a
server has to unquote the values it receives from a form (whether POST
or GET), but almost always the unquoted values are text, e.g.
someone's name or address, or a draft email message.

(Aside: I dislike functions that have a different return type based on
the value of a parameter.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jul 30 20:17:51 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Jul 2008 11:17:51 -0700
Subject: [Python-Dev] Fuzzing bugs: most bugs are closed
In-Reply-To: <20080721174141.GA16598@amk-desktop.matrixgroup.net>
References: <200807191323.13124.victor.stinner@haypocalc.com>
	<200807202245.39874.victor.stinner@haypocalc.com>
	<20080721133319.GA15912@amk-desktop.matrixgroup.net>
	<200807211554.11468.victor.stinner@haypocalc.com>
	<loom.20080721T154744-225@post.gmane.org>
	<20080721174141.GA16598@amk-desktop.matrixgroup.net>
Message-ID: <ca471dc20807301117y661306f4r365c08c7483b0bfc@mail.gmail.com>

On Mon, Jul 21, 2008 at 10:41 AM, A.M. Kuchling <amk at amk.ca> wrote:
> On Mon, Jul 21, 2008 at 03:53:18PM +0000, Antoine Pitrou wrote:
>> The underscore at the beginning of _sre clearly indicates that the module is
>> not recommended for direct consumption, IMO. Even the functions that don't
>> themselves start with an underscore...
>
> Sure, but if someone is trying to break in or DoS your application
> server, they don't care if the module starts with an underscore or
> not.
>
> To answer Victor's original question: the parser & compiler that turn
> a regex into bytecode is written in Python.  I can't think of a way to
> prevent other Python modules from importing _sre or accessing the
> compile() function; if nothing else, code could always do 'import re ;
> re.sre_compile._sre.compile(...)'.

I've written a re-code verifier for the Google App Engine. I have
permission to open source this, hopefully I will get to this before
2.6 beta 3.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From hall.jeff at gmail.com  Wed Jul 30 20:38:40 2008
From: hall.jeff at gmail.com (Jeff Hall)
Date: Wed, 30 Jul 2008 14:38:40 -0400
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
Message-ID: <1bc395c10807301138m5e39febbpf2c058720d9b38a0@mail.gmail.com>

>
>
> (Aside: I dislike functions that have a different return type based on
> the value of a parameter.)
>
>
I wanted to stay out of the whole discussion as it's largely over my head...
But I did want to express support for this idea which I think almost rises
to the level of a standard... I see more bugs created in our software
because of the above issues then anything else... I have no problem with
functions that accept various input but producing various outputs just seems
to wreak havoc...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080730/f17696aa/attachment-0001.htm>

From janssen at parc.com  Wed Jul 30 21:49:06 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 30 Jul 2008 12:49:06 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
Message-ID: <08Jul30.124910pdt."58698"@synergy1.parc.xerox.com>

> >  unquote() -- takes string, produces bytes or string
> >
> >     If optional "encoding" parameter is specified, decodes bytes with
> >     that encoding and returns string.  Otherwise, returns bytes.
> 
> The default of returning bytes will break almost all uses. Most code
> will uses the unquoted result as a text string, not as bytes -- e.g. a
> server has to unquote the values it receives from a form (whether POST
> or GET), but almost always the unquoted values are text, e.g.
> someone's name or address, or a draft email message.

I actually do know a lot about the uses of this function...

But:  OK, OK, I yield.  Though I still think this is a bad idea, I'll
shut up if we can also add "unquote_as_bytes" which returns a byte
sequence instead of a string.  I'll just change my code to use that.

> (Aside: I dislike functions that have a different return type based on
> the value of a parameter.)

Fair enough.

Bill

From guido at python.org  Wed Jul 30 22:32:24 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Jul 2008 13:32:24 -0700
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <7769003638770459002@unknownmsgid>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
Message-ID: <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>

On Wed, Jul 30, 2008 at 12:49 PM, Bill Janssen <janssen at parc.com> wrote:
>> >  unquote() -- takes string, produces bytes or string
>> >
>> >     If optional "encoding" parameter is specified, decodes bytes with
>> >     that encoding and returns string.  Otherwise, returns bytes.
>>
>> The default of returning bytes will break almost all uses. Most code
>> will uses the unquoted result as a text string, not as bytes -- e.g. a
>> server has to unquote the values it receives from a form (whether POST
>> or GET), but almost always the unquoted values are text, e.g.
>> someone's name or address, or a draft email message.
>
> I actually do know a lot about the uses of this function...
>
> But:  OK, OK, I yield.  Though I still think this is a bad idea, I'll
> shut up if we can also add "unquote_as_bytes" which returns a byte
> sequence instead of a string.  I'll just change my code to use that.
>
>> (Aside: I dislike functions that have a different return type based on
>> the value of a parameter.)
>
> Fair enough.

I think this is as close as consensus as we can get on this issue. Can
whoever wrote the patch adjust the patch to this outcome? (I think the
only change is to remove the encoding arguments and make separate
functions for bytes.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Thu Jul 31 02:25:25 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 30 Jul 2008 17:25:25 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<-6533647106000374959@unknownmsgid>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
Message-ID: <08Jul30.172526pdt."58698"@synergy1.parc.xerox.com>

> I think this is as close as consensus as we can get on this issue. Can
> whoever wrote the patch adjust the patch to this outcome? (I think the
> only change is to remove the encoding arguments and make separate
> functions for bytes.)

This is 2.7/3.1 only, right?  I'm looking at the bales of code I've
got that says something like,

  v = urlib.quote_plus(x.encode("UTF-8", "strict"))

then later on

  x = unicode(urllib.unquote_plus(v), "UTF-8", "strict")

Bill


From musiccomposition at gmail.com  Thu Jul 31 04:31:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 30 Jul 2008 21:31:37 -0500
Subject: [Python-Dev] critical issues for 2.6 and 3.0
Message-ID: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com>

I just went through the disturbingly long list of 67 open issues with
a "critical" priority pinging and trying to get things moving. There
are ~55 now; I was able to close some, but others I promoted to
release blocker for beta 3. Shouldn't all criticals be resolved by the
final?

I've never been through a Python release before, but I find these
statistics rather worrying if we want to make the October release
date. It doesn't help that we are low on active core developers,
presumably because they are taking full advantage of their summer
vacations. :) (Speaking of which, I'm leaving this Saturday.)

Please focus getting fixes reviewed, checked in, and their issue's
closed so we can bring beta 3 out on time!

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From brett at python.org  Thu Jul 31 04:51:38 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 30 Jul 2008 19:51:38 -0700
Subject: [Python-Dev] critical issues for 2.6 and 3.0
In-Reply-To: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com>
References: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com>
Message-ID: <bbaeab100807301951k511afae0g86d1e53c0943e199@mail.gmail.com>

On Wed, Jul 30, 2008 at 7:31 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> I just went through the disturbingly long list of 67 open issues with
> a "critical" priority pinging and trying to get things moving. There
> are ~55 now; I was able to close some, but others I promoted to
> release blocker for beta 3. Shouldn't all criticals be resolved by the
> final?
>

Probably, but at that point they will be promoated to release blocker
as necessary.

> I've never been through a Python release before, but I find these
> statistics rather worrying if we want to make the October release
> date.

If we don't make the release, then we don't make it. Plus this is one
of the more complicated releases that I have been through thanks to
the release of two simultaneous major revisions, so having a lot to do
is not a shock. But people tend to step up work when a beta release is
coming so when we get closer to b3 more work will probably land.

Another thing to keep in mind beyond the open issues is the code in
2.6 that is not 3.0 compatible when Python is run with -3. I just
finished running regrtest with -3 and have a text file listing all of
the code that has some warning thanks to -3. I will try to open an
issue with those files listed as some point soon, but I will hopefully
be able to plow through them rather quickly since most of them are
minor like dict.has_key(), etc.

-Brett

From matt.giuca at gmail.com  Thu Jul 31 05:49:17 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Thu, 31 Jul 2008 13:49:17 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
Message-ID: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>

> Con: URI encoding does not encode characters.

OK, for all the people who say URI encoding does not encode characters: yes
it does. This is not an encoding for binary data, it's an encoding for
character data, but it's unspecified how the strings map to octets before
being percent-encoded. From RFC 3986, section
1.2.1<http://tools.ietf.org/html/rfc3986#section-1.2.1>
:

Percent-encoded octets (Section 2.1) may be used within a URI to represent
> characters outside the range of the US-ASCII coded character set if this
> representation is allowed by the scheme or by the protocol element in which
> the URI is referenced.  Such a definition should specify the character
> encoding used to map those characters to octets prior to being
> percent-encoded for the URI.


So the string->string proposal is actually correct behaviour. I'm all in
favour of a bytes->string version as well, just not with the names "quote"
and "unquote".

I'll prepare a new patch shortly which has bytes->string and string->bytes
versions of the functions as well. (quote will accept either type, while
unquote will output a str, there will be a new function unquote_to_bytes
which outputs a bytes - is everyone happy with that?)

Guido says:

> Actually, we'd need to look at the various other APIs in Py3k before we can
> decide whether these should be considered taking or returning bytes or text.
> It looks like all other APIs in the Py3k version of urllib treat URLs as
> text.


Yes, as I said in the bug tracker, I've groveled over the entire stdlib to
see how my patch affects the behaviour of dependent code. Aside from a few
minor bits which assumed octets (and did their own encoding/decoding) (which
I fixed), all the code assumes strings and is very happy to go on assuming
this, as long as the URIs are encoded with UTF-8, which they almost
certainly are.

Guido says:

> I think the only change is to remove the encoding arguments and ...


You really want me to remove the encoding= named argument? And hard-code
UTF-8 into these functions? It seems like we may as well have the optional
encoding argument, as it does no harm and could be of significant benefit.
I'll post a patch with the unquote_to_bytes function, but leave the encoding
arguments in until this point is clarified.

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/209fd01b/attachment.htm>

From guido at python.org  Thu Jul 31 06:01:56 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Jul 2008 21:01:56 -0700
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
Message-ID: <ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com>

On Wed, Jul 30, 2008 at 8:49 PM, Matt Giuca <matt.giuca at gmail.com> wrote:
>
>> Con: URI encoding does not encode characters.
>
> OK, for all the people who say URI encoding does not encode characters: yes
> it does. This is not an encoding for binary data, it's an encoding for
> character data, but it's unspecified how the strings map to octets before
> being percent-encoded. From RFC 3986, section 1.2.1:
>
>> Percent-encoded octets (Section 2.1) may be used within a URI to represent
>> characters outside the range of the US-ASCII coded character set if this
>> representation is allowed by the scheme or by the protocol element in which
>> the URI is referenced.  Such a definition should specify the character
>> encoding used to map those characters to octets prior to being
>> percent-encoded for the URI.
>
> So the string->string proposal is actually correct behaviour. I'm all in
> favour of a bytes->string version as well, just not with the names "quote"
> and "unquote".
>
> I'll prepare a new patch shortly which has bytes->string and string->bytes
> versions of the functions as well. (quote will accept either type, while
> unquote will output a str, there will be a new function unquote_to_bytes
> which outputs a bytes - is everyone happy with that?)

I'd rather have two pairs of functions, so that those who want to give
the readers of their code a clue can do so. I'm not opposed to having
redundant functions that accept either string or bytes though, unless
others prefer not to.

> Guido says:
>>
>> Actually, we'd need to look at the various other APIs in Py3k before we
>> can decide whether these should be considered taking or returning bytes or
>> text. It looks like all other APIs in the Py3k version of urllib treat URLs
>> as text.
>
> Yes, as I said in the bug tracker, I've groveled over the entire stdlib to
> see how my patch affects the behaviour of dependent code. Aside from a few
> minor bits which assumed octets (and did their own encoding/decoding) (which
> I fixed), all the code assumes strings and is very happy to go on assuming
> this, as long as the URIs are encoded with UTF-8, which they almost
> certainly are.

Sorry, I have yet to look at the tracker (only so many minutes in a day...).

> Guido says:
>>
>> I think the only change is to remove the encoding arguments and ...
>
> You really want me to remove the encoding= named argument? And hard-code
> UTF-8 into these functions? It seems like we may as well have the optional
> encoding argument, as it does no harm and could be of significant benefit.
> I'll post a patch with the unquote_to_bytes function, but leave the encoding
> arguments in until this point is clarified.

I don't mind an encoding argument, as long as it isn't used to change
the return type (as Bill was proposing).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From sumant.gupta at aricent.com  Thu Jul 31 07:01:42 2008
From: sumant.gupta at aricent.com (Sumant Gupta)
Date: Thu, 31 Jul 2008 10:31:42 +0530
Subject: [Python-Dev] Memory Error while reading large file
Message-ID: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM>

Hi

I have a problem reading very large text file.
When I call the len function to get the total lines in python file.i get memory error .
I am reading the list of files in a loop ,2 files are read properly but when the third file is read ,
It gives an memory error .

Sumant Gupta
Software Engineer
Ext:5105


________________________________
"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error,please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility forloss or damage arising from the use of the information transmitted by this email including damage from virus."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/7ad461ea/attachment.htm>

From turnbull at sk.tsukuba.ac.jp  Thu Jul 31 08:36:30 2008
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Thu, 31 Jul 2008 15:36:30 +0900
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
Message-ID: <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp>

Matt Giuca writes:

 > OK, for all the people who say URI encoding does not encode characters: yes
 > it does. This is not an encoding for binary data, it's an encoding for
 > character data, but it's unspecified how the strings map to octets before
 > being percent-encoded.

In other words, it's an encoding for binary data, since the octet
sequences that might be encountered are completely unrestricted.  I
have to side with Bill on this.  URIs are sequences of characters, but
the character set used must contain the ASCII repertoire as a subset,
of which the URI delimiters must be mapped to the corresponding ASCII
codes, the rest of the set must be represented as sequences of octets
(which need not even be constant; you could gzip them first for all
URI-encoding cares).

URI-encoding itself is a purely mechanical process which transforms
reserved octets (not used as delimiters) to percent codes.

 > From RFC 3986, section
 > 1.2.1<http://tools.ietf.org/html/rfc3986#section-1.2.1>:

 > > Percent-encoded octets (Section 2.1) may be used within a URI to represent
 > > characters outside the range of the US-ASCII coded character set if this
 > > representation is allowed by the scheme or by the protocol element in which
 > > the URI is referenced.  Such a definition should specify the character
 > > encoding used to map those characters to octets prior to being
 > > percent-encoded for the URI.

This is kinda perverted, but suppose you have bytes which are actually a
Japanese string represented in packed EUC-JP.  AFAICS the paragraph above
does *not* say you can't transcode to UTF-8 before percent-encoding, and
in fact you might be required to by the definition of the scheme.

 > So the string->string proposal is actually correct behaviour.

Ye-e-es, but.  What the RFC clearly envisions is not that the
percent-encoder will be handed an unencoded string that looks like a
URI, but rather a sequence of octets representing one component
(scheme, authority, path, query, etc) of a URI.

In other words, a string->string URI encoder should only be called by
an URI builder, and never with a precomposed URI-like string.

Something like

def URIBuilder (strings):
    """Return an URI built from a list of strings.
    The first string *must* be the scheme.
    If the URI follows the generic URI syntax of RFC 3986, the
    remaining components should be given in the order authority, path,
    fragment, query part [, query part ...]."""

    def uriencode (s):
        """URI encode a string per RFC 3986 Section 3."""
        # We all know what this does.

    if strings[0] == "http":
        # HTTP scheme, delimiters and authority
        uri = "http://" + uriencode(strings[1]) + "/"
        # path, if present
        if strings[2]:
            uri = uri + uriencode(strings[2])
        # query, if present
        if  strings[4]:
            uri = uri + "?" + uriencode(strings[4])
        # further query parameters, if present
        for s in strings[4:]
            uri = uri + ";" + uriencode(s)
        # fragment, if present
        if strings[3]:
            uri = uri + "#" + uriencode(strings[3])
    else if strings[0] == "mailto":
        uri = "mailto:" + uriencode(strings[1])
    # etc etc

    return uri

I think you'd have a much easier time enforcing this pedantically
correct usage with a bytes->bytes encoder.

Of course, it's un-Pythonic to enforce pedantry, and we pedants can
use a string->string encoder correctly.

 > You really want me to remove the encoding= named argument? And hard-code
 > UTF-8 into these functions?

A quoting function that accepts bytes *must* have an encoding
argument.  There's no point to passing the quoter bytes unless the
text is represented in a non-Unicode encoding.


From steve at pearwood.info  Thu Jul 31 09:02:29 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 31 Jul 2008 17:02:29 +1000
Subject: [Python-Dev] Memory Error while reading large file
In-Reply-To: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM>
References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM>
Message-ID: <200807311702.30163.steve@pearwood.info>

On Thu, 31 Jul 2008 03:01:42 pm Sumant Gupta wrote:
> Hi
>
> I have a problem reading very large text file.
> When I call the len function to get the total lines in python file.i
> get memory error . I am reading the list of files in a loop ,2 files
> are read properly but when the third file is read , It gives an
> memory error .

I'm not completely sure, but I think that means you're out of memory.

If you have an actual question, I think you would be better off posting 
to the comp.lang.python newsgroup. This mailing list is for development 
of the Python compiler, not for writing Python programs.

I'll save you some time: when you post to comp.lang.python, you should 
post the actual error message you get, and the code that fails.


-- 
Steven

From janssen at parc.com  Thu Jul 31 09:39:29 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 31 Jul 2008 00:39:29 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
Message-ID: <08Jul31.003936pdt."58698"@synergy1.parc.xerox.com>

> Guido says:
> 
> > Actually, we'd need to look at the various other APIs in Py3k before we can
> > decide whether these should be considered taking or returning bytes or text.
> > It looks like all other APIs in the Py3k version of urllib treat URLs as
> > text.
> 
> 
> Yes, as I said in the bug tracker, I've groveled over the entire stdlib to
> see how my patch affects the behaviour of dependent code. Aside from a few
> minor bits which assumed octets (and did their own encoding/decoding) (which
> I fixed), all the code assumes strings and is very happy to go on assuming
> this, as long as the URIs are encoded with UTF-8, which they almost
> certainly are.

I'm not sure that's sufficient review, though I agree it's necessary.
The major consumers of quote/unquote are not in the Python standard
library.

> (quote will accept either type, while
> unquote will output a str, there will be a new function unquote_to_bytes
> which outputs a bytes - is everyone happy with that?)

No, so don't ask.

Bill

From janssen at parc.com  Thu Jul 31 09:47:12 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 31 Jul 2008 00:47:12 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
	<87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <08Jul31.004722pdt."58698"@synergy1.parc.xerox.com>

> Of course, it's un-Pythonic to enforce pedantry, and we pedants can
> use a string->string encoder correctly.

Sure.  All I was asking was that we not break the existing usage of
the standard library "unquote" by producing a string by *assuming* a
UTF-8 encoded string is what's in those percent-encoded bytes (instead
of, say, ISO 2022-JP).  Let the "new" function produce a string:
"unquote_as_string".

>  > You really want me to remove the encoding= named argument? And hard-code
>  > UTF-8 into these functions?
> 
> A quoting function that accepts bytes *must* have an encoding
> argument.

Huh?  What would it use it for?  The string, if string it is, is
already encoded as octets.  All it needs to do is percent-encode the
reserved octets.  So far as I can see, the "unquote_as_string" is the
function that needs the encoding.  Ah, it's too late, I'll pick this
up tomorrow.

Bill

From janssen at parc.com  Thu Jul 31 09:52:32 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 31 Jul 2008 00:52:32 PDT
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> 
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
	<87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <08Jul31.005237pdt."58698"@synergy1.parc.xerox.com>

Also see <http://en.wikipedia.org/wiki/Percent-encoding>.

Bill

From stephen at xemacs.org  Thu Jul 31 12:13:35 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 31 Jul 2008 19:13:35 +0900
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <08Jul31.004722pdt."58698"@synergy1.parc.xerox.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com>
	<200807301709.51528.nd@perlig.de>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
	<87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<08Jul31.004722pdt."58698"@synergy1.parc.xerox.com>
Message-ID: <87od4epdio.fsf@uwakimon.sk.tsukuba.ac.jp>

Bill Janssen writes:

 > > A quoting function that accepts bytes *must* have an encoding
 > > argument.
 > 
 > Huh?  What would it use it for?

Ah, you're right.  I was thinking in terms of an URI builder, where the
quoter would do any required conversion (eg, if the bytes represented
a string in Japanese) to another (possibly scheme-mandated) encoding
(typically UTF-8).  But that doesn't really make sense; the URI
builder should know what to do, and that's a better place to do such
conversions.


From matt.giuca at gmail.com  Thu Jul 31 14:07:36 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Thu, 31 Jul 2008 22:07:36 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
	<ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com>
Message-ID: <e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com>

Alright, I've uploaded the new patch which adds the two requested
bytes-oriented functions, as well as accompanying docs and tests.
http://bugs.python.org/issue3300
http://bugs.python.org/file11009/parse.py.patch6

I'd rather have two pairs of functions, so that those who want to give
> the readers of their code a clue can do so. I'm not opposed to having
> redundant functions that accept either string or bytes though, unless
> others prefer not to.
>

Yes, I was in a similar mindset. So the way I've implemented it, quote
accepts either a bytes or a str. Then there's a new function
quote_from_bytes, which is defined precisely like this:

quote_from_bytes = quote
>

So either name can be used on either input type, with the idea being that
you should use quote on a str, and quote_from_bytes on a bytes. Is this a
good idea or should it be rewritten so each function permits only one input
type?

Sorry, I have yet to look at the tracker (only so many minutes in a day...).


Ah, I didn't mean offense. Just that one could read the sordid details of my
investigation on the tracker if one so desired ;)

I don't mind an encoding argument, as long as it isn't used to change
> the return type (as Bill was proposing).


Yeah, my unquote always outputs a str, and unquote_to_bytes always outputs a
bytes.

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/85e6eb53/attachment.htm>

From matt.giuca at gmail.com  Thu Jul 31 14:25:09 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Thu, 31 Jul 2008 22:25:09 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <3166922734065744385@unknownmsgid>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
	<3166922734065744385@unknownmsgid>
Message-ID: <e6268af30807310525l255052deo5e9ad3dbde776bc6@mail.gmail.com>

Bill wrote:

I'm not sure that's sufficient review, though I agree it's necessary.
>
The major consumers of quote/unquote are not in the Python standard
>
library.


I figured that Python 3.0 is designed to fix things, with the breaking
third-party code being an acceptable side-effect of that. So the most
important thing when 3.0 is released is that the stdlib is internally
consistent. All other code is "allowed" to be broken. So I've investigated
all the code necessary.

Having said this, my patch breaks almost no code. Your suggestion breaks a
hell of a lot.

Sure.  All I was asking was that we not break the existing usage of
>
the standard library "unquote" by producing a string by *assuming* a
>
UTF-8 encoded string is what's in those percent-encoded bytes (instead
>
of, say, ISO 2022-JP).  Let the "new" function produce a string:
>
"unquote_as_string".


You're assuming that a Python 2.x "str" is the same thing as a Python 3.0
"bytes". It isn't. (If it was, this transition would be trivial). A Python 2
"str" is a non-Unicode string. It can be printed, concatenated with Unicode
strings, etc etc. It has the semantics of a string. The Python 3.0 "bytes"
is not a string at all.

What you're saying is "the old behaviour was to output a bytes, so the new
behaviour should be consistent". But that isn't true - the old behaviour was
to output a string (a non-Unicode one). People, and code, expect it to
output something with string semantics. So making unquote output a bytes is
just as big a change as making it output a (unicode) str. Python 3.0 doesn't
have a type which is like Python 2's "str" type (which is good - that type
was very messy). So the argument that "Python 2 unquote outputs a bytes, so
we should too" is not legitimate.



If you want to keep pushing this, please install my new patch (patch 6).
Then rename "unquote" to "unquote_to_string" and rename "unquote_to_bytes"
to "unquote", and witness the havoc that ensues. Firstly, you break most
Internet-related modules in the standard library.

10 tests failed:
>
    test_SimpleHTTPServer test_cgi test_email test_http_cookiejar
>
    test_httpservers test_robotparser test_urllib test_urllib2
>
    test_urllib2_localnet test_wsgiref
>

Fixing these isn't a matter of changing test cases (which all but one of my
fixes were). It would require changes to all the modules, to get them to
deal with bytes instead of strings (which would generally mean spraying
.decode("utf-8") all over the place). My code, on the other hand, "tends to
be" compatible with 2.x code.

Here I'm seeing:
BytesWarning: Comparison between bytes and string.
TypeError: expected an object with the buffer interface
http.client.BadStatusLine

For another example, try this:

>>> import http.server
>>> s = http.server.HTTPServer(('',8000),
http.server.SimpleHTTPRequestHandler)
>>> s.serve_forever()

The current (unpatched) build works, but links to files with non-ASCII
filenames (eg. '??') break, because of the URL. This is one example of my
patch directly fixing a bug in real code. With my patch applied, the links
work fine *because URL quoting and unquoting are consistent, and work on all
Unicode characters*.

If you change unquote to output a bytes, it breaks completely. You get a
"TypeError: expected an object with the buffer interface" as soon as the
user visits the page.

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/7bf35eb4/attachment.htm>

From ncoghlan at gmail.com  Thu Jul 31 15:51:42 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 31 Jul 2008 23:51:42 +1000
Subject: [Python-Dev] Matrix product
In-Reply-To: <b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>	<488A8302.3050408@canterbury.ac.nz>	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>	<3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>	<b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com>
	<b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com>
Message-ID: <4891C36E.4070200@gmail.com>

Sebastien Loisel wrote:
> Dear Raymond,
> 
> Thank you for your email.
> 
>> I think much of this thread is a repeat of conversations
>> that were held for PEP 225:
>> http://www.python.org/dev/peps/pep-0225/
>>
>> That PEP is marked as deferred.  Maybe it's time to
>> bring it back to life.
> 
> This is a much better PEP than the one I had found, and would solve
> all of the numpy problems. The PEP is very well thought-out.

A very interesting read! I wouldn't support some of the more exotic 
elements tacked on to the end (particularly the replacement of the now 
thoroughly entrenched bitwise operators), but the basic idea of 
providing ~op variants of several operators seems fairly sound. I'd be 
somewhat inclined to add ~not, ~and and ~or to the list  even though 
that would pretty much force the semantics to be elementwise for the ~ 
variants (since the standard not, and and or are always objectwise and 
without PEP 335 there's no way for an object to change that).

Cheers,
Nick.

From hall.jeff at gmail.com  Thu Jul 31 18:33:47 2008
From: hall.jeff at gmail.com (Jeff Hall)
Date: Thu, 31 Jul 2008 12:33:47 -0400
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<6046832403127758775@unknownmsgid>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
	<ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com>
	<e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com>
Message-ID: <1bc395c10807310933n519ac306v7656c83093e7cb6c@mail.gmail.com>

>
>
> quote_from_bytes = quote
>>
>
> So either name can be used on either input type, with the idea being that
> you should use quote on a str, and quote_from_bytes on a bytes. Is this a
> good idea or should it be rewritten so each function permits only one input
> type?
>
>
so you can use quote_from_bytes on strings? I assumed Guido meant it was
okay to have quote accept string/byte input and have a function that was
redundant but limited in what it accepted (i.e. quote_from_bytes accepts
only bytes)

I suppose your implementation doesn't break anything... it just strikes me
as "odd"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/e1c9be28/attachment.htm>

From matt.giuca at gmail.com  Thu Jul 31 19:19:57 2008
From: matt.giuca at gmail.com (Matt Giuca)
Date: Fri, 1 Aug 2008 03:19:57 +1000
Subject: [Python-Dev] urllib.quote and unquote - Unicode issues
In-Reply-To: <1bc395c10807310933n519ac306v7656c83093e7cb6c@mail.gmail.com>
References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com>
	<ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com>
	<3968642431361249330@unknownmsgid>
	<ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com>
	<7769003638770459002@unknownmsgid>
	<ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com>
	<e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com>
	<ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com>
	<e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com>
	<1bc395c10807310933n519ac306v7656c83093e7cb6c@mail.gmail.com>
Message-ID: <e6268af30807311019j14519f5eh233e827313b358e3@mail.gmail.com>

> so you can use quote_from_bytes on strings?

Yes, currently.

> I assumed Guido meant it was okay to have quote accept string/byte input and have a function that was redundant but limited in what it accepted (i.e. quote_from_bytes accepts only bytes)
>
> I suppose your implementation doesn't break anything... it just strikes me as "odd"

Yeah. I get exactly what you mean. Worse is it takes an
encoding/replace argument.

I'm in two minds about whether it should allow this or not. On one
hand, it kind of goes with the Python philosophy of not artificially
restricting the allowed types. And it avoids redundancy in the code.

But I'd be quite happy to let quote_from_bytes restrict its input to
just bytes, to avoid confusion. It would basically be a
slightly-modified version of quote:

def quote_from_bytes(s, safe = '/'):
    if isinstance(safe, str):
        safe = safe.encode('ascii', 'ignore')
    cachekey = (safe, always_safe)
    if not isinstance(s, bytes) or isinstance(s, bytearray):
        raise TypeError("quote_from_bytes() expected a bytes")
    try:
        quoter = _safe_quoters[cachekey]
    except KeyError:
        quoter = Quoter(safe)
        _safe_quoters[cachekey] = quoter
    res = map(quoter, s)
    return ''.join(res)

(Passes test suite).

I think I'm happier with this option. But the "if not isinstance(s,
bytes) or isinstance(s, bytearray)" is not very nice.
(The only difference to quote besides the missing arguments is the two
lines beginning "if not isinstance". Maybe we can generalise the rest
of the function).

From cesare at pronto.it  Thu Jul 31 21:25:57 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Thu, 31 Jul 2008 21:25:57 +0200
Subject: [Python-Dev] Matrix product
In-Reply-To: <4891C36E.4070200@gmail.com>
References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com>
	<488A8302.3050408@canterbury.ac.nz>
	<ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com>
	<3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com>
	<b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com>
	<b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com>
	<4891C36E.4070200@gmail.com>
Message-ID: <op.ue579jzuhlrjc9@conan>

Nick Coghlan write:

> Sebastien Loisel wrote:
>> Dear Raymond,
>>
>> Thank you for your email.
>>
>>> I think much of this thread is a repeat of conversations
>>> that were held for PEP 225:
>>> http://www.python.org/dev/peps/pep-0225/
>>>
>>> That PEP is marked as deferred.  Maybe it's time to
>>> bring it back to life.
>>
>> This is a much better PEP than the one I had found, and would solve
>> all of the numpy problems. The PEP is very well thought-out.
>
> A very interesting read! I wouldn't support some of the more exotic
> elements tacked on to the end (particularly the replacement of the now
> thoroughly entrenched bitwise operators), but the basic idea of
> providing ~op variants of several operators seems fairly sound. I'd be
> somewhat inclined to add ~not, ~and and ~or to the list  even though
> that would pretty much force the semantics to be elementwise for the ~
> variants (since the standard not, and and or are always objectwise and
> without PEP 335 there's no way for an object to change that).
>
> Cheers,
> Nick.

I agree: adding ~op will be very interesting.

For example, we can easily provide case insensitive comparisons for string:

if foo ~== 'Spam':
  print "It's spam!'

equivalent to:

if foo.upper() == 'SPAM:
  print "It's spam!'

we can save both CPU time and memory to build
a brand new string that will be discarded after the comparison...

It will be also useful to redefine /, // and ** operators to do some common operations:

'spam, egg' / ', '  could be equivalent to iter('spam, egg'.split(', ')) # Generates an iterator

'spam, egg' // ', '  could be equivalent to 'spam, egg'.split(', ') # Generates a list

and ', ' ** ('spam', 'egg') could be equivalent to ', '.join(('spam', 'egg'))

but unfortunately we know that at the moment buil-in types
cannot be "extended" through "monkey patching"...

Cesare

From martin at v.loewis.de  Thu Jul 31 22:35:50 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 31 Jul 2008 22:35:50 +0200
Subject: [Python-Dev] critical issues for 2.6 and 3.0
In-Reply-To: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com>
References: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com>
Message-ID: <48922226.90003@v.loewis.de>

> I've never been through a Python release before, but I find these
> statistics rather worrying if we want to make the October release
> date.

I don't worry. Every Python release had bugs, and there will be
2.6.1 and 3.0.1 releases.

The only sure way to resolve bugs is to revert features. If a certain
module is cause of too many serious bugs, it should be dropped from
the release (perhaps not from the source repository - just removed
from all build processes).

Regards,
Martin

From martin at v.loewis.de  Thu Jul 31 22:40:48 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 31 Jul 2008 22:40:48 +0200
Subject: [Python-Dev] Memory Error while reading large file
In-Reply-To: <200807311702.30163.steve@pearwood.info>
References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM>
	<200807311702.30163.steve@pearwood.info>
Message-ID: <48922350.9030102@v.loewis.de>

> If you have an actual question

I'd like to stress this point as well. Any good posting one
wants an answer to must include a question, and that question
must be explicitly phrased, and terminated with a question
mark.

(maybe the use of the question mark is more typical in German	
than in English; my stomach turns around when I read a question
that ends with a full stop)

Regards,
Martin

From scott+python-dev at scottdial.com  Thu Jul 31 23:38:42 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Thu, 31 Jul 2008 17:38:42 -0400
Subject: [Python-Dev] Memory Error while reading large file
In-Reply-To: <48922350.9030102@v.loewis.de>
References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM>	<200807311702.30163.steve@pearwood.info>
	<48922350.9030102@v.loewis.de>
Message-ID: <489230E2.501@scottdial.com>

Martin v. L?wis wrote:
> (maybe the use of the question mark is more typical in German	
> than in English; my stomach turns around when I read a question
> that ends with a full stop)

There is no loss in translation here. Proper English requires the use of 
a question mark just the same as German, but you can't assume proper 
English will be used on a forum of communication like this one. The OP 
stated his problem, and maybe he doesn't know enough English to actually 
ask his question (I'm guessing by the name "Sumant Gupta"). I don't 
believe you are native speaker yourself, and I would've expected more 
sympathy from you. Lord knows I hope the recipients of any German I 
write will have some.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From guido at python.org  Thu Jul 31 23:43:55 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 31 Jul 2008 14:43:55 -0700
Subject: [Python-Dev] Memory Error while reading large file
In-Reply-To: <489230E2.501@scottdial.com>
References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM>
	<200807311702.30163.steve@pearwood.info>
	<48922350.9030102@v.loewis.de> <489230E2.501@scottdial.com>
Message-ID: <ca471dc20807311443u514495eo1d59c913906a20ee@mail.gmail.com>

On Thu, Jul 31, 2008 at 2:38 PM, Scott Dial
<scott+python-dev at scottdial.com> wrote:
> Martin v. L?wis wrote:
>>
>> (maybe the use of the question mark is more typical in German
>> than in English; my stomach turns around when I read a question
>> that ends with a full stop)
>
> There is no loss in translation here. Proper English requires the use of a
> question mark just the same as German, but you can't assume proper English
> will be used on a forum of communication like this one. The OP stated his
> problem, and maybe he doesn't know enough English to actually ask his
> question (I'm guessing by the name "Sumant Gupta"). I don't believe you are
> native speaker yourself, and I would've expected more sympathy from you.
> Lord knows I hope the recipients of any German I write will have some.

On the level of mercy: (a) this is python-dev, which is explicitly
*not* for user questions; (b) the OP didn't show any actual code nor
error messages, which makes it impossible to help him unless you're
clairvoyant. Unfortunately we get quite a few of such ill-defined
problems in this list, despite it not being the wrong list, and my own
patience wears thin at times too. (I also get quite a bit of personal
mail of the same nature.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)