From guido at python.org Tue Jul 1 01:12:03 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jun 2008 16:12:03 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806300931x7635cee5t597c09ff7b06fc6f@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> <5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com> <ca471dc20806300853m7d2f269bibaf753c18c6f48b3@mail.gmail.com> <5c6f2a5d0806300931x7635cee5t597c09ff7b06fc6f@mail.gmail.com> Message-ID: <ca471dc20806301612m3019d300tabd552590146bcdc@mail.gmail.com> Mon, Jun 30, 2008 at 9:31 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > On Mon, Jun 30, 2008 at 4:53 PM, Guido van Rossum <guido at python.org> wrote: >> FWIW, I'm fine with making these methods on float -- a class method >> float.fromhex(...) echoes e.g. dict.fromkeys(...) and >> datetime.fromordinal(...). The to-hex conversion could be x.hex() -- >> we don't tend to use ".toxyz()" as a naming convention much in Python. > > Would it be totally outrageous for the float constructor to accept > hex strings directly? int('0x10') raises a ValueError as well. You might propose float('0x...p...', 16) but since the format is so specifically different I think that's not completely kosher. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Jul 1 22:16:37 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 1 Jul 2008 16:16:37 -0400 Subject: [Python-Dev] Second betas tomorrow Message-ID: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Wow, I bet this one crept up on you as quickly as it did me! We have our second planned beta releases for 2.6 and 3.0 tomorrow. As usual I will start looking at blockers and buildbots tomorrow afternoon (UTC-4 time) with a plan to start building things at about 6pm. Also, I will of course be in #python-dev on freenode to answer any questions, or get second opinions. PEP 361 claims that these will be the last betas. Whether that's true or not depends on how well the beta2's go. Please help review code or fix bugs. If you know of things that absolutely must go into beta2, be sure there is an open release-blocker bug on the issue. Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGqQpnEjvBPtnXfVAQLdagP/VooK8+AoPrb1bR7xAxGqg0vC1HOKw5qZ 8VQArzgldz1OnoG24PuKGdaEw7PbHjCMkD0/CyZWjH8/yWawcxV7hKl6RYHJ3GX9 keroo7wz3/NaptJtA9ldoKA5ekV8WVVC5OElgtjKr+v6HorPQSHzUgJiDHYUS1FW A8fdHipyZds= =vwYy -----END PGP SIGNATURE----- From guido at python.org Tue Jul 1 22:25:27 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 1 Jul 2008 13:25:27 -0700 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> Message-ID: <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> I think we should put this one off. The previous betas were done on June 18, and IMO the next beta should be about a month afterwards, not 2 weeks. On Tue, Jul 1, 2008 at 1:16 PM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Wow, I bet this one crept up on you as quickly as it did me! > > We have our second planned beta releases for 2.6 and 3.0 tomorrow. As > usual I will start looking at blockers and buildbots tomorrow afternoon > (UTC-4 time) with a plan to start building things at about 6pm. Also, I > will of course be in #python-dev on freenode to answer any questions, or get > second opinions. > > PEP 361 claims that these will be the last betas. Whether that's true or > not depends on how well the beta2's go. Please help review code or fix > bugs. If you know of things that absolutely must go into beta2, be sure > there is an open release-blocker bug on the issue. > > Thanks, > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iQCVAwUBSGqQpnEjvBPtnXfVAQLdagP/VooK8+AoPrb1bR7xAxGqg0vC1HOKw5qZ > 8VQArzgldz1OnoG24PuKGdaEw7PbHjCMkD0/CyZWjH8/yWawcxV7hKl6RYHJ3GX9 > keroo7wz3/NaptJtA9ldoKA5ekV8WVVC5OElgtjKr+v6HorPQSHzUgJiDHYUS1FW > A8fdHipyZds= > =vwYy > -----END PGP SIGNATURE----- > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080701/dda8165a/attachment.htm> From jnoller at gmail.com Tue Jul 1 22:28:11 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 1 Jul 2008 16:28:11 -0400 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <g4e3p1$77m$1@ger.gmane.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <g4e3p1$77m$1@ger.gmane.org> Message-ID: <4222a8490807011328v4df3eb9fl7de44be415b334ff@mail.gmail.com> On Tue, Jul 1, 2008 at 4:23 PM, Georg Brandl <g.brandl at gmx.net> wrote: > Barry Warsaw schrieb: >> >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Wow, I bet this one crept up on you as quickly as it did me! >> >> We have our second planned beta releases for 2.6 and 3.0 tomorrow. As >> usual I will start looking at blockers and buildbots tomorrow afternoon >> (UTC-4 time) with a plan to start building things at about 6pm. Also, I >> will of course be in #python-dev on freenode to answer any questions, or >> get second opinions. >> >> PEP 361 claims that these will be the last betas. Whether that's true or >> not depends on how well the beta2's go. Please help review code or fix >> bugs. If you know of things that absolutely must go into beta2, be sure >> there is an open release-blocker bug on the issue. > > May I ask if it really makes sense to release the beta tomorrow? Looking > at the Misc/NEWS files for 2.6 and 3.0, there are around 3-5 entries > for each release. I know it's good to follow the release plan, but it > also may save you, the release manager, work for the third beta (which > I think will be necessary if beta2 is released tomorrow). > > Georg > Speaking from my minor perspective - I've been sick and MIA, so there has not been a lot of movement on the pep 371 issues / multiprocessing bugs since Beta 1, there's still a fair amount of issues to close out. -jesse From barry at python.org Tue Jul 1 22:40:35 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 1 Jul 2008 16:40:35 -0400 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> Message-ID: <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 1, 2008, at 4:25 PM, Guido van Rossum wrote: > I think we should put this one off. The previous betas were done on > June 18, and IMO the next beta should be about a month afterwards, > not 2 weeks. I will not be able to make releases the weeks of July 21st and 28th. The next scheduled beta is August 6th. There are two options. I could shift everything forward 2 weeks and do the next betas on July 16th. Or we could wait until August 6th. That would mean 6 weeks between betas. It's fine with me either way. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGqWQ3EjvBPtnXfVAQK5owP/Yd1pwWtwelbstnb6xh/dEtILirAyhfyo kcfQSSFBX+GgkDIx99cxgmJ7nB+xSNSy1MlkXukDj41O2m+dCqcQaxhyim4yqBYC r/Zc7IIiPT/nNQ/l97z8w0FqBoS/bmk9pqckBzrJfRRW14LZD8m2E/aU+OZeGi6z 0GZn/zwQbYk= =yC2a -----END PGP SIGNATURE----- From musiccomposition at gmail.com Tue Jul 1 22:45:21 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 1 Jul 2008 15:45:21 -0500 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> Message-ID: <1afaf6160807011345n650a645fq661908f75a1f6a03@mail.gmail.com> On Tue, Jul 1, 2008 at 3:40 PM, Barry Warsaw <barry at python.org> wrote: > > There are two options. I could shift everything forward 2 weeks and do the > next betas on July 16th. Or we could wait until August 6th. That would > mean 6 weeks between betas. It's fine with me either way. I vote for shifting things 2 weeks forward. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From python at rcn.com Tue Jul 1 22:51:17 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 1 Jul 2008 13:51:17 -0700 Subject: [Python-Dev] [Python-3000] Second betas tomorrow References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org><ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> Message-ID: <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> From: "Barry Warsaw" <barry at python.org> > There are two options. I could shift everything forward 2 weeks and > do the next betas on July 16th. Or we could wait until August 6th. > That would mean 6 weeks between betas. It's fine with me either way. +1 for six weeks to allow the code to be more thoroughly exercised. Raymond From guido at python.org Tue Jul 1 22:54:54 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 1 Jul 2008 13:54:54 -0700 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> Message-ID: <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> wrote: > From: "Barry Warsaw" <barry at python.org> > >> There are two options. I could shift everything forward 2 weeks and do >> the next betas on July 16th. Or we could wait until August 6th. That >> would mean 6 weeks between betas. It's fine with me either way. >> > > +1 for six weeks to allow the code to be more thoroughly exercised. > In that case I'd rather insert an extra beta -- one in 2 weeks and one in 6 weeks. -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080701/641c2c52/attachment.htm> From barry at python.org Wed Jul 2 00:55:16 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 1 Jul 2008 18:55:16 -0400 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> Message-ID: <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote: > On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> > wrote: > From: "Barry Warsaw" <barry at python.org> > > There are two options. I could shift everything forward 2 weeks > and do the next betas on July 16th. Or we could wait until August > 6th. That would mean 6 weeks between betas. It's fine with me > either way. > > +1 for six weeks to allow the code to be more thoroughly exercised. > > In that case I'd rather insert an extra beta -- one in 2 weeks and > one in 6 weeks. Okay. I can't actually do it on July 16th, so the revised schedule will be: 15-Jul-2008 beta 2 23-Aug-2008 beta 3 03-Sep-2008 rc1 17-Sep-2008 rc2 01-Oct-2008 final releases I will update PEP 361 now. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGq11HEjvBPtnXfVAQJdYgP8DFVvCHeDzIDliY0bQuw+DXxMuGAxHWFO BZR2b4sEGFzMRfbGCJOi7wVubc4imwYDIpFXgzFHpWFMfUdBHGaSpnZJhGDxURqp 0vQQ3/nJLy7lpWfDYBy0Sps6XjANQF5SaqeW8KMVsa3X6Spw0fHTmF4xBIjiUaBy MvydyLNszY4= =9/s1 -----END PGP SIGNATURE----- From guido at python.org Wed Jul 2 01:04:04 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 1 Jul 2008 16:04:04 -0700 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> Message-ID: <ca471dc20807011604q77c00e59kdf0d8b86312a03e6@mail.gmail.com> On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote: > On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote: >> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> wrote: >> From: "Barry Warsaw" <barry at python.org> >> >> There are two options. I could shift everything forward 2 weeks and do >> the next betas on July 16th. Or we could wait until August 6th. That >> would mean 6 weeks between betas. It's fine with me either way. >> >> +1 for six weeks to allow the code to be more thoroughly exercised. >> >> In that case I'd rather insert an extra beta -- one in 2 weeks and one in >> 6 weeks. > > Okay. I can't actually do it on July 16th, so the revised schedule will be: > > 15-Jul-2008 beta 2 > 23-Aug-2008 beta 3 > 03-Sep-2008 rc1 > 17-Sep-2008 rc2 > 01-Oct-2008 final releases > > I will update PEP 361 now. +1 Thanks for being flexible! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Wed Jul 2 01:13:57 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 1 Jul 2008 19:13:57 -0400 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <ca471dc20807011604q77c00e59kdf0d8b86312a03e6@mail.gmail.com> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> <ca471dc20807011604q77c00e59kdf0d8b86312a03e6@mail.gmail.com> Message-ID: <535961AE-BCFE-4563-96E0-B883D97A1188@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 1, 2008, at 7:04 PM, Guido van Rossum wrote: > On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote: >> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote: >>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> >>> wrote: >>> From: "Barry Warsaw" <barry at python.org> >>> >>> There are two options. I could shift everything forward 2 weeks >>> and do >>> the next betas on July 16th. Or we could wait until August 6th. >>> That >>> would mean 6 weeks between betas. It's fine with me either way. >>> >>> +1 for six weeks to allow the code to be more thoroughly exercised. >>> >>> In that case I'd rather insert an extra beta -- one in 2 weeks and >>> one in >>> 6 weeks. >> >> Okay. I can't actually do it on July 16th, so the revised schedule >> will be: >> >> 15-Jul-2008 beta 2 >> 23-Aug-2008 beta 3 >> 03-Sep-2008 rc1 >> 17-Sep-2008 rc2 >> 01-Oct-2008 final releases >> >> I will update PEP 361 now. > > +1 > > Thanks for being flexible! Anything for a great release! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGq6NnEjvBPtnXfVAQKOlQP/RYlj6vxHEmlW/mVNIWqBYy/SmmMA6Qw4 hE3Bhb9QYGC5F0kEKyY5BmBVwETe70ahE1X3AOgmLrnHh5XwvGh8sNrFka/3s9sh vt6XAZh9IoXekZBIOGO4Gz0EtcURVUvAbCzCSXkHCQyL3qoV1r+mxsXVLRV2S4q0 UifMzkOm6WI= =wDrk -----END PGP SIGNATURE----- From brett at python.org Wed Jul 2 01:27:03 2008 From: brett at python.org (Brett Cannon) Date: Tue, 1 Jul 2008 16:27:03 -0700 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> Message-ID: <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote: > >> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> wrote: >> From: "Barry Warsaw" <barry at python.org> >> >> There are two options. I could shift everything forward 2 weeks and do >> the next betas on July 16th. Or we could wait until August 6th. That >> would mean 6 weeks between betas. It's fine with me either way. >> >> +1 for six weeks to allow the code to be more thoroughly exercised. >> >> In that case I'd rather insert an extra beta -- one in 2 weeks and one in >> 6 weeks. > > Okay. I can't actually do it on July 16th, so the revised schedule will be: > > 15-Jul-2008 beta 2 > 23-Aug-2008 beta 3 > 03-Sep-2008 rc1 > 17-Sep-2008 rc2 > 01-Oct-2008 final releases > > I will update PEP 361 now. Is a Google Calendar kept by anyone that lists stuff like planned release dates, etc.? -Brett From musiccomposition at gmail.com Wed Jul 2 01:28:59 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 1 Jul 2008 18:28:59 -0500 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> Message-ID: <1afaf6160807011628m43738e85x3f03064a6df307ef@mail.gmail.com> On Tue, Jul 1, 2008 at 6:27 PM, Brett Cannon <brett at python.org> wrote: > Is a Google Calendar kept by anyone that lists stuff like planned > release dates, etc.? It's on my personal one. :) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From barry at python.org Wed Jul 2 03:44:16 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 1 Jul 2008 21:44:16 -0400 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> Message-ID: <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote: > On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote: >> >>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> >>> wrote: >>> From: "Barry Warsaw" <barry at python.org> >>> >>> There are two options. I could shift everything forward 2 weeks >>> and do >>> the next betas on July 16th. Or we could wait until August 6th. >>> That >>> would mean 6 weeks between betas. It's fine with me either way. >>> >>> +1 for six weeks to allow the code to be more thoroughly exercised. >>> >>> In that case I'd rather insert an extra beta -- one in 2 weeks and >>> one in >>> 6 weeks. >> >> Okay. I can't actually do it on July 16th, so the revised schedule >> will be: >> >> 15-Jul-2008 beta 2 >> 23-Aug-2008 beta 3 >> 03-Sep-2008 rc1 >> 17-Sep-2008 rc2 >> 01-Oct-2008 final releases >> >> I will update PEP 361 now. > > Is a Google Calendar kept by anyone that lists stuff like planned > release dates, etc.? http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGrdcHEjvBPtnXfVAQJYxgQAh/+j8pF21H0k1vp+1znOh57MohU7gVP6 7fMnLSzOoA+9w7+pVvJVzWbr09vg41kO6OzqEAoMUPV2BK8ZHePuHZkLDwhCAAYk nixu2vRZZEGmT6aC0jejwOCY7vy5giTHelX442drKZcuSdNl4x1kvyohBnm0flIH 6B7HRL3Oo2Q= =5yqD -----END PGP SIGNATURE----- From brett at python.org Wed Jul 2 04:01:33 2008 From: brett at python.org (Brett Cannon) Date: Tue, 1 Jul 2008 19:01:33 -0700 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org> Message-ID: <bbaeab100807011901s5fd992cdn87c548bfdaa0866d@mail.gmail.com> On Tue, Jul 1, 2008 at 6:44 PM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote: > >> On Tue, Jul 1, 2008 at 3:55 PM, Barry Warsaw <barry at python.org> wrote: >>> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> On Jul 1, 2008, at 4:54 PM, Guido van Rossum wrote: >>> >>>> On Tue, Jul 1, 2008 at 1:51 PM, Raymond Hettinger <python at rcn.com> >>>> wrote: >>>> From: "Barry Warsaw" <barry at python.org> >>>> >>>> There are two options. I could shift everything forward 2 weeks and do >>>> the next betas on July 16th. Or we could wait until August 6th. That >>>> would mean 6 weeks between betas. It's fine with me either way. >>>> >>>> +1 for six weeks to allow the code to be more thoroughly exercised. >>>> >>>> In that case I'd rather insert an extra beta -- one in 2 weeks and one >>>> in >>>> 6 weeks. >>> >>> Okay. I can't actually do it on July 16th, so the revised schedule will >>> be: >>> >>> 15-Jul-2008 beta 2 >>> 23-Aug-2008 beta 3 >>> 03-Sep-2008 rc1 >>> 17-Sep-2008 rc2 >>> 01-Oct-2008 final releases >>> >>> I will update PEP 361 now. >> >> Is a Google Calendar kept by anyone that lists stuff like planned >> release dates, etc.? > > http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics Thanks, Barry! -Brett From brett at python.org Wed Jul 2 04:04:44 2008 From: brett at python.org (Brett Cannon) Date: Tue, 1 Jul 2008 19:04:44 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? Message-ID: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> I just committed r64651 which is my attempt to add support to fix_imports so that modules that have been split up in 3.0 can be properly fixed. 2to3's test suite passes and all, but I am not sure if I botched it somehow since I did the change slightly blind. Can someone just do a quick check to make sure I did it properly? Also, what order should renames be declared to give priority to certain renames (e.g., urllib should probably be renamed to urllib.requeste over urllib.error when not used in a ``from ... import`` statement). -Brett From musiccomposition at gmail.com Wed Jul 2 04:38:13 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 1 Jul 2008 21:38:13 -0500 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> Message-ID: <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote: > I just committed r64651 which is my attempt to add support to > fix_imports so that modules that have been split up in 3.0 can be > properly fixed. 2to3's test suite passes and all, but I am not sure if > I botched it somehow since I did the change slightly blind. Can > someone just do a quick check to make sure I did it properly? Also, > what order should renames be declared to give priority to certain > renames (e.g., urllib should probably be renamed to urllib.requeste > over urllib.error when not used in a ``from ... import`` statement). Well for starters, you know the test for fix_imports is disabled, right? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Wed Jul 2 04:42:33 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 1 Jul 2008 21:42:33 -0500 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org> Message-ID: <1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com> On Tue, Jul 1, 2008 at 8:44 PM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote: >> >> Is a Google Calendar kept by anyone that lists stuff like planned >> release dates, etc.? > > http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics Can I get the non-iCal version? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From barry at python.org Wed Jul 2 05:29:10 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 1 Jul 2008 23:29:10 -0400 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org> <1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com> Message-ID: <D8E48735-E192-486B-A3D3-CD50C5F99DBF@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 1, 2008, at 10:42 PM, Benjamin Peterson wrote: > On Tue, Jul 1, 2008 at 8:44 PM, Barry Warsaw <barry at python.org> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote: >>> >>> Is a Google Calendar kept by anyone that lists stuff like planned >>> release dates, etc.? >> >> http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics > > Can I get the non-iCal version? http://www.google.com/calendar/feeds/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic http://www.google.com/calendar/embed?src=b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com&ctz=America/New_York - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGr2BnEjvBPtnXfVAQJKBQP/bme7XNFS74SSmNNYX6Wz7Dq83VSQ8J6A hZf6k7tTx6I3qv0Xgc2jD9NnNuLmqG+Rw8Ag5CjBtZXgzAoyszluzddJfz3G0032 zPofZx/ekp22u4XJo9iQyrDKinp+qTlDqlQntsscY5l+KXR5P9ahWeWWM9aQw707 VYkxQ2yAA7g= =fzdc -----END PGP SIGNATURE----- From brett at python.org Wed Jul 2 05:36:59 2008 From: brett at python.org (Brett Cannon) Date: Tue, 1 Jul 2008 20:36:59 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> Message-ID: <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson <musiccomposition at gmail.com> wrote: > On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote: >> I just committed r64651 which is my attempt to add support to >> fix_imports so that modules that have been split up in 3.0 can be >> properly fixed. 2to3's test suite passes and all, but I am not sure if >> I botched it somehow since I did the change slightly blind. Can >> someone just do a quick check to make sure I did it properly? Also, >> what order should renames be declared to give priority to certain >> renames (e.g., urllib should probably be renamed to urllib.requeste >> over urllib.error when not used in a ``from ... import`` statement). > > Well for starters, you know the test for fix_imports is disabled, right? > Nope, I forgot and turning it on has it failing running under 2.5. -Brett From ismail at namtrac.org Wed Jul 2 07:25:19 2008 From: ismail at namtrac.org (=?UTF-8?Q?=C4=B0smail_D=C3=B6nmez?=) Date: Wed, 2 Jul 2008 08:25:19 +0300 Subject: [Python-Dev] py3k branch still using -fno-strict-aliasing Message-ID: <19e566510807012225v6da0e8b2jac05ef407caee1b4@mail.gmail.com> Hi, I remember discussing this before and coming to conclusion that -fno-strict-aliasing would be removed from py3k CFLAGS. But as of now its still used. I tested with gcc 4.3.1 on Linux x86_64 and there is no strict aliasing warning when this flag is removed. Also make testall passes. Is there any reason to keep this flag? If not see the attached patch. Regards, ismail -- Programmer Excuses number 45: I do object-oriented programming - if the customer objects, I do more programming. -------------- next part -------------- A non-text attachment was scrubbed... Name: strict-aliasing.patch Type: text/x-diff Size: 1064 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-dev/attachments/20080702/13911566/attachment.patch> From paddy3118 at googlemail.com Wed Jul 2 08:08:22 2008 From: paddy3118 at googlemail.com (Paddy 3118) Date: Wed, 2 Jul 2008 07:08:22 +0100 Subject: [Python-Dev] [issue3214] Suggest change to glossary explanation: "Duck Typing" Message-ID: <3f7cdd360807012308y6eb6f018l6341cb0ace73e4e@mail.gmail.com> Hi, I'd like extra opinions on this issue please: http://bugs.python.org/issue3214 It's about changing the definition of Duck typing to remove hasattr and leave just EAFP in the enablers - more detail is in the issue log. Thanks, Paddy. From brett at python.org Wed Jul 2 08:32:54 2008 From: brett at python.org (Brett Cannon) Date: Tue, 1 Jul 2008 23:32:54 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> Message-ID: <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com> On Tue, Jul 1, 2008 at 8:36 PM, Brett Cannon <brett at python.org> wrote: > On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson > <musiccomposition at gmail.com> wrote: >> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote: >>> I just committed r64651 which is my attempt to add support to >>> fix_imports so that modules that have been split up in 3.0 can be >>> properly fixed. 2to3's test suite passes and all, but I am not sure if >>> I botched it somehow since I did the change slightly blind. Can >>> someone just do a quick check to make sure I did it properly? Also, >>> what order should renames be declared to give priority to certain >>> renames (e.g., urllib should probably be renamed to urllib.requeste >>> over urllib.error when not used in a ``from ... import`` statement). >> >> Well for starters, you know the test for fix_imports is disabled, right? >> > > Nope, I forgot and turning it on has it failing running under 2.5. > And refactor.py cannot be run directly from 2.5 because of a relative import and in 2.6 (where runpy has extra smarts) it still doesn't work thanks to main() not being passed an argument is needs (Issue3131). Looks like 2to3 needs some TLC. -Brett From steve at holdenweb.com Wed Jul 2 13:23:48 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 02 Jul 2008 07:23:48 -0400 Subject: [Python-Dev] [issue3214] Suggest change to glossary explanation: "Duck Typing" In-Reply-To: <3f7cdd360807012308y6eb6f018l6341cb0ace73e4e@mail.gmail.com> References: <3f7cdd360807012308y6eb6f018l6341cb0ace73e4e@mail.gmail.com> Message-ID: <g4fog7$7ba$1@ger.gmane.org> Paddy 3118 wrote: > Hi, > I'd like extra opinions on this issue please: > http://bugs.python.org/issue3214 > > It's about changing the definition of Duck typing to remove hasattr and > leave just EAFP in the enablers - more detail is in the issue log. > The change seems to make sense. Use of hasattr() to determine method availability, while not strictly "look before you leap" because it doesn't test for a specific type, certainly isn't EAFP either. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From duncan.booth at suttoncourtenay.org.uk Wed Jul 2 13:47:02 2008 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Wed, 2 Jul 2008 11:47:02 +0000 (UTC) Subject: [Python-Dev] repeated keyword arguments References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <48659116.10302@canterbury.ac.nz> <200806281158.41558.steve@pearwood.info> Message-ID: <Xns9ACF5B23C38D1duncanrcpcouk@127.0.0.1> "Steven D'Aprano" <steve at pearwood.info> wrote: > It would be nice to be able to do this: > > defaults = dict(a=5, b=7) > f(**defaults, a=8) # override the value of a in defaults > > but unfortunately that gives a syntax error. Reversing the order would > override the wrong value. So as Python exists now, no, it's not > terribly useful. But it's not inherently a stupid idea. There is already an easy way to do that using functools.partial, and it is documented and therefore presumably deliberate behaviour "If additional keyword arguments are supplied, they extend and override keywords." >>> from functools import partial >>> def f(a=1, b=2, c=3): print a, b, c >>> g = partial(f, b=99) >>> g() 1 99 3 >>> g(a=100, b=101) 100 101 3 From ncoghlan at gmail.com Wed Jul 2 14:31:33 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 02 Jul 2008 22:31:33 +1000 Subject: [Python-Dev] Review needed: proposed fix for 2.6 __hash__ incompatibility (issue 2235) Message-ID: <486B7525.6070502@gmail.com> I've posted a possible fix for the __hash__ backwards incompatibilities described in issue 2235 [1]. The patch uses a model similar to that used in Py3k (using None is indicate "don't inherit __hash__"), but extends it to allowing Py_None to be explicitly stored in the tp_hash slot. The major downside is that we suffer the cost of an extra pointer comparison on every call to PyObject_Hash, but I wasn't able to come up with another solution that preserved backwards compatibility while still allowing collections.Hashable to function correctly. The patch involves a few changes to fairly deep components in typeobject.c though, so I'd like at least some kind of sanity check before I commit it. Cheers, Nick. [1] http://bugs.python.org/issue2235 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From asmodai at in-nomine.org Wed Jul 2 16:13:28 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 2 Jul 2008 16:13:28 +0200 Subject: [Python-Dev] UCS2/UCS4 default Message-ID: <20080702141328.GW62693@nexus.in-nomine.org> Guido (and others of course), back in 2001 you pointed out that you wanted to move to UCS4 completely as the ideal situation (http://mail.python.org/pipermail/i18n-sig/2001-June/001107.html) over the current default UCS2. Given 3.0 will use Unicode strings as the default, would it also not make sense to make the switch at this point as well? The current situation with UCS2 is particularly bad now that the CJK ideographs Extension B. has been produced (and C is under ballot and D is under development). Personally I use nothing else but UCS4 compiled Python binaries for the past years. See also http://www.python.org/dev/peps/pep-0261/ for background for the 2001 options. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Expansion of happiness is the purpose of life... From guido at python.org Wed Jul 2 16:13:59 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jul 2008 07:13:59 -0700 Subject: [Python-Dev] Review needed: proposed fix for 2.6 __hash__ incompatibility (issue 2235) In-Reply-To: <486B7525.6070502@gmail.com> References: <486B7525.6070502@gmail.com> Message-ID: <ca471dc20807020713y75da0bfsec3550ecd0dfac25@mail.gmail.com> On Wed, Jul 2, 2008 at 5:31 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: > I've posted a possible fix for the __hash__ backwards incompatibilities > described in issue 2235 [1]. > > The patch uses a model similar to that used in Py3k (using None is indicate > "don't inherit __hash__"), but extends it to allowing Py_None to be > explicitly stored in the tp_hash slot. The major downside is that we suffer > the cost of an extra pointer comparison on every call to PyObject_Hash, but > I wasn't able to come up with another solution that preserved backwards > compatibility while still allowing collections.Hashable to function > correctly. >From your description it seems storing Py_None in the slot acts as a magic value meaning "this is defined but not usable". However it used to be pretty common for various code around to call various slots directly (after a NULL) check. That would have disastrous results if the slot value was Py_None. Would it be terribly inconvenient if the magic value was in fact another function, with a public name), whose sole purpose was to raise an exception? > The patch involves a few changes to fairly deep components in typeobject.c > though, so I'd like at least some kind of sanity check before I commit it. > > Cheers, > Nick. > > [1] http://bugs.python.org/issue2235 I can't promise I'll have time to look at this before my EuroPython keynote, but it's important for me to get it right, so if nobody else jumps in, remind me Tuesday. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Wed Jul 2 16:36:18 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 03 Jul 2008 00:36:18 +1000 Subject: [Python-Dev] Review needed: proposed fix for 2.6 __hash__ incompatibility (issue 2235) In-Reply-To: <ca471dc20807020713y75da0bfsec3550ecd0dfac25@mail.gmail.com> References: <486B7525.6070502@gmail.com> <ca471dc20807020713y75da0bfsec3550ecd0dfac25@mail.gmail.com> Message-ID: <486B9262.80107@gmail.com> Guido van Rossum wrote: > On Wed, Jul 2, 2008 at 5:31 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: >> I've posted a possible fix for the __hash__ backwards incompatibilities >> described in issue 2235 [1]. >> >> The patch uses a model similar to that used in Py3k (using None is indicate >> "don't inherit __hash__"), but extends it to allowing Py_None to be >> explicitly stored in the tp_hash slot. The major downside is that we suffer >> the cost of an extra pointer comparison on every call to PyObject_Hash, but >> I wasn't able to come up with another solution that preserved backwards >> compatibility while still allowing collections.Hashable to function >> correctly. > >>From your description it seems storing Py_None in the slot acts as a > magic value meaning "this is defined but not usable". However it used > to be pretty common for various code around to call various slots > directly (after a NULL) check. That would have disastrous results if > the slot value was Py_None. Would it be terribly inconvenient if the > magic value was in fact another function, with a public name), whose > sole purpose was to raise an exception? Not only not inconvenient, but a significant improvement - as well as addressing your concern that I missed some code that calls tp_hash directly (a concern that I share, particularly since it could be an extension module we don't control that ends up doing it), it also gets rid of that extra pointer comparison in PyObject_Hash that was bothering me. >> The patch involves a few changes to fairly deep components in typeobject.c >> though, so I'd like at least some kind of sanity check before I commit it. >> >> Cheers, >> Nick. >> >> [1] http://bugs.python.org/issue2235 > > I can't promise I'll have time to look at this before my EuroPython > keynote, but it's important for me to get it right, so if nobody else > jumps in, remind me Tuesday. I'd now advise waiting until I have a chance to implement your idea anyway :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From collinw at gmail.com Wed Jul 2 18:30:14 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 2 Jul 2008 09:30:14 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> Message-ID: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson <musiccomposition at gmail.com> wrote: > On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote: >> I just committed r64651 which is my attempt to add support to >> fix_imports so that modules that have been split up in 3.0 can be >> properly fixed. 2to3's test suite passes and all, but I am not sure if >> I botched it somehow since I did the change slightly blind. Can >> someone just do a quick check to make sure I did it properly? Also, >> what order should renames be declared to give priority to certain >> renames (e.g., urllib should probably be renamed to urllib.requeste >> over urllib.error when not used in a ``from ... import`` statement). > > Well for starters, you know the test for fix_imports is disabled, right? Why was this test disabled, rather than fixed? That seems a rather poor solution to the problem of it taking longer than desired to run. Collin From musiccomposition at gmail.com Wed Jul 2 18:34:00 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 2 Jul 2008 11:34:00 -0500 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> Message-ID: <1afaf6160807020934h37f6e989i772da74b68ced414@mail.gmail.com> On Wed, Jul 2, 2008 at 11:30 AM, Collin Winter <collinw at gmail.com> wrote: > On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson >> Well for starters, you know the test for fix_imports is disabled, right? > > Why was this test disabled, rather than fixed? That seems a rather > poor solution to the problem of it taking longer than desired to run. I believe Martin was the one who disabled it. > > Collin > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From collinw at gmail.com Wed Jul 2 18:36:30 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 2 Jul 2008 09:36:30 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com> Message-ID: <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com> On Tue, Jul 1, 2008 at 11:32 PM, Brett Cannon <brett at python.org> wrote: > On Tue, Jul 1, 2008 at 8:36 PM, Brett Cannon <brett at python.org> wrote: >> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson >> <musiccomposition at gmail.com> wrote: >>> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote: >>>> I just committed r64651 which is my attempt to add support to >>>> fix_imports so that modules that have been split up in 3.0 can be >>>> properly fixed. 2to3's test suite passes and all, but I am not sure if >>>> I botched it somehow since I did the change slightly blind. Can >>>> someone just do a quick check to make sure I did it properly? Also, >>>> what order should renames be declared to give priority to certain >>>> renames (e.g., urllib should probably be renamed to urllib.requeste >>>> over urllib.error when not used in a ``from ... import`` statement). >>> >>> Well for starters, you know the test for fix_imports is disabled, right? >>> >> >> Nope, I forgot and turning it on has it failing running under 2.5. >> > > And refactor.py cannot be run directly from 2.5 because of a relative > import and in 2.6 (where runpy has extra smarts) it still doesn't work > thanks to main() not being passed an argument is needs (Issue3131). Why are you trying to run refactor.py directly, rather than using 2to3 (http://svn.python.org/view/sandbox/trunk/2to3/2to3) as an entry point? > Looks like 2to3 needs some TLC. Agreed. A lot of the pending bugs seem to be related to the version of lib2to3 in the stdlib, rather than the stand-alone product. Neal Norwitz and I have been working to turn parts of 2to3 into a more general refactoring library; once that's done (or even preferably before), lib2to3 should be removed from the stdlib. It's causing far more trouble than it's worth. Collin From guido at python.org Wed Jul 2 19:08:01 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jul 2008 10:08:01 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080702141328.GW62693@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> Message-ID: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> I think we should continue to leave this up to the distribution. AFAIK many Linux distros already use UCS4 for everything anyway. The alternative (no matter what the configure flag is called) is UTF-16, not UCS-2 though: there is support for surrogate pairs in various places, including the \U escape and the UTF-8 codec. I don't want to rule out UTF-16 as internal representation from the Python language spec, because JVM- and .NET-based implementations pretty much have no choice in the matter if they want to be compatible with the native string type (which is very important for performance and compatibility with other languages on those platforms). For that reason I think it's also better that the configure script continues to default to UTF-16 -- this will give the UTF-16 support code the necessary exercise. (It is mostly a superset of the UCS-4 support code, so I'm less worried about the latter getting enough exercise.) --Guido On Wed, Jul 2, 2008 at 7:13 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > > Guido (and others of course), > > back in 2001 you pointed out that you wanted to move to UCS4 completely as > the ideal situation > (http://mail.python.org/pipermail/i18n-sig/2001-June/001107.html) over the > current default UCS2. > > Given 3.0 will use Unicode strings as the default, would it also not make > sense to make the switch at this point as well? > > The current situation with UCS2 is particularly bad now that the CJK > ideographs Extension B. has been produced (and C is under ballot and D is > under development). > > Personally I use nothing else but UCS4 compiled Python binaries for the past > years. > > See also http://www.python.org/dev/peps/pep-0261/ for background for the > 2001 options. > > -- > Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai > ????? ?????? ??? ?? ?????? > http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B > Expansion of happiness is the purpose of life... > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 2 19:13:45 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jul 2008 10:13:45 -0700 Subject: [Python-Dev] Who wants to work with Klocwork? Message-ID: <ca471dc20807021013u36df2645k644f5e4ac40fe519@mail.gmail.com> I've got an offer from Klocwork (a static source code analysis company, www.klocwork.com) to give some developers free access to their findings from running their bug-finding software over Python source code. I don't have the bandwidth to deal with this myself, but I think it would be valuable if we could get some folks to look at their findings. We have a similar relationship with one of Klocwork's competitors. In my experience, each vendor's tool has a different strength, and it is likely that each will find some important bugs that the other didn't flag. So IMO it's useful to do this with each vendor that offers... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Wed Jul 2 19:19:57 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 2 Jul 2008 19:19:57 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> Message-ID: <20080702171957.GY62693@nexus.in-nomine.org> -On [20080702 19:08], Guido van Rossum (guido at python.org) wrote: >I think we should continue to leave this up to the distribution. AFAIK >many Linux distros already use UCS4 for everything anyway. FreeBSD's ports makes it a configure option. >For that reason I think it's also better that the configure script >continues to default to UTF-16 -- this will give the UTF-16 support >code the necessary exercise. (It is mostly a superset of the UCS-4 >support code, so I'm less worried about the latter getting enough >exercise.) I was under the impression that it was still UCS2 and thus limiting things to the BMP only. So you are saying it's UTF-16 nowadays? For both 2.6 and 3.0? -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Nature does nothing uselessly... From guido at python.org Wed Jul 2 19:42:13 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jul 2008 10:42:13 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080702171957.GY62693@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> Message-ID: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> On Wed, Jul 2, 2008 at 10:19 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > -On [20080702 19:08], Guido van Rossum (guido at python.org) wrote: >>I think we should continue to leave this up to the distribution. AFAIK >>many Linux distros already use UCS4 for everything anyway. > > FreeBSD's ports makes it a configure option. > >>For that reason I think it's also better that the configure script >>continues to default to UTF-16 -- this will give the UTF-16 support >>code the necessary exercise. (It is mostly a superset of the UCS-4 >>support code, so I'm less worried about the latter getting enough >>exercise.) > > I was under the impression that it was still UCS2 and thus limiting things > to the BMP only. So you are saying it's UTF-16 nowadays? For both 2.6 and > 3.0? Yes. At least in the sense that \Uxxxxxxxx gets translated to a surrogate pair, and that the UTF-8 codec supports surrogate pairs in both directions. It's been like this for a long time. What else would you expect from UTF-16 support? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 2 20:17:16 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jul 2008 11:17:16 -0700 Subject: [Python-Dev] Who wants to work with Klocwork? In-Reply-To: <ca471dc20807021013u36df2645k644f5e4ac40fe519@mail.gmail.com> References: <ca471dc20807021013u36df2645k644f5e4ac40fe519@mail.gmail.com> Message-ID: <ca471dc20807021117k150f43d1r6fcabbbdc4405a10@mail.gmail.com> Followup: Neal Norwitz <nnorwitz at gmail.com> will coordinate this, send mail to him if you're interested, not to me. :-) On Wed, Jul 2, 2008 at 10:13 AM, Guido van Rossum <guido at python.org> wrote: > I've got an offer from Klocwork (a static source code analysis > company, www.klocwork.com) to give some developers free access to > their findings from running their bug-finding software over Python > source code. I don't have the bandwidth to deal with this myself, but > I think it would be valuable if we could get some folks to look at > their findings. > > We have a similar relationship with one of Klocwork's competitors. In > my experience, each vendor's tool has a different strength, and it is > likely that each will find some important bugs that the other didn't > flag. So IMO it's useful to do this with each vendor that offers... > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Wed Jul 2 20:22:15 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 2 Jul 2008 20:22:15 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> Message-ID: <20080702182215.GA62693@nexus.in-nomine.org> -On [20080702 19:42], Guido van Rossum (guido at python.org) wrote: >Yes. At least in the sense that \Uxxxxxxxx gets translated to a >surrogate pair, and that the UTF-8 codec supports surrogate pairs in >both directions. It's been like this for a long time. What else would >you expect from UTF-16 support? Well, unless I misunderstand things, a Python 3 compiled with the default Unicode option gives this: >>> len("\N{MUSICAL SYMBOL G CLEF}") 2 Whereas a Python 3 with --with-wide-unicode gives: >>> len("\N{MUSICAL SYMBOL G CLEF}") 1 This, of course, causes problems with splitting, finding, and so on. So that means that a Python 3 with only 2 byte Unicode support is not to be used/recommended for Unicode outside of the BMP. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Tomorrow's battle is won during today's practice... From guido at python.org Wed Jul 2 20:27:42 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jul 2008 11:27:42 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080702182215.GA62693@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> Message-ID: <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> On Wed, Jul 2, 2008 at 11:22 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > -On [20080702 19:42], Guido van Rossum (guido at python.org) wrote: >>Yes. At least in the sense that \Uxxxxxxxx gets translated to a >>surrogate pair, and that the UTF-8 codec supports surrogate pairs in >>both directions. It's been like this for a long time. What else would >>you expect from UTF-16 support? > > Well, unless I misunderstand things, a Python 3 compiled with the default > Unicode option gives this: > >>>> len("\N{MUSICAL SYMBOL G CLEF}") > 2 > > Whereas a Python 3 with --with-wide-unicode gives: > > >>>> len("\N{MUSICAL SYMBOL G CLEF}") > 1 > > This, of course, causes problems with splitting, finding, and so on. Understood. > So that > means that a Python 3 with only 2 byte Unicode support is not to be > used/recommended for Unicode outside of the BMP. I disagree. Instead, I would say that such code needs to be aware of surrogates. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Wed Jul 2 20:35:41 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 2 Jul 2008 20:35:41 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> Message-ID: <20080702183541.GB62693@nexus.in-nomine.org> -On [20080702 20:27], Guido van Rossum (guido at python.org) wrote: >I disagree. Instead, I would say that such code needs to be aware of >surrogates. Just to make sure I understood you: Python's code needs to be made aware of surrogates? If so, do you want me to log issues for the things encountered? -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Learn from the past -- don't wear it like a yoke around your neck... From guido at python.org Wed Jul 2 20:47:02 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 2 Jul 2008 11:47:02 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080702183541.GB62693@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> Message-ID: <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> On Wed, Jul 2, 2008 at 11:35 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > -On [20080702 20:27], Guido van Rossum (guido at python.org) wrote: >>I disagree. Instead, I would say that such code needs to be aware of >>surrogates. > > Just to make sure I understood you: > > Python's code needs to be made aware of surrogates? No, Python already is aware of surrogates. I meant applications processing non-BMP text should beware of them. > If so, do you want me to log issues for the things encountered? If you find places where the Python core or standard library is doing Unicode processing that would break when surrogates are present you should file a bug. However this does not mean that every bit of code that slices a string at an arbitrary point (and hence risks slicing in the middle of a surrogate) is incorrect -- it all depends on what is done next with the slice. I'd also prefer to receive bug reports about breakages actually encountered in the wild than purely theoretical issues. And in all cases a fragment of test code to reproduce the problem would be appreciated. > -- > Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai > ????? ?????? ??? ?? ?????? > http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B > Learn from the past -- don't wear it like a yoke around your neck... > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Jul 2 21:02:46 2008 From: brett at python.org (Brett Cannon) Date: Wed, 2 Jul 2008 12:02:46 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> Message-ID: <bbaeab100807021202n1743d899ge7425ae92f8e555c@mail.gmail.com> On Wed, Jul 2, 2008 at 9:30 AM, Collin Winter <collinw at gmail.com> wrote: > On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson > <musiccomposition at gmail.com> wrote: >> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote: >>> I just committed r64651 which is my attempt to add support to >>> fix_imports so that modules that have been split up in 3.0 can be >>> properly fixed. 2to3's test suite passes and all, but I am not sure if >>> I botched it somehow since I did the change slightly blind. Can >>> someone just do a quick check to make sure I did it properly? Also, >>> what order should renames be declared to give priority to certain >>> renames (e.g., urllib should probably be renamed to urllib.requeste >>> over urllib.error when not used in a ``from ... import`` statement). >> >> Well for starters, you know the test for fix_imports is disabled, right? > > Why was this test disabled, rather than fixed? That seems a rather > poor solution to the problem of it taking longer than desired to run. I think it may have been to turn off a failing test just before a release and it was just never switched back on. -Brett From brett at python.org Wed Jul 2 21:06:01 2008 From: brett at python.org (Brett Cannon) Date: Wed, 2 Jul 2008 12:06:01 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com> <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com> Message-ID: <bbaeab100807021206l6b47141fq5a214c73948db361@mail.gmail.com> On Wed, Jul 2, 2008 at 9:36 AM, Collin Winter <collinw at gmail.com> wrote: > On Tue, Jul 1, 2008 at 11:32 PM, Brett Cannon <brett at python.org> wrote: >> On Tue, Jul 1, 2008 at 8:36 PM, Brett Cannon <brett at python.org> wrote: >>> On Tue, Jul 1, 2008 at 7:38 PM, Benjamin Peterson >>> <musiccomposition at gmail.com> wrote: >>>> On Tue, Jul 1, 2008 at 9:04 PM, Brett Cannon <brett at python.org> wrote: >>>>> I just committed r64651 which is my attempt to add support to >>>>> fix_imports so that modules that have been split up in 3.0 can be >>>>> properly fixed. 2to3's test suite passes and all, but I am not sure if >>>>> I botched it somehow since I did the change slightly blind. Can >>>>> someone just do a quick check to make sure I did it properly? Also, >>>>> what order should renames be declared to give priority to certain >>>>> renames (e.g., urllib should probably be renamed to urllib.requeste >>>>> over urllib.error when not used in a ``from ... import`` statement). >>>> >>>> Well for starters, you know the test for fix_imports is disabled, right? >>>> >>> >>> Nope, I forgot and turning it on has it failing running under 2.5. >>> >> >> And refactor.py cannot be run directly from 2.5 because of a relative >> import and in 2.6 (where runpy has extra smarts) it still doesn't work >> thanks to main() not being passed an argument is needs (Issue3131). > > Why are you trying to run refactor.py directly, rather than using 2to3 > (http://svn.python.org/view/sandbox/trunk/2to3/2to3) as an entry > point? > Because I honestly did not see it yesterday in my terminal. I blame it on Canada Day. =) >> Looks like 2to3 needs some TLC. > > Agreed. A lot of the pending bugs seem to be related to the version of > lib2to3 in the stdlib, rather than the stand-alone product. Neal > Norwitz and I have been working to turn parts of 2to3 into a more > general refactoring library; once that's done (or even preferably > before), lib2to3 should be removed from the stdlib. It's causing far > more trouble than it's worth. Fine by me. -Brett From martin at v.loewis.de Wed Jul 2 21:51:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 02 Jul 2008 21:51:49 +0200 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> Message-ID: <486BDC55.6070506@v.loewis.de> > Why was this test disabled, rather than fixed? That seems a rather > poor solution to the problem of it taking longer than desired to run. I disabled it because I didn't know how to fix it, and created bug reports 2968 and 2969 in return. It is policy that tests that break get disabled, rather than keeping them broken. Regards, Martin From martin at v.loewis.de Wed Jul 2 22:09:57 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 02 Jul 2008 22:09:57 +0200 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com> <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com> Message-ID: <486BE095.2030801@v.loewis.de> > Agreed. A lot of the pending bugs seem to be related to the version of > lib2to3 in the stdlib, rather than the stand-alone product. Neal > Norwitz and I have been working to turn parts of 2to3 into a more > general refactoring library; once that's done (or even preferably > before), lib2to3 should be removed from the stdlib. It's causing far > more trouble than it's worth. I disagree. I think it is quite useful that distutils is able to invoke it, and other people also asked for that feature on PyCon. Why do you think the trouble wouldn't be caused if it wasn't a standard library package? Regards, Martin From collinw at gmail.com Wed Jul 2 22:39:58 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 2 Jul 2008 13:39:58 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <486BDC55.6070506@v.loewis.de> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <43aa6ff70807020930j32ab1564n33ea41b9db487400@mail.gmail.com> <486BDC55.6070506@v.loewis.de> Message-ID: <43aa6ff70807021339j56d81522hbbfcb754fa027150@mail.gmail.com> On Wed, Jul 2, 2008 at 12:51 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote: >> Why was this test disabled, rather than fixed? That seems a rather >> poor solution to the problem of it taking longer than desired to run. > > I disabled it because I didn't know how to fix it, and created bug > reports 2968 and 2969 in return. So you did. I didn't notice them, sorry. > It is policy that tests that break > get disabled, rather than keeping them broken. From collinw at gmail.com Wed Jul 2 22:40:42 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 2 Jul 2008 13:40:42 -0700 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <486BE095.2030801@v.loewis.de> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com> <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com> <486BE095.2030801@v.loewis.de> Message-ID: <43aa6ff70807021340w76804278rd0b90c31a8175faa@mail.gmail.com> On Wed, Jul 2, 2008 at 1:09 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote: >> Agreed. A lot of the pending bugs seem to be related to the version of >> lib2to3 in the stdlib, rather than the stand-alone product. Neal >> Norwitz and I have been working to turn parts of 2to3 into a more >> general refactoring library; once that's done (or even preferably >> before), lib2to3 should be removed from the stdlib. It's causing far >> more trouble than it's worth. > > I disagree. I think it is quite useful that distutils is able to > invoke it, and other people also asked for that feature on PyCon. But distutils currently *doesn't* invoke it, AFAICT (unless that support is implemented somewhere other than trunk/Lib/distutils/), and no-one has stepped up to make that happen in the months since PyCon. Moreover, as I told those people who asked for this at PyCon, 2to3 is and will never be perfect, meaning that at best, distutils/2to3 integration would look like "python setup.py run2to3", where distutils is just a long-hand way of running 2to3 over your code. This strikes me as a waste of time. > Why do you think the trouble wouldn't be caused if it wasn't > a standard library package? Problems with the current setup: 1) People are currently confused as to where they should be commit fixes. 2) Changes to the sandbox version have to be manually merged into the stdlib version, which is more overhead than I think it's worth. In addition, the stdlib version lags the sandbox version. 3) At least one bug report (issue3131) has mentioned problems with the stdlib 2to3 exhibiting problems that the stand-alone version does not. This is again extra overhead. 4) The test_imports test was commented out because of stdlib test policy. I'd rather not have that policy imposed on 2to3. Collin From martin at v.loewis.de Thu Jul 3 00:36:59 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 03 Jul 2008 00:36:59 +0200 Subject: [Python-Dev] Can someone check my lib2to3 change for fix_imports? In-Reply-To: <43aa6ff70807021340w76804278rd0b90c31a8175faa@mail.gmail.com> References: <bbaeab100807011904i91bf75bped74b8be68c808f7@mail.gmail.com> <1afaf6160807011938r44a58880n60e696b25c098617@mail.gmail.com> <bbaeab100807012036i456b079t47d9e2f2fd762935@mail.gmail.com> <bbaeab100807012332l614ad502qb411b50442f20d7f@mail.gmail.com> <43aa6ff70807020936j6bfa578cm9b260d363f34e5f@mail.gmail.com> <486BE095.2030801@v.loewis.de> <43aa6ff70807021340w76804278rd0b90c31a8175faa@mail.gmail.com> Message-ID: <486C030B.6060405@v.loewis.de> > But distutils currently *doesn't* invoke it, AFAICT Sure. In 3k, look at Lib/distutils/command/build.py:build_py_2to3. That's how I ported Django to Py3k. > 1) People are currently confused as to where they should be commit fixes. Sure, but it only happens rarely. > 2) Changes to the sandbox version have to be manually merged into the > stdlib version, which is more overhead than I think it's worth. In > addition, the stdlib version lags the sandbox version. It's not a real problem, IMO, using msgmerge is fairly straight-forward. > 3) At least one bug report (issue3131) has mentioned problems with the > stdlib 2to3 exhibiting problems that the stand-alone version does not. > This is again extra overhead. I think the 2to3 packaging issue is otherwise unresolved. Do you want 2to3 to be excluded completely from 2.6 and 3.1 releases? If not, how do you want them packaged? Will it work if packaged in that way? > 4) The test_imports test was commented out because of stdlib test > policy. I'd rather not have that policy imposed on 2to3. It would be possible to comment out the test only in the copy in the stdlib version, or to omit testing 2to3 in the stdlib altogether, if that helps. Regards, Martin From asmodai at in-nomine.org Thu Jul 3 12:48:13 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 12:48:13 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> Message-ID: <20080703104813.GF62693@nexus.in-nomine.org> My apologies for hammering on this, but I think it is quite important and currently Python 3.0 seems confused about UCS-2 versus UTF-16. -On [20080702 20:47], Guido van Rossum (guido at python.org) wrote: >No, Python already is aware of surrogates. I meant applications >processing non-BMP text should beware of them. Just to make sure people are fully aware of the distinctions: UCS-2 uses 16 bits to encode Unicode data, does NOT support surrogate pairs and therefore CANNOT represent data beyond U+FFFF (thus only supporting the Basic Multilingual Plane, BMP). It is a fixed-length character encoding. UTF-16 also uses 16 bits to encode Unicode data, but DOES support surrogate pairs and therefore CAN represent data beyond U+FFFF by using said surrogate pairs (thus supporting all planes). It is a variable-length character encoding. So a string representation in UCS-2 means every character occupies 16 bits. A string representation in UTF-16 means characters can occupy 16 bits or 32-bits. If one stays within the BMP than all is well, but when you move beyond the BMP (U+10000 - U+10FFFF) then Python needs to correctly check the string for surrogate pairs and deal with them internally. >If you find places where the Python core or standard library is doing >Unicode processing that would break when surrogates are present you >should file a bug. However this does not mean that every bit of code >that slices a string at an arbitrary point (and hence risks slicing in >the middle of a surrogate) is incorrect -- it all depends on what is >done next with the slice. Basically everything but string forming or string printing seems to be broken for surrogate pairs, from what I can tell. Also, I think you are confused about slicing in the middle of a surrogate pair, from a UTF-16 perspective this is 1 codepoint! And as such Python needs to treat it as one character/codepoint in a string, dealing with slicing as appropriate. The way you currently describe it is that UTF-16 strings will be treated as UCS-2 when it comes to slicing and the likes. >From a UTF-16 point of view such slicing can NEVER occur unless you are bit or byte slicing instead of character/codepoint slicing. The documentation for len() says: Return the length (the number of items) of an object. I think it can be fairly said that an item in a string is a character or codepoint. Take for example the following string: a = '\U00020045\u942a' # Two hanzi/kanji/hanja >From a Unicode perspective we are looking at two characters/codepoints. When we use a 4-byte Python 3.0 binary we get (as expected): >>> len(a) 2 When we use a 2-byte Python 3.0 binary (the default) we get (not as expected): >>> len(a) 3 >From a UTF-16 perspective a surrogate pair is one character/codepoint and as such len() should have reported 2 as well. That the sequence is stored internally as 0xd840 0xdc45 0x942a and occupies 3 bytes is not interesting. But it seems as if len() is treating the string as being in UCS-2 (fixed-length), which is the only logical explanation for the number 3, instead of treating it as UTF-16 (variable-length) and reporting the number 2. Subsequently doing a: print a[1] to get the 0x942a (?) actually requires a[2] on the 2-byte Python 3.0. As such the code you write for 2-byte and 4-byte Python 3.0 is *different* when you have to deal with the same Unicode strings! This cannot be the desired situation, can it? Two more examples: >>> a.find('?') # 4-byte 1 >>> a.find('?') # 2-byte 2 >>> import re # 4-byte >>> m = re.search('?', a) >>> m.start() 1 >>> import re # 2-byte >>> m = re.search('?', a) >>> m.start() 2 This, in my opinion, has nothing to do with the application writers, but more with Python's internals being confused about UCS-2 and UTF-16. We accept full 32-bit codepoints with the \U escape in strings, and we may even store it as UTF-16 internally, but we clearly do not deal with it properly as UTF-16, but rather as UCS-2, when it comes to using said strings with core functions and modules. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B For wouldst thou not carve at my Soul with thine sword of Supreme Truth? From solipsis at pitrou.net Thu Jul 3 13:58:17 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 3 Jul 2008 11:58:17 +0000 (UTC) Subject: [Python-Dev] UCS2/UCS4 default References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> Message-ID: <loom.20080703T115255-50@post.gmane.org> Hi, > Subsequently doing a: print a[1] to get the 0x942a (?) actually requires > a[2] on the 2-byte Python 3.0. How is it annoying *in practice*? In actual code the index, instead of being a constant, will be retrieved through various means such as .find() or re.search().start()... as you show yourself later in your message. What is primordial is that Python shows a consistent behaviour, and it does, since indices returned by .find() et al. have the same meaning as indices you can use with the [] operator. AFAIK that's why Guido asked for real-world rather than theoretical examples. Regards Antoine. From ncoghlan at gmail.com Thu Jul 3 14:39:29 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 03 Jul 2008 22:39:29 +1000 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703104813.GF62693@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> Message-ID: <486CC881.5090902@gmail.com> Jeroen Ruigrok van der Werven wrote: > The documentation for len() says: > Return the length (the number of items) of an object. So what this tells us is that in a UCS-2 build of Python, the "items" in a unicode string are not, strictly speaking, Unicode code points or characters. Instead, they are successive 16-bit fragments of a UTF-16 encoded string (which correspond to characters only if there are no surrogate pairs present in the string). Let's look at the options here: 1. System is NOT memory limited (i.e. most desktops): use a UCS-4 Python build, which is what most Linux distributions do (I'm not sure about the pydotorg provided Windows or Mac OS X builds). 2. System is memory limited, only BMP Unicode code points are used: use a UCS-2 Python build, limit yourself to characters on the BMP (possibly enforced by use of an appropriate codec to decode input text). 3. System is memory limited, but needs to support characters beyond the BMP: use a UCS-2 Python build, handling any codepoints outside the BMP in application code. The current Python approach handles all three cases relatively gracefully and with minimal overhead. Dealing natively with surrogate pair issues could easily result in pointless complexity for cases 1 and 2, while completely disallowing codepoints beyond the BMP in a UCS-2 build would needlessly rule out option 3. So here's the challenge: 1. If you are advocating disallowing the use of characters outside the BMP in a UCS-2 build, enumerate the advantages of doing so (paying particular attention to any advantages which cannot be obtained simply by using an appropriate codec that disallows non-BMP characters). 2. If you are advocating making the "items" in a Unicode string code points even in a UCS-2 build, enumerate all of the string behaviours that would have to change, as well as indicating how to avoid causing a reduction in speed for cases 1 and 2 above. Sure, option 2 might be nice to have, but the purity argument isn't going to be anywhere near enough motivation to justify the additional code complexity - there need to be practical benefits that aren't better met just by sacrificing a bit of memory efficiency and switching to a UCS-4 build. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Thu Jul 3 15:00:22 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Jul 2008 15:00:22 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486CC881.5090902@gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> Message-ID: <486CCD66.70906@egenix.com> I think the discussion is going in the wrong direction: The choice between UCS2 and UCS4 builds is really only meant to enhance the possibility to interface to native OS or application APIs, e.g. Windows LIBC and Java use UTF-16, glibc on Unix uses UCS4. The problem of slicing Unicode objects is far more complicated than just breaking a surrogate pair. Unicode if full of combining code points - if you break such a sequence, the output will be just as wrong; regardless of UCS2 vs. UCS4. A long time ago we had a discussion about these problems. I had suggested a new module (unicodeindex IIRC) which takes care of indexing Unicode strings based on code points (which support for surrogates), glyphs (taking combining code points into account) and words (with support for various breaking/non-breaking separation code points). Trying to solve such issues at the storage level is the wrong approach, since the problem is application specific and thus requires a higher-level set of possible solutions. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 3 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From djarb at highenergymagic.org Thu Jul 3 15:14:47 2008 From: djarb at highenergymagic.org (Daniel Arbuckle) Date: Thu, 3 Jul 2008 06:14:47 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486CC881.5090902@gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> Message-ID: <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> On Thu, Jul 3, 2008 at 5:39 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: > 1. If you are advocating disallowing the use of characters outside the BMP > in a UCS-2 build, enumerate the advantages of doing so (paying particular > attention to any advantages which cannot be obtained simply by using an > appropriate codec that disallows non-BMP characters). Right now, the same python code has different meaning, depending on a compile-time option that most users didn't even set for themselves. Moreover, the errors caused by this semantic difference are not reported. There's just no way to justify that. You can't solve this problem by saying 'programmers should choose a codec that limits them to the BMP when they target 2-byte python,' because the problem specifically arises when code that works correctly in a 4-byte python is placed into a 2-byte python, an operation performed by the users rather than by programmers. Since 2-byte python is apparently a holdover for memory-limited (and presumably CPU-limited as well) systems, it doesn't make sense to impose on it the requirement of correctly dealing with surrogate pairs. Given that, it seems to me that the best solution would be to make 4-byte python the default, and also to make 2-byte python raise an exception when it encounters characters outside the BMP. This way, a mysterious and unreported semantic error becomes an explicit syntactic error. For programmers who want to target a 2-byte format (for win32 compatibility, for example), the correct choice of codec is a superior solution to forcing a 2-byte internal representation on python. From asmodai at in-nomine.org Thu Jul 3 15:21:46 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 15:21:46 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486CCD66.70906@egenix.com> References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> Message-ID: <20080703132146.GI62693@nexus.in-nomine.org> -On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote: >Unicode if full of combining code points - if you break such a sequence, >the output will be just as wrong; regardless of UCS2 vs. UCS4. In my opinion you are confusing two related, but very separated things here. Combining characters have nothing to do with breaking up the encoding of a single codepoint. Sure enough, if you arbitrary slice up codepoints that consist of combining characters then your result is indeed odd looking. I never said that nor is that the point I am making. Guido points out that Python supports surrogate pairs and says that if Python is dealing wrongly with this in the core than it needs to be fixed. I am pointing out that given the fact we allow surrogate pairs we deal rather simplistic with it in the core. In fact, we do not consider them at all. In essence: though we may accept full 21-bit codepoints in the form of \U00000000 escape sequences and store them internally as UTF-16 (which I still need to verify) we subsequently deal with them programmatically as UCS-2, which is plain silly. You either commit yourself fully to UTF-16 and surrogate pairs or not. Not some form in-between, because that will ultimately lead to more confusion due to the difference in results when dealing with Unicode. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Believe in Angels... From mhammond at skippinet.com.au Thu Jul 3 15:42:58 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 3 Jul 2008 23:42:58 +1000 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> Message-ID: <031901c8dd12$b7258b60$2570a220$@com.au> > For programmers who want to target a 2-byte format (for win32 > compatibility, for example) As MAL said, this is taking the discussion in the wrong direction. For people on Windows, win32 isn't a "compatibility" consideration. I suspect most users of the other platforms MAL mentioned and all others with their own native unicode implementations would agree. Cheers, Mark From mal at egenix.com Thu Jul 3 15:57:41 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Jul 2008 15:57:41 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703132146.GI62693@nexus.in-nomine.org> References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> <20080703132146.GI62693@nexus.in-nomine.org> Message-ID: <486CDAD5.1060506@egenix.com> On 2008-07-03 15:21, Jeroen Ruigrok van der Werven wrote: > -On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote: >> Unicode if full of combining code points - if you break such a sequence, >> the output will be just as wrong; regardless of UCS2 vs. UCS4. > > In my opinion you are confusing two related, but very separated things here. > Combining characters have nothing to do with breaking up the encoding of a > single codepoint. Sure enough, if you arbitrary slice up codepoints that > consist of combining characters then your result is indeed odd looking. > > I never said that nor is that the point I am making. Please remember that lone surrogate pair code points are perfectly valid Unicode code points, nevertheless. Just as a lone combining code point is valid on its own. > Guido points out that Python supports surrogate pairs and says that if > Python is dealing wrongly with this in the core than it needs to be fixed. > I am pointing out that given the fact we allow surrogate pairs we deal > rather simplistic with it in the core. In fact, we do not consider them at > all. In essence: though we may accept full 21-bit codepoints in the form of > \U00000000 escape sequences and store them internally as UTF-16 (which I > still need to verify) we subsequently deal with them programmatically as > UCS-2, which is plain silly. Python applies conversion from non-BMP code points to surroagtes for UCS builds in a few places and I agree that we should probably do that at a few more places. However, these are mainly conversion issues of encoded Unicode representations vs. the internal Unicode storage where you want to avoid exceptions in favor of finding a solution that preserves data. To make it clear: UCS2 builds of Python do not support non-BMP code points out of the box. A programmer will always have to use a codec to map the internal storage on these builds to the full Unicode code point range. The following codecs support surrogates on UCS2 builds: * UTF-8 * UTF-16 * UTF-32 * unicode-escape * raw-unicode-escape > You either commit yourself fully to UTF-16 and surrogate pairs or not. Not > some form in-between, because that will ultimately lead to more confusion > due to the difference in results when dealing with Unicode. Programmers will have to be aware of the fact that on UCS2 builds of Python non-BMP code points will have to be treated differently than on UCS4 builds. I don't see that as a problem. It is in a way similar to 32-bit vs. 64-bit builds of Python or the fact that floating point numbers work differently depending on the Python platform or compiler being used. BTW: Have you ever run into any problems with UCS2 vs. UCS4 in practice that were not easy to solve ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 3 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From guido at python.org Thu Jul 3 15:58:26 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jul 2008 06:58:26 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703104813.GF62693@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> Message-ID: <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> On Thu, Jul 3, 2008 at 3:48 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > My apologies for hammering on this, but I think it is quite important and > currently Python 3.0 seems confused about UCS-2 versus UTF-16. [...] Your seem to be suggesting that len(u"\U00012345") should return 1 on a system that internally uses UTF-16 and hence represents this string as a surrogate pair. This is not going to happen. You may as well complain to the authors of the Java standard about the corresponding problem there. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From djarb at highenergymagic.org Thu Jul 3 16:38:47 2008 From: djarb at highenergymagic.org (Daniel Arbuckle) Date: Thu, 3 Jul 2008 07:38:47 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <031901c8dd12$b7258b60$2570a220$@com.au> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> Message-ID: <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> On Thu, Jul 3, 2008 at 6:42 AM, Mark Hammond <mhammond at skippinet.com.au> wrote: > For people on Windows, win32 isn't a "compatibility" consideration. I > suspect most users of the other platforms MAL mentioned and all others with > their own native unicode implementations would agree. I'm sorry, but you're wrong. Interfacing python to interoperate with the underlying system is compatibility. Surely your own win32 extensions already address this necessity. Regardless, as I said before, nothing justifies silently changing the meaning of a program based on an option that most users don't set for themselves and are not aware of. When such a change would take place, it should be reported explicitly as an error. From asmodai at in-nomine.org Thu Jul 3 16:46:48 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 16:46:48 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> Message-ID: <20080703144648.GA34192@nexus.in-nomine.org> -On [20080703 15:58], Guido van Rossum (guido at python.org) wrote: >Your seem to be suggesting that len(u"\U00012345") should return 1 on >a system that internally uses UTF-16 and hence represents this string >as a surrogate pair. >From a Unicode and UTF-16 point of view that makes the most sense. So yes, I am suggesting that. >This is not going to happen. You may as well complain to the authors >of the Java standard about the corresponding problem there. Why would I need to complain to them? They already fixed it since 1.5.0. Java 1.5.0's release notes (http://java.sun.com/developer/technicalArticles/releases/j2se15/): Supplementary Character Support 32-bit supplementary character support has been carefully added to the platform as part of the transition to Unicode 4.0 support. Supplementary characters are encoded as a special pair of UTF16 values to generate a different character, or codepoint. A surrogate pair is a combination of a high UTF16 value and a following low UTF16 value. The high and low values are from a special range of UTF16 values. In general, when using a String or sequence of characters, the core API libraries will transparently handle the new supplementary characters for you. See also http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html The methods that accept an int value support all Unicode characters, including supplementary characters. For example, Character.isLetter(0x2F81A) returns true because the code point value represents a letter (a CJK ideograph). -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Life can only be understood backwards, but it must be lived forwards... From guido at python.org Thu Jul 3 17:03:55 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jul 2008 08:03:55 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703144648.GA34192@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> Message-ID: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> On Thu, Jul 3, 2008 at 7:46 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > -On [20080703 15:58], Guido van Rossum (guido at python.org) wrote: >>Your seem to be suggesting that len(u"\U00012345") should return 1 on >>a system that internally uses UTF-16 and hence represents this string >>as a surrogate pair. > > From a Unicode and UTF-16 point of view that makes the most sense. So yes, I > am suggesting that. > >>This is not going to happen. You may as well complain to the authors >>of the Java standard about the corresponding problem there. > > Why would I need to complain to them? They already fixed it since 1.5.0. > > Java 1.5.0's release notes > (http://java.sun.com/developer/technicalArticles/releases/j2se15/): > > Supplementary Character Support > > 32-bit supplementary character support has been carefully added to the > platform as part of the transition to Unicode 4.0 support. Supplementary > characters are encoded as a special pair of UTF16 values to generate a > different character, or codepoint. A surrogate pair is a combination of a > high UTF16 value and a following low UTF16 value. The high and low values > are from a special range of UTF16 values. > > In general, when using a String or sequence of characters, the core API > libraries will transparently handle the new supplementary characters for > you. > > See also http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html > > The methods that accept an int value support all Unicode characters, > including supplementary characters. For example, Character.isLetter(0x2F81A) > returns true because the code point value represents a letter (a CJK > ideograph). I don't see an answer there to the question of whether the length() method of a Java String object containing a single surrogate pair returns 1 or 2; I suspect it returns 2. Python 3 supports things like chr(0x12345) and ord("\U00012345"). (And so does Python 2, using unichr and unicode literals.) The one thing that may be missing from Python is things like interpretation of surrogates by functions like isalpha() and I'm okay with adding that (since those have to loop over the entire string anyway). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amauryfa at gmail.com Thu Jul 3 17:31:57 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 3 Jul 2008 17:31:57 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> Message-ID: <e27efe130807030831q57b907e4h69fe63e4cf3e51ab@mail.gmail.com> Hello, 2008/7/3 Guido van Rossum <guido at python.org>: > I don't see an answer there to the question of whether the length() > method of a Java String object containing a single surrogate pair > returns 1 or 2; I suspect it returns 2. Python 3 supports things like > chr(0x12345) and ord("\U00012345"). (And so does Python 2, using > unichr and unicode literals.) python2.6 support for supplementary characters is not ideal: >>> unichr(0x2f81a) ValueError: unichr() arg not in range(0x10000) (narrow Python build) >>> ord(u'\U0002F81A') TypeError: ord() expected a character, but string of length 2 found. \Uxxxxxxxx seems the only way to enter these characters. 3.0 is much better and passes the two tests above. The unicodedata module gives good results in both versions: >>> unicodedata.name(u'\U0002F81A') 'CJK COMPATIBILITY IDEOGRAPH-2F81A' [34311 refs] >>> unicodedata.category(u'\U0002F81A') 'Lo' With python 3.0, I found only two places that refuse large code points on narrow builds: the "%c" format, and Py_BuildValue('C'). They should be fixed. > The one thing that may be missing from Python is things like > interpretation of surrogates by functions like isalpha() and I'm okay > with adding that (since those have to loop over the entire string > anyway). In this case, a new .isascii() method would be needed for some uses. -- Amaury Forgeot d'Arc From p.f.moore at gmail.com Thu Jul 3 17:32:37 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 3 Jul 2008 16:32:37 +0100 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> Message-ID: <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> On 03/07/2008, Guido van Rossum <guido at python.org> wrote: > I don't see an answer there to the question of whether the length() > method of a Java String object containing a single surrogate pair > returns 1 or 2; I suspect it returns 2. It appears you're right: >type testucs.java class testucs { public static void main(String[] args) { StringBuilder s = new StringBuilder("Hello, "); s.appendCodePoint(0x2F81A); System.out.println(s); // Display the string. System.out.println(s.length()); } } >java testucs Hello, ? 9 >java -version java version "1.6.0_05" Java(TM) SE Runtime Environment (build 1.6.0_05-b13) Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing) > Python 3 supports things like > chr(0x12345) and ord("\U00012345"). (And so does Python 2, using > unichr and unicode literals.) And Java doesn't appear to - that appendCodePoint() method was wonderfully hard to find :-) Paul. From armin.ronacher at active-4.com Thu Jul 3 18:30:09 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Thu, 3 Jul 2008 16:30:09 +0000 (UTC) Subject: [Python-Dev] UCS2/UCS4 default References: <20080702141328.GW62693@nexus.in-nomine.org> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> Message-ID: <loom.20080703T162738-186@post.gmane.org> Guido van Rossum <guido <at> python.org> writes: > The one thing that may be missing from Python is things like > interpretation of surrogates by functions like isalpha() and I'm okay > with adding that (since those have to loop over the entire string > anyway). That and methods to safely iterate and slice strings by codepoint. Java supports that via String.codePointCount / String.codePointAt / String.codePointBefore / String.offsetByCodepoints. Maybe not on the unicode/str object itself but as part of unicodedata that would make sense for applications that have to deal with unicode on that level. Regards, Armin From steve at holdenweb.com Thu Jul 3 18:35:29 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 03 Jul 2008 12:35:29 -0400 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> Message-ID: <g4iv4n$3oq$1@ger.gmane.org> Paul Moore wrote: > On 03/07/2008, Guido van Rossum <guido at python.org> wrote: >> I don't see an answer there to the question of whether the length() >> method of a Java String object containing a single surrogate pair >> returns 1 or 2; I suspect it returns 2. > > It appears you're right: > >> type testucs.java > class testucs { > public static void main(String[] args) { > StringBuilder s = new StringBuilder("Hello, "); > s.appendCodePoint(0x2F81A); > System.out.println(s); // Display the string. > System.out.println(s.length()); > } > } > >> java testucs > Hello, ? > 9 > >> java -version > java version "1.6.0_05" > Java(TM) SE Runtime Environment (build 1.6.0_05-b13) > Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing) > >> Python 3 supports things like >> chr(0x12345) and ord("\U00012345"). (And so does Python 2, using >> unichr and unicode literals.) > > And Java doesn't appear to - that appendCodePoint() method was > wonderfully hard to find :-) > There's also the issue of indexing the Unicode strings. If we are going to insist that len(u) counts surrogate pairs as one character then random access to the characters of a string is going to be an extremely inefficient operation. Surely it's desirable under all circumstances that len(u) == sum(1 for c in u) and that [c for c in u] == [c[i] for i in range(*len(u))] How would that play under Jeroen's proposed change? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From asmodai at in-nomine.org Thu Jul 3 18:41:32 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 18:41:32 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> References: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> Message-ID: <20080703164132.GB34192@nexus.in-nomine.org> -On [20080703 17:32], Paul Moore (p.f.moore at gmail.com) wrote: > System.out.println(s.length()); I think you want to use codePointCount() to count the Unicode code points. length() returns Unicode code units. As http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html explains: In the J2SE API documentation, Unicode code point is used for character values in the range between U+0000 and U+10FFFF, and Unicode code unit is used for 16-bit char values that are code units of the UTF-16 encoding. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Man is the measure of all things... From guido at python.org Thu Jul 3 18:48:38 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jul 2008 09:48:38 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <g4iv4n$3oq$1@ger.gmane.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> <g4iv4n$3oq$1@ger.gmane.org> Message-ID: <ca471dc20807030948x418a1607yf928d2e92fa5e54d@mail.gmail.com> On Thu, Jul 3, 2008 at 9:35 AM, Steve Holden <steve at holdenweb.com> wrote: > Paul Moore wrote: >> >> On 03/07/2008, Guido van Rossum <guido at python.org> wrote: >>> >>> I don't see an answer there to the question of whether the length() >>> method of a Java String object containing a single surrogate pair >>> returns 1 or 2; I suspect it returns 2. >> >> It appears you're right: >> >>> type testucs.java >> >> class testucs { >> public static void main(String[] args) { >> StringBuilder s = new StringBuilder("Hello, "); >> s.appendCodePoint(0x2F81A); >> System.out.println(s); // Display the string. >> System.out.println(s.length()); >> } >> } >> >>> java testucs >> >> Hello, ? >> 9 >> >>> java -version >> >> java version "1.6.0_05" >> Java(TM) SE Runtime Environment (build 1.6.0_05-b13) >> Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing) >> >>> Python 3 supports things like >>> chr(0x12345) and ord("\U00012345"). (And so does Python 2, using >>> unichr and unicode literals.) >> >> And Java doesn't appear to - that appendCodePoint() method was >> wonderfully hard to find :-) >> > There's also the issue of indexing the Unicode strings. If we are going to > insist that len(u) counts surrogate pairs as one character then random > access to the characters of a string is going to be an extremely inefficient > operation. But my whole point is that len(u) should count surrogate pairs as TWO! > Surely it's desirable under all circumstances that > > len(u) == sum(1 for c in u) > > and that > > [c for c in u] == [c[i] for i in range(*len(u))] > > How would that play under Jeroen's proposed change? I am not considering such a change. At best there will be some helper function in unicodedata, or perhaps a helper method on the 3.0 str type to iterate over characters instead of 16-bit values. Whether that iterator should yield 21-bit integer values or strings containing one character (i.e. perhaps a surrogate pair) and what it would do with lone surrogate halves is up to the committee to design this API. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Thu Jul 3 18:51:40 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 18:51:40 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> References: <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> Message-ID: <20080703165140.GD34192@nexus.in-nomine.org> -On [20080703 17:03], Guido van Rossum (guido at python.org) wrote: >I don't see an answer there to the question of whether the length() >method of a Java String object containing a single surrogate pair >returns 1 or 2; I suspect it returns 2. As http://java.sun.com/j2se/1.5.0/docs/api/java/lang/CharSequence.html#length() states: int length() Returns the length of this character sequence. The length is the number of 16-bit chars in the sequence. But since Java switched to full UTF-16 support in 1.5.0 they extended their API since the existing methods have probably come too ingrained. E.g. codePointCount() http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html#codePointCount(char[],%20int,%20int) >The one thing that may be missing from Python is things like >interpretation of surrogates by functions like isalpha() and I'm okay >with adding that (since those have to loop over the entire string >anyway). Those would be welcome already, yes. I'll see if I can help out. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Fallen into ever-mourn, with these wings so torn, after your day my dawn... From asmodai at in-nomine.org Thu Jul 3 19:01:30 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 19:01:30 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net> References: <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net> Message-ID: <20080703170130.GE34192@nexus.in-nomine.org> -On [20080703 18:45], James Y Knight (foom at fuhm.net) wrote: >I think this is misguided. Only trying to at least correct the current situation, which I consider a bit of a mess, personally. (Although it seems others share my view.) >I'd like to have 3 levels of access available: >1) "byte"-level. In a new implementation I'd probably choose to make >all my strings stored in UTF-8, but UTF-16 is fine too. >2) codepoint-level. >3) grapheme-level. Sounds interesting as well and I can very much see the advantages of such levels and their methods. Especially in the i18n/l10n work I do. >You should be able to iterate over the string at any of the levels, >ask for the nearest codepoint/grapheme boundary to the left or right >of an index at a different level, etc. [snip] Actually it seems Java already has a lot of similar methods. >There are a few more desirable operations, to manipulate strings at >the grapheme level (because unlike for UTF-8/UTF-16 codepoints, >graphemes don't have the nice property of not containing prefixes >which are themselves valid graphemes). So, you want a find (and >everything else that implicitly does a find operation, like split, >replace, strip, etc) which requires that both endpoints of its match >are on a grapheme-boundary. [[Probably the easiest way to implement >this would be in the regexp engine.]] Well, your ideas and seeing Java's stuff actually got me excited to work on these kind of ideas, next to my datetime revamp. What would the chances for inclusion in Python be if such a PEP + code would be presented Guido? -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Beware of the fury of the patient man... From foom at fuhm.net Thu Jul 3 18:45:39 2008 From: foom at fuhm.net (James Y Knight) Date: Thu, 3 Jul 2008 12:45:39 -0400 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703144648.GA34192@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> Message-ID: <F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net> On Jul 3, 2008, at 10:46 AM, Jeroen Ruigrok van der Werven wrote: > -On [20080703 15:58], Guido van Rossum (guido at python.org) wrote: >> Your seem to be suggesting that len(u"\U00012345") should return 1 on >> a system that internally uses UTF-16 and hence represents this string >> as a surrogate pair. > > From a Unicode and UTF-16 point of view that makes the most sense. > So yes, I > am suggesting that. I think this is misguided. IMO, basically every programming language gets string handling wrong. (maybe with the exception of the unreleased perl6? it had some interesting moves in this area, but I haven't really been paying attention.) Everyone treats strings as arrays, but they are used quite differently. For a string, there is hardly ever a time when a programmer needs to index it with an arbitrary offset in number of codepoints, and the length-in-codepoints is pretty non-useful as well. Constant-time access to arbitrary codepoints in a string is pretty much unimportant. What *is* of utmost importantance is constant-time access to previously-returned points in the string. I'd like to have 3 levels of access available: 1) "byte"-level. In a new implementation I'd probably choose to make all my strings stored in UTF-8, but UTF-16 is fine too. 2) codepoint-level. 3) grapheme-level. You should be able to iterate over the string at any of the levels, ask for the nearest codepoint/grapheme boundary to the left or right of an index at a different level, etc. Python could probably still be made to work kinda like this. I think a language designed as such in the first place could be nicer, with opaque index objects into the string rather than integers, and such, but...whatever. Let's assume python is changed to always store strings in UTF-16. All it would take is adding a few more functions to the str object to operate on the higher levels. Wherever I say "pos" I mean an integer index into the string, at the UTF-16 level. That may sometimes be unaligned with the boundary of the representation you're asking about, and behavior in that case needs to be specified as well. .nextcodepoint(curpos, how_many=1) -> returns an index into the string how_many codepoints to the right (or left if negative) of the index curpos. .nextgrapheme(curpos, how_many=1) -> returns an index into the string how_many graphemes to the right (or left if negative) of the index curpos. .codepoints(from_pos=0, to_pos=None) -> return an iterator of codepoints from 'from_pos' to 'to_pos'. I think codepoints could be represented as strings themselves (so usually one character, sometimes two character strings). .graphemes(from_pos=0, to_pos=None) -> return an iterator of graphemes from 'from_pos' to 'to_pos'. Also could be represented by strings. The returned graphemes should probably be normalized. There are a few more desirable operations, to manipulate strings at the grapheme level (because unlike for UTF-8/UTF-16 codepoints, graphemes don't have the nice property of not containing prefixes which are themselves valid graphemes). So, you want a find (and everything else that implicitly does a find operation, like split, replace, strip, etc) which requires that both endpoints of its match are on a grapheme-boundary. [[Probably the easiest way to implement this would be in the regexp engine.]] A concrete example of that: u'A\N{COMBINING TILDE}\N{COMBINING MACRON BELOW}'.find(u'A\N{COMBINING TILDE}') returns 0. But you want a way to ask for only a *actual* "A with tilde", not an "A with tilde and macron". Anyhow, I'm not going to tackle this issue or try to push it further, but if someone does tackle it, python could grow to have the best unicode available. :) James From guido at python.org Thu Jul 3 19:10:07 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jul 2008 10:10:07 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703170130.GE34192@nexus.in-nomine.org> References: <20080702171957.GY62693@nexus.in-nomine.org> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <F4FF747D-1FA6-4C75-ADB2-76B794E6EFAF@fuhm.net> <20080703170130.GE34192@nexus.in-nomine.org> Message-ID: <ca471dc20807031010w4af82a4cn60a275a13638deb9@mail.gmail.com> On Thu, Jul 3, 2008 at 10:01 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > What would the chances for inclusion in Python be if such a PEP + code would > be presented Guido? As long as it is clear that the len() function and the basic slicing and indexing operations on strings continue to work in code units (i.e. 16-bit quantities) and the APIs for dealing with code points (i.e. treating surrogate pairs as a single character) are a separate API, there is a chance. Existing code using the existing APIs should not change its behavior (even if you consider the existing behavior broken), with the exception of isalpha() and similar APIs, which can IMO safely be extended to consider surrogate pairs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From facundobatista at gmail.com Thu Jul 3 19:12:41 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 3 Jul 2008 14:12:41 -0300 Subject: [Python-Dev] us.pycon.org down? Message-ID: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com> (sorry for the crossposting) Do you know what happened with "http://us.pycon.org/"? Thank you! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From rhamph at gmail.com Thu Jul 3 19:21:24 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 3 Jul 2008 11:21:24 -0600 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486CDAD5.1060506@egenix.com> References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> <20080703132146.GI62693@nexus.in-nomine.org> <486CDAD5.1060506@egenix.com> Message-ID: <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com> On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote: > On 2008-07-03 15:21, Jeroen Ruigrok van der Werven wrote: >> >> -On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote: >>> >>> Unicode if full of combining code points - if you break such a sequence, >>> the output will be just as wrong; regardless of UCS2 vs. UCS4. >> >> In my opinion you are confusing two related, but very separated things >> here. >> Combining characters have nothing to do with breaking up the encoding of a >> single codepoint. Sure enough, if you arbitrary slice up codepoints that >> consist of combining characters then your result is indeed odd looking. >> >> I never said that nor is that the point I am making. > > Please remember that lone surrogate pair code points are perfectly > valid Unicode code points, nevertheless. Just as a lone combining > code point is valid on its own. That is a big part of these problems. For all practical purposes, a surrogate is like a UTF-8 code unit, and must be handled the same way, so why the heck do they confuse everybody by saying "oh, it's a code point too!"? -- Adam Olsen, aka Rhamphoryncus From martin at v.loewis.de Thu Jul 3 19:31:14 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 03 Jul 2008 19:31:14 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703104813.GF62693@nexus.in-nomine.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> Message-ID: <486D0CE2.6010909@v.loewis.de> > Basically everything but string forming or string printing seems to be > broken for surrogate pairs, from what I can tell. We probably disagree what "it works correctly" means. I think everything works correctly. > Also, I think you are confused about slicing in the middle of a surrogate > pair, from a UTF-16 perspective this is 1 codepoint! Yes, but it is two code units. Python's UTF-16 implementation operates on code units, not code points. > And as such Python > needs to treat it as one character/codepoint in a string, dealing with > slicing as appropriate. It does. However, functions such as len, and all indexing, operate in code units, not code points. > The way you currently describe it is that UTF-16 > strings will be treated as UCS-2 when it comes to slicing and the likes. No. In UCS-2, the surrogate range is reserved (for UTF-16). In Python, it's not reserved, but interpreted as UTF-16. > From a UTF-16 point of view such slicing can NEVER occur unless you are bit > or byte slicing instead of character/codepoint slicing. It most certainly can. UTF-16 is not a character set, but a character encoding form (unlike UCS-2, which is a coded character set). Slicing *can* occur at the code unit level. UTF-16 is also understood as a character encoding scheme (by means of the BOM), then slicing can occur even on the byte level. > I think it can be fairly said that an item in a string is a character or > codepoint. Not in Python - it's a code unit. Regards, Martin From goodger at python.org Thu Jul 3 19:32:41 2008 From: goodger at python.org (David Goodger) Date: Thu, 3 Jul 2008 13:32:41 -0400 Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down? In-Reply-To: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com> References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com> Message-ID: <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com> On Thu, Jul 3, 2008 at 13:12, Facundo Batista <facundobatista at gmail.com> wrote: > (sorry for the crossposting) > > Do you know what happened with "http://us.pycon.org/"? Not sure. The machine is still up (it serves www.pycon.org as well). Either something is misconfigured, or a process can't start, or something... I'll ask Jeff Rush (whose machine it's on) and Doug Napoleone (who knows more about the server than I, and has admin access) to look into it. -- David Goodger <http://python.net/~goodger> From martin at v.loewis.de Thu Jul 3 19:33:23 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 03 Jul 2008 19:33:23 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486CC881.5090902@gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> Message-ID: <486D0D63.3090807@v.loewis.de> > 1. System is NOT memory limited (i.e. most desktops): use a UCS-4 Python > build, which is what most Linux distributions do (I'm not sure about the > pydotorg provided Windows or Mac OS X builds). The Windows builds must continue to use a two-byte representation, as otherwise PythonWin will break (and anything else that tries to pass Unicode strings directly to a Win32 *W function). Regards, Martin From asmodai at in-nomine.org Thu Jul 3 19:35:45 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 19:35:45 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com> References: <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> <20080703132146.GI62693@nexus.in-nomine.org> <486CDAD5.1060506@egenix.com> <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com> Message-ID: <20080703173545.GF34192@nexus.in-nomine.org> -On [20080703 19:21], Adam Olsen (rhamph at gmail.com) wrote: >On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote: >> Please remember that lone surrogate pair code points are perfectly >> valid Unicode code points, nevertheless. Just as a lone combining >> code point is valid on its own. > >That is a big part of these problems. For all practical purposes, a >surrogate is like a UTF-8 code unit, and must be handled the same way, >so why the heck do they confuse everybody by saying "oh, it's a code >point too!"? Because surrogate code points are not Unicode scalar values, isolated UTF-16 code units in the range 0xd800-0xdfff are ill-formed. (D91 from Unicode 5.0/5.1, section 3.9) So, no, it is not a code point too. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Als men blijft geloven kan de zwaarste steen niet zinken... From martin at v.loewis.de Thu Jul 3 19:36:03 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 03 Jul 2008 19:36:03 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486CDAD5.1060506@egenix.com> References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> <20080703132146.GI62693@nexus.in-nomine.org> <486CDAD5.1060506@egenix.com> Message-ID: <486D0E03.8020007@v.loewis.de> > Please remember that lone surrogate pair code points are perfectly > valid Unicode code points, nevertheless. Just as a lone combining > code point is valid on its own. Actually, I think they aren't (not any more than an invalid codepoint, or an unassigned codepoint). They are reserved for UTF-16 only. I would have to lookup the exact Unicode terminology, but "valid" is probably not a predicate that they would use. Regards, Martin From martin at v.loewis.de Thu Jul 3 19:39:00 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 03 Jul 2008 19:39:00 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703164132.GB34192@nexus.in-nomine.org> References: <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> <20080703164132.GB34192@nexus.in-nomine.org> Message-ID: <486D0EB4.10901@v.loewis.de> > I think you want to use codePointCount() to count the Unicode code points. > length() returns Unicode code units. > > As http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html explains: > > In the J2SE API documentation, Unicode code point is used for character > values in the range between U+0000 and U+10FFFF, and Unicode code unit is > used for 16-bit char values that are code units of the UTF-16 encoding. So you would like to contribute a function codePointCount to Python's standard library? Go ahead. Regards, Martin From janssen at parc.com Thu Jul 3 19:43:58 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 3 Jul 2008 10:43:58 PDT Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <g4iv4n$3oq$1@ger.gmane.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <ca471dc20807030658o6ed24172x30733bf955d37221@mail.gmail.com> <20080703144648.GA34192@nexus.in-nomine.org> <ca471dc20807030803v2c7fe62fv713e28816671440c@mail.gmail.com> <79990c6b0807030832k48bb408bmddb3911dc931a839@mail.gmail.com> <g4iv4n$3oq$1@ger.gmane.org> Message-ID: <08Jul3.104407pdt."58698"@synergy1.parc.xerox.com> > Surely it's desirable under all circumstances that > > len(u) == sum(1 for c in u) > > and that > > [c for c in u] == [c[i] for i in range(*len(u))] > > How would that play under Jeroen's proposed change? Yes, but I think the argument is about what "c" is -- a character or a codepoint. Your point about efficiency is well-taken; I doubt that random access to a particular character in a string has to be efficient -- kind of a dying technique these days -- but slices and regexp performance need efficiency guarantees. Bill From tjreedy at udel.edu Thu Jul 3 19:44:19 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 03 Jul 2008 13:44:19 -0400 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> Message-ID: <g4j35h$i4p$1@ger.gmane.org> Daniel Arbuckle wrote: > Regardless, as I said before, nothing justifies silently changing the > meaning of a program based on an option that most users don't set for > themselves and are not aware of. The premise of this thread seems to be that the majority should suffer for the benefit of a few. That is not Python's philosophy. Python hides many system differences. It is gradually hiding more. For instance, float('nan') works uniformly in 2.6 (with little performance hit), whereas it was system specific in 2.5 But Python does not promise to hide all system differences. If the possible effects of (unicode) string build choice are not properly documented, then I agree that they should be, just as they are (or, in some cases, I presume are) the effects of underlying OS, processor integer and pointer size, float scheme, garbage collection scheme, and perhaps something I forgot. Suggested documentation changes can be submitted to the tracker as specific ascii text targeted at a specific location. If accepted, the doc maintainers will adapt submitted text to 'doc style' and add the needed markup. Current response time is usually under a week, perhaps even a day. Documented effects are not 'silent'. But I am sure they could be made a bit louder. Perhaps someday someone will volunteer to contribute a chapter to Using Python on Possible Semantic Variations that would run through the issues listed above so they are gathered together in one place as well as scattered throughout the Language and Library Reference manuals. > When such a change would take place, > it should be reported explicitly as an error. No, possible changes should be documented so that they are not silent. (But I am curious, by 'would' do you mean 'would with the current data' or 'theoretically could with chosen data'?) Terry Jan Reedy From guido at python.org Thu Jul 3 19:55:24 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jul 2008 10:55:24 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <g4j35h$i4p$1@ger.gmane.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> Message-ID: <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> On Thu, Jul 3, 2008 at 10:44 AM, Terry Reedy <tjreedy at udel.edu> wrote: > The premise of this thread seems to be that the majority should suffer for > the benefit of a few. That is not Python's philosophy. Who are the many here? Who are the few? I'd venture that (at least for the foreseeable future, say, until China will finally have taken over the role of the US as the de-facto dominant super power :-) the many are people whose app will never see a Unicode character outside the BMP, or who do such minimal string processing that their code doesn't care whether it's handling UTF-16-encoded data. Python's philosophy is also Practicality Beats Purity. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From doug.napoleone at gmail.com Thu Jul 3 19:53:46 2008 From: doug.napoleone at gmail.com (doug.napoleone at gmail.com) Date: Thu, 3 Jul 2008 13:53:46 -0400 Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down? In-Reply-To: <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com> References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com> <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com> Message-ID: <f08f4fbb0807031053p59fdd3f4qf2ea7b1d22234db@mail.gmail.com> In Montana visiting. Will be back at the hotel in about 4 hours. Looks like base site include is missing or has wrong permissions. On 7/3/08, David Goodger <goodger at python.org> wrote: > On Thu, Jul 3, 2008 at 13:12, Facundo Batista <facundobatista at gmail.com> > wrote: >> (sorry for the crossposting) >> >> Do you know what happened with "http://us.pycon.org/"? > > Not sure. The machine is still up (it serves www.pycon.org as well). > Either something is misconfigured, or a process can't start, or > something... > > I'll ask Jeff Rush (whose machine it's on) and Doug Napoleone (who > knows more about the server than I, and has admin access) to look into > it. > > -- > David Goodger <http://python.net/~goodger> > _______________________________________________ > PyCon-organizers mailing list > PyCon-organizers at python.org > http://mail.python.org/mailman/listinfo/pycon-organizers > From asmodai at in-nomine.org Thu Jul 3 20:39:02 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 3 Jul 2008 20:39:02 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486D0CE2.6010909@v.loewis.de> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702171957.GY62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486D0CE2.6010909@v.loewis.de> Message-ID: <20080703183902.GG34192@nexus.in-nomine.org> -On [20080703 19:31], "Martin v. L?wis" (martin at v.loewis.de) wrote: >Yes, but it is two code units. Python's UTF-16 implementation operates >on code units, not code points. Thank you, that is the single most important piece of information I got about this entire thing because it does change the entire approach. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Knowledge comes, but Wisdom lingers... From goodger at python.org Thu Jul 3 20:40:35 2008 From: goodger at python.org (David Goodger) Date: Thu, 3 Jul 2008 14:40:35 -0400 Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down? In-Reply-To: <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com> References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com> <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com> Message-ID: <4335d2c40807031140o61bed415s2588f1e97f2aa655@mail.gmail.com> On Thu, Jul 3, 2008 at 13:32, David Goodger <goodger at python.org> wrote: > On Thu, Jul 3, 2008 at 13:12, Facundo Batista <facundobatista at gmail.com> wrote: >> (sorry for the crossposting) >> >> Do you know what happened with "http://us.pycon.org/"? > > Not sure. The machine is still up (it serves www.pycon.org as well). > Either something is misconfigured, or a process can't start, or > something... > > I'll ask Jeff Rush (whose machine it's on) and Doug Napoleone (who > knows more about the server than I, and has admin access) to look into > it. Jeff fixed it. URL rewriting was off by mistake. -- David Goodger <http://python.net/~goodger> From rhamph at gmail.com Thu Jul 3 20:50:57 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 3 Jul 2008 12:50:57 -0600 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703173545.GF34192@nexus.in-nomine.org> References: <20080702182215.GA62693@nexus.in-nomine.org> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> <20080703132146.GI62693@nexus.in-nomine.org> <486CDAD5.1060506@egenix.com> <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com> <20080703173545.GF34192@nexus.in-nomine.org> Message-ID: <aac2c7cb0807031150w5a9fcdb6jee5c9adf892390f1@mail.gmail.com> On Thu, Jul 3, 2008 at 11:35 AM, Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> wrote: > -On [20080703 19:21], Adam Olsen (rhamph at gmail.com) wrote: >>On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote: >>> Please remember that lone surrogate pair code points are perfectly >>> valid Unicode code points, nevertheless. Just as a lone combining >>> code point is valid on its own. >> >>That is a big part of these problems. For all practical purposes, a >>surrogate is like a UTF-8 code unit, and must be handled the same way, >>so why the heck do they confuse everybody by saying "oh, it's a code >>point too!"? > > Because surrogate code points are not Unicode scalar values, isolated UTF-16 > code units in the range 0xd800-0xdfff are ill-formed. (D91 from Unicode > 5.0/5.1, section 3.9) > > So, no, it is not a code point too. UTF-16 D91 UTF-16 encoding form: The Unicode encoding form that assigns each Unicode scalar value in the ranges U+0000..U+D7FF and U+E000..U+FFFF to a single unsigned 16-bit code unit with the same numeric value as the Unicode scalar value, and that assigns each Unicode scalar value in the range U+10000..U+10FFFF to a surrogate pair, according to Table 3-5. ? In UTF-16, the code point sequence <004D, 0430, 4E8C, 10302> is represented as <004D 0430 4E8C D800 DF02>, where <D800 DF02> corresponds to U+10302. ? Because surrogate code points are not Unicode scalar values, isolated UTF-16 code units in the range D80016..DFFF16 are ill-formed. In the context of UTF-8 or UTF-32, a Unicode scalar value is a single code point of a valid character (more or less) and a code unit is the base unit (1 and 4 bytes respectively) of which 1 or more combine to form a code point. In UTF-16, code point becomes synonymous with code unit and Unicode scalar value becomes one or more code points. WTF? -- Adam Olsen, aka Rhamphoryncus From mal at egenix.com Thu Jul 3 21:07:04 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Jul 2008 21:07:04 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com> References: <ca471dc20807021008x6b78f420k51e00f4dc2c491aa@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> <20080703132146.GI62693@nexus.in-nomine.org> <486CDAD5.1060506@egenix.com> <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com> Message-ID: <486D2358.1010604@egenix.com> On 2008-07-03 19:21, Adam Olsen wrote: > On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote: >> On 2008-07-03 15:21, Jeroen Ruigrok van der Werven wrote: >>> -On [20080703 15:00], M.-A. Lemburg (mal at egenix.com) wrote: >>>> Unicode if full of combining code points - if you break such a sequence, >>>> the output will be just as wrong; regardless of UCS2 vs. UCS4. >>> In my opinion you are confusing two related, but very separated things >>> here. >>> Combining characters have nothing to do with breaking up the encoding of a >>> single codepoint. Sure enough, if you arbitrary slice up codepoints that >>> consist of combining characters then your result is indeed odd looking. >>> >>> I never said that nor is that the point I am making. >> Please remember that lone surrogate pair code points are perfectly >> valid Unicode code points, nevertheless. Just as a lone combining >> code point is valid on its own. > > That is a big part of these problems. For all practical purposes, a > surrogate is like a UTF-8 code unit, and must be handled the same way, > so why the heck do they confuse everybody by saying "oh, it's a code > point too!"? You have to take that up with the Unicode consortium :-) It would have been better not to add surrogates to the standard at all. To be fair, I don't think that anybody seriously assumed at the time that more than 16 bits would be needed. In practice, you do need to be able to build Unicode strings that contain half a surrogate (ie. a single code point) or a combining code point without its anchor code point, so trying to be smart about detecting surrogates is going to create more confusion than do good, e.g. >>> x1 = u'\udbc0' >>> x2 = u'\udc00' >>> x1 u'\udbc0' >>> x2 u'\udc00' >>> len(x1) 1 >>> len(x2) 1 Having len(x1+x2) == 1 wouldn't be right and break all sorts of assumptions you normally make about string concatenation. Which is why len(x1+x2) gives 2 in both UCS2 and UCS4 builds. The fact that u'\U00100000' can map to a length 1 Unicode string in UCS4 builds and a length 2 string in UCS2 builds is merely due to the fact that the unicode-escape codec (which converts the escaped string literal to a Unicode object) does know about surrogates and uses them to avoid exceptions. Programmers need to be aware of this fact, that's all... just like they need to aware of differences between integer and float division, different behavior of classic and new-style classes, etc. etc. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 3 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Thu Jul 3 21:16:03 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Jul 2008 21:16:03 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <20080703173545.GF34192@nexus.in-nomine.org> References: <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <486CCD66.70906@egenix.com> <20080703132146.GI62693@nexus.in-nomine.org> <486CDAD5.1060506@egenix.com> <aac2c7cb0807031021j768ec436y24b71d360967a0b9@mail.gmail.com> <20080703173545.GF34192@nexus.in-nomine.org> Message-ID: <486D2573.3070301@egenix.com> On 2008-07-03 19:35, Jeroen Ruigrok van der Werven wrote: > -On [20080703 19:21], Adam Olsen (rhamph at gmail.com) wrote: >> On Thu, Jul 3, 2008 at 7:57 AM, M.-A. Lemburg <mal at egenix.com> wrote: >>> Please remember that lone surrogate pair code points are perfectly >>> valid Unicode code points, nevertheless. Just as a lone combining >>> code point is valid on its own. >> That is a big part of these problems. For all practical purposes, a >> surrogate is like a UTF-8 code unit, and must be handled the same way, >> so why the heck do they confuse everybody by saying "oh, it's a code >> point too!"? > > Because surrogate code points are not Unicode scalar values, isolated UTF-16 > code units in the range 0xd800-0xdfff are ill-formed. (D91 from Unicode > 5.0/5.1, section 3.9) True. They are not valid UTF-16 code units, but a code unit is just a storage byte representation of a Unicode tranformation... """ Code Unit. The minimal bit combination that can represent a unit of encoded text for processing or interchange. The Unicode Standard uses 8-bit code units in the UTF-8 encoding form, 16-bit code units in the UTF-16 encoding form, and 32-bit code units in the UTF-32 encoding form. (See definition D77 in Section 3.9, Unicode Encoding Forms.) """ That's not the same thing as a code point which is an assignment of a slot in the Unicode character set... """ Code Point. Any value in the Unicode codespace; that is, the range of integers from 0 to 10FFFF16. (See definition D10 in Section 3.4, Characters and Encoding.) """ Reference: http://www.unicode.org/glossary/ Also see Chapter 3.4 (http://www.unicode.org/versions/Unicode5.0.0/ch03.pdf#G2212): """ Surrogate code points and noncharacters are considered assigned code points, but not assigned characters. """ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 3 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From facundobatista at gmail.com Thu Jul 3 21:16:07 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 3 Jul 2008 16:16:07 -0300 Subject: [Python-Dev] [PyCon-Organizers] us.pycon.org down? In-Reply-To: <4335d2c40807031140o61bed415s2588f1e97f2aa655@mail.gmail.com> References: <e04bdf310807031012v2c6b106ey3862b209ede251bf@mail.gmail.com> <4335d2c40807031032v6804a6e0pc75f02204de9bb3d@mail.gmail.com> <4335d2c40807031140o61bed415s2588f1e97f2aa655@mail.gmail.com> Message-ID: <e04bdf310807031216w2b5df96bgabefb77d828462fb@mail.gmail.com> 2008/7/3 David Goodger <goodger at python.org>: > Jeff fixed it. URL rewriting was off by mistake. Thanks! :) -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From mal at egenix.com Thu Jul 3 21:24:40 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 03 Jul 2008 21:24:40 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <g4j35h$i4p$1@ger.gmane.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> Message-ID: <486D2778.8090503@egenix.com> On 2008-07-03 19:44, Terry Reedy wrote: > The premise of this thread seems to be that the majority should suffer > for the benefit of a few. That is not Python's philosophy. In reality, most Unixes ship with UCS4 builds of Python. Windows and Mac OS X ship with UCS2 builds. Still, anyone is free to build their own favorite version - that's freedom of choice, which is good. Programmers just need to be made aware of the differences in UCS2 and UCS4 builds and deal with it. Here's talk I've given many many times over the years which explains some of the details that a Python programmer needs to know when dealing with Unicode: http://www.egenix.com/files/python/PyConUK2007-Developing-Unicode-aware-applications-in-Python.pdf Perhaps I should add a section on UCS2 vs. UCS4 the next time around ;-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 3 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From jeremy at 54oaks.com Thu Jul 3 21:35:33 2008 From: jeremy at 54oaks.com (Jeremy Link) Date: Thu, 3 Jul 2008 12:35:33 -0700 Subject: [Python-Dev] problems compiling ctypes Message-ID: <010601c8dd43$f71ab890$8101a8c0@bocaron> I've grabbed the latest libffi that contains support for the ARM processor. I then enable FFI_CLOSURES in the arm/ffi.c file. When I do this, I get compilation errors that it is missing ffi_prep_closure. Is ffi.c up to date for supporting the ARM platform? Not sure if there is a simple configuration change in one of the files that will fix *everything* or if ffi.c just doesn't support ARM yet and so it needs be developed/revamped. Thanks for any help. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080703/a6a04fab/attachment.htm> From steve at holdenweb.com Thu Jul 3 21:59:08 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 03 Jul 2008 15:59:08 -0400 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <486D2778.8090503@egenix.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <486D2778.8090503@egenix.com> Message-ID: <g4jb2l$f3g$1@ger.gmane.org> M.-A. Lemburg wrote: > On 2008-07-03 19:44, Terry Reedy wrote: >> The premise of this thread seems to be that the majority should suffer >> for the benefit of a few. That is not Python's philosophy. > > In reality, most Unixes ship with UCS4 builds of Python. Windows > and Mac OS X ship with UCS2 builds. Still, anyone is free to build > their own favorite version - that's freedom of choice, which is good. > > Programmers just need to be made aware of the differences in UCS2 > and UCS4 builds and deal with it. > > Here's talk I've given many many times over the years which explains > some of the details that a Python programmer needs to know when dealing > with Unicode: > > http://www.egenix.com/files/python/PyConUK2007-Developing-Unicode-aware-applications-in-Python.pdf > > > Perhaps I should add a section on UCS2 vs. UCS4 the next time around ;-) > The indications are that would be helpful to many people (including myself). regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From tjreedy at udel.edu Thu Jul 3 23:01:48 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 03 Jul 2008 17:01:48 -0400 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> Message-ID: <g4jenq$rks$1@ger.gmane.org> Guido van Rossum wrote: > On Thu, Jul 3, 2008 at 10:44 AM, Terry Reedy <tjreedy at udel.edu> wrote: >> The premise of this thread seems to be that the majority should suffer for >> the benefit of a few. That is not Python's philosophy. The premise is the OP's idea that Python should switch to all UCS4 to create a more pure ('ideal') situation or the idea that len(s) should count codepoints (correct term?) for all builds as a matter of purity even though on it would be time-costly on 16-bit builds as a matter of practicality. > Who are the many here? Those who are happy with 3.0 strings as they are for their systems and who would not benefit from the proposed change. In other words, what you say below. > Who are the few? Those who are stuck with 16-bit builds and who would benefit from 32-bits builds because they need to use non basic plane chars and need to use the operations for which a change would make a positive difference. In my opinion, such people with Windows should at least install Linux + UCS4 Python as an alternate install. > I'd venture that (at least for > the foreseeable future, say, until China will finally have taken over > the role of the US as the de-facto dominant super power :-) the many > are people whose app will never see a Unicode character outside the > BMP, or who do such minimal string processing that their code doesn't > care whether it's handling UTF-16-encoded data. Just what I meant. > Python's philosophy is also Practicality Beats Purity. Just what I meant, in the form 'Purity does not beat Practicality'. Having summarized, perhaps too briefly, why Python's basic unicode implementation would not change in the near future, I went on to my main point, which is that better docs might be an alternative solution to the problems raised. tjr From martin at v.loewis.de Thu Jul 3 23:15:40 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 03 Jul 2008 23:15:40 +0200 Subject: [Python-Dev] problems compiling ctypes In-Reply-To: <010601c8dd43$f71ab890$8101a8c0@bocaron> References: <010601c8dd43$f71ab890$8101a8c0@bocaron> Message-ID: <486D417C.9060503@v.loewis.de> > Thanks for any help. This list (python-dev) is not for getting help, but for providing it. So if you have patches that you would like to discuss, please go ahead. As you are seeking help, please use python-list at python.org (aka news:comp.lang.python) instead. Regards, Martin From rhamph at gmail.com Fri Jul 4 00:00:56 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 3 Jul 2008 16:00:56 -0600 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <g4jenq$rks$1@ger.gmane.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> <g4jenq$rks$1@ger.gmane.org> Message-ID: <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> On Thu, Jul 3, 2008 at 3:01 PM, Terry Reedy <tjreedy at udel.edu> wrote: > > The premise is the OP's idea that Python should switch to all UCS4 to create > a more pure ('ideal') situation or the idea that len(s) should count > codepoints (correct term?) for all builds as a matter of purity even though > on it would be time-costly on 16-bit builds as a matter of practicality. Wrong term - code units and code points are equivalent in UTF-16 and UTF-32. What you're looking for is unicode scalar values. -- Adam Olsen, aka Rhamphoryncus From guido at python.org Fri Jul 4 00:21:46 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jul 2008 15:21:46 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> <g4jenq$rks$1@ger.gmane.org> <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> Message-ID: <ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com> On Thu, Jul 3, 2008 at 3:00 PM, Adam Olsen <rhamph at gmail.com> wrote: > On Thu, Jul 3, 2008 at 3:01 PM, Terry Reedy <tjreedy at udel.edu> wrote: >> >> The premise is the OP's idea that Python should switch to all UCS4 to create >> a more pure ('ideal') situation or the idea that len(s) should count >> codepoints (correct term?) for all builds as a matter of purity even though >> on it would be time-costly on 16-bit builds as a matter of practicality. > > Wrong term - code units and code points are equivalent in UTF-16 and > UTF-32. What you're looking for is unicode scalar values. I don't think so. I have in my lap the Unicode 5.0 standard, which on page 102, under UTF-16, states (amongst others): """ * In UTF-16, the code point sequence <004D, 0430, 4E8C, 10302> is represented as <004D 0439 4E8C D800 DF02>, where <D800 DF02> corresponds to U+10302. * Because surrogate code points are not Unicode scalar values, isolated UTF-16 code units in the range D800[16]..DFFF[16] are ill-formed. """ >From this I understand they distinguish carefully between code points and code units -- D800 is a code unit but not a code point, 10302 is a code point but not a (UTF-16) code unit. OTOH outside the context of UTF-8, the surrogates are also referred to as "reserved code points" (e.g. in Table 2-3 on page 27, "Types of Code Points"). I think the best thing we can do is to use "code points" to refer to characters and "code units" to the individual 16-bit values in the UTF-16 encoding; this seems compatible with usage elsewhere in this thread by most folks. Also see http://unicode.org/glossary/: """ Code Point. Any value in the Unicode codespace; that is, the range of integers from 0 to 10FFFF16. (See definition D10 in Section 3.4, Characters and Encoding.) . . . Code Unit. The minimal bit combination that can represent a unit of encoded text for processing or interchange. The Unicode Standard uses 8-bit code units in the UTF-8 encoding form, 16-bit code units in the UTF-16 encoding form, and 32-bit code units in the UTF-32 encoding form. (See definition D77 in Section 3.9, Unicode Encoding Forms.) """ -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Fri Jul 4 00:31:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 04 Jul 2008 00:31:49 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> <g4jenq$rks$1@ger.gmane.org> <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> Message-ID: <486D5355.4000104@v.loewis.de> > Wrong term - code units and code points are equivalent in UTF-16 and > UTF-32. What you're looking for is unicode scalar values. How so? Section 2.5, UTF-16 says "code points in the supplementary planes, in the range U+10000..U+10FFFF, are represented as pairs of 16-bit code units." So clearly, code points in Unicode range from U+0000..U+10FFFF, independent of encoding form. In UTF-16, code units range from 0..65535. OTOH, "unicode scalar value" is nearly synonymous to "code point": D76 Unicode Scalar Value. Any Unicode code point except high-surrogate and low-surrogate code points. So codepoint in Terry's message was the right term. Regards, Martin From rhamph at gmail.com Fri Jul 4 01:50:51 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 3 Jul 2008 17:50:51 -0600 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> <g4jenq$rks$1@ger.gmane.org> <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> <ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com> Message-ID: <aac2c7cb0807031650u7abe63d2q7371bdc1782b7d8@mail.gmail.com> On Thu, Jul 3, 2008 at 4:21 PM, Guido van Rossum <guido at python.org> wrote: > On Thu, Jul 3, 2008 at 3:00 PM, Adam Olsen <rhamph at gmail.com> wrote: >> On Thu, Jul 3, 2008 at 3:01 PM, Terry Reedy <tjreedy at udel.edu> wrote: >>> >>> The premise is the OP's idea that Python should switch to all UCS4 to create >>> a more pure ('ideal') situation or the idea that len(s) should count >>> codepoints (correct term?) for all builds as a matter of purity even though >>> on it would be time-costly on 16-bit builds as a matter of practicality. >> >> Wrong term - code units and code points are equivalent in UTF-16 and >> UTF-32. What you're looking for is unicode scalar values. > > I don't think so. I have in my lap the Unicode 5.0 standard, which on > page 102, under UTF-16, states (amongst others): > > """ > * In UTF-16, the code point sequence <004D, 0430, 4E8C, 10302> is > represented as <004D 0439 4E8C D800 DF02>, where <D800 DF02> > corresponds to U+10302. The literal interpretation is that the U+10302 code point should get expanded into <D800 DF02>. It doesn't say if <D800 DF02> is a pair of code units or a pair of code points. > * Because surrogate code points are not Unicode scalar values, > isolated UTF-16 code units in the range D800[16]..DFFF[16] are > ill-formed. > """ So a lone surrogate code unit is not a valid scalar. It also implies surrogate code points exist, rather than ruling them out. > From this I understand they distinguish carefully between code points > and code units -- D800 is a code unit but not a code point, 10302 is a > code point but not a (UTF-16) code unit. I disagree. They switch between code point and code unit arbitrarily, never than saying surrogate code points don't exist. > OTOH outside the context of UTF-8, the surrogates are also referred to > as "reserved code points" (e.g. in Table 2-3 on page 27, "Types of > Code Points"). You mean outside the context of UTF-16? Regarding them as reserved and lone surrogates as ill-formed code units would have been simpler, but alas, is not the case. Regarding changes in 5.1 (http://www.unicode.org/versions/Unicode5.1.0/), I can find this bit to give some context: Rendering Default Ignorable Code Points Update the last paragraph on p. 192 of The Unicode Standard, Version 5.0, in Section 5.20, Default Ignorable Code Points, to read as follows: Replacement Text An implementation should ignore all default ignorable code points in rendering whenever it does not support those code points, whether they are assigned or not. In previous versions of the Unicode Standard, surrogate code points, private use code points, and some control characters were also default ignorable code points. However, to avoid security problems, such characters always should be displayed with a missing glyph, so that there is a visible indication of their presence in the text. In Unicode 5.1 these code points are no longer default ignorable code points. For more information, see UTR #36, "Unicode Security Considerations." Clearly they act as if surrogate code points exist. Finally, we find this in the glossary: Unicode Scalar Value. Any Unicode code point except high-surrogate and low-surrogate code points. In other words, the ranges of integers 0 to D7FF16 and E00016 to 10FFFF16 inclusive. (See definition D76 in Section 3.9, Unicode Encoding Forms.) Clearly, each surrogate is a valid code point, regardless of encoding. A surrogate pair simultaneously represents both one code point (the scalar value) and two code points (the surrogate code points). To be unambiguous you must instead use either code units (always 2 for UTF-16) or scalar values (always 1 in any encoding). The OP wanted it to always be 1, so the correct unambiguous term is scalar value. > I think the best thing we can do is to use "code points" to refer to > characters and "code units" to the individual 16-bit values in the > UTF-16 encoding; this seems compatible with usage elsewhere in this > thread by most folks. > > Also see http://unicode.org/glossary/: > > """ > Code Point. Any value in the Unicode codespace; that is, the range of > integers from 0 to 10FFFF16. (See definition D10 in Section 3.4, > Characters and Encoding.) > . > . > . > Code Unit. The minimal bit combination that can represent a unit of > encoded text for processing or interchange. The Unicode Standard uses > 8-bit code units in the UTF-8 encoding form, 16-bit code units in the > UTF-16 encoding form, and 32-bit code units in the UTF-32 encoding > form. (See definition D77 in Section 3.9, Unicode Encoding Forms.) > """ > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- Adam Olsen, aka Rhamphoryncus From guido at python.org Fri Jul 4 05:26:16 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 3 Jul 2008 20:26:16 -0700 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <aac2c7cb0807031650u7abe63d2q7371bdc1782b7d8@mail.gmail.com> References: <20080702141328.GW62693@nexus.in-nomine.org> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> <g4jenq$rks$1@ger.gmane.org> <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> <ca471dc20807031521q77be0814ia864bc532fda2a2f@mail.gmail.com> <aac2c7cb0807031650u7abe63d2q7371bdc1782b7d8@mail.gmail.com> Message-ID: <ca471dc20807032026r1dfca0a5h2799180dbbed54ca@mail.gmail.com> On Thu, Jul 3, 2008 at 4:50 PM, Adam Olsen <rhamph at gmail.com> wrote: > Clearly, each surrogate is a valid code point, regardless of encoding. > A surrogate pair simultaneously represents both one code point (the > scalar value) and two code points (the surrogate code points). To be > unambiguous you must instead use either code units (always 2 for > UTF-16) or scalar values (always 1 in any encoding). > > The OP wanted it to always be 1, so the correct unambiguous term is > scalar value. Fine, if you want to be completely unambiguous you apparently you can't use the word code point but you have to use either scalar values (always Unicode characters) or code units (always part of an encoding, and 8, 16 or 32 bits). Regardless of what the OP might want, len() of a surrogate pair will return 2 (since it counts code units), and we'll have to provide another API to count scalar values / characters that sees a surrogate pair as one. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Fri Jul 4 11:39:55 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 4 Jul 2008 10:39:55 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <e8a0972d0806281912w4328c55tfc55c5a298c3333c@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> <e8a0972d0806281912w4328c55tfc55c5a298c3333c@mail.gmail.com> Message-ID: <5c6f2a5d0807040239s22fc6b7cx31f27e916b6ba6a2@mail.gmail.com> On Sun, Jun 29, 2008 at 3:12 AM, Alex Martelli <aleaxit at gmail.com> wrote: > On Sat, Jun 28, 2008 at 4:46 PM, Raymond Hettinger <python at rcn.com> wrote: >> Is everyone agreed on a tohex/fromhex pair using the C99 notation as >> recommended in 754R? > > Dunno about everyone, but I'm +1 on that. > > >> Are you thinking of math module functions or as a method and classmethod on >> floats? > > I'd prefer math modules functions. I'm halfway through implementing this as a pair of float methods. Are there compelling reasons to prefer math module functions over float methods, or vice versa? Personally, I'm leaning slightly towards float methods: for me, these conversions are important enough to belong in the core language. But I don't have strong feelings either way. Mark From mal at egenix.com Fri Jul 4 12:08:14 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Jul 2008 12:08:14 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <g4jb2l$f3g$1@ger.gmane.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021042rde10916mb21ec88e13d8977f@mail.gmail.com> <20080702182215.GA62693@nexus.in-nomine.org> <ca471dc20807021127v695f9da8tc8ad305999c5d522@mail.gmail.com> <20080702183541.GB62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <486D2778.8090503@egenix.com> <g4jb2l$f3g$1@ger.gmane.org> Message-ID: <486DF68E.7090000@egenix.com> On 2008-07-03 21:59, Steve Holden wrote: > M.-A. Lemburg wrote: >> On 2008-07-03 19:44, Terry Reedy wrote: >>> The premise of this thread seems to be that the majority should >>> suffer for the benefit of a few. That is not Python's philosophy. >> >> In reality, most Unixes ship with UCS4 builds of Python. Windows >> and Mac OS X ship with UCS2 builds. Still, anyone is free to build >> their own favorite version - that's freedom of choice, which is good. >> >> Programmers just need to be made aware of the differences in UCS2 >> and UCS4 builds and deal with it. >> >> Here's talk I've given many many times over the years which explains >> some of the details that a Python programmer needs to know when dealing >> with Unicode: >> >> http://www.egenix.com/files/python/PyConUK2007-Developing-Unicode-aware-applications-in-Python.pdf >> >> >> Perhaps I should add a section on UCS2 vs. UCS4 the next time around ;-) > > The indications are that would be helpful to many people (including > myself). Ok, I'll add one for one of the next conferences. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 04 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 2 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From guido at python.org Fri Jul 4 15:49:07 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Jul 2008 06:49:07 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0807040239s22fc6b7cx31f27e916b6ba6a2@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> <e8a0972d0806281912w4328c55tfc55c5a298c3333c@mail.gmail.com> <5c6f2a5d0807040239s22fc6b7cx31f27e916b6ba6a2@mail.gmail.com> Message-ID: <ca471dc20807040649r467d8554n18aa8e7f41daaa9b@mail.gmail.com> Float methods are fine. On Fri, Jul 4, 2008 at 2:39 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > On Sun, Jun 29, 2008 at 3:12 AM, Alex Martelli <aleaxit at gmail.com> wrote: >> On Sat, Jun 28, 2008 at 4:46 PM, Raymond Hettinger <python at rcn.com> wrote: >>> Is everyone agreed on a tohex/fromhex pair using the C99 notation as >>> recommended in 754R? >> >> Dunno about everyone, but I'm +1 on that. >> >> >>> Are you thinking of math module functions or as a method and classmethod on >>> floats? >> >> I'd prefer math modules functions. > > I'm halfway through implementing this as a pair of float methods. Are there > compelling reasons to prefer math module functions over float methods, or > vice versa? > > Personally, I'm leaning slightly towards float methods: for me, these > conversions are important enough to belong in the core language. But I > don't have strong feelings either way. > > Mark > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mail at timgolden.me.uk Fri Jul 4 17:24:32 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Fri, 04 Jul 2008 16:24:32 +0100 Subject: [Python-Dev] ctypes assertion failure Message-ID: <486E40B0.4020608@timgolden.me.uk> This problem was raised on the comtypes-users list as it prevents comtypes from being imported on Python 2.6 at the moment. http://bugs.python.org/issue3258 I'll try to find the time to step through to code to work out what's going on, but it's inside the innards of ctypes which I've never looked into before. Could someone confirm at a glance whether this should be given a high priority, please? It results in an assertion error in debug mode, a SystemError in non-debug referring to a NULL return without an Exception set. Thanks TJG From status at bugs.python.org Fri Jul 4 18:06:28 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 4 Jul 2008 18:06:28 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080704160628.7A783780B1@psf.upfronthosting.co.za> ACTIVITY SUMMARY (06/27/08 - 07/04/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1941 open (+33) / 13165 closed (+34) / 15106 total (+67) Open issues with patches: 607 Average duration of open issues: 704 days. Median duration of open issues: 1570 days. Open Issues Breakdown open 1915 (+33) pending 26 ( +0) Issues Created Or Reopened (71) _______________________________ isinstance(anything, MetaclassThatDefinesInstancecheck) raises i 06/30/08 http://bugs.python.org/issue2325 reopened jyasskin patch test_multiprocessing hangs on OS X 10.5.3 07/02/08 http://bugs.python.org/issue3088 reopened jnoller patch test_multiprocessing causes test_ctypes to fail 07/02/08 http://bugs.python.org/issue3125 reopened jnoller patch "Quick search" box renders too long on FireFox 3 06/27/08 http://bugs.python.org/issue3154 reopened benjamin.peterson make text is broken 06/27/08 CLOSED http://bugs.python.org/issue3217 created benjamin.peterson 2to3 Fix_imports optimization 06/27/08 http://bugs.python.org/issue3218 created nedds patch repeated keyword arguments 06/27/08 CLOSED http://bugs.python.org/issue3219 created gangesmaster patch Improve Bytes and Byte Array Methods doc 06/27/08 CLOSED http://bugs.python.org/issue3220 created tjreedy SystemError: Parent module 'foo' not loaded on import statement 06/27/08 http://bugs.python.org/issue3221 created schmir inf*inf gives inf, but inf**2 gives overflow error 06/27/08 CLOSED http://bugs.python.org/issue3222 created ms py3k warn on use of frame.f_exc* 06/27/08 http://bugs.python.org/issue3223 created benjamin.peterson easy Small typo in 2.6 what's new 06/28/08 CLOSED http://bugs.python.org/issue3224 created catlee backport python 3.0 language functionality to python 2.5 by addi 06/28/08 CLOSED http://bugs.python.org/issue3225 created kaizhu can't install on OSX 10.4 06/28/08 http://bugs.python.org/issue3226 created benjamin.peterson os.environ.clear has no effect on child processes 06/28/08 CLOSED http://bugs.python.org/issue3227 created joe.p.cool mailbox.mbox creates files with execute bit set 06/28/08 http://bugs.python.org/issue3228 created pl Language reference, class definitions: missing text in "Programm 06/28/08 CLOSED http://bugs.python.org/issue3229 created oefe dictobject.c: inappropriate use of PySet_GET_SIZE? 06/28/08 CLOSED http://bugs.python.org/issue3230 created oefe re.compile fails with some bytes patterns 06/28/08 http://bugs.python.org/issue3231 created pitrou patch Wrong str->bytes conversion in Lib/encodings/idna.py 06/29/08 http://bugs.python.org/issue3232 created pitrou Timestamp stored in ZIP file not correct ? 06/29/08 CLOSED http://bugs.python.org/issue3233 created pythonmeister subprocess.py strips last character when raising an AttributeErr 06/29/08 CLOSED http://bugs.python.org/issue3234 created mmokrejs Improve subprocess module usage 06/29/08 CLOSED http://bugs.python.org/issue3235 created mmokrejs ints contructed from strings don't use the smallint constants 06/29/08 CLOSED http://bugs.python.org/issue3236 created pitrou idlelib3.0 still using xrange 06/29/08 CLOSED http://bugs.python.org/issue3237 created tjreedy backport python 3.0 language functionality to python 2.6 by addi 06/29/08 http://bugs.python.org/issue3238 created kaizhu curses/textpad.py incorrectly and redundantly imports ascii 06/30/08 http://bugs.python.org/issue3239 reopened facundobatista patch IDLE environment corrupts string.letters 06/30/08 CLOSED http://bugs.python.org/issue3240 created rupole warnings module prints garbage 06/30/08 CLOSED http://bugs.python.org/issue3241 created schmir Segfault in PyFile_SoftSpace/PyEval_EvalFrameEx with sys.stdout 06/30/08 CLOSED http://bugs.python.org/issue3242 reopened benjamin.peterson patch Support iterable bodies in httplib 06/30/08 http://bugs.python.org/issue3243 created catlee multipart/form-data encoding 06/30/08 http://bugs.python.org/issue3244 created catlee Memory leak on OS X 06/30/08 CLOSED http://bugs.python.org/issue3245 created fiddlerwoaroof configure: WARNING: sys/socket.h: present but cannot be compiled 06/30/08 http://bugs.python.org/issue3246 created rrochele dir of an "_sre.SRE_Match" object not working 07/01/08 CLOSED http://bugs.python.org/issue3247 created vizcayno ScrolledText can't be placed in a PanedWindow 07/01/08 http://bugs.python.org/issue3248 created gpolo patch bug adding datetime.timedelta to datetime.date 07/01/08 CLOSED http://bugs.python.org/issue3249 created cjw296 datetime.time does not support arithmetic 07/01/08 http://bugs.python.org/issue3250 created cjw296 patch references are case insensitive 07/01/08 CLOSED http://bugs.python.org/issue3251 created tds333 str.tobytes() and bytes/bytearray.tostr() 07/01/08 CLOSED http://bugs.python.org/issue3252 created mark shutil.move bahave unexpected in fat32 07/01/08 http://bugs.python.org/issue3253 created grissiom Suggestion: change default behavior of __ne__ 07/01/08 CLOSED http://bugs.python.org/issue3254 created cvp [proposal] alternative for re.sub 07/02/08 http://bugs.python.org/issue3255 created ocean-city Multiprocessing docs are not 3.0-ready 07/02/08 http://bugs.python.org/issue3256 created mishok13 "#define socklen_t int" in pyconfig.h 07/02/08 CLOSED http://bugs.python.org/issue3257 created fgoujeon ctypes assertion failure in trunk 07/02/08 http://bugs.python.org/issue3258 created tim.golden fix_imports needs to be using the 'as' keyword 07/02/08 CLOSED http://bugs.python.org/issue3259 created brett.cannon fix_imports does not handle intra-package renames 07/02/08 http://bugs.python.org/issue3260 created brett.cannon Lib/test/test_cookielib declares utf-8 encoding, but contains no 07/02/08 CLOSED http://bugs.python.org/issue3261 created leosoto re.split doesn't split with zero-width regex 07/02/08 http://bugs.python.org/issue3262 created mrabarnett patch Odd code fragment in ABC definitions 07/02/08 CLOSED http://bugs.python.org/issue3263 created rhettinger Use -lcrypto instead of -lcrypt on Solaris 2.6 when available 07/02/08 CLOSED http://bugs.python.org/issue3264 created mmokrejs Python-2.5.2/Modules/_ctypes/malloc_closure.c:70: error: `MAP_AN 07/03/08 http://bugs.python.org/issue3265 created mmokrejs Python-2.5.2/Modules/mmapmodule.c:915: error: `O_RDWR' undeclare 07/03/08 http://bugs.python.org/issue3266 created mmokrejs yield in list comprehensions possibly broken in 3.0 07/03/08 CLOSED http://bugs.python.org/issue3267 created erickt Cleanup of tp_basicsize inheritance 07/03/08 http://bugs.python.org/issue3268 created Rhamphoryncus patch strptime() makes an error concerning second in arg 07/03/08 CLOSED http://bugs.python.org/issue3269 created nevgor test_multiprocessing: test_listener_client flakiness 07/03/08 http://bugs.python.org/issue3270 created jnoller patch iter.next() or iter.__next__() ? 07/03/08 CLOSED http://bugs.python.org/issue3271 created vizcayno Multiprocessing hangs when multiprocessing.Pool methods are call 07/03/08 http://bugs.python.org/issue3272 created mishok13 multiprocessing and meaningful errors 07/03/08 http://bugs.python.org/issue3273 created mishok13 Py_CLEAR(tmp) seg faults 07/03/08 http://bugs.python.org/issue3274 created stutzbach Control flow not optimized 07/03/08 CLOSED http://bugs.python.org/issue3275 created quotemstr httplib.HTTPConnection._send_request should not blindly assume d 07/04/08 http://bugs.python.org/issue3276 created ludvig.ericson patch socket's OOB data management is broken on FreeBSD 07/04/08 http://bugs.python.org/issue3277 created giampaolo.rodola socket's SO_OOBINLINE option does not work on FreeBSD 07/04/08 http://bugs.python.org/issue3278 created giampaolo.rodola import of site.py fails on startup 07/04/08 http://bugs.python.org/issue3279 created rupole %c format does not accept large numbers on ucs-2 builds 07/04/08 http://bugs.python.org/issue3280 created amaury.forgeotdarc support r"\" 07/04/08 CLOSED http://bugs.python.org/issue3281 created lidaobing Undefined unicode characters should be non-printable 07/04/08 CLOSED http://bugs.python.org/issue3282 created amaury.forgeotdarc multiprocessing.connection doesn't import AuthenticationError, w 07/04/08 http://bugs.python.org/issue3283 created mishok13 patch Issues Now Closed (61) ______________________ DeprecationWarning in zipfile.py while zipping 113000 files 216 days http://bugs.python.org/issue1526 loewis zipfile hangs on certain zip files 202 days http://bugs.python.org/issue1622 loewis patch Move Demo/classes/Rat.py to Lib/fractions.py and fix it up. 6 days http://bugs.python.org/issue1682 marketdickinson patch ZIP files with archive comments longer than 4k not recognized as 179 days http://bugs.python.org/issue1746 loewis test_audioop.py converted to unittest 142 days http://bugs.python.org/issue2042 benjamin.peterson patch urlparse() does not handle URLs with port numbers properly 127 days http://bugs.python.org/issue2195 facundobatista subprocess.Popen.communicate takes bytes, not str 68 days http://bugs.python.org/issue2683 georg.brandl patch pickling of large recursive structures crashes cPickle 4 days http://bugs.python.org/issue2702 facundobatista patch asynchat forgets packets when push is called from a thread 54 days http://bugs.python.org/issue2808 josiahcarlson Copy cgi.parse_qs() to urllib.parse 50 days http://bugs.python.org/issue2829 benjamin.peterson tests for sys.getsizeof fail on win64 9 days http://bugs.python.org/issue3147 schuppenies 3.0b1 doesn't seem to install on macs 8 days http://bugs.python.org/issue3174 benjamin.peterson Pydoc should ignore __package__ attributes 8 days http://bugs.python.org/issue3190 ncoghlan round docstring is inaccurate 7 days http://bugs.python.org/issue3191 georg.brandl Documentation for fractions module needs work 2 days http://bugs.python.org/issue3197 marketdickinson patch Can't import sqlite3 in Python 2.6b1 3 days http://bugs.python.org/issue3215 loewis make text is broken 4 days http://bugs.python.org/issue3217 georg.brandl repeated keyword arguments 4 days http://bugs.python.org/issue3219 benjamin.peterson patch Improve Bytes and Byte Array Methods doc 4 days http://bugs.python.org/issue3220 georg.brandl inf*inf gives inf, but inf**2 gives overflow error 1 days http://bugs.python.org/issue3222 marketdickinson Small typo in 2.6 what's new 0 days http://bugs.python.org/issue3224 benjamin.peterson backport python 3.0 language functionality to python 2.5 by addi 0 days http://bugs.python.org/issue3225 loewis os.environ.clear has no effect on child processes 0 days http://bugs.python.org/issue3227 benjamin.peterson Language reference, class definitions: missing text in "Programm 0 days http://bugs.python.org/issue3229 benjamin.peterson dictobject.c: inappropriate use of PySet_GET_SIZE? 0 days http://bugs.python.org/issue3230 rhettinger Timestamp stored in ZIP file not correct ? 0 days http://bugs.python.org/issue3233 loewis subprocess.py strips last character when raising an AttributeErr 0 days http://bugs.python.org/issue3234 benjamin.peterson Improve subprocess module usage 3 days http://bugs.python.org/issue3235 mmokrejs ints contructed from strings don't use the smallint constants 1 days http://bugs.python.org/issue3236 loewis idlelib3.0 still using xrange 0 days http://bugs.python.org/issue3237 benjamin.peterson IDLE environment corrupts string.letters 2 days http://bugs.python.org/issue3240 loewis warnings module prints garbage 0 days http://bugs.python.org/issue3241 brett.cannon Segfault in PyFile_SoftSpace/PyEval_EvalFrameEx with sys.stdout 1 days http://bugs.python.org/issue3242 amaury.forgeotdarc patch Memory leak on OS X 0 days http://bugs.python.org/issue3245 benjamin.peterson dir of an "_sre.SRE_Match" object not working 2 days http://bugs.python.org/issue3247 amaury.forgeotdarc bug adding datetime.timedelta to datetime.date 3 days http://bugs.python.org/issue3249 cjw296 references are case insensitive 0 days http://bugs.python.org/issue3251 georg.brandl str.tobytes() and bytes/bytearray.tostr() 0 days http://bugs.python.org/issue3252 lemburg Suggestion: change default behavior of __ne__ 0 days http://bugs.python.org/issue3254 cvp "#define socklen_t int" in pyconfig.h 0 days http://bugs.python.org/issue3257 amaury.forgeotdarc fix_imports needs to be using the 'as' keyword 0 days http://bugs.python.org/issue3259 brett.cannon Lib/test/test_cookielib declares utf-8 encoding, but contains no 0 days http://bugs.python.org/issue3261 brett.cannon Odd code fragment in ABC definitions 0 days http://bugs.python.org/issue3263 gvanrossum Use -lcrypto instead of -lcrypt on Solaris 2.6 when available 0 days http://bugs.python.org/issue3264 mmokrejs yield in list comprehensions possibly broken in 3.0 0 days http://bugs.python.org/issue3267 brett.cannon strptime() makes an error concerning second in arg 0 days http://bugs.python.org/issue3269 nevgor iter.next() or iter.__next__() ? 0 days http://bugs.python.org/issue3271 benjamin.peterson Control flow not optimized 1 days http://bugs.python.org/issue3275 georg.brandl support r"\" 0 days http://bugs.python.org/issue3281 facundobatista Undefined unicode characters should be non-printable 0 days http://bugs.python.org/issue3282 georg.brandl rlcompleter add "(" to callables feature 2520 days http://bugs.python.org/issue449227 facundobatista patch, easy Docs don't define sequence-ness very well 1979 days http://bugs.python.org/issue678464 benjamin.peterson asyncore.dispactcher: incorrect connect 1613 days http://bugs.python.org/issue889153 josiahcarlson catch invalid chunk length in httplib read routine 1590 days http://bugs.python.org/issue900744 pdorrell patch asyncore fixes and improvements 1583 days http://bugs.python.org/issue909005 josiahcarlson patch asyncore.file_dispatcher should not take fd as argument 1393 days http://bugs.python.org/issue1025525 josiahcarlson asyncore should handle ECONNRESET in send 1331 days http://bugs.python.org/issue1063924 josiahcarlson Add notes to the manual about `is` and methods 893 days http://bugs.python.org/issue1410739 georg.brandl patch, easy 2.4.2 file.read caches EOF state 715 days http://bugs.python.org/issue1523853 georg.brandl asyncore/asynchat patches 387 days http://bugs.python.org/issue1736190 josiahcarlson patch asynchat should call "handle_close" 379 days http://bugs.python.org/issue1740572 josiahcarlson Top Issues Most Discussed (10) ______________________________ 35 test_multiprocessing hangs on OS X 10.5.3 2 days open http://bugs.python.org/issue3088 24 math test fails on Solaris 10 13 days open http://bugs.python.org/issue3167 18 cmath test fails on Solaris 10 13 days open http://bugs.python.org/issue3168 11 Let bin/oct/hex show floats 10 days open http://bugs.python.org/issue3008 10 Use -lcrypto instead of -lcrypt on Solaris 2.6 when available 0 days closed http://bugs.python.org/issue3264 10 __eq__ / __hash__ check doesn't take inheritance into account 122 days open http://bugs.python.org/issue2235 8 Segfault in PyFile_SoftSpace/PyEval_EvalFrameEx with sys.stdout 1 days closed http://bugs.python.org/issue3242 8 "Quick search" box renders too long on FireFox 3 7 days pending http://bugs.python.org/issue3154 7 re.IGNORECASE not Unicode-ready 53 days open http://bugs.python.org/issue2834 6 ints contructed from strings don't use the smallint constants 1 days closed http://bugs.python.org/issue3236 From unknown_kev_cat at hotmail.com Sat Jul 5 01:20:34 2008 From: unknown_kev_cat at hotmail.com (Joe Smith) Date: Fri, 4 Jul 2008 23:20:34 +0000 (UTC) Subject: [Python-Dev] UCS2/UCS4 default References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> <g4jenq$rks$1@ger.gmane.org> <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> <486D5355.4000104@v.loewis.de> Message-ID: <loom.20080704T225603-695@post.gmane.org> Martin v. L?wis <martin <at> v.loewis.de> writes: > > > Wrong term - code units and code points are equivalent in UTF-16 and > > UTF-32. What you're looking for is unicode scalar values. > > How so? Section 2.5, UTF-16 says > > "code points in the supplementary planes, in the range > U+10000..U+10FFFF, are represented as pairs of 16-bit code units." > > So clearly, code points in Unicode range from U+0000..U+10FFFF, > independent of encoding form. > > In UTF-16, code units range from 0..65535. > > OTOH, "unicode scalar value" is nearly synonymous to "code point": > > D76 Unicode Scalar Value. Any Unicode code point except high-surrogate > and low-surrogate code points. > > So codepoint in Terry's message was the right term. > No Terry did definitely mean Unicode scalar values. He was describing the "pure" but impractical "len()" that would count a surrogate pair as "1", not 2, even in the 32-bit builds. For what it is worth: Code point: a number between 0 and 1114111. Scalar Value: a code point, except the surrogate code points. Code unit: The basic unit of the encoding. One code unit is always sufficient to encode some Unicode Scalar values. However, other Unicode scalar values may require multiple Code units. Note that a scalar value is a code point. A code point may or may not be a scalar value. Practical len() returns the number of code units of the internal storage format. Pure len() allegedly would return the number of Unicode scalar values (obviously a surrogate pair would be considered a single Unicode scalar value). Please keep in mind that encodings encode Unicode scalar values. Thus a utf-8 code unit sequence (or UTF-32 code unit) that would give a code point in the surrogate sections is technically in error. (Although python would do well to ignore this restriction as there may be valid reasons to have a utf-8 sequence that is not a valid encoded Unicode text sequence) From martin at v.loewis.de Sat Jul 5 07:35:18 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 05 Jul 2008 07:35:18 +0200 Subject: [Python-Dev] UCS2/UCS4 default In-Reply-To: <loom.20080704T225603-695@post.gmane.org> References: <20080702141328.GW62693@nexus.in-nomine.org> <ca471dc20807021147i76e08c08i41bf3290e9042e4c@mail.gmail.com> <20080703104813.GF62693@nexus.in-nomine.org> <486CC881.5090902@gmail.com> <e4b9b0040807030614h7d7a754dv4a6da6d3cbe82ba3@mail.gmail.com> <031901c8dd12$b7258b60$2570a220$@com.au> <e4b9b0040807030738x3fc7d4c1p9b97530901c16150@mail.gmail.com> <g4j35h$i4p$1@ger.gmane.org> <ca471dc20807031055pfaa317ua254fd3e6f483107@mail.gmail.com> <g4jenq$rks$1@ger.gmane.org> <aac2c7cb0807031500k427ac560j91220a916c710136@mail.gmail.com> <486D5355.4000104@v.loewis.de> <loom.20080704T225603-695@post.gmane.org> Message-ID: <486F0816.8070707@v.loewis.de> >> The premise is the OP's idea that Python should switch to all UCS4 to >> create a more pure ('ideal') situation or the idea that len(s) should >> count codepoints (correct term?) for all builds as a matter of purity >> even though on it would be time-costly on 16-bit builds as a matter >> of practicality. > No Terry did definitely mean Unicode scalar values. True. However, using the word "code point" to refer to "Unicode scalar values" is also correct. He (rather, the OP) wanted to count code points (i.e. not count code units). > Practical len() returns the number of code units of the internal storage format. No, it returns the number of code units. > Pure len() allegedly would return the number of Unicode scalar values (obviously > a surrogate pair would be considered a single Unicode scalar value). Perhaps-not-so-obviously-but-still-intendended, a pure len counting surrogate pairs as one would *also* count code points. > Please keep in mind that encodings encode Unicode scalar values. A "coded character set" is "a character set in which each character is assigned a numeric code point". So clearly, a character encoding form encodeds code points. > Thus a utf-8 > code unit sequence (or UTF-32 code unit) that would give a code point in the > surrogate sections is technically in error. Sure, but this has nothing to do with Terry's terminology use. Regards, Martin From dickinsm at gmail.com Sat Jul 5 11:39:34 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 5 Jul 2008 10:39:34 +0100 Subject: [Python-Dev] C99 code in the Python core? Message-ID: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com> I have a general question and a specific question. First the general one: (1) When is it okay to use C99 code in the Python core? More particularly, is it considered acceptable to use widely-implemented library functions that are specified in C99 but not ANSI C, or widely-implemented features that are new to C99? Or is C99 code now acceptable pretty much anywhere? If so, should PEP 7 be updated? It currently says: """Use ANSI/ISO standard C (the 1989 version of the standard).""" I think there are some C99 features that still aren't implemented everywhere, even on major platforms. (Examples are the inverse hyperbolic trig functions in math.h.) And the specific question: (2) Is it okay to use the '%a' format specifier for sprintf, sscanf and friends. Are there major platforms where this isn't implemented? (Using '%a' would make the issue implementation much simpler.) Mark From matthieu.brucher at gmail.com Sat Jul 5 11:59:13 2008 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Sat, 5 Jul 2008 11:59:13 +0200 Subject: [Python-Dev] C99 code in the Python core? In-Reply-To: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com> References: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com> Message-ID: <e76aa17f0807050259xf2ae911v3100af01606cd450@mail.gmail.com> 2008/7/5 Mark Dickinson <dickinsm at gmail.com>: > I have a general question and a specific question. First the general one: > > (1) When is it okay to use C99 code in the Python core? More particularly, > is it considered acceptable to use widely-implemented library functions that > are specified in C99 but not ANSI C, or widely-implemented features that > are new to C99? > > Or is C99 code now acceptable pretty much anywhere? If so, should > PEP 7 be updated? It currently says: """Use ANSI/ISO standard C > (the 1989 version of the standard).""" > > I think there are some C99 features that still aren't implemented > everywhere, even on major platforms. (Examples are the inverse hyperbolic > trig functions in math.h.) Hi, I don't think that C99 is not supported by Visual Studio and there are no plan for Microsoft to do so. Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher From martin at v.loewis.de Sat Jul 5 12:46:53 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Jul 2008 12:46:53 +0200 Subject: [Python-Dev] C99 code in the Python core? In-Reply-To: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com> References: <5c6f2a5d0807050239s61e00dd9ub315bcac59299bdc@mail.gmail.com> Message-ID: <486F511D.50809@v.loewis.de> > (1) When is it okay to use C99 code in the Python core? More particularly, > is it considered acceptable to use widely-implemented library functions that > are specified in C99 but not ANSI C, or widely-implemented features that > are new to C99? [C99 is also ANSI C, IIUC. ANSI has adopted ISO/IEC 9899:1999 as a U.S. national standard.] It's ok to use functions of the C99 standard library if you have a configure test and a fall-back implementation, or if you know that the function is available on all systems we care about. > Or is C99 code now acceptable pretty much anywhere? No. As others have pointed out, Microsoft still hasn't implemented in Visual C. > If so, should > PEP 7 be updated? It currently says: """Use ANSI/ISO standard C > (the 1989 version of the standard).""" No. > (2) Is it okay to use the '%a' format specifier for sprintf, sscanf and friends. > Are there major platforms where this isn't implemented? (Using > '%a' would make the issue implementation much simpler.) It's implemented in VS 2008, see http://msdn.microsoft.com/en-us/library/hf4y5e3w.aspx On the other hand, people still might try to run Python on older versions of Solaris, such as Solaris 2.6 (which was released 1997). I don't know when Solaris' CRT first started to support this. I'd add a configure test, and, at run-time, raise an exception if the C library doesn't support it yet somebody tries to use it. Regards, Martin From greg at krypto.org Sat Jul 5 21:00:35 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 5 Jul 2008 12:00:35 -0700 Subject: [Python-Dev] [Python-3000] Second betas tomorrow In-Reply-To: <D8E48735-E192-486B-A3D3-CD50C5F99DBF@python.org> References: <2865D095-DA45-4875-AE40-8A5F8C81C299@python.org> <ca471dc20807011325p693e47b1v104e134ec4d21c6f@mail.gmail.com> <A072A9E2-D3C9-4737-992A-3373FC8634A0@python.org> <9B1D90667C03488BB57E09808F4C9B64@RaymondLaptop1> <ca471dc20807011354v2499a923yd52da679c6c1cb73@mail.gmail.com> <8A43F3E7-BEE8-4656-833D-867328D16D52@python.org> <bbaeab100807011627p658d0aeax5cdcbdf0dec2c244@mail.gmail.com> <79F76ED3-D21C-4D1D-B6B0-ECEBDFCEDDE3@python.org> <1afaf6160807011942r738aae8u6c84aab03d463d76@mail.gmail.com> <D8E48735-E192-486B-A3D3-CD50C5F99DBF@python.org> Message-ID: <52dc1c820807051200v9bb3673g967aa6d596520f6a@mail.gmail.com> On Tue, Jul 1, 2008 at 8:29 PM, Barry Warsaw <barry at python.org> wrote: > On Jul 1, 2008, at 10:42 PM, Benjamin Peterson wrote: > >> On Tue, Jul 1, 2008 at 8:44 PM, Barry Warsaw <barry at python.org> wrote: >> >>> On Jul 1, 2008, at 7:27 PM, Brett Cannon wrote: >>> >>>> >>>> Is a Google Calendar kept by anyone that lists stuff like planned >>>> release dates, etc.? >>>> >>> >>> >>> http://www.google.com/calendar/ical/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic.ics >>> >> >> Can I get the non-iCal version? >> > > > http://www.google.com/calendar/feeds/b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com/public/basic > > > http://www.google.com/calendar/embed?src=b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com&ctz=America/New_York > > - -Barry > > And for anyone who hasn't already figured it out.. you can just add b6v58qvojllt0i6ql654r1vh00 at group.calendar.google.com<http://www.google.com/calendar/embed?src=b6v58qvojllt0i6ql654r1vh00%40group.calendar.google.com&ctz=America/New_York>as a friend in your existing google calendar to see the release schedule calendar alongside your own. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080705/ed3611be/attachment.htm> From solipsis at pitrou.net Sat Jul 5 21:10:37 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 5 Jul 2008 19:10:37 +0000 (UTC) Subject: [Python-Dev] bytearray and array.array are not thread-safe Message-ID: <loom.20080705T185033-556@post.gmane.org> Hello, Short story: bytearray and array.array by construction allow user code to reallocate their internal memory buffer. But a raw pointer to the said buffer can also be obtained by another thread, and used after releasing the GIL (for CPU-intensive operations like compression). As a consequence, the interpreter crashes. Was it envisioned? I see no warning in the docs for the array.array type (although it has been there for quite some time). See http://bugs.python.org/issue3139 (reported by Amaury) Regards Antoine. From martin at v.loewis.de Sun Jul 6 00:28:43 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 06 Jul 2008 00:28:43 +0200 Subject: [Python-Dev] bytearray and array.array are not thread-safe In-Reply-To: <loom.20080705T185033-556@post.gmane.org> References: <loom.20080705T185033-556@post.gmane.org> Message-ID: <486FF59B.6090609@v.loewis.de> > Short story: bytearray and array.array by construction allow user code to > reallocate their internal memory buffer. But a raw pointer to the said buffer > can also be obtained by another thread, and used after releasing the GIL (for > CPU-intensive operations like compression). As a consequence, the interpreter > crashes. > > Was it envisioned? I guess this wasn't considered. For t#, there is a comment from Travis that it really shouldn't release the buffer yet, but it does, anyway. I propose that new codes s*, t*, w* are added, and that s#,t#,w# refuses objects which implement a releasebuffer procedure (alternatively, s# etc might be removed altogether right away). Users of s* then need to pass in a Py_Buffer view pointer that gets filled, and need to explicitly release the buffer. For convenience, it might help if the Py_buffer structure includes a borrowed PyObject* to the underlying object, along with a PyBuffer_Release procedure/macro. Regards, Martin From grig.gheorghiu at gmail.com Sun Jul 6 03:02:31 2008 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Sat, 5 Jul 2008 18:02:31 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> On Thu, Jun 26, 2008 at 8:18 AM, <glyph at divmod.com> wrote: > Today on planetpython.org, Doug Hellman announced the June issue of Python > magazine. The cover story this month is about Pybots, "the fantastic > automation system that has been put in place to make sure new releases of > Python software are as robust and stable as possible". > > Last week, there was a "beta" release of Python which, according to the > community buildbots, cannot run any existing python software. Normally I'd > be complaining here about Twisted, but in fact Twisted is doing relatively > well right now; only 80 failing tests. Django apparently cannot even be > imported. > > The community buildbots have been in a broken state for months now[1]. I've > been restraining myself from whinging about this, but now that it's getting > close to release, it's time to get these into shape, or it's time to get rid > of them. Hi all, Sorry for not replying sooner, I was on vacation when this thread started and I only got back in town yesterday. To bring my $0.02 to this discussion: the Pybots 'community buildbots' turned out largely to be a failure. Why? Because there was never really a 'community' around it, especially a community of project leaders who would be interested in the state of their projects' tests. All the machines donated for the Pybots farm belong to people who just happen to be interested in given projects, but are not really the leaders of those projects. The only project who constantly stayed on top of the buildbot status was Twisted, represented by JP Calderone (although even there the tests were running on my machine, and not on a machine contributed by the Twisted folks.) I still haven't given up, and I hope this thread will spur project leaders into donating time, or resources, to the Pybots project. It has been my bitter observation about the Open Source world that people just LOVE to get stuff for free. As soon as you mention more involvement from them in the form of time, money, hardware resources, etc., the same brave proponents of cool things to be done are nowhere to be found. To come back to this thread, I don't think it's reasonable to expect the Python core developers to be that interested in the status of the community buildbots. It is again up to the project leaders to step up to the plate, donate machines to Pybots, and stay on top of any breakages that result from Python core checkins. It seems to me that the Python core developers have always responded promptly and favorably to reports of breakages coming from the Pybots farm. Grig From josiah.carlson at gmail.com Sun Jul 6 03:52:26 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sat, 5 Jul 2008 18:52:26 -0700 Subject: [Python-Dev] Packing and unpacking integers Message-ID: <e6511dbf0807051852i34226efpf3de82ce1ff2599a@mail.gmail.com> A few years ago (yes, it's been that long), I proposed adding a new format code to struct that would pack integers as strings, similar to the 's' format code. In particular, struct.pack('>60G', v) would be a 60-byte big-endian unsigned integer as a string. The feature request is http://bugs.python.org/issue1023290 . Shortly thereafter, it was decided that it wouldn't become a struct format code, but instead would find itself as part of binhex. Raymond Hettinger was supposed to write the function a couple years ago for, I believe, Python 2.4 . It never happened. It still hasn't happened for Python 2.5 or 2.6 . I believe there is still a need for packing integers as strings and unpacking strings as integers, more specifically, offering to Python an interface to _PyLong_FromByteArray() and _PyLong_AsByteArray(). I would be happy to write the functionality and unittests this coming week for 2.6 and 3.0 if I get the ok. If not, I can write it for 2.7 and 3.1 . - Josiah From solipsis at pitrou.net Sun Jul 6 13:17:23 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 06 Jul 2008 13:17:23 +0200 Subject: [Python-Dev] bytearray and array.array are not thread-safe In-Reply-To: <486FF59B.6090609@v.loewis.de> References: <loom.20080705T185033-556@post.gmane.org> <486FF59B.6090609@v.loewis.de> Message-ID: <1215343043.5983.7.camel@fsol> Le dimanche 06 juillet 2008 ? 00:28 +0200, "Martin v. L?wis" a ?crit : > I propose that new codes s*, t*, w* are added, and that s#,t#,w# refuses > objects which implement a releasebuffer procedure (alternatively, s# etc > might be removed altogether right away). Users of s* then need to pass > in a Py_Buffer view pointer that gets filled, and need to explicitly > release the buffer. For convenience, it might help if the Py_buffer > structure includes a borrowed PyObject* to the underlying object, along > with a PyBuffer_Release procedure/macro. Why a borrowed reference rather than a new one? It could be decref'ed as part as the proposed PyBuffer_Release procedure. Overall it sounds like a clean resolution of the problem. Regards Antoine. From glyph at divmod.com Sun Jul 6 17:46:30 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Sun, 06 Jul 2008 15:46:30 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> Message-ID: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> On 01:02 am, grig.gheorghiu at gmail.com wrote: >To bring my $0.02 to this discussion: the Pybots 'community buildbots' >turned out largely to be a failure. Let's not say it's a failure. Let's instead say that it hasn't yet become a success :-). >I still haven't given up, and I hope this thread will spur project >leaders into donating time, or resources, to the Pybots project. It >has been my bitter observation about the Open Source world that people >just LOVE to get stuff for free. As soon as you mention more >involvement from them in the form of time, money, hardware resources, >etc., the same brave proponents of cool things to be done are nowhere >to be found. I think this list is the wrong place to go to reach the people who need to be reached. It's python core developers and other people already involved in and aware of core development. That said I'm not sure what the *right* place is; I think your blog is syndicated on the unofficial planet python, so maybe that's a good place to start. Sadly, the right thing to do in terms of drumming up support is to get someone interested in PR and have them go to each project individually, but that might be more effort than setting up the buildbots themselves, at least initially... However, let's say that this were tremendously successful, and lots of people start paying attention. I think pybots.org needs to be updated to say exactly what a participant interested in python testing needs to do, beyond "here's how you set up a buildbot" (a page that is actually a daunting-looking blog post which admits it may be somewhat outdated), because setting up a buildbot might not be the only thing that the project needs. It's one thing to tell people that they need to be helping out (and I'm sure you're right) but it's much more useful to get the message out that "we really need people to do X, Y, and Z". One thing I will definitely commit to is that if you make a "cry for help" page, I'll blog about it to drive attention to it, and I'll encourage the other, perhaps better-read Python bloggers I know to do so as well. My personal interest at the moment is to get all of the irrelevant red off of the community builders page. Whether or not you believe in an XP "green bar" philosophy, the large number of spurious failures is distracting. Who is it that is capable of making appropriate changes? Is there something I could do to help with that? Note that I'm committing to say that I can do *that*, but, at least you could shut me up by making it my fault ;-). (I'd also like to improve the labels of the build slaves. What exactly is "x86 Red Hat 9 trunk" testing? Trunk of what? What project?) It would be good to remove the perception that it's somebody else's problem as much as possible. Right now, all these dead buildbots suggest to the various communities, "oh, I guess that guy who runs that buildbot needs to fix it". The dead bots should just be killed off, and their projects removed from the list, so that if someone wants to get involved and set up a bot for lxml, they're not put off of it by the fact that it might be rude to the guy who is currently (allegedly) running it. From martin at v.loewis.de Sun Jul 6 19:25:34 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 06 Jul 2008 19:25:34 +0200 Subject: [Python-Dev] bytearray and array.array are not thread-safe In-Reply-To: <1215343043.5983.7.camel@fsol> References: <loom.20080705T185033-556@post.gmane.org> <486FF59B.6090609@v.loewis.de> <1215343043.5983.7.camel@fsol> Message-ID: <4871000E.2060800@v.loewis.de> > Why a borrowed reference rather than a new one? It could be decref'ed as > part as the proposed PyBuffer_Release procedure. The question is a) whether a Py_Buffer remains valid even if the object goes away. That seems not to be the case, i.e. the caller of getbuffer needs to hold onto the object, anyway. b) whether it would still be correct to call releasebuffer explicitly. Of course, as getbuffer would have to fill the object into the view, releasebuffer could also DECREF the included object. Alternatively, there could be a pair of functions PyBuffer_Get and PyBuffer_Release, which would fill the object into the view itself. So I withdraw issue b; the real question remains whether it is desired that a buffer will remain alive as long as there is a view to it. That is a question for the buffer experts to answer; it may also have impacts on cyclic garbage collection (as inclusion of a view into an object will mean that the tp_traverse function must also Py_VISIT the embedded object). > Overall it sounds like a clean resolution of the problem. Unfortunately, it's also a significant change at this point. I personally won't have time to provide a patch, but I think a patch is needed before the last beta. IOW, the issue should become a release blocker. Regards, Martin From stephen at xemacs.org Sun Jul 6 19:40:14 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 07 Jul 2008 02:40:14 +0900 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> Message-ID: <87y74enc75.fsf@uwakimon.sk.tsukuba.ac.jp> glyph at divmod.com writes: > On 01:02 am, grig.gheorghiu at gmail.com wrote: > >To bring my $0.02 to this discussion: the Pybots 'community buildbots' > >turned out largely to be a failure. > > Let's not say it's a failure. Let's instead say that it hasn't yet > become a success :-). +1 > >I still haven't given up, and I hope this thread will spur project > >leaders into donating time, or resources, to the Pybots project. > I think this list is the wrong place to go to reach the people who need > to be reached. It's python core developers and other people already > involved in and aware of core development. That said I'm not sure what > the *right* place is; I think your blog is syndicated on the unofficial > planet python, so maybe that's a good place to start. Sadly, the right > thing to do in terms of drumming up support is to get someone interested > in PR and have them go to each project individually, but that might be > more effort than setting up the buildbots themselves, at least > initially... Exactly, and that's why nobody should be "bitter" about it. The problem is that the while overall the effort and rewards look to be balanced in favor of the rewards, the startup costs for individuals are quite high. I think this *is* the place to start, though. The project leaders "should" be, and probably generally are, "here". They have the strongest interest in any individual 'bot, while Guido is quite correct in saying python-dev can't afford to have strong interest in all the bots. > However, let's say that this were tremendously successful, and lots of > people start paying attention. I think pybots.org needs to be updated > to say exactly what a participant interested in python testing needs to > do, beyond "here's how you set up a buildbot" (a page that is actually a > daunting-looking blog post which admits it may be somewhat outdated), > because setting up a buildbot might not be the only thing that the > project needs. It's one thing to tell people that they need to be > helping out (and I'm sure you're right) but it's much more useful to get > the message out that "we really need people to do X, Y, and Z". One > thing I will definitely commit to is that if you make a "cry for help" > page, I'll blog about it to drive attention to it, and I'll encourage > the other, perhaps better-read Python bloggers I know to do so as > well. Two suggestions in this vein: First, I think it's established that some but not all "red community bots" *are* of interest to Python core development. While I'm not aware of the technical details, I estimate that triaging the community 'bot failures is probably similar to reviewing bugs in the Python tracker. Perhaps Martin van Loewis and others who have offered the 5-for-1 review deal would be willing to extend the definition of "review" to include initial bug reports based on a red community bot (ie, you review the community 'bot failure and decide it is something that should concern Python core development). Perhaps that's not appropriate, but a similar system could be set up. Second, something intermediate between the occasional half-hour of triaging bugs and a full-blown PR campaign at the projects would be documenting the criteria for reporting a failure on a community 'bot to the Python tracker as a bug, etc. This would also serve as a basis for talking to project lurker who might have the odd half-hour to do some "red bot" triaging. (By criteria I mean the kinds of things that Python core considers necessary breakage in new versions that downstream must address in their own code, vs. regressions in a x.y.z patchlevel, etc. The kind of thing that glyph and Guido were discussing earlier.) > It would be good to remove the perception that it's somebody else's > problem as much as possible. To the extent that a 'bot is running prerelease project against prerelease Python, this is probably not very doable. If Python is stable and the project version is prerelease, it's the project's bug until proven otherwise, and vice versa. If both are stable, again some expertise is probably needed for triage. I guess that means that one important task is to classfy the bots in a two-by-two matrix according to stability of project and Python respectively. From martin at v.loewis.de Sun Jul 6 19:29:34 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 06 Jul 2008 19:29:34 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> Message-ID: <487100FE.508@v.loewis.de> > (I'd also like to improve the labels of the build slaves. What exactly > is "x86 Red Hat 9 trunk" testing? Trunk of what? What project?) It seems like you would like to edit the master configuration file. That can be arranged fairly easily. Regards, Martin From martin at v.loewis.de Sun Jul 6 19:34:06 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 06 Jul 2008 19:34:06 +0200 Subject: [Python-Dev] Packing and unpacking integers In-Reply-To: <e6511dbf0807051852i34226efpf3de82ce1ff2599a@mail.gmail.com> References: <e6511dbf0807051852i34226efpf3de82ce1ff2599a@mail.gmail.com> Message-ID: <4871020E.5070802@v.loewis.de> > I believe there is still a need for packing integers as strings and > unpacking strings as integers, more specifically, offering to Python > an interface to _PyLong_FromByteArray() and _PyLong_AsByteArray(). I > would be happy to write the functionality and unittests this coming > week for 2.6 and 3.0 if I get the ok. If not, I can write it for 2.7 > and 3.1 . I think it needs to be deferred to the next releases, given that the beta release already happened. If you have any spare time, please look into some of the real serious, release-blocking bug reports. Regards, Martin From grig.gheorghiu at gmail.com Sun Jul 6 23:09:37 2008 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Sun, 6 Jul 2008 14:09:37 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> Message-ID: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> On Sun, Jul 6, 2008 at 8:46 AM, <glyph at divmod.com> wrote: > > However, let's say that this were tremendously successful, and lots of > people start paying attention. I think pybots.org needs to be updated to > say exactly what a participant interested in python testing needs to do, > beyond "here's how you set up a buildbot" (a page that is actually a > daunting-looking blog post which admits it may be somewhat outdated), > because setting up a buildbot might not be the only thing that the project > needs. It's one thing to tell people that they need to be helping out (and > I'm sure you're right) but it's much more useful to get the message out that > "we really need people to do X, Y, and Z". One thing I will definitely > commit to is that if you make a "cry for help" page, I'll blog about it to > drive attention to it, and I'll encourage the other, perhaps better-read > Python bloggers I know to do so as well. I have posted 'cries for help' repeatedly on my blog, with generally little success. See http://agiletesting.blogspot.com/search?q=pybots . But I will post again. If you can include here a paragraph of what your vision of the 'X, Y and Z' above is, that'd help too. I think I've been pretty clear about the benefits that the Pybots farm can bring to a given project, so all project leaders on this list should be aware of them IMO. If not, I'd be happy to rehash them. But the home page of pybots.org is pretty self-explanatory I think. > > My personal interest at the moment is to get all of the irrelevant red off > of the community builders page. Whether or not you believe in an XP "green > bar" philosophy, the large number of spurious failures is distracting. Who > is it that is capable of making appropriate changes? Is there something I > could do to help with that? Note that I'm committing to say that I can do > *that*, but, at least you could shut me up by making it my fault ;-). > I'll send a message to the pybots mailing list asking people whose buildbots are turned off if they're still interested in running them. Negative or no answers will mean we can remove them from the farm. > (I'd also like to improve the labels of the build slaves. What exactly is > "x86 Red Hat 9 trunk" testing? Trunk of what? What project?) > It's not only a question of changing a static label in this case. A given buildslave can run the tests for multiple projects, in which case it becomes tricky to change the label on the fly accordingly. As an aside, the slave you mention was running on my machine, and I used it to run the Twisted tests, but I shut it down a while ago because the buildbot process was taking too many resources. If the Twisted project can donate a machine, I'd be happy to include it in the Pybots farm ASAP. > It would be good to remove the perception that it's somebody else's problem > as much as possible. Right now, all these dead buildbots suggest to the > various communities, "oh, I guess that guy who runs that buildbot needs to > fix it". The dead bots should just be killed off, and their projects > removed from the list, so that if someone wants to get involved and set up a > bot for lxml, they're not put off of it by the fact that it might be rude to > the guy who is currently (allegedly) running it. As I said, I'll see what the current owners have to say, and then I'll report back to this list. Thanks for offering your help! Grig From victor.stinner at haypocalc.com Mon Jul 7 01:11:52 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 7 Jul 2008 01:11:52 +0200 Subject: [Python-Dev] Play with fuzzing Message-ID: <200807070111.52959.victor.stinner@haypocalc.com> Hi, I wrote a fuzzing "framework" called Fusil and this week I wrote a fuzzer for Python. The idea is quite simple: for a module, - list all functions, classes and class methods - call a function with random arguments (of random types) - instanciate a class with random arguments - if the class is created correctly, call methods with random arguments Example: --------------------- 8< ----------------------------------- print "Call 39/40: linuxaudiodev.open()" try: linuxaudiodev.open( # argument 1/2 u"\u62C0\uFBD7\uB46A\u55E0\uFB7B\uD392\u7CEE", # argument 2/2 52.682, ) except Exception, err: print >>stderr, "ERROR: %s" % err --------------------- 8< ----------------------------------- I tried it on CPython 2.5 and then on CPython trunk (future 2.6). I found some bugs, see last bug entries in Python bugtracker. Just an example: http://bugs.python.org/issue3304 -> invalid call to PyMem_Free() in fileio_init() Most bugs crash with a segmentation fault, abort or a denial of service. If you would like to try my fuzzer, use: (1) svn co http://fusil.hachoir.org/svn/trunk fusil (2) cd fusil (3) ./run_fusil.sh -p projects/python.py --fast --remove ALL The option --fast goes faster, --remove does remove session directory even if Python generated some files, and "ALL" test all modules. FUSIL IS NOT SAFE! So run it under a different user using to avoid dangerous call to os.unlink(). The module list is hardcoded: it's the list of CPython modules written in C. More informations about Fusil: http://fusil.hachoir.org/trac -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From brett at python.org Mon Jul 7 01:33:14 2008 From: brett at python.org (Brett Cannon) Date: Sun, 6 Jul 2008 16:33:14 -0700 Subject: [Python-Dev] Play with fuzzing In-Reply-To: <200807070111.52959.victor.stinner@haypocalc.com> References: <200807070111.52959.victor.stinner@haypocalc.com> Message-ID: <bbaeab100807061633x1c3f794and24234b39916bd4b@mail.gmail.com> On Sun, Jul 6, 2008 at 4:11 PM, Victor Stinner <victor.stinner at haypocalc.com> wrote: > Hi, > > I wrote a fuzzing "framework" called Fusil and this week I wrote a fuzzer for > Python. The idea is quite simple: for a module, > - list all functions, classes and class methods > - call a function with random arguments (of random types) > - instanciate a class with random arguments > - if the class is created correctly, call methods with random arguments > > Example: > --------------------- 8< ----------------------------------- > print "Call 39/40: linuxaudiodev.open()" > try: > linuxaudiodev.open( > # argument 1/2 > u"\u62C0\uFBD7\uB46A\u55E0\uFB7B\uD392\u7CEE", > # argument 2/2 > 52.682, > ) > except Exception, err: > print >>stderr, "ERROR: %s" % err > --------------------- 8< ----------------------------------- > > I tried it on CPython 2.5 and then on CPython trunk (future 2.6). I found some > bugs, see last bug entries in Python bugtracker. Just an example: > > http://bugs.python.org/issue3304 > -> invalid call to PyMem_Free() in fileio_init() > You can use http://bugs.python.org/issue?%40search_text=&title=&%40columns=title&id=&%40columns=id&creation=&creator=haypo&activity=2008-07-06&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&%40pagesize=50&%40startwith=0&%40queryname=&%40old-queryname=&%40action=search to see all of the bugs Victor has filed today (looks like eight). -Brett From martin at v.loewis.de Mon Jul 7 06:37:10 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Jul 2008 06:37:10 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> Message-ID: <48719D76.7040806@v.loewis.de> > It's not only a question of changing a static label in this case. A > given buildslave can run the tests for multiple projects, in which > case it becomes tricky to change the label on the fly accordingly. I think you could set up different builders for a single slave in that case (use a slave lock to make them run sequentially). > As > an aside, the slave you mention was running on my machine, and I used > it to run the Twisted tests, but I shut it down a while ago because > the buildbot process was taking too many resources. If the Twisted > project can donate a machine, I'd be happy to include it in the Pybots > farm ASAP. Please talk to Trent Nelson. He has a Windows machine that he donated precisely for that kind of activity. Regards, Martin From martin at v.loewis.de Mon Jul 7 06:38:48 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 07 Jul 2008 06:38:48 +0200 Subject: [Python-Dev] Play with fuzzing In-Reply-To: <200807070111.52959.victor.stinner@haypocalc.com> References: <200807070111.52959.victor.stinner@haypocalc.com> Message-ID: <48719DD8.4070807@v.loewis.de> > I wrote a fuzzing "framework" called Fusil and this week I wrote a fuzzer for > Python. The idea is quite simple: for a module, > - list all functions, classes and class methods > - call a function with random arguments (of random types) > - instanciate a class with random arguments > - if the class is created correctly, call methods with random arguments I was already wondering how you found out all these things. It's quite amazing! Thanks, Martin From solipsis at pitrou.net Mon Jul 7 13:57:57 2008 From: solipsis at pitrou.net (Antoine) Date: Mon, 7 Jul 2008 13:57:57 +0200 (CEST) Subject: [Python-Dev] bytearray and array.array are not thread-safe In-Reply-To: <4871000E.2060800@v.loewis.de> References: <loom.20080705T185033-556@post.gmane.org> <486FF59B.6090609@v.loewis.de> <1215343043.5983.7.camel@fsol> <4871000E.2060800@v.loewis.de> Message-ID: <2463911e698f87e837b4296cb13810ea.squirrel@webmail.nerim.net> > Unfortunately, it's also a significant change at this point. I > personally won't have time to provide a patch, but I think a patch > is needed before the last beta. IOW, the issue should become a > release blocker. Agreed. Unfortunately I don't have much time to write a patch either. Perhaps in one or two weeks, but it would be better if someone beats me to it. Regards Antoine. From solipsis at pitrou.net Mon Jul 7 14:39:33 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 7 Jul 2008 12:39:33 +0000 (UTC) Subject: [Python-Dev] buildbots References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> Message-ID: <loom.20080707T123338-370@post.gmane.org> Hello, As someone who could (perhaps) (potentially) provide a buildbot machine, there are several questions which need answering before I take a decision: - are more buildbots needed and if so, which kinds of platforms/architectures? - for which software? Python itself? third-party apps and libraries? - how resource-consuming is it? CPU? memory? disk space? can it run along other services fine or does it need the whole machine for itself? - how time-consuming is it (in terms of human work)? I may spend a bit of time at the start to set it up but I'd like it to it run quite flawlessly afterward. I'm really not a sysadmin at heart... I suppose other interested people could ask themselves the same questions... Just my 2 cents. Antoine. From grig.gheorghiu at gmail.com Mon Jul 7 20:49:05 2008 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Mon, 7 Jul 2008 11:49:05 -0700 Subject: [Python-Dev] buildbots In-Reply-To: <loom.20080707T123338-370@post.gmane.org> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <loom.20080707T123338-370@post.gmane.org> Message-ID: <3f09d5a00807071149r42b7b7ddl4eb5a91d21e662b0@mail.gmail.com> On Mon, Jul 7, 2008 at 5:39 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > > > - are more buildbots needed and if so, which kinds of platforms/architectures? I can't really answer that question for the python code buildbot farm, but for the Pybots community project, the platforms we currently have are in a table on this page: http://pybots.org/ If you are able to offer something that's not on the list, that'd be good. But any help at all is appreciated. I believe Windows has traditionally been under-represented in all buildbot farms, and it's likely to stay that way... > - for which software? Python itself? third-party apps and libraries? For Pybots, we're testing third-party apps and libraries against changes made to Python core. If you're interested in a 3rd party project, and you're willing to stay on top of that project's buildbot status, and notify both the project leaders and the Python core devs whenever you notice an ugly breakage -- then you're exactly the kind of guy we need on the Pybots project :-) > - how resource-consuming is it? CPU? memory? disk space? can it run along other > services fine or does it need the whole machine for itself? In my experience, buildbot runs fine on newer hardware. It does consume CPU, so if you have a slow machine, it might start impacting your other processes. > - how time-consuming is it (in terms of human work)? I may spend a bit of time > at the start to set it up but I'd like it to it run quite flawlessly afterward. > I'm really not a sysadmin at heart... The initial learning curve can be a bit steep, but I'm here to help. Once you add your buildslave to the buildbot farm, things should run fairly smoothly. You will get notified via email / RSS about breakages, and then you'll have to invest the time to see what kind of breakage it is, and to notify the interested parties. > > I suppose other interested people could ask themselves the same questions... > > Just my 2 cents. > > Antoine. Thanks for the questions, they really help IMO. I also hope the answers helped. Grig From grig.gheorghiu at gmail.com Mon Jul 7 21:54:23 2008 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Mon, 7 Jul 2008 12:54:23 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> Message-ID: <3f09d5a00807071254x5b5ee091ye24781a1dd3afa07@mail.gmail.com> On Sun, Jul 6, 2008 at 2:09 PM, Grig Gheorghiu <grig.gheorghiu at gmail.com> wrote: > I'll send a message to the pybots mailing list asking people whose > buildbots are turned off if they're still interested in running them. > Negative or no answers will mean we can remove them from the farm. > OK, I posted a message to the pybots mailing list and I removed 2 slaves. Out of the 6 remaining, 4 are currently active, and one will hopefully soon be active starting next week. This leaves just one unanswered for so far. I also got an email from another person volunteering a buildslave, so we'll soon have 7 machines. As I said, if anybody else wants to participate in the Pybots project, please let me know! I'll also post a blog entry on this soon. Grig From priyarp.tech at gmail.com Mon Jul 7 22:48:45 2008 From: priyarp.tech at gmail.com (Pree Raj) Date: Mon, 7 Jul 2008 13:48:45 -0700 Subject: [Python-Dev] __module__ not found on ported Python Message-ID: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com> Hi, I am trying to port Python to ThreadX. I have managed to get the prompt. However when I try "import sys" or any built in module I get an error __import__ not found. initmain() in the Initialization code is commented out at present because of some errors. Could it be because of this ? Also, I would like to know which are the MUST HAVE built in modules to be included for normal working of my ported version of Python. Thanks, Priya -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080707/e21c2cce/attachment.htm> From brett at python.org Mon Jul 7 23:15:32 2008 From: brett at python.org (Brett Cannon) Date: Mon, 7 Jul 2008 14:15:32 -0700 Subject: [Python-Dev] __module__ not found on ported Python In-Reply-To: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com> References: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com> Message-ID: <bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com> On Mon, Jul 7, 2008 at 1:48 PM, Pree Raj <priyarp.tech at gmail.com> wrote: [SNIP] > Also, I would like to know which are the MUST HAVE built in modules to be > included for normal working of my ported version of Python. You can look at sys.builtin_module_names to see what CPython compiles in. Otherwise you just have to go based on what error messages say. =) -Brett From priyarp.tech at gmail.com Tue Jul 8 01:39:39 2008 From: priyarp.tech at gmail.com (Pree Raj) Date: Mon, 7 Jul 2008 16:39:39 -0700 Subject: [Python-Dev] __module__ not found on ported Python In-Reply-To: <bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com> References: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com> <bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com> Message-ID: <8bb4faa80807071639l7879595ejb2a49affa9536442@mail.gmail.com> Thanks Brett. I have been able to do initmain() now. However, if I do "import sys" from the python prompt I still get ImportError: __import__ not found I am not sure where the initialization is going wrong for this error to show up. Can someone please help. On Mon, Jul 7, 2008 at 2:15 PM, Brett Cannon <brett at python.org> wrote: > On Mon, Jul 7, 2008 at 1:48 PM, Pree Raj <priyarp.tech at gmail.com> wrote: > [SNIP] > > Also, I would like to know which are the MUST HAVE built in modules to be > > included for normal working of my ported version of Python. > > You can look at sys.builtin_module_names to see what CPython compiles > in. Otherwise you just have to go based on what error messages say. =) > > -Brett > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080707/b325d8b0/attachment.htm> From martin at v.loewis.de Tue Jul 8 07:32:29 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Jul 2008 07:32:29 +0200 Subject: [Python-Dev] __module__ not found on ported Python In-Reply-To: <8bb4faa80807071639l7879595ejb2a49affa9536442@mail.gmail.com> References: <8bb4faa80807071348u3c296026w303f6fe433bedf67@mail.gmail.com> <bbaeab100807071415p638b6138haefe8d0b890a23df@mail.gmail.com> <8bb4faa80807071639l7879595ejb2a49affa9536442@mail.gmail.com> Message-ID: <4872FBED.50402@v.loewis.de> > ImportError: __import__ not found > I am not sure where the initialization is going wrong for this error to > show up. > Can someone please help. This isn't really the right list to ask for help, at least without studying some source code prior to posting. The specific error message is produced in ceval.c, IMPORT_NAME. Use debugging technologies to trace through the code to find out what went wrong. Regards, Martin From solipsis at pitrou.net Tue Jul 8 11:33:02 2008 From: solipsis at pitrou.net (Antoine) Date: Tue, 8 Jul 2008 11:33:02 +0200 (CEST) Subject: [Python-Dev] buildbots In-Reply-To: <3f09d5a00807071149r42b7b7ddl4eb5a91d21e662b0@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <loom.20080707T123338-370@post.gmane.org> <3f09d5a00807071149r42b7b7ddl4eb5a91d21e662b0@mail.gmail.com> Message-ID: <3baff3efe0bbb965dd60c3b8a996e9f3.squirrel@webmail.nerim.net> Hi and thanks for your answers, > If you are able to offer something that's not on the list, that'd be > good. But any help at all is appreciated. > > I believe Windows has traditionally been under-represented in all > buildbot farms, and it's likely to stay that way... Well what I could provide is a 32-bit x86 Debian stable. Rather common I fear... > For Pybots, we're testing third-party apps and libraries against > changes made to Python core. If you're interested in a 3rd party > project, and you're willing to stay on top of that project's buildbot > status, and notify both the project leaders and the Python core devs > whenever you notice an ugly breakage Not interested /enough/ I think... by your description it sounds the job should really be done by a core developer of each of those packages (even if the machine is donated by someone else). What I could be interested in is to provide a buildbot for Python itself, but I don't know if that's needed right now. Especially for such a common platform as a x86 Debian. Regards Antoine. From jeremy at alum.mit.edu Tue Jul 8 14:49:02 2008 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 8 Jul 2008 08:49:02 -0400 Subject: [Python-Dev] [Python-checkins] buildbot failure in sparc Debian 3.0 In-Reply-To: <20080702202345.DAD571E400A@bag.python.org> References: <20080702202345.DAD571E400A@bag.python.org> Message-ID: <e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com> Does anyone have a clue about why this test fails only on this platform? The test is question is verifying that URLError gets raised. From the traceback, it appears that there is an uncaught exception (URLError) but it fails in an assertRaises() check for URLError. That doesn't make much sense unless the variable URLError refers to different objects, but both modules use "from urllib.error import URLError". And, of course, the test works fine on other platforms. Jeremy On Wed, Jul 2, 2008 at 4:23 PM, <buildbot at python.org> wrote: > The Buildbot has detected a new failure of sparc Debian 3.0. > Full details are available at: > http://www.python.org/dev/buildbot/all/sparc%20Debian%203.0/builds/330 > > Buildbot URL: http://www.python.org/dev/buildbot/all/ > > Buildslave for this Build: klose-debian-sparc > > Build Reason: > Build Source Stamp: [branch branches/py3k] HEAD > Blamelist: benjamin.peterson > > BUILD FAILED: failed test > > Excerpt from the test logfile: > 1 test failed: > test_urllib2 > > ====================================================================== > ERROR: test_badly_named_methods (test.test_urllib2.OpenerDirectorTests) > ---------------------------------------------------------------------- > > Traceback (most recent call last): > File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/test/test_urllib2.py", line 408, in test_badly_named_methods > self.assertRaises(URLError, o.open, scheme+"://example.com/") > File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/unittest.py", line 311, in failUnlessRaises > callableObj(*args, **kwargs) > File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 356, in open > response = self._open(req, data) > File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 379, in _open > 'unknown_open', req) > File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 334, in _call_chain > result = func(*args) > File "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py", line 1102, in unknown_open > raise URLError('unknown url type: %s' % type) > urllib.error.URLError: <urlopen error unknown url type: do> > > make: *** [buildbottest] Error 1 > > sincerely, > -The Buildbot > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From martin at v.loewis.de Tue Jul 8 21:15:08 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 08 Jul 2008 21:15:08 +0200 Subject: [Python-Dev] [Python-checkins] buildbot failure in sparc Debian 3.0 In-Reply-To: <e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com> References: <20080702202345.DAD571E400A@bag.python.org> <e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com> Message-ID: <4873BCBC.9040104@v.loewis.de> Jeremy Hylton wrote: > Does anyone have a clue about why this test fails only on this > platform? The test is question is verifying that URLError gets > raised. From the traceback, it appears that there is an uncaught > exception (URLError) but it fails in an assertRaises() check for > URLError. That doesn't make much sense unless the variable URLError > refers to different objects, but both modules use "from urllib.error > import URLError". And, of course, the test works fine on other > platforms. It might be a transient error, and a complete cleanup of the tree might fix it. To do so, build a non-existent branch through the web ui, then build the original branch again; this will cause a fresh checkout. If the error then persists, my guess it's some kind of compiler issue, which can be investigated only with access to the machine. Regards, Martin From glyph at divmod.com Tue Jul 8 21:30:04 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Tue, 08 Jul 2008 19:30:04 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <487100FE.508@v.loewis.de> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> <487100FE.508@v.loewis.de> Message-ID: <20080708193004.25821.1534535725.divmod.xquotient.12490@joule.divmod.com> On 6 Jul, 05:29 pm, martin at v.loewis.de wrote: >>(I'd also like to improve the labels of the build slaves. What >>exactly >>is "x86 Red Hat 9 trunk" testing? Trunk of what? What project?) > >It seems like you would like to edit the master configuration file. >That can be arranged fairly easily. How shall we arrange it, then? :) Whoever is interested, I've got a recent SSH key on https://launchpad.net/~glyph/+sshkeys From glyph at divmod.com Tue Jul 8 21:56:56 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Tue, 08 Jul 2008 19:56:56 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> Message-ID: <20080708195656.25821.1158092892.divmod.xquotient.12536@joule.divmod.com> On 6 Jul, 09:09 pm, grig.gheorghiu at gmail.com wrote: >On Sun, Jul 6, 2008 at 8:46 AM, <glyph at divmod.com> wrote: >> >>It's one thing to tell people that they need to be helping out (and >>I'm sure you're right) but it's much more useful to get the message >>out that >>"we really need people to do X, Y, and Z". >I have posted 'cries for help' repeatedly on my blog, with generally >little success. See http://agiletesting.blogspot.com/search?q=pybots . >But I will post again. If you can include here a paragraph of what >your vision of the 'X, Y and Z' above is, that'd help too It looks to me like the main thing that Pybots needs is help with maintenance. Getting someone to set up an individual buildbot is one thing, but keeping it working is the important bit and it looks like people are not doing that. The project would also be greatly aided by having dedicated people diagnose errors, report bugs against Python core if they're real and report bugs against Pybots if they're spurious. It would be good to have this effort be centralized and directed because it would otherwise be too easy to file duplicate bug reports, or to assume "oh, this has been failing for months, someone must have filed a bug already". >It's not only a question of changing a static label in this case. A >given buildslave can run the tests for multiple projects, in which >case it becomes tricky to change the label on the fly accordingly. I'm a bit confused about how the projects being tested changes on the fly... but then, this level of specifics is probably best left to the pybots mailing list. Hopefully sometime soon I'll have the time to add yet another subscription. Thanks for cleaning up the buildbots though! I can see a lot more tests actually running now :). From grig.gheorghiu at gmail.com Tue Jul 8 22:47:28 2008 From: grig.gheorghiu at gmail.com (Grig Gheorghiu) Date: Tue, 8 Jul 2008 13:47:28 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080708195656.25821.1158092892.divmod.xquotient.12536@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <3f09d5a00807051802y20a86ceax2b3d0267ae6b6682@mail.gmail.com> <20080706154630.25821.930077863.divmod.xquotient.12261@joule.divmod.com> <3f09d5a00807061409y42f17e8lba152f2bfe5025e4@mail.gmail.com> <20080708195656.25821.1158092892.divmod.xquotient.12536@joule.divmod.com> Message-ID: <3f09d5a00807081347x631b7ca3w37effe6e75f6fe6@mail.gmail.com> On Tue, Jul 8, 2008 at 12:56 PM, <glyph at divmod.com> wrote: > It looks to me like the main thing that Pybots needs is help with > maintenance. Getting someone to set up an individual buildbot is one thing, > but keeping it working is the important bit and it looks like people are not > doing that. The project would also be greatly aided by having dedicated > people diagnose errors, report bugs against Python core if they're real and > report bugs against Pybots if they're spurious. > > It would be good to have this effort be centralized and directed because it > would otherwise be too easy to file duplicate bug reports, or to assume "oh, > this has been failing for months, someone must have filed a bug already". I agree with all you're saying here. As usual, the devil is in the details. Finding those 'dedicated people' and also people who would act as the central point of contact for bug reports etc. turns out to be very hard in practice. If you have any ideas, I'd be glad to hear them. Grig From tseaver at palladion.com Fri Jul 11 05:08:11 2008 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 10 Jul 2008 23:08:11 -0400 Subject: [Python-Dev] [Python-checkins] buildbot failure in sparc Debian 3.0 In-Reply-To: <4873BCBC.9040104@v.loewis.de> References: <20080702202345.DAD571E400A@bag.python.org> <e8bf7a530807080549g640f2517v7b6ee99eb52a21@mail.gmail.com> <4873BCBC.9040104@v.loewis.de> Message-ID: <4876CE9B.8050402@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: > Jeremy Hylton wrote: >> Does anyone have a clue about why this test fails only on this >> platform? The test is question is verifying that URLError gets >> raised. From the traceback, it appears that there is an uncaught >> exception (URLError) but it fails in an assertRaises() check for >> URLError. That doesn't make much sense unless the variable URLError >> refers to different objects, but both modules use "from urllib.error >> import URLError". And, of course, the test works fine on other >> platforms. > > It might be a transient error, and a complete cleanup of the tree > might fix it. To do so, build a non-existent branch through the web ui, > then build the original branch again; this will cause a fresh checkout. > > If the error then persists, my guess it's some kind of compiler issue, > which can be investigated only with access to the machine. I would also be on the lookout for stale .pyc / .pyo files: I saw a similar failure recently while testing third-party code, and suspected both causes: cleaning out the .pyc files and carefully removing aliased imports eventually got the problem to go away (at which point I could no longer reproduce it at all). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIds6b+gerLs4ltQ4RAmIRAJ4pxs0sWLDrpOAilqV+Mx8vKJzeEQCeLMoX gsFhfjJ4bxwAxgBji7/Jzvw= =bMRD -----END PGP SIGNATURE----- From dickinsm at gmail.com Fri Jul 11 10:37:36 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 11 Jul 2008 09:37:36 +0100 Subject: [Python-Dev] patch review request: float.hex and float.fromhex Message-ID: <5c6f2a5d0807110137l3fffd090mbc2c9c80d1f7b05b@mail.gmail.com> Does anyone have time to review the patch http://bugs.python.org/file10876/hex_float5.patch for issue 3008 (float <-> hexadecimal string conversion): http://bugs.python.org/issue3008 ? It would be really good if this could go in before next week's beta. Guido's approved the idea in principle, but I still need to: - get permission from Barry to check in a new feature this late in the release cycle, and - persuade some other developer to review the patch. I'll gladly 'pay' for a patch review by reviewing one or more of someone else's patches. Mark From python at rcn.com Fri Jul 11 11:06:51 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 11 Jul 2008 12:06:51 +0300 Subject: [Python-Dev] patch review request: float.hex and float.fromhex References: <5c6f2a5d0807110137l3fffd090mbc2c9c80d1f7b05b@mail.gmail.com> Message-ID: <3A2DC3264D7A4DCD9D64416A9441B8A1@RaymondLaptop1> From: "Mark Dickinson" <dickinsm at gmail.com> > Does anyone have time to review the patch > > http://bugs.python.org/file10876/hex_float5.patch > > for issue 3008 (float <-> hexadecimal string conversion): I'll look at it today and tomorrow. Raymond From python at rcn.com Fri Jul 11 13:24:39 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 11 Jul 2008 14:24:39 +0300 Subject: [Python-Dev] Running Py2.6 with the -3 option Message-ID: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> Some effort needs to be made to clear the standard library of -3 warnings. Running -3 on production code usually involves exercising library code so the useful result is obscured by Python complaining about itself. Since that use case involves the users own tests, I don't think the effort needs to be extended to our own unittest suite. But the rest of the library could likely benefit from a good -3 cleanup. Raymond From kirkshorts at hotmail.co.uk Fri Jul 11 13:27:44 2008 From: kirkshorts at hotmail.co.uk (Andy Scott) Date: Fri, 11 Jul 2008 11:27:44 +0000 Subject: [Python-Dev] A proposed solution for Issue 502236: Asyncrhonous exceptions between threads Message-ID: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl> [OK so a newbie post here so many apologies if I am doing this wrong] Quick Synopsis: A child thread in an executing Python program can not safely shutdown the program. The issue URL is: http://bugs.python.org/issue502236 So my proposal is: Example: We have three threads - t0 - Main system thread t1 - Worker thread t2 - Worker thread t1 encounters an issue that means it wants to shut down the application in as safe a way as possible A Solution: 1. Put in place a new function call sys.exitapplication, what this would do is: a. Mark a flag in t0's data structure saying a request to shutdown has been made b. Raise a new exception, SystemShuttingDown, in t1. 2. As the main interpreter executes it checks the "shutting down flag" in the per thread data and follows one of two paths: If it is t0: a. Stops execution of the current code sequence b. Iterates over all extant threads setting the "system shutdown" flag in the per thread data structure. Setting this flag is a one time deal - it can not be undone once set. (And to avoid issues with multiple threads setting it - it can only ever be a single fixed value so setting it multiple times results in the same answer) c. Enters a timed wait loop where it will allow the other threads time to see the signal. It will iterate this loop a set number of times to avoid being blocked on any given thread. d. When all threads have exited, or been forcefully closed, raise the SystemShuttingDown exception If it is not t0: a. Stops execution of the current code sequence b. Raises the exception, SystemShuttingDown. There are problems with this approach, as I see it they are (but please see the assumptions I have made): P1. If the thread is in a tight loop will it see the exception? Or more generally: when should the exception be raised? P2. When should the interpreter check this flag? I think the answer to both of these problems is to: Check the flag, and hence raise the exception, in the following circumstances: - When the interpreter executes a back loop. So this should catch the jump back to the top of a "while True:" loop - Just before the interpreter makes a call to a hooked in non-Python system function, e.g. file I/O, networking &c. Checking at these points should be the minimal required, I think, to ensure that a given thread can not ignore the exception. It may be possible, or even required, to perform the check every time a Python function call is made. I think this approach would then allow for the finally handlers to be called. Assumptions: [Here I must admit to a large amount of ignorance of the internals of Python at this time. So if my assumptions are incorrect I would greatly appreciate being told so :-) Preferably as polite as possible and any code pointers while welcome unless they point to some very esoteric and arcane area would be best kept general so I feel more of a spur to go learn the code base] 1. The Python interpreter has per thread information. 2. The Python interpreter can tell if the system, t0, thread is running. 3. The Python engine has (or can easily obtain) a list of all threads it created. 4. It is possible to raise exceptions as the byte code is executing. I am mailing this out as: A. I have no idea if my thoughts are correct or total un-mitigated rubbish :-) B. I believe the introduction of this proposal (if I am correct) will require a PEP being raised, which aiui requires building community support (which is very fair imo) so this is me trying to do so :-) So apologies if this post has been total spam (but no eggs) or too long - give a little whistle and it will all be OK again. Andy --------------------------------------Brain chemistry is not just for Christmas _________________________________________________________________ Play and win great prizes with Live Search and Kung Fu Panda http://clk.atdmt.com/UKM/go/101719966/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080711/2e69e93e/attachment.htm> From musiccomposition at gmail.com Fri Jul 11 14:19:18 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 11 Jul 2008 07:19:18 -0500 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> Message-ID: <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote: > Some effort needs to be made to clear the standard library of -3 warnings. > Running -3 on production code usually involves exercising library code so > the useful result is obscured by Python complaining about itself. Since > that use case involves the users own tests, I don't think the effort needs > to be extended to our own unittest suite. But the rest of the library could > likely benefit from a good -3 cleanup. Yes, indeed. We should make sure, however, that the changes in the 2.6 libraries are the absolute minimum to get the job done. (I'm trying to pretend like this isn't violating the prohibition on all-inclusive overhauls in the stdlib.) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at holdenweb.com Fri Jul 11 15:02:21 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 11 Jul 2008 09:02:21 -0400 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> Message-ID: <g57ll0$kqn$1@ger.gmane.org> Benjamin Peterson wrote: > On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote: >> Some effort needs to be made to clear the standard library of -3 warnings. >> Running -3 on production code usually involves exercising library code so >> the useful result is obscured by Python complaining about itself. Since >> that use case involves the users own tests, I don't think the effort needs >> to be extended to our own unittest suite. But the rest of the library could >> likely benefit from a good -3 cleanup. > > Yes, indeed. We should make sure, however, that the changes in the 2.6 > libraries are the absolute minimum to get the job done. (I'm trying to > pretend like this isn't violating the prohibition on all-inclusive > overhauls in the stdlib.) > The prohibition is on *gratuitous* changes, basically along the lines of "if it ain't broke, don't fix it". The stdlib is definitely broken if it raises warnings of that kind. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From musiccomposition at gmail.com Fri Jul 11 16:50:21 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 11 Jul 2008 09:50:21 -0500 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <g57ll0$kqn$1@ger.gmane.org> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> Message-ID: <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com> On Fri, Jul 11, 2008 at 8:02 AM, Steve Holden <steve at holdenweb.com> wrote: > Benjamin Peterson wrote: >> >> Yes, indeed. We should make sure, however, that the changes in the 2.6 >> libraries are the absolute minimum to get the job done. (I'm trying to >> pretend like this isn't violating the prohibition on all-inclusive >> overhauls in the stdlib.) >> > The prohibition is on *gratuitous* changes, basically along the lines of "if > it ain't broke, don't fix it". The stdlib is definitely broken if it raises > warnings of that kind. Just because it's massive breakage fixage doesn't mean that it's unlikely to break something else. :) > > regards > Steve > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > Holden Web LLC http://www.holdenweb.com/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at holdenweb.com Fri Jul 11 17:08:41 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 11 Jul 2008 11:08:41 -0400 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com> Message-ID: <48777779.70302@holdenweb.com> Benjamin Peterson wrote: > On Fri, Jul 11, 2008 at 8:02 AM, Steve Holden <steve at holdenweb.com> wrote: >> Benjamin Peterson wrote: >>> Yes, indeed. We should make sure, however, that the changes in the 2.6 >>> libraries are the absolute minimum to get the job done. (I'm trying to >>> pretend like this isn't violating the prohibition on all-inclusive >>> overhauls in the stdlib.) >>> >> The prohibition is on *gratuitous* changes, basically along the lines of "if >> it ain't broke, don't fix it". The stdlib is definitely broken if it raises >> warnings of that kind. > > Just because it's massive breakage fixage doesn't mean that it's > unlikely to break something else. :) > I agree but, contrariwise, just because we are likely to break other things doesn't mean we shouldn't fix the massive breakage. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jul 11 17:08:41 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 11 Jul 2008 11:08:41 -0400 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <1afaf6160807110750m45ccf92ek37a255fef025832b@mail.gmail.com> Message-ID: <48777779.70302@holdenweb.com> Benjamin Peterson wrote: > On Fri, Jul 11, 2008 at 8:02 AM, Steve Holden <steve at holdenweb.com> wrote: >> Benjamin Peterson wrote: >>> Yes, indeed. We should make sure, however, that the changes in the 2.6 >>> libraries are the absolute minimum to get the job done. (I'm trying to >>> pretend like this isn't violating the prohibition on all-inclusive >>> overhauls in the stdlib.) >>> >> The prohibition is on *gratuitous* changes, basically along the lines of "if >> it ain't broke, don't fix it". The stdlib is definitely broken if it raises >> warnings of that kind. > > Just because it's massive breakage fixage doesn't mean that it's > unlikely to break something else. :) > I agree but, contrariwise, just because we are likely to break other things doesn't mean we shouldn't fix the massive breakage. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From status at bugs.python.org Fri Jul 11 18:06:49 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 11 Jul 2008 18:06:49 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080711160649.5EDC0782BC@psf.upfronthosting.co.za> ACTIVITY SUMMARY (07/04/08 - 07/11/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1967 open (+43) / 13199 closed (+17) / 15166 total (+60) Open issues with patches: 621 Average duration of open issues: 700 days. Median duration of open issues: 1604 days. Open Issues Breakdown open 1939 (+42) pending 28 ( +1) Issues Created Or Reopened (60) _______________________________ Delete obsolete 'Unicode' in Py3 str docstrings; related fixes 07/04/08 CLOSED http://bugs.python.org/issue3284 created tjreedy easy Fraction.from_any() 07/04/08 CLOSED http://bugs.python.org/issue3285 created rhettinger patch IDLE opens window too low on Windows 07/04/08 http://bugs.python.org/issue3286 created tjreedy Fraction constructor should raise TypeError instead of Attribute 07/04/08 CLOSED http://bugs.python.org/issue3287 created rhettinger patch float.as_integer_ratio method is not documented 07/05/08 http://bugs.python.org/issue3288 created marketdickinson unnecessary call to time and localtime slows time.mktime 07/05/08 CLOSED http://bugs.python.org/issue3289 created nother_jnelson python-config --cflags includes irrelevant flags 07/05/08 http://bugs.python.org/issue3290 created sacha rlcompleter doesn't work anymore 07/05/08 CLOSED http://bugs.python.org/issue3291 created pitrou patch Position index limit; s.insert(i,x) not same as s[i:i]=[x] 07/05/08 http://bugs.python.org/issue3292 created tjreedy incorrect comments for PyObject_ReleaseBuffer 07/05/08 http://bugs.python.org/issue3293 created pitrou SVN repository contains an incorrect symbolic link 07/05/08 CLOSED http://bugs.python.org/issue3294 created pitrou PyExc_BufferError is declared but nowhere defined 07/05/08 CLOSED http://bugs.python.org/issue3295 created pitrou patch print function not executed in python 3.0 tutorial 07/05/08 CLOSED http://bugs.python.org/issue3296 created segfaulthunter Python interpreter uses Unicode surrogate pairs only before the 07/06/08 http://bugs.python.org/issue3297 created ezio.melotti Multiline string with quotes is not parsed correctly. 07/06/08 CLOSED http://bugs.python.org/issue3298 created Stavros invalid object destruction in re.finditer() 07/06/08 http://bugs.python.org/issue3299 created haypo patch urllib.quote and unquote - Unicode issues 07/06/08 http://bugs.python.org/issue3300 created mgiuca patch DoS when lo is negative in bisect.insort_right() / _left() 07/06/08 CLOSED http://bugs.python.org/issue3301 created haypo patch segfault on gettext(None) 07/06/08 http://bugs.python.org/issue3302 created haypo patch invalid ref count on locale.strcoll() error 07/06/08 http://bugs.python.org/issue3303 created haypo patch invalid call to PyMem_Free() in fileio_init() 07/06/08 http://bugs.python.org/issue3304 created haypo patch, easy Use Py_XDECREF() instead of Py_DECREF() in MultibyteCodec and Mu 07/06/08 http://bugs.python.org/issue3305 created haypo patch audioop.findmax() crashs with negative length 07/06/08 CLOSED http://bugs.python.org/issue3306 created haypo patch invalid check of _bsddb creation failure 07/06/08 http://bugs.python.org/issue3307 created haypo patch MinGW built extensions do not load (specified procedure cannot b 07/07/08 CLOSED http://bugs.python.org/issue3308 created rogerbinns missing lock release in BZ2File_iternext() 07/07/08 http://bugs.python.org/issue3309 created haypo patch, easy Out-of-date example 3.0b1 Tutorial Classes page, 'issubclass' 07/07/08 http://bugs.python.org/issue3310 created alexis.layton block operation on closed socket/pipe for multiprocessing 07/07/08 http://bugs.python.org/issue3311 created haypo patch bugs in _sqlite module 07/07/08 http://bugs.python.org/issue3312 created haypo patch dlopen() error with no error message from dlerror() 07/07/08 http://bugs.python.org/issue3313 created haypo patch urllib.parse doesn't import sys 07/07/08 CLOSED http://bugs.python.org/issue3314 created mgiuca patch abc.rst little error 07/07/08 CLOSED http://bugs.python.org/issue3315 created mishok13 patch Proposal for fix_urllib 07/07/08 http://bugs.python.org/issue3316 created nedds patch duplicate lines in zipfile.py 07/07/08 http://bugs.python.org/issue3317 created amaury.forgeotdarc patch Documentation: timeit: "lower bound" should read "upper bound" 07/08/08 http://bugs.python.org/issue3318 created unutbu pystone.main(10) causes ZeroDivisionError 07/08/08 http://bugs.python.org/issue3319 created mokeefe patch various doc typos 07/08/08 http://bugs.python.org/issue3320 created dsm001 patch _multiprocessing.Connection() doesn't check handle 07/08/08 http://bugs.python.org/issue3321 created haypo patch bugs in scanstring_str() and scanstring_unicode() of _json modul 07/08/08 http://bugs.python.org/issue3322 created haypo Clarify __slots__ behaviour when inheriting 07/09/08 http://bugs.python.org/issue3323 created strangefeatures Broken link in online doc 07/09/08 http://bugs.python.org/issue3324 created ThomasH use of cPickle in multiprocessing 07/09/08 http://bugs.python.org/issue3325 created mishok13 patch py3k shouldn't use -fno-strict-aliasing anymore 07/09/08 http://bugs.python.org/issue3326 created cartman patch NULL member in modules_by_index 07/09/08 http://bugs.python.org/issue3327 created krisvale When PyObject_CallMethod fails, refcount is incorrect 07/09/08 http://bugs.python.org/issue3328 created dominic.lavoie API for setting the memory allocator used by Python 07/09/08 http://bugs.python.org/issue3329 created jlaurila webbrowser module doesn't correctly handle '|' character. 07/09/08 http://bugs.python.org/issue3330 created AdrianP Possible inconsistency in behavior of list comprehensions vs. ge 07/10/08 http://bugs.python.org/issue3331 created carlj DocTest and dict sort. 07/10/08 CLOSED http://bugs.python.org/issue3332 created jedie Need -3 warning for exec statement becoming a function 07/10/08 CLOSED http://bugs.python.org/issue3333 created rhettinger 2to3 looses indentation on import fix 07/10/08 http://bugs.python.org/issue3334 created ctheune subprocess lib - opening same command fails 07/10/08 http://bugs.python.org/issue3335 created gtg944q datetime weekday() function 07/10/08 http://bugs.python.org/issue3336 created ryanboesch Fixer for dbm is failing 07/11/08 CLOSED http://bugs.python.org/issue3337 created brett.cannon cPickle segfault with deep recursion 07/11/08 http://bugs.python.org/issue3338 created esrever_otua dummy_thread LockType.acquire() always returns None, should be T 07/11/08 http://bugs.python.org/issue3339 created toymachine patch optparse print_usage(),.. methods are not documented 07/11/08 http://bugs.python.org/issue3340 created techtonik "Suggest a change" link 07/11/08 http://bugs.python.org/issue3341 created techtonik Tracebacks are not properly indented 07/11/08 http://bugs.python.org/issue3342 created amaury.forgeotdarc patch Py_DisplaySourceLine is not documented 07/11/08 http://bugs.python.org/issue3343 created amaury.forgeotdarc Issues Now Closed (36) ______________________ async_chat.__init__() parameters 221 days http://bugs.python.org/issue1519 josiahcarlson Error when printing an exception containing a Unicode string 99 days http://bugs.python.org/issue2517 ncoghlan patch performance problem in socket._fileobject.read 82 days http://bugs.python.org/issue2632 gregory.p.smith patch shutil.copytree glob-style filtering [patch] 76 days http://bugs.python.org/issue2663 georg.brandl patch "Report bug" links 61 days http://bugs.python.org/issue2823 techtonik cleanup of freelist management 52 days http://bugs.python.org/issue2862 gregory.p.smith patch, patch By default, HTTPSConnection should send header "Host: somehost" 24 days http://bugs.python.org/issue3094 gregory.p.smith patch, easy glob.py improvements 20 days http://bugs.python.org/issue3159 facundobatista patch cmath test fails on Solaris 10 14 days http://bugs.python.org/issue3168 MrJean1 patch sha modules & Modules/Setup.dist 13 days http://bugs.python.org/issue3183 gregory.p.smith float('infinity') should be valid 11 days http://bugs.python.org/issue3188 marketdickinson patch Improve subprocess module usage 6 days http://bugs.python.org/issue3235 georg.brandl curses/textpad.py incorrectly and redundantly imports ascii 5 days http://bugs.python.org/issue3239 facundobatista patch socket's OOB data management is broken on OS X and FreeBSD 3 days http://bugs.python.org/issue3277 gregory.p.smith socket's SO_OOBINLINE option does not work 3 days http://bugs.python.org/issue3278 gregory.p.smith %c format does not accept large numbers on ucs-2 builds 1 days http://bugs.python.org/issue3280 amaury.forgeotdarc Delete obsolete 'Unicode' in Py3 str docstrings; related fixes 0 days http://bugs.python.org/issue3284 benjamin.peterson easy Fraction.from_any() 6 days http://bugs.python.org/issue3285 rhettinger patch Fraction constructor should raise TypeError instead of Attribute 6 days http://bugs.python.org/issue3287 rhettinger patch unnecessary call to time and localtime slows time.mktime 0 days http://bugs.python.org/issue3289 facundobatista rlcompleter doesn't work anymore 0 days http://bugs.python.org/issue3291 benjamin.peterson patch SVN repository contains an incorrect symbolic link 0 days http://bugs.python.org/issue3294 benjamin.peterson PyExc_BufferError is declared but nowhere defined 0 days http://bugs.python.org/issue3295 benjamin.peterson patch print function not executed in python 3.0 tutorial 0 days http://bugs.python.org/issue3296 benjamin.peterson Multiline string with quotes is not parsed correctly. 0 days http://bugs.python.org/issue3298 Stavros DoS when lo is negative in bisect.insort_right() / _left() 4 days http://bugs.python.org/issue3301 rhettinger patch audioop.findmax() crashs with negative length 1 days http://bugs.python.org/issue3306 facundobatista patch MinGW built extensions do not load (specified procedure cannot b 1 days http://bugs.python.org/issue3308 loewis urllib.parse doesn't import sys 0 days http://bugs.python.org/issue3314 facundobatista patch abc.rst little error 0 days http://bugs.python.org/issue3315 benjamin.peterson patch DocTest and dict sort. 0 days http://bugs.python.org/issue3332 rhettinger Need -3 warning for exec statement becoming a function 1 days http://bugs.python.org/issue3333 rhettinger Fixer for dbm is failing 0 days http://bugs.python.org/issue3337 brett.cannon asyncore.py and "handle_error" 1839 days http://bugs.python.org/issue760475 josiahcarlson asyncore misses socket closes when poll is used 1515 days http://bugs.python.org/issue953599 josiahcarlson asyncore should handle also ECONNABORTED in recv 390 days http://bugs.python.org/issue1736101 josiahcarlson patch Top Issues Most Discussed (10) ______________________________ 14 threading module can deadlock after fork 1643 days open http://bugs.python.org/issue874900 11 MinGW built extensions do not load (specified procedure cannot 1 days closed http://bugs.python.org/issue3308 10 urllib.quote and unquote - Unicode issues 5 days open http://bugs.python.org/issue3300 9 Crash in PyObject_Malloc 356 days open http://bugs.python.org/issue1758146 8 urllib2 header capitalization 122 days open http://bugs.python.org/issue2275 7 bytearrays are not thread safe 22 days open http://bugs.python.org/issue3139 7 test_multiprocessing hangs intermittently on POSIX platforms 9 days open http://bugs.python.org/issue3088 6 API for setting the memory allocator used by Python 2 days open http://bugs.python.org/issue3329 6 duplicate lines in zipfile.py 4 days open http://bugs.python.org/issue3317 6 Let bin/oct/hex show floats 17 days open http://bugs.python.org/issue3008 From rhamph at gmail.com Fri Jul 11 21:26:33 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 11 Jul 2008 13:26:33 -0600 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <g57ll0$kqn$1@ger.gmane.org> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> Message-ID: <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> On Fri, Jul 11, 2008 at 7:02 AM, Steve Holden <steve at holdenweb.com> wrote: > Benjamin Peterson wrote: >> >> On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote: >>> >>> Some effort needs to be made to clear the standard library of -3 >>> warnings. >>> Running -3 on production code usually involves exercising library code >>> so >>> the useful result is obscured by Python complaining about itself. Since >>> that use case involves the users own tests, I don't think the effort >>> needs >>> to be extended to our own unittest suite. But the rest of the library >>> could >>> likely benefit from a good -3 cleanup. >> >> Yes, indeed. We should make sure, however, that the changes in the 2.6 >> libraries are the absolute minimum to get the job done. (I'm trying to >> pretend like this isn't violating the prohibition on all-inclusive >> overhauls in the stdlib.) >> > The prohibition is on *gratuitous* changes, basically along the lines of "if > it ain't broke, don't fix it". The stdlib is definitely broken if it raises > warnings of that kind. Is the stdlib broken or is it the warnings that are broken? The code is just fine in 2.6. Adding pragmas to disable warnings would be just fine. Or we could hardcode some warnings as "already seen". -- Adam Olsen, aka Rhamphoryncus From brett at python.org Fri Jul 11 22:16:30 2008 From: brett at python.org (Brett Cannon) Date: Fri, 11 Jul 2008 13:16:30 -0700 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> Message-ID: <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> On Fri, Jul 11, 2008 at 12:26 PM, Adam Olsen <rhamph at gmail.com> wrote: > On Fri, Jul 11, 2008 at 7:02 AM, Steve Holden <steve at holdenweb.com> wrote: >> Benjamin Peterson wrote: >>> >>> On Fri, Jul 11, 2008 at 6:24 AM, Raymond Hettinger <python at rcn.com> wrote: >>>> >>>> Some effort needs to be made to clear the standard library of -3 >>>> warnings. >>>> Running -3 on production code usually involves exercising library code >>>> so >>>> the useful result is obscured by Python complaining about itself. Since >>>> that use case involves the users own tests, I don't think the effort >>>> needs >>>> to be extended to our own unittest suite. But the rest of the library >>>> could >>>> likely benefit from a good -3 cleanup. >>> >>> Yes, indeed. We should make sure, however, that the changes in the 2.6 >>> libraries are the absolute minimum to get the job done. (I'm trying to >>> pretend like this isn't violating the prohibition on all-inclusive >>> overhauls in the stdlib.) >>> >> The prohibition is on *gratuitous* changes, basically along the lines of "if >> it ain't broke, don't fix it". The stdlib is definitely broken if it raises >> warnings of that kind. > > Is the stdlib broken or is it the warnings that are broken? Nothing is broken, per se, but the stdlib emits a ton of warnings through basic usage for Py3K-related changes. We are telling people to run their code in 2.6 with -3 and to eliminate all warnings in order to have 2to3 work to transition to 3.0. Having the stdlib itself emit warnings is just not reasonable. > The code > is just fine in 2.6. Adding pragmas to disable warnings would be just > fine. Or we could hardcode some warnings as "already seen". > No, we should eat our own dog food and transition the code over. If anything it will help with code maintenance between 2.x and 3.x. -Brett From josiah.carlson at gmail.com Sat Jul 12 04:51:58 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 11 Jul 2008 19:51:58 -0700 Subject: [Python-Dev] A proposed solution for Issue 502236: Asyncrhonous exceptions between threads In-Reply-To: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl> References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl> Message-ID: <e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com> This doesn't need to be an interpreter thing; it's easy to implement by the user (I've done it about a dozen times using a single global flag). If you want it to be automatic, it's even possible to make it happen automatically using sys.settrace() and friends (you can even make it reasonably fast if you use a C callback). - Josiah On Fri, Jul 11, 2008 at 4:27 AM, Andy Scott <kirkshorts at hotmail.co.uk> wrote: > [OK so a newbie post here so many apologies if I am doing this wrong] > > Quick Synopsis: > > A child thread in an executing Python program can not safely shutdown the > program. The issue URL is: http://bugs.python.org/issue502236 > > So my proposal is: > > Example: > > We have three threads - > t0 - Main system thread > t1 - Worker thread > t2 - Worker thread > > t1 encounters an issue that means it wants to shut down the application in > as safe a way as possible > > > A Solution: > > 1. Put in place a new function call sys.exitapplication, what this would do > is: > a. Mark a flag in t0's data structure saying a request to shutdown has > been made > b. Raise a new exception, SystemShuttingDown, in t1. > 2. As the main interpreter executes it checks the "shutting down flag" in > the per thread data and follows one of two paths: > If it is t0: > a. Stops execution of the current code sequence > b. Iterates over all extant threads setting the "system shutdown" flag > in the per thread data structure. Setting this flag is a one time deal - it > can not be undone once set. (And to avoid issues with multiple threads > setting it - it can only ever be a single fixed value so setting it multiple > times results in the same answer) > c. Enters a timed wait loop where it will allow the other threads time > to see the signal. It will iterate this loop a set number of times to avoid > being blocked on any given thread. > d. When all threads have exited, or been forcefully closed, raise the > SystemShuttingDown exception > > If it is not t0: > a. Stops execution of the current code sequence > b. Raises the exception, SystemShuttingDown. > > There are problems with this approach, as I see it they are (but please see > the assumptions I have made): > > P1. If the thread is in a tight loop will it see the exception? Or more > generally: when should the exception be raised? > P2. When should the interpreter check this flag? > > I think the answer to both of these problems is to: > > Check the flag, and hence raise the exception, in the following > circumstances: > > - When the interpreter executes a back loop. So this should catch the jump > back to the top of a "while True:" loop > - Just before the interpreter makes a call to a hooked in non-Python > system function, e.g. file I/O, networking &c. > > Checking at these points should be the minimal required, I think, to ensure > that a given thread can not ignore the exception. It may be possible, or > even required, to perform the check every time a Python function call is > made. > > I think this approach would then allow for the finally handlers to be > called. > > Assumptions: > > [Here I must admit to a large amount of ignorance of the internals of Python > at this time. So if my assumptions are incorrect I would greatly appreciate > being told so :-) Preferably as polite as possible and any code pointers > while welcome unless they point to some very esoteric and arcane area would > be best kept general so I feel more of a spur to go learn the code base] > > 1. The Python interpreter has per thread information. > 2. The Python interpreter can tell if the system, t0, thread is running. > 3. The Python engine has (or can easily obtain) a list of all threads it > created. > 4. It is possible to raise exceptions as the byte code is executing. > > I am mailing this out as: > > A. I have no idea if my thoughts are correct or total un-mitigated rubbish > :-) > B. I believe the introduction of this proposal (if I am correct) will > require a PEP being raised, which aiui requires building community support > (which is very fair imo) so this is me trying to do so :-) > > So apologies if this post has been total spam (but no eggs) or too long - > give a little whistle and it will all be OK again. > > Andy > -------------------------------------- > Brain chemistry is not just for Christmas > > > ________________________________ > Get Messenger on your Mobile! Get it now! > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com > > From fumanchu at aminus.org Sat Jul 12 19:08:31 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Sat, 12 Jul 2008 10:08:31 -0700 Subject: [Python-Dev] A proposed solution for Issue 502236: Asyncrhonousexceptions between threads In-Reply-To: <e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com> References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl> <e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com> Message-ID: <F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local> Josiah Carlson wrote: > This doesn't need to be an interpreter thing; it's easy to implement > by the user (I've done it about a dozen times using a single global > flag). If you want it to be automatic, it's even possible to make it > happen automatically using sys.settrace() and friends (you can even > make it reasonably fast if you use a C callback). Agreed. If someone wants a small library to help do this, especially in web servers, the latest version of Cherrpy includes a 'process' subpackage under a generous license. It does all the things Andy describes via a Bus object: > Andy Scott wrote: > > 1. Put in place a new function call sys.exitapplication, what this > > would do is: > > a. Mark a flag in t0's data structure saying a request to > > shutdown has been made This is bus.exit(), which publishes a 'stop' message to all subscribed 'stop' listeners, and then an 'exit' message to any 'exit' listeners. > > b. Raise a new exception, SystemShuttingDown, in t1. That's up to the listener. > > 2. As the main interpreter executes it checks the "shutting down > > flag" in the per thread data and follows one of two paths: > > If it is t0: > > a. Stops execution of the current code sequence > > b. Iterates over all extant threads ... > > c. Enters a timed wait loop where it will allow the other > > threads time to see the signal. It will iterate this loop > > a set number of times to avoid being blocked on any given > > thread. This is implemented as [t.join() for t in threading.enumerate()] in the main thread. > > d. When all threads have exited, or been forcefully closed, > > raise the SystemShuttingDown exception The bus just lets the main thread exit at this point. > > P1. If the thread is in a tight loop will it see the exception? Or > > more generally: when should the exception be raised? That's dependent enough on what work the thread is doing that a completely generic approach is generally not sufficient. Therefore, the process.bus sends a 'stop' message, and leaves the implementation of the receiver up to the author of that thread's logic. Presumably, one wouldn't register a listener for the 'stop' message unless one knew how to actually stop. > > P2. When should the interpreter check this flag? > > > > I think the answer to both of these problems is to check the flag, > > and hence raise the exception, in the following circumstances: > > - When the interpreter executes a back loop. So this should catch > > the jump back to the top of a "while True:" loop > > - Just before the interpreter makes a call to a hooked in non- > > Python system function, e.g. file I/O, networking &c. This is indeed how most well-written apps do it already. > > Checking at these points should be the minimal required, I think, to > > ensure that a given thread can not ignore the exception. It may be > > possible, or even required, to perform the check every time a Python > > function call is made. PLEASE don't make Python function calls slower. > > 1. The Python interpreter has per thread information. > > 2. The Python interpreter can tell if the system, t0, thread is > > running. > > 3. The Python engine has (or can easily obtain) a list of all > > threads it created. > > 4. It is possible to raise exceptions as the byte code is executing. Replace 'Python interpreter' with 'your application' and those become relatively simple architectural issues: maintain a list of threads, have them expose an interface to determine if they're running, and make them monitor a flag to know when another thread is asking them to stop. Robert Brewer fumanchu at aminus.org From matt.giuca at gmail.com Sat Jul 12 19:27:16 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Sun, 13 Jul 2008 03:27:16 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues Message-ID: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> Hi all, My first post to the list. In fact, first time Python hacker, long-time Python user though. (Melbourne, Australia). Some of you may have seen for the past week or so my bug report on Roundup, http://bugs.python.org/issue3300 I've spent a heap of effort on this patch now so I'd really like to get some more opinions and have this patch considered for Python 3.0. Basically, urllib.quote and unquote seem not to have been updated since Python 2.5, and because of this they implicitly perform Latin-1 encoding and decoding (with respect to percent-encoded characters). I think they should default to UTF-8 for a number of reasons, including that's what other software such as web browsers use. I've submitted a patch which fixes quote and unquote to use UTF-8 by default. I also added extra arguments allowing the caller to choose the encoding (after discussion, there was some consensus that this would be beneficial). I have now completed updating the documentation, writing extensive test cases, and testing the rest of the standard library for code breakage - with the result being there wasn't really any, everything seems to just work nicely with UTF-8. You can read the sordid details of my investigation in the tracker. Firstly, it'd be nice to hear if people think this is desirable behaviour. Secondly, if it's feasible to get this patch in Python 3.0. (I think if it were delayed to Python 3.1, the code breakage wouldn't justify it). And thirdly, if the first two are positive, if anyone would like to review this patch and check it in. I have extensively tested it, and am now pretty confident that it won't cause any grief if it's checked in. Thanks very much, Matt Giuca -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/d6f74f48/attachment.htm> From brett at python.org Sat Jul 12 20:46:48 2008 From: brett at python.org (Brett Cannon) Date: Sat, 12 Jul 2008 11:46:48 -0700 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> Message-ID: <bbaeab100807121146k22046188v858bc70d138a833c@mail.gmail.com> On Sat, Jul 12, 2008 at 10:27 AM, Matt Giuca <matt.giuca at gmail.com> wrote: > Hi all, > > My first post to the list. In fact, first time Python hacker, long-time > Python user though. (Melbourne, Australia). > Welcome! > Some of you may have seen for the past week or so my bug report on Roundup, > http://bugs.python.org/issue3300 > > I've spent a heap of effort on this patch now so I'd really like to get some > more opinions and have this patch considered for Python 3.0. > Hopefully we can get to it in the near future. Since we are having two more betas (one of this is next week) hopefully there is enough time before hitting a release candidate to have this looked at. > Basically, urllib.quote and unquote seem not to have been updated since > Python 2.5, and because of this they implicitly perform Latin-1 encoding and > decoding (with respect to percent-encoded characters). I think they should > default to UTF-8 for a number of reasons, including that's what other > software such as web browsers use. > > I've submitted a patch which fixes quote and unquote to use UTF-8 by > default. I also added extra arguments allowing the caller to choose the > encoding (after discussion, there was some consensus that this would be > beneficial). I have now completed updating the documentation, writing > extensive test cases, and testing the rest of the standard library for code > breakage - with the result being there wasn't really any, everything seems > to just work nicely with UTF-8. You can read the sordid details of my > investigation in the tracker. > > Firstly, it'd be nice to hear if people think this is desirable behaviour. Based on what is said in this email, it sounds reasonable. > Secondly, if it's feasible to get this patch in Python 3.0. (I think if it > were delayed to Python 3.1, the code breakage wouldn't justify it). If what you are saying is true, then it can probably go in as a bug fix (unless someone else knows something about Latin-1 on the Net that makes this not true). > And > thirdly, if the first two are positive, if anyone would like to review this > patch and check it in. > That I can't say I can necessarily due; have my own bug reports to work through this weekend. =) -Brett From janssen at parc.com Sat Jul 12 23:07:09 2008 From: janssen at parc.com (Bill Janssen) Date: Sat, 12 Jul 2008 14:07:09 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> Message-ID: <08Jul12.140711pdt."58698"@synergy1.parc.xerox.com> > Basically, urllib.quote and unquote seem not to have been updated since > Python 2.5, and because of this they implicitly perform Latin-1 encoding and > decoding (with respect to percent-encoded characters). I think they should > default to UTF-8 for a number of reasons, including that's what other > software such as web browsers use. The standard here is RFC 3986, from Jan 2005, which says, ``When a new URI scheme defines a component that represents textual data consisting of characters from the Universal Character Set [UCS], the data should first be encoded as octets according to the UTF-8 character encoding [STD63]; then only those octets that do not correspond to characters in the unreserved set should be percent-encoded.'' The "unreserved set" consists of the following ASCII characters: ``Characters that are allowed in a URI but do not have a reserved purpose are called unreserved. These include uppercase and lowercase letters, decimal digits, hyphen, period, underscore, and tilde. unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" '' There are a few other wrinkles; it's worth reading section 2.5 carefully. I'd say, treat the incoming data as either Unicode (if it's a Unicode string), or some unknown superset of ASCII (which includes both Latin-1 and UTF-8) if it's a byte-string (and thus in some unknown encoding), and apply the appropriate transformation. Bill From asmodai at in-nomine.org Sun Jul 13 00:28:01 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 13 Jul 2008 00:28:01 +0200 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> Message-ID: <20080712222801.GC27106@nexus.in-nomine.org> -On [20080712 19:27], Matt Giuca (matt.giuca at gmail.com) wrote: >Basically, urllib.quote and unquote seem not to have been updated since Python >2.5, and because of this they implicitly perform Latin-1 encoding and decoding >(with respect to percent-encoded characters). I think they should default to >UTF-8 for a number of reasons, including that's what other software such as web >browsers use. Very nice, I had this somewhere on my todo list to work on. I'm very much in favour, especially since it synchronizes us with the RFCs (for all I remember reading about it last time). -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Can your hear the Dolphin's cry..? From martin at v.loewis.de Sun Jul 13 01:10:01 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 13 Jul 2008 01:10:01 +0200 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <20080712222801.GC27106@nexus.in-nomine.org> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> Message-ID: <487939C9.3040703@v.loewis.de> > Very nice, I had this somewhere on my todo list to work on. I'm very much > in favour, especially since it synchronizes us with the RFCs (for all I > remember reading about it last time). I still think that it doesn't. The RFCs haven't changed, and can't change for compatibility reasons. The encoding of non-ASCII characters in URLs remains as underspecified as it always was. Now, with IRIs, the situation is different, but I don't think the patch claims to implement IRIs (and if so, it perhaps shouldn't change URL processing in doing so). Regards, Martin From matt.giuca at gmail.com Sun Jul 13 01:15:18 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Sun, 13 Jul 2008 09:15:18 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <20080712222801.GC27106@nexus.in-nomine.org> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> Message-ID: <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> Thanks for all the replies, and making me feel welcome :) > > If what you are saying is true, then it can probably go in as a bug > fix (unless someone else knows something about Latin-1 on the Net that > makes this not true). > Well from what I've seen, the only time Latin-1 naturally appears on the net is when you have a web page in Latin-1 (either explicit or inferred; and note that a browser like Firefox will infer Latin-1 if it sees only ASCII characters) with a form in it. Submitting the form, the browser will use Latin-1 to percent-encode the query string. So if you write a web app and you don't have any non-ASCII characters or mention the charset, chances are you'll get Latin-1. But I would argue you're leaving things to chance and you deserve to get funny behaviour. If you do any of the following: - Use a non-ASCII character, encoded as UTF-8 on the page. - Send a Content-Type: xxxx; charset=utf-8. - In HTML, set a <meta http-equiv="Content-Type: xxxx; charset=utf-8" />. - In the form itself, set <form accept-encoding="utf-8">. then the browser will encode the form data as UTF-8. And most "proper" web pages should get themselves explicitly served as UTF-8. That I can't say I can necessarily due; have my own bug reports to > work through this weekend. =) OK well I'm busy for the next few days; after that I can do a patch trade with someone. (That is if I am allowed to do reviews; not sure since I don't have developer privileges). On Sun, Jul 13, 2008 at 5:58 AM, Mark Hammond <mhammond at skippinet.com.au> wrote: > > My first post to the list. In fact, first time Python hacker, > > long-time Python user though. (Melbourne, Australia). > > Cool - where exactly? I'm in Wantirna (although not at this very moment - > I'm in Lithuania, but home again in a couple of days) Cool :) Balwyn. > * Please take Martin with a grain of salt ( \I would say "ignore him", but > that is too strong ;) Lol, he is a hard man to please, but he's given some good feedback. On Sun, Jul 13, 2008 at 7:07 AM, Bill Janssen <janssen at parc.com> wrote: > > The standard here is RFC 3986, from Jan 2005, which says, > > ``When a new URI scheme defines a component that represents textual > data consisting of characters from the Universal Character Set [UCS], > the data should first be encoded as octets according to the UTF-8 > character encoding [STD63]; then only those octets that do not > correspond to characters in the unreserved set should be > percent-encoded.'' Ah yes, I was originally hung up on the idea that "URLs had to be encoded in UTF-8", till Martin pointed out that it only says "new URI scheme" there. It's perfectly valid to have non-UTF-8-encoded URIs. However in practice they're almost always UTF-8. So I think introducing the new encoding argument and having it default to "utf-8" is quite reasonable. I'd say, treat the incoming data as either Unicode (if it's a Unicode > string), or some unknown superset of ASCII (which includes both > Latin-1 and UTF-8) if it's a byte-string (and thus in some unknown > encoding), and apply the appropriate transformation. > Ah there may be some confusion here. We're only dealing with str->str transformations (which in Python 3 means Unicode strings). You can't put a bytes in or get a bytes out of either of these functions. I suggested a "quote_raw" and "unquote_raw" function which would let you do this. The issue is with the percent-encoded characters in the URI string, which must be interpreted as bytes, not code points. How then do you convert these into a Unicode string? (Python 2 did not have this problem, since you simply output a byte string without caring about the encoding). On Sun, Jul 13, 2008 at 9:10 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote: > > Very nice, I had this somewhere on my todo list to work on. I'm very much > > in favour, especially since it synchronizes us with the RFCs (for all I > > remember reading about it last time). > > I still think that it doesn't. The RFCs haven't changed, and can't > change for compatibility reasons. The encoding of non-ASCII characters > in URLs remains as underspecified as it always was. Correct. But my patch brings us in-line with that unspecification. The unpatched version forces you to use Latin-1. My patch lets you specify the encoding to use. > Now, with IRIs, the situation is different, but I don't think the patch > claims to implement IRIs (and if so, it perhaps shouldn't change URL > processing in doing so). True. I don't claim to have implemented IRIs or even know enough about them to do that. I'll read up on these things in the next few days. However, this is a URI library, not IRI. From what I've seen, it's percent-encoded URIs coming in from the browser, not IRIs. We just need to make sure with this patch that IRIs don't become less-supported than they were before; don't need to explicitly support them. Cheers, Matt Giuca -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/6a007f88/attachment-0001.htm> From nd at perlig.de Sun Jul 13 01:55:48 2008 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Sun, 13 Jul 2008 01:55:48 +0200 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> Message-ID: <200807130155.48346@news.perlig.de> * Matt Giuca wrote: > Well from what I've seen, the only time Latin-1 naturally appears on the > net is when you have a web page in Latin-1 (either explicit or inferred; > and note that a browser like Firefox will infer Latin-1 if it sees only > ASCII characters) with a form in it. Submitting the form, the browser > will use Latin-1 to percent-encode the query string. This POV is way too browser-centric... > So if you write a web app and you don't have any non-ASCII characters or > mention the charset, chances are you'll get Latin-1. But I would argue > you're leaving things to chance and you deserve to get funny behaviour. > If you do any of the following: > > - Use a non-ASCII character, encoded as UTF-8 on the page. > - Send a Content-Type: xxxx; charset=utf-8. > - In HTML, set a <meta http-equiv="Content-Type: xxxx; charset=utf-8" > />. - In the form itself, set <form accept-encoding="utf-8">. > > then the browser will encode the form data as UTF-8. And most "proper" > web pages should get themselves explicitly served as UTF-8. ... because 1) URL encoding is not limited to web forms at all 2) The web form encoding depends on the browser settings as well (for example, try playing around with the internet explorer settings regarding query encoding) 3) The process submitting the form may not be a browser at all 4) The web form may not be under your own control (Search engine forms are a common example here, e.g. "put this google form snippet onto your webpage") 5) Different cultures do not choose necessarily between latin-1 and utf-8. They deal more with things like, say KOI8-R or Big5. etc pp Besides all that and without any offense: "most proper" and "should do" and the implication that all web browsers behave the same way are not a good location to argue from when talking about implementing a standard ;) nd -- Wenn nur Ingenieure mit Diplom programmieren w?rden, h?tten wir wahrscheinlich weniger schlechte Software. Wir h?tten allerdings auch weniger gute Software. -- Felix von Leitner in dasr From matt.giuca at gmail.com Sun Jul 13 02:24:11 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Sun, 13 Jul 2008 10:24:11 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <200807130155.48346@news.perlig.de> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <200807130155.48346@news.perlig.de> Message-ID: <e6268af30807121724l470e32agb23a018a5f83c757@mail.gmail.com> > This POV is way too browser-centric... > This is but one example. Note that I found web forms to be the least clear-cut example of choosing an encoding. Most of the time applications seem to be using UTF-8, and all the standards I have read are moving towards specifying UTF-8 (from being unspecified). I've never seen a standard specify or even recommend Latin-1. Where web forms are concerned, basically setting the form accept-charset or the page charset is the *maximum amount* of control you have over the encoding. As you say, it can be encoded by another page or the user can override their settings. Then what can you do as the server? Nothing ... there's no way to predict the encoding. So you just handle the cases you have control over. 5) Different cultures do not choose necessarily between latin-1 and utf-8. > They deal more with things like, say KOI8-R or Big5. Exactly. This is exactly my point - Latin-1 is arbitrary from a standards point of view. It's just one of the many legacy encodings we'd like to forget. The UTFs are the only options which support all languages, and UTF-8 is the only ASCII-compatible (and therefore URI-compatible) encoding. So we should aim to support that as the default. Besides all that and without any offense: "most proper" and "should do" and > the implication that all web browsers behave the same way are not a good > location to argue from when talking about implementing a standard ;) I agree. However if there *was* a proper standard we wouldn't have to argue! "Most proper" and "should do" is the most confident we can be when dealing with this standard, as there is no correct encoding. Does anyone have a suggestion which will be more compatible with the rest of the world than allowing the user to select an encoding, and defaulting to "utf-8"? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/d022d9f6/attachment.htm> From bignose+hates-spam at benfinney.id.au Sun Jul 13 11:29:22 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Sun, 13 Jul 2008 19:29:22 +1000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> Message-ID: <8763radte5.fsf@benfinney.id.au> "Guido van Rossum" <guido at python.org> writes: > Same here; let's tread carefully here and not change this with 3.0. > Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. I'm very glad this is on the table. Even though I'd really like to see the names become PEP-8-conformant in the 2.x series, the arguments against such haste are compelling. So I'm convinced to be +1 on post-3.0 for changing the unittest API. > I like using only the assertKeyword variants, removing assert_, fail*, > and assertEquals. I'm the opposite. I prefer the 'fail*' variants over the 'assert*' variants, because "fail" tells me exactly what the function will *do*, while "assert" leaves it implicit what will actually happen if the assertion is false. For this reason, I'd prefer the "fail*" names to be recommended, and the "assert*" names to be, if not deprecated, then at least second-class citizens. -- \ ?I think there is a world market for maybe five computers.? | `\ ?Thomas Watson, chairman of IBM, 1943 | _o__) | Ben Finney From andrew-pythondev at puzzling.org Sun Jul 13 13:46:53 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Sun, 13 Jul 2008 21:46:53 +1000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <8763radte5.fsf@benfinney.id.au> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> Message-ID: <20080713114653.GE9654@steerpike.home.puzzling.org> Ben Finney wrote: > "Guido van Rossum" <guido at python.org> writes: [...] > > I like using only the assertKeyword variants, removing assert_, fail*, > > and assertEquals. > > I'm the opposite. I prefer the 'fail*' variants over the 'assert*' > variants, because "fail" tells me exactly what the function will *do*, > while "assert" leaves it implicit what will actually happen if the > assertion is false. > > For this reason, I'd prefer the "fail*" names to be recommended, and > the "assert*" names to be, if not deprecated, then at least > second-class citizens. On the other hand, ?assert*? names are positive statements of what the behaviour of the system-under-test is. And conversely, ?fail*? names are a bit like implementation details of how the test is written. So I personally have a mild preference for the assert* names. My suspicion is that it will be very hard to get consensus on the colour of this particular bike shed :) -Andrew. From solipsis at pitrou.net Sun Jul 13 14:25:35 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 13 Jul 2008 12:25:35 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?unittest=27s_redundant_assertions=3A_asser?= =?utf-8?q?ts=09vs=2E=09failIf/Unlesses?= References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> Message-ID: <loom.20080713T121738-271@post.gmane.org> Andrew Bennetts <andrew-pythondev <at> puzzling.org> writes: > > On the other hand, ?assert*? names are positive statements of what the > behaviour of the system-under-test is. And conversely, ?fail*? names are a > bit like implementation details of how the test is written. So I personally > have a mild preference for the assert* names. The problem with "fail*" is that you get names like "failIfNotEqual" (or perhaps even "failUnlessNotEqual") which are double negatives and therefore much more annoying to read and decipher. I always had the same problem when reading Perl's "unless" statements. They are, IMO, useless complication. "assert*" names are, well, assertive :-) (not to mention "assert" is a widely established name in various languages - including Python - for checking that things went as expected) From bignose+hates-spam at benfinney.id.au Sun Jul 13 14:36:27 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Sun, 13 Jul 2008 22:36:27 +1000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> Message-ID: <87prpic65w.fsf@benfinney.id.au> Antoine Pitrou <solipsis at pitrou.net> writes: > The problem with "fail*" is that you get names like "failIfNotEqual" That would better be written (preferring PEP 8 names) "fail_unless_equal". > (or perhaps even "failUnlessNotEqual") idem, "fail_if_equal". > which are double negatives Exactly. With "if" and "unless" at our disposal, we can avoid such double negatives. > (not to mention "assert" is a widely established name in various > languages - including Python - for checking that things went as > expected) That's another reason to avoid "assert" in the name: these methods *don't* necessarily use the 'assert' statement. Avoiding the implication that they do use that is a good thing. -- \ ?Never do anything against conscience even if the state demands | `\ it.? ?Albert Einstein | _o__) | Ben Finney From bignose+hates-spam at benfinney.id.au Sun Jul 13 14:38:39 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Sun, 13 Jul 2008 22:38:39 +1000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> Message-ID: <87lk06c628.fsf@benfinney.id.au> Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > On the other hand, ?assert*? names are positive statements of what > the behaviour of the system-under-test is. That statement is better made by the name of the test case. The method used for implementing the test shouldn't need to be part of that statement. > And conversely, ?fail*? names are a bit like implementation > details of how the test is written. Entirely appropriate, since those names are used exactly at the point of implementing the test, and the names are not visible outside that implementation. No? > My suspicion is that it will be very hard to get consensus on the > colour of this particular bike shed :) Perhaps so :-) -- \ ?Quidquid latine dictum sit, altum viditur.? (?Whatever is | `\ said in Latin, sounds profound.?) ?anonymous | _o__) | Ben Finney From ncoghlan at gmail.com Sun Jul 13 14:51:30 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 13 Jul 2008 22:51:30 +1000 Subject: [Python-Dev] AMD64-W2k8 buildbot wedged Message-ID: <4879FA52.4040706@gmail.com> The compilation step on this buildbot is failing because it can't delete or overwrite any of the Python DLLs [1]. Is there any way to fix this remotely, or at least to tell why kill_python isn't doing the trick? (Going back a bit further, it looks like test_multiprocessing is the culprit as the original hanging test. The number of 64-bit safeness warnings being emitted by the current trunk is also fairly worrying) Cheers, Nick. [1] http://www.python.org/dev/buildbot/all/AMD64%20W2k8%20trunk/builds/757/step-compile/0 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From solipsis at pitrou.net Sun Jul 13 15:07:05 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 13 Jul 2008 13:07:05 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?unittest=27s_redundant_assertions=3A=09ass?= =?utf-8?q?erts=09vs=2E=09failIf/Unlesses?= References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> Message-ID: <loom.20080713T130302-168@post.gmane.org> Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes: > > That would better be written (preferring PEP 8 names) > "fail_unless_equal". Which is still a double negative ("fail" and "unless" are both negative words). > That's another reason to avoid "assert" in the name: these methods > *don't* necessarily use the 'assert' statement. But all those constructs (assert, assertEqual, etc.) raise the same exception type named AssertionError which, if you care for naming consistency, is more important than whether or not they are implemented in terms of each other. :-) From steve at holdenweb.com Sun Jul 13 15:31:35 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 13 Jul 2008 09:31:35 -0400 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <loom.20080713T130302-168@post.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> Message-ID: <g5d03p$air$1@ger.gmane.org> Antoine Pitrou wrote: > Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes: >> That would better be written (preferring PEP 8 names) >> "fail_unless_equal". > > Which is still a double negative ("fail" and "unless" are both negative words). > "Fail" isn't a negative. As Guido said, it's a description of the test behavior under particular circumstances. "fail_unless_equal" says quite clearly that the test requires equality of the values. >> That's another reason to avoid "assert" in the name: these methods >> *don't* necessarily use the 'assert' statement. > > But all those constructs (assert, assertEqual, etc.) raise the same exception > type named AssertionError which, if you care for naming consistency, is more > important than whether or not they are implemented in terms of each other. :-) > But the important behavior is the failure of the test, not the specific exception that is raised to fail it. Or would you prefer tests that raise other exceptions to succeed? The exception type is an implementation detail, and a fairly unimportant one as far as the purpose of testing goes. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From bignose+hates-spam at benfinney.id.au Sun Jul 13 15:34:51 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Sun, 13 Jul 2008 23:34:51 +1000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> Message-ID: <87ej5xdi10.fsf@benfinney.id.au> Antoine Pitrou <solipsis at pitrou.net> writes: > Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes: > > That would better be written (preferring PEP 8 names) > > "fail_unless_equal". > > Which is still a double negative ("fail" and "unless" are both > negative words). Hmm, not to this native-English-speaker's ear. "fail" is a verb stating what will be done, while "unless" and "if" are the conditions under which it will be done. > > That's another reason to avoid "assert" in the name: these methods > > *don't* necessarily use the 'assert' statement. > > But all those constructs (assert, assertEqual, etc.) raise the same > exception type named AssertionError Only by default. They can be overridden to raise any exception type. The only thing they have in common at that point (when the exception is raised) is that they have failed the test. -- \ ?First things first, but not necessarily in that order.? ?The | `\ Doctor, _Doctor Who_ | _o__) | Ben Finney From Scott.Daniels at Acm.Org Sun Jul 13 16:02:01 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun, 13 Jul 2008 07:02:01 -0700 Subject: [Python-Dev] Update bz2 library? Message-ID: <g5d18v$dav$1@ger.gmane.org> I just noticed that the bz2lib version was updated to 1.0.5 in December of 2007, for security reasons. I suspect it would be good to be sure to ship this with 2.6 or 3.0. --Scott David Daniels Scott.Daniels at Acm.Org From solipsis at pitrou.net Sun Jul 13 16:39:48 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 13 Jul 2008 14:39:48 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?unittest=27s_redundant_assertions=3A_asser?= =?utf-8?q?ts_vs=2E=09failIf/Unlesses?= References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> Message-ID: <loom.20080713T135317-118@post.gmane.org> Let's split hairs a little... Steve Holden <steve <at> holdenweb.com> writes: > "Fail" isn't a negative. As Guido said, it's a description of the test > behavior under particular circumstances. In most circumstances, "fail" is a negative word defined as the contrary of something else (that is, as the "failure to pass/succeed/perform/achieve/..."), while the reverse is not true (few people would define "success" or "passing a test" as the negative of "failure", except in desperate circumstances). Although I'm not a native English speaker, I don't think our respective languages and cultures differ on this point. > "fail_unless_equal" says quite > clearly that the test requires equality of the values. Actually, saying "that the test requires equality of the values" translates directly into an "assert equals" (or "enforce equals" if you want a stronger word) rather than a "fail if not equal". It is a grammatical fact... In other words, if you express a requirement, you intent to say how the implementation under test is supposed to behave for it to be considered successful, not the conditions under which its behaviour constitutes a failure. As you said, if an exception is thrown which isn't part of the testing protocol (e.g. something other than an AssertionError), the test is still said to fail... if the intent of testing were to test for failure conditions, on the contrary, the test would be said to be passed (because it wouldn't have met the failure conditions). Regards Antoine. From fuzzyman at voidspace.org.uk Sun Jul 13 16:45:58 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 13 Jul 2008 15:45:58 +0100 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <loom.20080713T135317-118@post.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> <loom.20080713T135317-118@post.gmane.org> Message-ID: <487A1526.6030401@voidspace.org.uk> Antoine Pitrou wrote: > Let's split hairs a little... > > Steve Holden <steve <at> holdenweb.com> writes: > >> "Fail" isn't a negative. As Guido said, it's a description of the test >> behavior under particular circumstances. >> > > In most circumstances, "fail" is a negative word defined as the contrary of > something else (that is, as the "failure to pass/succeed/perform/achieve/..."), > while the reverse is not true (few people would define "success" or "passing a > test" as the negative of "failure", except in desperate circumstances). Although > I'm not a native English speaker, I don't think our respective languages and > cultures differ on this point. > > > >> "fail_unless_equal" says quite >> clearly that the test requires equality of the values. >> > > Actually, saying "that the test requires equality of the values" translates > directly into an "assert equals" (or "enforce equals" if you want a stronger > word) rather than a "fail if not equal". It is a grammatical fact... > > In other words, if you express a requirement, you intent to say how the > implementation under test is supposed to behave for it to be considered > successful, not the conditions under which its behaviour constitutes a failure. > Agreed. I tend to think of testing as action followed by assertion - I do this and this should have happened. Your tests usually define 'expected behaviour' rather than defining how your code won't fail... Michael > As you said, if an exception is thrown which isn't part of the testing protocol > (e.g. something other than an AssertionError), the test is still said to fail... > if the intent of testing were to test for failure conditions, on the contrary, > the test would be said to be passed (because it wouldn't have met the failure > conditions). > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From kirkshorts at hotmail.co.uk Sun Jul 13 17:28:16 2008 From: kirkshorts at hotmail.co.uk (Andy Scott) Date: Sun, 13 Jul 2008 15:28:16 +0000 Subject: [Python-Dev] A proposed solution for Issue 502236: Asyncrhonousexceptions between threads In-Reply-To: <F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local> References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl> <e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com> <F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local> Message-ID: <BLU136-W52FFE760D80D27784A462788920@phx.gbl> Thanks Guys So it seems as if I've made some pretty basic newbie mistakes then :-$ 1. Posting to the wrong list 2. Posting about something that already has a work around oh well never mind - better luck next time ;-) Only one thing comes to my mind about the comments made saying that this is no longer an issue: All of the solutions presented rely on the use of a third party library to control the threading. It appears as if there have been no basic changes to the thread package that ships with Python to resolve this. While it is possible to set and read global variables &c to do something close - it doesn't strike me as a nice solution as Python offers in so many other places. However, I am willing to concede that as the issue has been dormant (or appears to be) now for ~2years that it can not be a particularly burning issue. So I'll go back and trawl the open issues for something else to help my favourite language get better :-) Andy ----- Brain chemistry is not just for Christmas [snipped the replies to cut down on re-quote spam - cos I kind of hate that] <snip> _________________________________________________________________ Invite your Facebook friends to chat on Messenger http://clk.atdmt.com/UKM/go/101719649/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/86826b85/attachment.htm> From steve at pearwood.info Sun Jul 13 17:34:53 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 14 Jul 2008 01:34:53 +1000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <487A1526.6030401@voidspace.org.uk> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <loom.20080713T135317-118@post.gmane.org> <487A1526.6030401@voidspace.org.uk> Message-ID: <200807140134.54676.steve@pearwood.info> On Mon, 14 Jul 2008 12:45:58 am Michael Foord wrote: > I tend to think of testing as action followed by assertion - > I do this and this should have happened. Your tests usually define > 'expected behaviour' rather than defining how your code won't fail... Who is the "your" that you are speaking for there? It isn't me. I tend to think of tests as *both* expected behaviour and unexpected behaviour: sometimes a test is most naturally written as "this should happen" and sometimes as "this shouldn't happen". It depends on what I'm testing. When it comes to test-driven development, I will often start by thinking "What test can I write to make this code fail?" -- not "what test can I write to make this code pass?". Having come up with a failure mode, the test is often most naturally written as "fail if" or "fail unless", and I resent having to turn the condition around into a "assert if not" test just to satisfy others who are never going to read my code. I wish people would go find another bike shed to interfere with. -- Steven From steve at holdenweb.com Sun Jul 13 17:56:45 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 13 Jul 2008 11:56:45 -0400 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <loom.20080713T135317-118@post.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> <loom.20080713T135317-118@post.gmane.org> Message-ID: <g5d8k2$ha$1@ger.gmane.org> Antoine Pitrou wrote: > Let's split hairs a little... > > Steve Holden <steve <at> holdenweb.com> writes: >> "Fail" isn't a negative. As Guido said, it's a description of the test >> behavior under particular circumstances. > > In most circumstances, "fail" is a negative word defined as the contrary of > something else (that is, as the "failure to pass/succeed/perform/achieve/..."), > while the reverse is not true (few people would define "success" or "passing a > test" as the negative of "failure", except in desperate circumstances). Although > I'm not a native English speaker, I don't think our respective languages and > cultures differ on this point. > "The test will fail" is an assertion (in English, not in Python :). It is not a negative. "The test will not fail" is an assertion containing a negative. "The test will not not fail" is an assertion containing a double negative. > >> "fail_unless_equal" says quite >> clearly that the test requires equality of the values. > > Actually, saying "that the test requires equality of the values" translates > directly into an "assert equals" (or "enforce equals" if you want a stronger > word) rather than a "fail if not equal". It is a grammatical fact... > Right. For extra points, discuss the differences between "fail_unless_equal", "fail_if_not_equal", assert_equals" and "assert_unequal". > In other words, if you express a requirement, you intent to say how the > implementation under test is supposed to behave for it to be considered > successful, not the conditions under which its behaviour constitutes a failure. > > As you said, if an exception is thrown which isn't part of the testing protocol > (e.g. something other than an AssertionError), the test is still said to fail... > if the intent of testing were to test for failure conditions, on the contrary, > the test would be said to be passed (because it wouldn't have met the failure > conditions). > But Guido said: > I like using only the assertKeyword variants, removing assert_, fail*, > and assertEquals. So we are now no longer discussing what color to paint the bike shed, we are discussing how to describe the color. Let's stop. This is getting silly. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at pearwood.info Sun Jul 13 18:20:52 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 14 Jul 2008 02:20:52 +1000 Subject: [Python-Dev] =?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?= =?iso-8859-1?q?asserts_vs=2E=09failIf/Unlesses?= In-Reply-To: <loom.20080713T135317-118@post.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <g5d03p$air$1@ger.gmane.org> <loom.20080713T135317-118@post.gmane.org> Message-ID: <200807140220.53574.steve@pearwood.info> On Mon, 14 Jul 2008 12:39:48 am Antoine Pitrou wrote: > Let's split hairs a little... > > Steve Holden <steve <at> holdenweb.com> writes: > > "Fail" isn't a negative. As Guido said, it's a description of the > > test behavior under particular circumstances. > > In most circumstances, "fail" is a negative word defined as the > contrary of something else (that is, as the "failure to > pass/succeed/perform/achieve/..."), while the reverse is not true > (few people would define "success" or "passing a test" as the > negative of "failure", except in desperate circumstances). "Few people"? Do you have studies to support your claim, or are you just projecting your own opinion as if it were an objective fact? I often consider "success" as the negative of failure: my code didn't fail. The bridge didn't fall down. The invasion didn't get bogged down in a long and pointless guerrilla war. The medicine didn't have massive side-effects. These are all successes. assert_bridge_not_collapsed() assert_no_guerrilla_war() assert_if_no_side-effects() fail_if_bridge_collapsed() fail_if_guerrilla_war() fail_if_side_effects() Speaking for myself, the last three are far more natural to me. I'll also point out that in science, success is often considered the opposite of failure. You design your experiments to disprove the theory, to fail, and if they don't fail, the theory is considered successful. A theory is considered "correct" when it has failed to fail. Some philosophers of science, like the late Karl Popper, consider this falsification to be the defining characteristic of science. [...] > In other words, if you express a requirement, you intent to say how > the implementation under test is supposed to behave for it to be > considered successful, not the conditions under which its behaviour > constitutes a failure. Please don't tell me what my intent is. Unless you are a mind-reader, you have no way of telling what my intent is. When I write tests, my intent is often to specify the conditions that constitute a failure. I don't want to negate it to specify a success just to satisfy you. -- Steven From martin at v.loewis.de Sun Jul 13 18:35:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 13 Jul 2008 18:35:49 +0200 Subject: [Python-Dev] AMD64-W2k8 buildbot wedged In-Reply-To: <4879FA52.4040706@gmail.com> References: <4879FA52.4040706@gmail.com> Message-ID: <487A2EE5.9020302@v.loewis.de> Nick Coghlan wrote: > The compilation step on this buildbot is failing because it can't delete > or overwrite any of the Python DLLs [1]. Is there any way to fix this > remotely, or at least to tell why kill_python isn't doing the trick? That is in the log: TerminateProcess failed: 5 where 5 is ERROR_ACCESS_DENIED. Now, *why* the system is refusing to terminate the process is difficult to say. It's a general Windows problem: the system doesn't support forced process termination in a reliable manner. In any case, Trent seems to have fixed the problem. > The number of 64-bit safeness > warnings being emitted by the current trunk is also fairly worrying) Do you have a specific one in mind? The ones truncating size_t/ssize_t should only matter when you actually do have data larger than 2GiB. Regards, Martin From martin at v.loewis.de Sun Jul 13 18:37:06 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 13 Jul 2008 18:37:06 +0200 Subject: [Python-Dev] Update bz2 library? In-Reply-To: <g5d18v$dav$1@ger.gmane.org> References: <g5d18v$dav$1@ger.gmane.org> Message-ID: <487A2F32.1000707@v.loewis.de> > I just noticed that the bz2lib version was updated to 1.0.5 in December > of 2007, for security reasons. I suspect it would be good to be sure > to ship this with 2.6 or 3.0. Python 2.6b1 shipped with bzip2 1.0.5. Regards, Martin From mhammond at skippinet.com.au Sun Jul 13 20:19:57 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Sun, 13 Jul 2008 21:19:57 +0300 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <200807140134.54676.steve@pearwood.info> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <loom.20080713T135317-118@post.gmane.org> <487A1526.6030401@voidspace.org.uk> <200807140134.54676.steve@pearwood.info> Message-ID: <004001c8e515$0fb08640$2f1192c0$@com.au> > On Mon, 14 Jul 2008 12:45:58 am Michael Foord wrote: > > > I tend to think of testing as action followed by assertion - > > I do this and this should have happened. Your tests usually define > > 'expected behaviour' rather than defining how your code won't fail... > > Who is the "your" that you are speaking for there? It isn't me. Although Michael unfortunately chose to use "Your", it is clear to me that the context implies he actually meant "My" (as in, his) > I resent having to turn the condition around into > a "assert if not" test just to satisfy others who are never going to > read my code. I wish people would go find another bike shed to > interfere with. Oh please, just get over it, and try to stick to the issue at hand rather than trying to score points or acting under the assumption that everyone is out to give you cause to "resent" things... As Steve said, this is getting silly... Mark From nd at perlig.de Sun Jul 13 20:54:52 2008 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Sun, 13 Jul 2008 20:54:52 +0200 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807121724l470e32agb23a018a5f83c757@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <200807130155.48346@news.perlig.de> <e6268af30807121724l470e32agb23a018a5f83c757@mail.gmail.com> Message-ID: <200807132054.52146@news.perlig.de> * Matt Giuca wrote: > > This POV is way too browser-centric... > > This is but one example. Note that I found web forms to be the least > clear-cut example of choosing an encoding. Most of the time applications > seem to be using UTF-8, and all the standards I have read are moving > towards specifying UTF-8 (from being unspecified). I've never seen a > standard specify or even recommend Latin-1. Ahem. The HTTP standard does ;-) > Where web forms are concerned, basically setting the form accept-charset > or the page charset is the *maximum amount* of control you have over the > encoding. As you say, it can be encoded by another page or the user can > override their settings. Then what can you do as the server? Nothing ... Guessing works pretty well in most of the cases. > Exactly. This is exactly my point - Latin-1 is arbitrary from a standards > point of view. It's just one of the many legacy encodings we'd like to > forget. The UTFs are the only options which support all languages, and > UTF-8 is the only ASCII-compatible (and therefore URI-compatible) > encoding. So we should aim to support that as the default. Latin-1 is not exactly arbitray. Besides being a charset - it maps one-to-one to octet values, hence it's commonly used to encode octets and is therefore a better fallback than every other encoding. > I agree. However if there *was* a proper standard we wouldn't have to > argue! "Most proper" and "should do" is the most confident we can be when > dealing with this standard, as there is no correct encoding. Well, the standard says, there are octets to be encoded. I find that proper enough. > Does anyone have a suggestion which will be more compatible with the rest > of the world than allowing the user to select an encoding, and defaulting > to "utf-8"? Default to latin-1 for decoding and utf-8 for encoding. This might be confusing though, so maybe you've asked the wrong question ;) nd -- Real programmers confuse Christmas and Halloween because DEC 25 = OCT 31. -- Unknown (found in ssl_engine_mutex.c) From rrr at ronadam.com Sun Jul 13 20:55:24 2008 From: rrr at ronadam.com (Ron Adam) Date: Sun, 13 Jul 2008 13:55:24 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <loom.20080713T130302-168@post.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> Message-ID: <487A4F9C.6090007@ronadam.com> Antoine Pitrou wrote: > Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes: >> That would better be written (preferring PEP 8 names) >> "fail_unless_equal". > > Which is still a double negative ("fail" and "unless" are both negative words). > >> That's another reason to avoid "assert" in the name: these methods >> *don't* necessarily use the 'assert' statement. > > But all those constructs (assert, assertEqual, etc.) raise the same exception > type named AssertionError which, if you care for naming consistency, is more > important than whether or not they are implemented in terms of each other. :-) Regarding: "all those constructs ... raise the same exception type named AssertionError.." As an experiment a while back (out of frustration), I created some tests that used more specific (local unittest module only) exceptions. The clarity of the failed errors and the mental effort to debug the problems was much improved for me. I sub-classed unittest.TestCase, and added two new methods and a few local and unique test-only exceptions. * test -> a test function to call * ref -> addition helpful debug info assertTestReturns(test, expect, ref="") raises on failure: Wrong_Result_Returned Unexpected_Exception_Raised assertTestRaises(test, expect, ref="") raises on failure: No_Exception_Raised Wrong_Exception_Raised These more specific exceptions (over plain AssertionError), allowed for more specific error reports. A testcase with an error would produce a standard python exception so you know instantly that you need to fix the test rather than the code being tested. Also the more specific exception code can better handle some cases where a wrong traceback would be returned. ie.. the test code traceback rather than the code being tested traceback. Is there any interest in going in this direction with unittests? Ron From rrr at ronadam.com Sun Jul 13 20:55:24 2008 From: rrr at ronadam.com (Ron Adam) Date: Sun, 13 Jul 2008 13:55:24 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <loom.20080713T130302-168@post.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> Message-ID: <487A4F9C.6090007@ronadam.com> Antoine Pitrou wrote: > Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes: >> That would better be written (preferring PEP 8 names) >> "fail_unless_equal". > > Which is still a double negative ("fail" and "unless" are both negative words). > >> That's another reason to avoid "assert" in the name: these methods >> *don't* necessarily use the 'assert' statement. > > But all those constructs (assert, assertEqual, etc.) raise the same exception > type named AssertionError which, if you care for naming consistency, is more > important than whether or not they are implemented in terms of each other. :-) Regarding: "all those constructs ... raise the same exception type named AssertionError.." As an experiment a while back (out of frustration), I created some tests that used more specific (local unittest module only) exceptions. The clarity of the failed errors and the mental effort to debug the problems was much improved for me. I sub-classed unittest.TestCase, and added two new methods and a few local and unique test-only exceptions. * test -> a test function to call * ref -> addition helpful debug info assertTestReturns(test, expect, ref="") raises on failure: Wrong_Result_Returned Unexpected_Exception_Raised assertTestRaises(test, expect, ref="") raises on failure: No_Exception_Raised Wrong_Exception_Raised These more specific exceptions (over plain AssertionError), allowed for more specific error reports. A testcase with an error would produce a standard python exception so you know instantly that you need to fix the test rather than the code being tested. Also the more specific exception code can better handle some cases where a wrong traceback would be returned. ie.. the test code traceback rather than the code being tested traceback. Is there any interest in going in this direction with unittests? Ron From josiah.carlson at gmail.com Sun Jul 13 21:15:29 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 13 Jul 2008 12:15:29 -0700 Subject: [Python-Dev] A proposed solution for Issue 502236: Asyncrhonousexceptions between threads In-Reply-To: <BLU136-W52FFE760D80D27784A462788920@phx.gbl> References: <BLU136-W35B41F13B127D65D8CA01188900@phx.gbl> <e6511dbf0807111951h25097659j116dbcf18e7b9642@mail.gmail.com> <F1962646D3B64642B7C9A06068EE1E6403F8903C@ex10.hostedexchange.local> <BLU136-W52FFE760D80D27784A462788920@phx.gbl> Message-ID: <e6511dbf0807131215y29f7ca7an9a72d7ada15528ea@mail.gmail.com> Andy, You had an idea, and it was a pretty good idea, but the practical considerations made it a nonstarter. That's ok, it happens all the time, among both new and seasoned Python developers alike. Search for "a case for top and bottom values" on Google for a bit of a laugh ;) . - Josiah On Sun, Jul 13, 2008 at 8:28 AM, Andy Scott <kirkshorts at hotmail.co.uk> wrote: > Thanks Guys > > So it seems as if I've made some pretty basic newbie mistakes then :-$ > > 1. Posting to the wrong list > 2. Posting about something that already has a work around > > oh well never mind - better luck next time ;-) > > Only one thing comes to my mind about the comments made saying that this is > no longer an issue: > > All of the solutions presented rely on the use of a third party library to > control the threading. It appears as if there have been no basic changes to > the thread package that ships with Python to resolve this. While it is > possible to set and read global variables &c to do something close - it > doesn't strike me as a nice solution as Python offers in so many other > places. > > However, I am willing to concede that as the issue has been dormant (or > appears to be) now for ~2years that it can not be a particularly burning > issue. So I'll go back and trawl the open issues for something else to help > my favourite language get better :-) > > > Andy > ----- > Brain chemistry is not just for Christmas > > [snipped the replies to cut down on re-quote spam - cos I kind of hate that] > ________________________________ > <snip> > > ________________________________ > Get fish-slapping on Messenger! Play Now From solipsis at pitrou.net Sun Jul 13 22:15:01 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 13 Jul 2008 20:15:01 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?unittest=27s_redundant_assertions=3A_asser?= =?utf-8?q?ts_vs=2E=09failIf/Unlesses?= References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> <loom.20080713T135317-118@post.gmane.org> <g5d8k2$ha$1@ger.gmane.org> Message-ID: <loom.20080713T200212-415@post.gmane.org> Steve Holden <steve <at> holdenweb.com> writes: > But Guido said: > > > I like using only the assertKeyword variants, removing assert_, fail*, > > and assertEquals. > > So we are now no longer discussing what color to paint the bike shed, we > are discussing how to describe the color. Let's stop. This is getting silly. It's certainly getting silly - the language is named Python after all! However, the whole thread started by someone opposing Guido's statement above, which at least explains (if not justifies) the origins of this particular instance of silliness... (and I've probably made my point explicitly enough, so I won't insist, even if I sometimes enjoy arguing on my spare time ;-)) cheers Antoine. From janssen at parc.com Sun Jul 13 22:36:06 2008 From: janssen at parc.com (Bill Janssen) Date: Sun, 13 Jul 2008 13:36:06 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> Message-ID: <08Jul13.133612pdt."58698"@synergy1.parc.xerox.com> > Ah there may be some confusion here. We're only dealing with str->str > transformations (which in Python 3 means Unicode strings). You can't put a > bytes in or get a bytes out of either of these functions. I suggested a > "quote_raw" and "unquote_raw" function which would let you do this. Ah, well, that's a problem. Clearly the unquote is str->bytes, while the quote is (bytes OR str)->str. You can't pass a Unicode string back as the result of unquote *without* passing in an encoding specifier, because the character set is application-specific. Bill From thebeatles42 at gmail.com Sun Jul 13 22:58:15 2008 From: thebeatles42 at gmail.com (Tom Mullins) Date: Sun, 13 Jul 2008 16:58:15 -0400 Subject: [Python-Dev] Exception tracebacks Message-ID: <cb7a3820807131358h590b353clb4d58b12452943a1@mail.gmail.com> I do not know where cleanup_traceback (in run.py) is called, or really anything about the inner workings of python, but there is a problem with sys.tracebacklimit. If it is set to one or zero, "** IDLE Internal Exception" is printed in the traceback for any exception, not internal ones. I think tracebacklimit is applied to the traceback before cleanup_traceback is called, but should be applied after. I imagine the solution is not that simple, and you may already be aware of the problem, but thanks anyway. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/193ff0ac/attachment.htm> From aahz at pythoncraft.com Sun Jul 13 23:42:09 2008 From: aahz at pythoncraft.com (Aahz) Date: Sun, 13 Jul 2008 14:42:09 -0700 Subject: [Python-Dev] Exception tracebacks In-Reply-To: <cb7a3820807131358h590b353clb4d58b12452943a1@mail.gmail.com> References: <cb7a3820807131358h590b353clb4d58b12452943a1@mail.gmail.com> Message-ID: <20080713214209.GA29334@panix.com> On Sun, Jul 13, 2008, Tom Mullins wrote: > > I do not know where cleanup_traceback (in run.py) is called, or really > anything about the inner workings of python, but there is a problem with > sys.tracebacklimit. If it is set to one or zero, "** IDLE Internal > Exception" is printed in the traceback for any exception, not internal ones. > I think tracebacklimit is applied to the traceback before cleanup_traceback > is called, but should be applied after. I imagine the solution is not that > simple, and you may already be aware of the problem, but thanks anyway. If you care about this issue, please file a report at bugs.python.org -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ I support the RKAB From ncoghlan at gmail.com Mon Jul 14 00:08:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 14 Jul 2008 08:08:05 +1000 Subject: [Python-Dev] AMD64-W2k8 buildbot wedged In-Reply-To: <487A2EE5.9020302@v.loewis.de> References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de> Message-ID: <487A7CC5.5070504@gmail.com> Martin v. L?wis wrote: > Nick Coghlan wrote: >> The number of 64-bit safeness >> warnings being emitted by the current trunk is also fairly worrying) > > Do you have a specific one in mind? The ones truncating size_t/ssize_t > should only matter when you actually do have data larger than 2GiB. Nothing specific, I just don't think I've ever actually looked at the output of a 64-bit build before and was a little surprised at the number of such warnings. I guess they were mostly in modules which are probably going to struggle with handling 2+ GiB chunks of data anyway. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From bignose+hates-spam at benfinney.id.au Mon Jul 14 00:28:41 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 08:28:41 +1000 Subject: [Python-Dev] PEP 8 conformance of unittest (was: unittest's redundant assertions: asserts vs. failIf/Unlesses) References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> Message-ID: <87y745bequ.fsf_-_@benfinney.id.au> "Guido van Rossum" <guido at python.org> writes: > Same here; let's tread carefully here and not change this with 3.0. > Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. > I like using only the assertKeyword variants, removing assert_, fail*, > and assertEquals. Are we to interpret the above ("? using only the assertKeyword variants, removing assert_, ?") as "we should actively remove names that are PEP 8 compatible from 'unittest', instead preferring names that go against PEP 8? I really hope I'm misinterpreting this and that there's another explanation. Preferably one that includes "we have a plan to make 'unittest' conform with the existing PEP 8". -- \ ?Pinky, are you pondering what I'm pondering?? ?Umm, I think | `\ so, Brain, but three men in a tub? Ooh, that's unsanitary!? | _o__) ?_Pinky and The Brain_ | Ben Finney From fuzzyman at voidspace.org.uk Mon Jul 14 00:35:30 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 13 Jul 2008 23:35:30 +0100 Subject: [Python-Dev] PEP 8 conformance of unittest In-Reply-To: <87y745bequ.fsf_-_@benfinney.id.au> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <87y745bequ.fsf_-_@benfinney.id.au> Message-ID: <487A8332.1030102@voidspace.org.uk> Ben Finney wrote: > "Guido van Rossum" <guido at python.org> writes: > > >> Same here; let's tread carefully here and not change this with 3.0. >> Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. >> I like using only the assertKeyword variants, removing assert_, fail*, >> and assertEquals. >> > > Are we to interpret the above ("? using only the assertKeyword > variants, removing assert_, ?") as "we should actively remove names > that are PEP 8 compatible from 'unittest', instead preferring names > that go against PEP 8? > > I really hope I'm misinterpreting this and that there's another > explanation. Preferably one that includes "we have a plan to make > 'unittest' conform with the existing PEP 8". > > "we have a plan to make 'unittest' conform with the existing PEP 8" - that was the outcome of the discussion. PEP 8'ification to begin in 2.7 / 3.1 with the deprecation of non-PEP8 compliant method names. There was some added functionality discussed, but it is too late to get this into 2.6 / 3.0. I volunteered to take it on, and have a record of the whole discussion. Unfortunately my writing commitment is going to keep me occupied until August - after which it will be one of my highest priorities. Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From bignose+hates-spam at benfinney.id.au Mon Jul 14 00:44:59 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 08:44:59 +1000 Subject: [Python-Dev] PEP 8 conformance of unittest References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <87y745bequ.fsf_-_@benfinney.id.au> <487A8332.1030102@voidspace.org.uk> Message-ID: <87lk05bdzo.fsf@benfinney.id.au> Michael Foord <fuzzyman at voidspace.org.uk> writes: > "we have a plan to make 'unittest' conform with the existing PEP 8" > - that was the outcome of the discussion. PEP 8'ification to begin > in 2.7 / 3.1 with the deprecation of non-PEP8 compliant method > names. Thanks, that's exactly the reassurance I was hoping for :-) > I volunteered to take it on, and have a record of the whole > discussion. Unfortunately my writing commitment is going to keep me > occupied until August - after which it will be one of my highest > priorities. I've contacted you yesterday regarding this, but to reiterate: This issue is of interest to me, and I'd like to help with it if I can. Please contact me privately if I can be of assistance, especially if I can help during your busy period. -- \ ?The trouble with the rat race is that even if you win, you're | `\ still a rat.? ?Jane Wagner, via Lily Tomlin | _o__) | Ben Finney From fuzzyman at voidspace.org.uk Mon Jul 14 00:51:44 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 13 Jul 2008 23:51:44 +0100 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <20080713093936.GA3623@benfinney.id.au> References: <20080713093936.GA3623@benfinney.id.au> Message-ID: <487A8700.8000200@voidspace.org.uk> Ben Finney wrote: > Howdy Michael, > > I'm interested in the changes you're proposing for Python's 'unittest' > module. I am (like, I suspect, many Python coders) maintaining my own > set of extensions to the module across many projects, so I'd really > like to see many of the improvements you discuss actually in the > standard library. > > What assistance can I offer to help on this issue? > > I intend to start working on them in August, after I have finished my current writing commitments. The full list of changes proposed (feel free to start - but ping me or the list) and not shot down was something like: Documenting that the assert method names are to be preferred over the 'FailUnless' names (this stirred up some controversy this weekend so should probably not happen). Adding the following new asserts: assertIn (member, container, msg=None) assertNotIn (member, container, msg=None) assertIs (first, second, msg=None) assertNotIs (first, second, msg=None) assertRaisesWithMessage (exc_class, message, callable, *args, **keywargs) A simple 'RunTests' function that takes a collection of modules, test suites or test cases and runs them with TextTestRunner - passing on keyword arguments to the test runner. This makes running a test suite easier - once you have collected all your test cases together you can just pass them to this function so long as you are happy with the default runner (possibly allowing an alternative runner class to be provided as a keyword argument). Make the error messages for "assertEquals" and "assertNotEquals" more useful - showing the objects that compare incorrectly even if an explicit message is passed in. Additionally, when comparing lists and tuples that are the same length show the members (and indices?) that were different. Possibly even providing a diff in the case of comparing strings (we have an implementation of this at work). assertLessThan assertGreaterThan assertLessThanOrEquals assertGreaterThanOrEquals Guido suggested various variants on assertEquals: assertListEqual(self, list1, list2, msg=None): assertDictEqual(self, d1, d2, msg=None): assertMultiLineEqual(self, first, second, msg=Non In my opinion these can all be provided by improving the messages from assertEquals and don't require new methods. assertSameElements(self, expected_seq, actual_seq, msg=None): I usually achieve this with: assertEquals(set(actual), set(expected)) A convenience method might be nice, but I'm worried about the API growing in an unbounded way. Other suggestions that weren't controversial but I might not get to: assertRaisesWithMessage taking a regex to match the error message expect methods that collect failures and report at the end of the test (allowing an individual test method to raise several errors without stopping) assertIsInstance and assertIsSubclass Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From bignose+hates-spam at benfinney.id.au Mon Jul 14 01:20:23 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 09:20:23 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> Message-ID: <87hcatbcco.fsf@benfinney.id.au> Michael Foord <fuzzyman at voidspace.org.uk> writes: > Adding the following new asserts: > > assertIn (member, container, msg=None) > assertNotIn (member, container, msg=None) > assertIs (first, second, msg=None) > assertNotIs (first, second, msg=None) > assertRaisesWithMessage (exc_class, message, callable, *args, > **keywargs) [?] > assertLessThan > assertGreaterThan > assertLessThanOrEquals > assertGreaterThanOrEquals [?] > assertListEqual(self, list1, list2, msg=None): > assertDictEqual(self, d1, d2, msg=None): > assertMultiLineEqual(self, first, second, msg=Non [?] > assertSameElements(self, expected_seq, actual_seq, msg=None): All these are new, so there is no existing expectation of these names from users of the standard library 'unittest' module (i.e. no backward-compatibility concern since these are new methods). If we're planning to deprecate the existing non-PEP-8 names in 2.7 and 3.1, why would we introduce new names that are non-PEP-8? Wouldn't it be better to add these as PEP-8 compatible names in the first instance? -- \ ?You've got the brain of a four-year-old boy, and I'll bet he | `\ was glad to get rid of it.? ?Groucho Marx | _o__) | Ben Finney From steve at holdenweb.com Mon Jul 14 01:27:55 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 13 Jul 2008 19:27:55 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487A8700.8000200@voidspace.org.uk> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> Message-ID: <487A8F7B.1090901@holdenweb.com> Michael Foord wrote: > Ben Finney wrote: >> Howdy Michael, >> >> I'm interested in the changes you're proposing for Python's 'unittest' >> module. I am (like, I suspect, many Python coders) maintaining my own >> set of extensions to the module across many projects, so I'd really >> like to see many of the improvements you discuss actually in the >> standard library. >> >> What assistance can I offer to help on this issue? >> >> > I intend to start working on them in August, after I have finished my > current writing commitments. > > The full list of changes proposed (feel free to start - but ping me or > the list) and not shot down was something like: > > Documenting that the assert method names are to be preferred over the > 'FailUnless' names (this stirred up some controversy this weekend so > should probably not happen). > > > Adding the following new asserts: > > assertIn (member, container, msg=None) > assertNotIn (member, container, msg=None) > assertIs (first, second, msg=None) > assertNotIs (first, second, msg=None) Please, let's call this one "assertIsNot". I know it's valid Python to say if a not is b: but it's a much less natural way of expressing the condition, and (for all I know) might even introduce an extra negation operation. "is not" is, I believe, treated as a single operator. > [...] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Mon Jul 14 01:27:55 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 13 Jul 2008 19:27:55 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487A8700.8000200@voidspace.org.uk> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> Message-ID: <487A8F7B.1090901@holdenweb.com> Michael Foord wrote: > Ben Finney wrote: >> Howdy Michael, >> >> I'm interested in the changes you're proposing for Python's 'unittest' >> module. I am (like, I suspect, many Python coders) maintaining my own >> set of extensions to the module across many projects, so I'd really >> like to see many of the improvements you discuss actually in the >> standard library. >> >> What assistance can I offer to help on this issue? >> >> > I intend to start working on them in August, after I have finished my > current writing commitments. > > The full list of changes proposed (feel free to start - but ping me or > the list) and not shot down was something like: > > Documenting that the assert method names are to be preferred over the > 'FailUnless' names (this stirred up some controversy this weekend so > should probably not happen). > > > Adding the following new asserts: > > assertIn (member, container, msg=None) > assertNotIn (member, container, msg=None) > assertIs (first, second, msg=None) > assertNotIs (first, second, msg=None) Please, let's call this one "assertIsNot". I know it's valid Python to say if a not is b: but it's a much less natural way of expressing the condition, and (for all I know) might even introduce an extra negation operation. "is not" is, I believe, treated as a single operator. > [...] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From stephen at xemacs.org Mon Jul 14 01:51:27 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 14 Jul 2008 08:51:27 +0900 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <g5d03p$air$1@ger.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> Message-ID: <87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp> Steve Holden writes: > "Fail" isn't a negative. As Guido said, it's a description of the test > behavior under particular circumstances. This is not true, however. "Fail" is a description of a potentailly very large class of behaviors. Knowing that the test failed does not tell you which of the failure behaviors occurred, only that the success behavior did not occur. The analogy to the fact that True != not not 10 is telling, I think. From steve at pearwood.info Mon Jul 14 01:46:03 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 14 Jul 2008 09:46:03 +1000 Subject: [Python-Dev] =?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?= =?iso-8859-1?q?asserts_vs=2E=09failIf/Unlesses?= In-Reply-To: <004001c8e515$0fb08640$2f1192c0$@com.au> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <200807140134.54676.steve@pearwood.info> <004001c8e515$0fb08640$2f1192c0$@com.au> Message-ID: <200807140946.05034.steve@pearwood.info> On Mon, 14 Jul 2008 04:19:57 am Mark Hammond wrote: > try to stick to the issue at hand I'm sorry, did I reply to the wrong message? I thought this was part of a thread titled "unittest's redundant assertions: asserts vs. failIf/Unlesses". What *is* the issue at hand if not the question of positive assertions versus fail if/unless? > As Steve said, this is getting silly... It certainly is. Python may be Guido's language, and if he wants to use his dictatorial powers to say that tests must be written as positive assertions because that's the way he likes it, that's his prerogative. But let's not pretend that this particular bike shed colour has any objectively rational reason, or that the change won't force some people to have to change the way they think about tests. -- Steven From steve at holdenweb.com Mon Jul 14 01:54:16 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 13 Jul 2008 19:54:16 -0400 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <200807140946.05034.steve@pearwood.info> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <200807140134.54676.steve@pearwood.info> <004001c8e515$0fb08640$2f1192c0$@com.au> <200807140946.05034.steve@pearwood.info> Message-ID: <g5e4ja$8rg$1@ger.gmane.org> Steven D'Aprano wrote: > On Mon, 14 Jul 2008 04:19:57 am Mark Hammond wrote: > >> try to stick to the issue at hand > > I'm sorry, did I reply to the wrong message? I thought this was part of > a thread titled "unittest's redundant assertions: asserts vs. > failIf/Unlesses". What *is* the issue at hand if not the question of > positive assertions versus fail if/unless? > >> As Steve said, this is getting silly... > > It certainly is. > > Python may be Guido's language, and if he wants to use his dictatorial > powers to say that tests must be written as positive assertions because > that's the way he likes it, that's his prerogative. But let's not > pretend that this particular bike shed colour has any objectively > rational reason, or that the change won't force some people to have to > change the way they think about tests. > But sometimes even the wrong decision is better than no decision, and I suspect most if us can agree that it's better if everyone thinks about tests the same way. Otherwise test code is difficult to understand for some of its user population. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From bignose+hates-spam at benfinney.id.au Mon Jul 14 02:06:48 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 10:06:48 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> Message-ID: <87d4lhba7b.fsf@benfinney.id.au> Steve Holden <steve at holdenweb.com> writes: > Michael Foord wrote: > > Adding the following new asserts: > > > > assertIn (member, container, msg=None) > > assertNotIn (member, container, msg=None) > > assertIs (first, second, msg=None) > > assertNotIs (first, second, msg=None) > > Please, let's call this one "assertIsNot". I know it's valid Python > to say > > if a not is b: > > but it's a much less natural way of expressing the condition, and > (for all I know) might even introduce an extra negation operation. > "is not" is, I believe, treated as a single operator. Dang. You're exactly right. The problem is, that makes it quite inconsistent with other "not" uses (such as "assert_not_equal", "assert_not_in", etc.) I would really prefer that all these "not" uses be gramatically consistent for predictability. Is this a case where "assert_is_not" should exist alongside "assert_not_is"? I know that part of the goal here is to have "preferably only one obvious way to do it", but I can see *both* those names as "the obvious way to do it". Is this an instance where the "preferably" clause must be exercised in the negative? -- \ ?Every sentence I utter must be understood not as an | `\ affirmation, but as a question.? ?Niels Bohr | _o__) | Ben Finney From mithrandi-python-dev at mithrandi.za.net Mon Jul 14 01:41:23 2008 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Mon, 14 Jul 2008 01:41:23 +0200 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> <87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080713234123.GA11170@mithrandi.net> * Stephen J. Turnbull <stephen at xemacs.org> [2008-07-14 08:51:27 +0900]: > The analogy to the fact that True != not not 10 is telling, I think. What? >>> True == (not not 10) True -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: <http://mail.python.org/pipermail/python-dev/attachments/20080714/94860626/attachment.pgp> From tim.peters at gmail.com Mon Jul 14 02:32:12 2008 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 13 Jul 2008 20:32:12 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487A8F7B.1090901@holdenweb.com> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> Message-ID: <1f7befae0807131732x54f5ed4em23b42b5d5a23dcad@mail.gmail.com> [Michael Foord] >> ... >> Adding the following new asserts: >> >> ... >> assertNotIs (first, second, msg=None) [Steve Holden] > Please, let's call this one "assertIsNot". +1 > I know it's valid Python to say > > if a not is b: Nope, that's a syntax error. > but it's a much less natural way of expressing the condition, and (for all I > know) might even introduce an extra negation operation. "is not" is, I > believe, treated as a single operator. "is not" and "not in" are both binary infix operators, not to be confused with the distinct use of "not" on its own as a unary prefix operator. "not is" and "in not" are both gibberish. >>> 1 is not 2 True >>> 1 is (not 2) False >>> 1 not is 2 SyntaxError: invalid syntax >>> 1 not in [2] True >>> 1 in not [2] SyntaxError: invalid syntax >>> 1 in (not [2]) Traceback (most recent call last): ... TypeError: argument of type 'bool' is not iterable From exarkun at divmod.com Mon Jul 14 02:58:48 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sun, 13 Jul 2008 20:58:48 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487A8700.8000200@voidspace.org.uk> Message-ID: <20080714005848.4714.1762448186.divmod.quotient.20097@ohm> On Sun, 13 Jul 2008 23:51:44 +0100, Michael Foord <fuzzyman at voidspace.org.uk> wrote: >Ben Finney wrote: >>Howdy Michael, >> >>I'm interested in the changes you're proposing for Python's 'unittest' >>module. I am (like, I suspect, many Python coders) maintaining my own set >>of extensions to the module across many projects, so I'd really like to see >>many of the improvements you discuss actually in the standard library. >> >>What assistance can I offer to help on this issue? >> >> >I intend to start working on them in August, after I have finished my >current writing commitments. > >The full list of changes proposed (feel free to start - but ping me or the >list) and not shot down was something like: > >Documenting that the assert method names are to be preferred over the >'FailUnless' names (this stirred up some controversy this weekend so should >probably not happen). > > >Adding the following new asserts: > > assertIn (member, container, msg=None) > assertNotIn (member, container, msg=None) > assertIs (first, second, msg=None) > assertNotIs (first, second, msg=None) > assertRaisesWithMessage (exc_class, message, callable, *args, >**keywargs) > Several of these are implemented in other libraries (Twisted, at least). You might save some time by grabbing them and their unit tests, rather than re-implementing them. Twisted calls `assertIs? `assertIdentical?, by the way. > [snip] > >Other suggestions that weren't controversial but I might not get to: > > assertRaisesWithMessage taking a regex to match the error message > Actually, I remember that someone raised an object to this as being not as flexible as some might want - an objection I agree with. Perhaps that was overruled, but I didn't want this to slip by as "not controversial". > expect methods that collect failures and report at the end of the test >(allowing an individual test method to raise several errors without >stopping) > > assertIsInstance and assertIsSubclass > The former of these is also in Twisted already, if you want to copy it. Jean-Paul From steve at holdenweb.com Mon Jul 14 03:06:10 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 13 Jul 2008 21:06:10 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <1f7befae0807131732x54f5ed4em23b42b5d5a23dcad@mail.gmail.com> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <1f7befae0807131732x54f5ed4em23b42b5d5a23dcad@mail.gmail.com> Message-ID: <g5e8q5$hjl$1@ger.gmane.org> Tim Peters wrote: > [Michael Foord] >>> ... >>> Adding the following new asserts: >>> >>> ... >>> assertNotIs (first, second, msg=None) > > [Steve Holden] >> Please, let's call this one "assertIsNot". > > +1 > >> I know it's valid Python to say >> >> if a not is b: > > Nope, that's a syntax error. > Rats, naturally I was thinking of "if not (a is b):" >> but it's a much less natural way of expressing the condition, and (for all I >> know) might even introduce an extra negation operation. "is not" is, I >> believe, treated as a single operator. > > "is not" and "not in" are both binary infix operators, not to be > confused with the distinct use of "not" on its own as a unary prefix > operator. "not is" and "in not" are both gibberish. > >>>> 1 is not 2 > True >>>> 1 is (not 2) > False >>>> 1 not is 2 > SyntaxError: invalid syntax > >>>> 1 not in [2] > True >>>> 1 in not [2] > SyntaxError: invalid syntax >>>> 1 in (not [2]) > Traceback (most recent call last): > ... > TypeError: argument of type 'bool' is not iterable regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From greg.ewing at canterbury.ac.nz Mon Jul 14 03:05:34 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 14 Jul 2008 13:05:34 +1200 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <87prpic65w.fsf@benfinney.id.au> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> Message-ID: <487AA65E.6080007@canterbury.ac.nz> Ben Finney wrote: > That's another reason to avoid "assert" in the name: these methods > *don't* necessarily use the 'assert' statement. Avoiding the > implication that they do use that is a good thing. Perhaps some positive alternative such as "verify" could be used instead. -- Greg From greg.ewing at canterbury.ac.nz Mon Jul 14 03:14:28 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 14 Jul 2008 13:14:28 +1200 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <g5d03p$air$1@ger.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> Message-ID: <487AA874.2010205@canterbury.ac.nz> Steve Holden wrote: > "Fail" isn't a negative. That depends on what you're trying to find out by reading the code. If you're trying to find out under what conditions the test succeeds, then it succeeds if it doesn't fail, so you have a negative. Whichever convention is chosen, there will be situations in which you want to mentally negate it. If you start with a positive, the mental negation produces a single negative. If you start with a negative, the mental negation produces a double negative. Therefore it is less confusing overall to start with as few negatives as possible. -- Greg From matt.giuca at gmail.com Mon Jul 14 05:22:23 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Mon, 14 Jul 2008 13:22:23 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <8249917582531821653@unknownmsgid> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> Message-ID: <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> On Mon, Jul 14, 2008 at 4:54 AM, Andr? Malo <nd at perlig.de> wrote: > > Ahem. The HTTP standard does ;-) > Really? Can you include a quotation please? The HTTP standard talks a lot about ISO-8859-1 (Latin-1) in terms of actually raw encoded bytes, but not in terms of URI percent-encoding (a different issue) as far as I can tell. > > > Where web forms are concerned, basically setting the form accept-charset > > or the page charset is the *maximum amount* of control you have over the > > encoding. As you say, it can be encoded by another page or the user can > > override their settings. Then what can you do as the server? Nothing ... > > Guessing works pretty well in most of the cases. > Are you suggesting that urllib.unquote guess the encoding? It could do that but it would make things rather unpredictable. I think if this was an application (such as a web browser), then guessing is OK. But this is a library function. Library functions should not make arbitrary decisions; they should be well-specified. Latin-1 is not exactly arbitray. Besides being a charset - it maps > one-to-one to octet values, hence it's commonly used to encode octets and > is therefore a better fallback than every other encoding. > True. So the only advantage I see to the current implementation is that if you really want to, you can take the Latin-1-decoded URI (from unquote) and explicitly encode it as Latin-1 and then decode it again as whatever encoding you want. But that would be a hack, would it not? I'd prefer if the library didn't require a hack just to get the extremely common use case (UTF-8). > > > I agree. However if there *was* a proper standard we wouldn't have to > > argue! "Most proper" and "should do" is the most confident we can be when > > dealing with this standard, as there is no correct encoding. > > Well, the standard says, there are octets to be encoded. I find that proper > enough. Yes but unfortunately we aren't talking about octets any more in Python 3, but characters. If we're going to follow the standard and encode octets, then we should be accepting (for quote) and returning (for unquote) bytes objects, not strings. But as that's going to break most existing code and be extremely confusing, I think it's best we try and solve this problem for Unicode strings. > > Does anyone have a suggestion which will be more compatible with the rest > > of the world than allowing the user to select an encoding, and defaulting > > to "utf-8"? > > Default to latin-1 for decoding and utf-8 for encoding. This might be > confusing though, so maybe you've asked the wrong question ;) > :o that would break so so much existing code, not to mention being horribly inconsistent and confusing. Having said that, that's almost what the current behaviour is (quote uses Latin-1 for characters < 256, and UTF-8 for characters above; unquote uses Latin-1). Again I bring up the http server example. If you go to a directory, create a file with a name such as '??', and then run this code in Python 3.0 from that directory: import http.server s = http.server.HTTPServer(('',8000), http.server.SimpleHTTPRequestHandler) s.serve_forever() You'll see the file in the directory listing - its HTML will be <a href="%E6%BC%A2%E5%AD%97">??</a>. But if you click it, you get a 404 because the server will look for the file named unquote("%E6%BC%A2%E5%AD%97") = '????\xad\x97'. If you apply my patch (patch5) *everything* *just* *works*. On Mon, Jul 14, 2008 at 6:36 AM, Bill Janssen <janssen at parc.com> wrote: > > Ah there may be some confusion here. We're only dealing with str->str > > transformations (which in Python 3 means Unicode strings). You can't put > a > > bytes in or get a bytes out of either of these functions. I suggested a > > "quote_raw" and "unquote_raw" function which would let you do this. > > Ah, well, that's a problem. Clearly the unquote is str->bytes, while > the quote is (bytes OR str)->str. > OK so for quote, you're suggesting that we accept either a bytes or a str object. That sounds quite reasonable (though neither the unpatched or patched versions accept a bytes at the moment). I'd simply change the code in quote (from patch5) to do this: if isinstance(s, str): s = s.encode(encoding, errors) .... res = map(quoter, s) Now you get this behaviour by default (which may appear confusing but I'd argue correct given the different semantics of 'h\xfcllo' and b'h\xfcllo'): >>> urllib.parse.quote(b'h\xfcllo') 'h%FCllo' # Directly-encoded octets >>> urllib.parse.quote('h\xfcllo') 'h%C3%BCllo' # UTF-8 encoded string, then encoded octets Clearly the unquote is str->bytes, <snip> You can't pass a Unicode string > back > as the result of unquote *without* passing in an encoding specifier, > because the character set is application-specific. > So for unquote you're suggesting that it always return a bytes object UNLESS an encoding is specified? As in: >>> urllib.parse.unquote('h%C3%BCllo') b'h\xc3\xbcllo' I would object to that on two grounds. Firstly, I wouldn't expect or desire a bytes object. The vast majority of uses for unquote will be to get a character string out, not bytes. Secondly, there is a mountain of code (including about 12 modules in the standard library) which call unquote and don't give the user the encoding option, so it's best if we pick a default that is what the majority of users will expect. I argue that that's UTF-8. I'd prefer having a separate unquote_raw function which is str->bytes, and the unquote function performs the same role as it always have, which is str->str. But I agree on quote, I think it can be (bytes OR str)->str. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080714/e2c64a4a/attachment-0001.htm> From schmir at gmail.com Mon Jul 14 07:03:55 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Mon, 14 Jul 2008 07:03:55 +0200 Subject: [Python-Dev] AMD64-W2k8 buildbot wedged In-Reply-To: <487A2EE5.9020302@v.loewis.de> References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de> Message-ID: <932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com> On Sun, Jul 13, 2008 at 6:35 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote: > Nick Coghlan wrote: >> The compilation step on this buildbot is failing because it can't delete >> or overwrite any of the Python DLLs [1]. Is there any way to fix this >> remotely, or at least to tell why kill_python isn't doing the trick? > > That is in the log: > > TerminateProcess failed: 5 > > where 5 is ERROR_ACCESS_DENIED. Now, *why* the system is refusing to > terminate the process is difficult to say. It's a general Windows > problem: the system doesn't support forced process termination in a > reliable manner. > > In any case, Trent seems to have fixed the problem. > >> The number of 64-bit safeness >> warnings being emitted by the current trunk is also fairly worrying) > > Do you have a specific one in mind? The ones truncating size_t/ssize_t > should only matter when you actually do have data larger than 2GiB. > http://bugs.python.org/issue3026 comes to mind. And I would rather use a little bit different wording: The ones truncating size_t/ssize_t do matter, unless you know in advance that you will always deal with data lesser than 2GiB. Regards, - Ralf From martin at v.loewis.de Mon Jul 14 07:16:20 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 14 Jul 2008 07:16:20 +0200 Subject: [Python-Dev] AMD64-W2k8 buildbot wedged In-Reply-To: <932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com> References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de> <932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com> Message-ID: <487AE124.1030301@v.loewis.de> > http://bugs.python.org/issue3026 comes to mind. > > And I would rather use a little bit different wording: The ones > truncating size_t/ssize_t do matter, unless you know in advance that > you will always deal with data lesser than 2GiB. I thought Nick's comment was in the context of the buildbots hanging in the multiprocessing tests, which I know has only data smaller than 2GiB. Regards, Martin From stephen at xemacs.org Mon Jul 14 08:27:40 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 14 Jul 2008 15:27:40 +0900 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <20080713234123.GA11170@mithrandi.net> References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com> <8763radte5.fsf@benfinney.id.au> <20080713114653.GE9654@steerpike.home.puzzling.org> <loom.20080713T121738-271@post.gmane.org> <87prpic65w.fsf@benfinney.id.au> <loom.20080713T130302-168@post.gmane.org> <g5d03p$air$1@ger.gmane.org> <87prph5on4.fsf@uwakimon.sk.tsukuba.ac.jp> <20080713234123.GA11170@mithrandi.net> Message-ID: <87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp> Tristan Seligmann writes: > * Stephen J. Turnbull <stephen at xemacs.org> [2008-07-14 08:51:27 +0900]: > > > The analogy to the fact that True != not not 10 is telling, I think. > > What? > > >>> True == (not not 10) > True FWIW, I meant 10 != not not 10. From ncoghlan at gmail.com Mon Jul 14 11:14:20 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 14 Jul 2008 19:14:20 +1000 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <87d4lhba7b.fsf@benfinney.id.au> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> Message-ID: <487B18EC.6040104@gmail.com> Ben Finney wrote: > Steve Holden <steve at holdenweb.com> writes: > >> Michael Foord wrote: >>> Adding the following new asserts: >>> >>> assertIn (member, container, msg=None) >>> assertNotIn (member, container, msg=None) >>> assertIs (first, second, msg=None) >>> assertNotIs (first, second, msg=None) >> Please, let's call this one "assertIsNot". I know it's valid Python >> to say >> >> if a not is b: >> >> but it's a much less natural way of expressing the condition, and >> (for all I know) might even introduce an extra negation operation. >> "is not" is, I believe, treated as a single operator. > > Dang. You're exactly right. > > The problem is, that makes it quite inconsistent with other "not" uses > (such as "assert_not_equal", "assert_not_in", etc.) I would really > prefer that all these "not" uses be gramatically consistent for > predictability. Is this a case where "assert_is_not" should exist > alongside "assert_not_is"? If we can flip the word order in the language syntax, we can sure as heck flip it in a method name :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Mon Jul 14 11:16:33 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 14 Jul 2008 19:16:33 +1000 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487A8700.8000200@voidspace.org.uk> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> Message-ID: <487B1971.105@gmail.com> Michael Foord wrote: > Ben Finney wrote: >> Howdy Michael, >> >> I'm interested in the changes you're proposing for Python's 'unittest' >> module. I am (like, I suspect, many Python coders) maintaining my own >> set of extensions to the module across many projects, so I'd really >> like to see many of the improvements you discuss actually in the >> standard library. >> >> What assistance can I offer to help on this issue? >> >> > I intend to start working on them in August, after I have finished my > current writing commitments. Would it be worth Ben collating your current notes into a draft PEP targeting 2.7/3.1? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From hrvoje.niksic at avl.com Mon Jul 14 10:50:50 2008 From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=) Date: Mon, 14 Jul 2008 10:50:50 +0200 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <4856CCBC.5040003@egenix.com> References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com> <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <48551A56.1020704@vector-seven.com> <g338l9$be$1@ger.gmane.org> <485529EF.3080209@vector-seven.com> <g33a2m$46q$1@ger.gmane.org> <4856CCBC.5040003@egenix.com> Message-ID: <1216025450.8943.4.camel@localhost> On Mon, 2008-06-16 at 22:27 +0200, M.-A. Lemburg wrote: > On 2008-06-15 16:47, Georg Brandl wrote: > > Thomas Lee schrieb: > >> Georg Brandl wrote: > >>> Remember that it must still be possible to write (in 2.6) > >>> > >>> True = 0 > >>> assert not True > >> > >> Ah of course. Looks like I should just avoid optimizations of > >> Name("True") and Name("False") all together. That's a shame! > > > > We can of course decide to make assignment to True and False > > illegal in 2.7 :) > > Raising a run-time exception would be fine, but not a SyntaxError at > compile time - this would effectively make it impossible to keep > code compatible to Python 2.1. Maybe it wouldn't. Something like: try: True, False except NameError: globals()['True'] = 1 globals()['False'] should still work for compatibility. It would break *existing* code that tries to be compatible with old releases, but that is unavoidable in making something illegal that was previously legal. In this case, the price is justified by being able to optimize access to None, True, and False without having to resort to dirty tricks such as "none=None" (previously None=None) in functions. It should also enable the compiler to optimize "while True:" the way it optimizes "while 1:". From ncoghlan at gmail.com Mon Jul 14 11:22:37 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 14 Jul 2008 19:22:37 +1000 Subject: [Python-Dev] AMD64-W2k8 buildbot wedged In-Reply-To: <487AE124.1030301@v.loewis.de> References: <4879FA52.4040706@gmail.com> <487A2EE5.9020302@v.loewis.de> <932f8baf0807132203lf83aa30u3e35eaa87ac3738f@mail.gmail.com> <487AE124.1030301@v.loewis.de> Message-ID: <487B1ADD.6030109@gmail.com> Martin v. L?wis wrote: >> http://bugs.python.org/issue3026 comes to mind. >> >> And I would rather use a little bit different wording: The ones >> truncating size_t/ssize_t do matter, unless you know in advance that >> you will always deal with data lesser than 2GiB. > > I thought Nick's comment was in the context of the buildbots hanging > in the multiprocessing tests, which I know has only data smaller than > 2GiB. Ah, sorry about the confusion - my chain of thought was a little more convoluted than that. The wedged buildbot caused the compile to fail after a checkin of mine, which lead to me looking at that buildbot's compile log, which had all sorts of warnings which I had never seen before because my development machine is a 32-bit Linux box. Given that I thought all those warnings had been cleared out when Py_ssize_t was first added, I was a little surprised by the quantity of them. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From bignose+hates-spam at benfinney.id.au Mon Jul 14 12:05:22 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 20:05:22 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> Message-ID: <874p6sbx25.fsf@benfinney.id.au> Nick Coghlan <ncoghlan at gmail.com> writes: > Ben Finney wrote: > > The problem is, that makes it quite inconsistent with other "not" > > uses (such as "assert_not_equal", "assert_not_in", etc.) I would > > really prefer that all these "not" uses be gramatically consistent > > for predictability. Is this a case where "assert_is_not" should > > exist alongside "assert_not_is"? > > If we can flip the word order in the language syntax, we can sure as > heck flip it in a method name :) To be clear, I take it you're in favour of the following names (with no aliases): assert_equal assert_not_equal assert_is assert_is_not assert_in assert_not_in assert_almost_equal assert_not_almost_equal and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by the others, in the interest of matching Python's 'is not' grammar. -- \ ?Instead of having ?answers? on a math test, they should just | `\ call them ?impressions?, and if you got a different | _o__) ?impression?, so what, can't we all be brothers?? ?Jack Handey | Ben Finney From steve at holdenweb.com Mon Jul 14 14:45:06 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 14 Jul 2008 08:45:06 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <874p6sbx25.fsf@benfinney.id.au> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au> Message-ID: <g5fhol$411$1@ger.gmane.org> Ben Finney wrote: > Nick Coghlan <ncoghlan at gmail.com> writes: > >> Ben Finney wrote: >>> The problem is, that makes it quite inconsistent with other "not" >>> uses (such as "assert_not_equal", "assert_not_in", etc.) I would >>> really prefer that all these "not" uses be gramatically consistent >>> for predictability. Is this a case where "assert_is_not" should >>> exist alongside "assert_not_is"? >> If we can flip the word order in the language syntax, we can sure as >> heck flip it in a method name :) > > To be clear, I take it you're in favour of the following names (with > no aliases): > > assert_equal assert_not_equal > assert_is assert_is_not > assert_in assert_not_in > assert_almost_equal assert_not_almost_equal > > and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by > the others, in the interest of matching Python's 'is not' grammar. > Well, I'd have said "in the interest of reading correctly in English", though I have to acknowledge this may not be an issue for many Python users whose first language not is English. "assert_not_is" is just dissonant to my ears. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From bignose+hates-spam at benfinney.id.au Mon Jul 14 15:17:32 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 23:17:32 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487B1971.105@gmail.com> Message-ID: <87zloka9lf.fsf@benfinney.id.au> Nick Coghlan <ncoghlan at gmail.com> writes: > Would it be worth Ben collating your current notes into a draft PEP > targeting 2.7/3.1? I'll do it and we'll find out. -- \ ?A fine is a tax for doing wrong. A tax is a fine for doing | `\ well.? ?anonymous | _o__) | Ben Finney From ncoghlan at gmail.com Mon Jul 14 15:17:59 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 14 Jul 2008 23:17:59 +1000 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <g5fhol$411$1@ger.gmane.org> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org> Message-ID: <487B5207.2020904@gmail.com> Steve Holden wrote: > Ben Finney wrote: >> To be clear, I take it you're in favour of the following names (with >> no aliases): >> >> assert_equal assert_not_equal >> assert_is assert_is_not >> assert_in assert_not_in >> assert_almost_equal assert_not_almost_equal >> >> and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by >> the others, in the interest of matching Python's 'is not' grammar. >> > Well, I'd have said "in the interest of reading correctly in English", > though I have to acknowledge this may not be an issue for many Python > users whose first language not is English. "assert_not_is" is just > dissonant to my ears. The two reasons aren't that far apart, given that Python's grammar uses "is not" because it makes more sense in English. One thing to remember is that the word 'is' is actually implied in all of the contracted phrases above other than those already including it explicitly. "x is equal to y" "x is not equal to y" "x is y" "x is not y" "x is in y" "x is not in y" "x is almost equal to y" "x is not almost equal to y" As for which phrasing I personally prefer, unit tests and method names are areas where I'm quite happy to paint the bike shed the same colour as the house :) Cheers, Nick. P.S. Deciphering that somewhat strained metaphor: I don't have a strong preference with regards to the unit test method names. While I tend to go with the assert* variants when left to my own devices, I have no problem sticking to the fail* variants when updating a test that uses them. Camel-case vs underscores in method names isn't something that particularly worries me either. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From bignose+hates-spam at benfinney.id.au Mon Jul 14 15:41:19 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 23:41:19 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org> Message-ID: <87r69wa8hs.fsf@benfinney.id.au> Steve Holden <steve at holdenweb.com> writes: > Ben Finney wrote: > > and so on; i.e. that 'assert_is_not' breaks the obvious pattern > > set by the others, in the interest of matching Python's 'is not' > > grammar. > > Well, I'd have said "in the interest of reading correctly in English", > though I have to acknowledge this may not be an issue for many Python > users whose first language not is English. "assert_not_is" is just > dissonant to my ears. I'd count this as another (minor) point in favour of making the 'fail*' methods canonical: the names are consistent *and* gramatically sensible: fail_if_equal fail_unless_equal fail_if_is fail_unless_is fail_if_in fail_unless_in fail_if_almost_equal fail_unless_almost_equal -- \ ?We are not gonna be great; we are not gonna be amazing; we are | `\ gonna be *amazingly* amazing!? ?Zaphod Beeblebrox, _The | _o__) Hitch-Hiker's Guide To The Galaxy_, Douglas Adams | Ben Finney From bignose+hates-spam at benfinney.id.au Mon Jul 14 15:43:14 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 23:43:14 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> Message-ID: <87mykka8el.fsf@benfinney.id.au> Michael Foord <fuzzyman at voidspace.org.uk> writes: > The full list of changes proposed (feel free to start - but ping me or > the list) and not shot down was something like: [?] Thanks. I'm working these into another draft PEP that I hope to have up in a day or two. -- \ ?[W]e are still the first generation of users, and for all that | `\ we may have invented the net, we still don't really get it.? | _o__) ?Douglas Adams | Ben Finney From python at rcn.com Mon Jul 14 15:59:22 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Jul 2008 06:59:22 -0700 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> Message-ID: <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> > Michael Foord <fuzzyman at voidspace.org.uk> writes: > >> The full list of changes proposed (feel free to start - but ping me or >> the list) and not shot down was something like: > [?] > > Thanks. I'm working these into another draft PEP that I hope to have > up in a day or two. Given all of the language changes in 2.6 and 3.0, I would think that it is dangerous to make any changes at all to the unittest API. That module is the one anchor in a sea of change. Raymond From fuzzyman at voidspace.org.uk Mon Jul 14 16:21:47 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 14 Jul 2008 15:21:47 +0100 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> Message-ID: <487B60FB.5010602@voidspace.org.uk> Raymond Hettinger wrote: >> Michael Foord <fuzzyman at voidspace.org.uk> writes: >> >>> The full list of changes proposed (feel free to start - but ping me or >>> the list) and not shot down was something like: >> [?] >> >> Thanks. I'm working these into another draft PEP that I hope to have >> up in a day or two. > > > Given all of the language changes in 2.6 and 3.0, I would think > that it is dangerous to make any changes at all to the unittest API. > That module is the one anchor in a sea of change. > As proposed the changes don't remove or rename anything - so there will be no code breakage, just additional test methods. However, as we're into the beta phase I don't think these changes can make 2.6 / 3.0 anyway. Michael > > Raymond > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From janssen at parc.com Mon Jul 14 19:39:42 2008 From: janssen at parc.com (Bill Janssen) Date: Mon, 14 Jul 2008 10:39:42 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> Message-ID: <08Jul14.103943pdt."58698"@synergy1.parc.xerox.com> >> Clearly the unquote is str->bytes, <snip> You can't pass a Unicode string back >> as the result of unquote *without* passing in an encoding specifier, >> because the character set is application-specific. > So for unquote you're suggesting that it always return a bytes object > UNLESS an encoding is specified? As in: > >> urllib.parse.unquote('h%C3%BCllo') > b'h\xc3\xbcllo' Yes, that's correct. That's what the RFC says we have to do. > I would object to that on two grounds. Firstly, I wouldn't expect or > desire a bytes object. The vast majority of uses for unquote will be > to get a character string out, not bytes. Secondly, there is a > mountain of code (including about 12 modules in the standard library) > which call unquote and don't give the user the encoding option, so > it's best if we pick a default that is what the majority of users will > expect. I argue that that's UTF-8. Unfortunately, despite your expectations or desires, the spec doesn't allow us that luxury. It's bytes out, and they may even be in a non-standard (not registered with IANA) encoding. There's no way to safely and correctly turn that sequence of bytes into a string. If other modules have been mis-using the interface, they are buggy and should be fixed. There's a lot of buggy stdlib code in Python around the older Web standards. I think it would be great to have another function, unquote_to_string, which took an extra "encoding" parameter, and returned a string. It would also be OK to add a keyword parameter to "unquote", I think, which provides an encoding, and causes unquote to return a string. But the standard behavior has to be to return bytes. > I'd prefer having a separate unquote_raw function which is > str->bytes, and the unquote function performs the same role as it > always have, which is str->str. Actually, it was originally bytes->bytes, because there was no notion of Unicode strings when it was added. It perhaps got misunderstood during the addition of Unicode support to Python; many people have had trouble wrapping their heads around all this, myself included. Bill From ben+python at benfinney.id.au Mon Jul 14 15:25:32 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 14 Jul 2008 23:25:32 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module Message-ID: <87vdz8a983.fsf@benfinney.id.au> :PEP: XXX :Title: Consolidating names and classes in the `unittest` module :Version: 0.0 :Last-Modified: 2008-07-14 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: test/x-rst :Created: 2008-07-14 :Post-History: .. contents:: Abstract ======== This PEP proposes to consolidate the names and classes that constitute the API of the standard library `unittest` module, with the goal of removing redundant names, conforming with PEP 8, and eliminating classic classes. Motivation ========== The normal use case for the `unittest` module is to subclass its classes, overriding and re-using its functios and methods. This draws constant attention to the fact that the existing implementation fails several current Python standards: * It does not use new-style classes, preventing e.g. straightforward use of ``super`` for calling the fixture set-up and tear-down methods. * It does not conform to PEP 8, requiring users to write their own non-PEP-8 conformant names when overriding methods, and encouraging extensions to further depart from PEP 8. * It has many synonyms in its API, which goes against the Zen of Python (specifically, that "there should be one, and preferably only one, obvious way to do it"). Specification ============= Use new-style classes throughout -------------------------------- The following classes will inherit explicitly from the built-in `object` type, to make all classes in the module part of the new-style type hierarchy. * ``TestResult`` * ``TestCase`` * ``TestSuite`` * ``TestLoader`` * ``_WritelnDecorator`` * ``TextTestRunner`` * ``TestProgram`` Remove obsolete names --------------------- The following attributes are not documented as part of the API and are marked as obsolete in the implementation. They will be removed. * ``_makeLoader`` * ``getTestCaseNames`` * ``makeSuite`` * ``findTestCases`` Remove redundant names ---------------------- The following attribute names exist only as synonyms for other names. They are to be removed, leaving only one name for each attribute in the API. ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ * ``assertEqual`` * ``assertEquals`` * ``assertNotEqual`` * ``assertNotEquals`` * ``assertAlmostEqual`` * ``assertAlmostEquals`` * ``assertNotAlmostEqual`` * ``assertNotAlmostEquals`` * ``assertRaises`` * ``assert_`` * ``assertTrue`` * ``assertFalse`` Conform API with PEP 8 ---------------------- The following names are to be introduced, each replacing an existing name, to make all names in the module conform with PEP 8. Each name is shown with the existing name that it replaces. Where function parameters are to be renamed also, they are shown. Where function parameters are not to be renamed, they are elided with the ellipse ("?") symbol. Module attributes ~~~~~~~~~~~~~~~~~ ``_make_loader(prefix, sort_using, suite_class)`` Replaces ``_makeLoader (prefix, sortUsing, suiteClass)`` ``find_test_cases(module, prefix, sort_using, suite_class)`` Replaces ``findTestCases(module, prefix, sortUsing, suiteClass)`` ``get_test_case_names(test_case_class, prefix, sort_using)`` Replaces ``getTestCaseNames(testCaseClass, prefix, sortUsing)`` ``make_suite(test_case_class, prefix, sort_using, suite_class)`` Replaces ``makeSuite(testCaseClass, prefix, sortUsing, suiteClass)`` ``default_test_loader`` Replaces ``defaultTestLoader`` ``TestResult`` atributes ~~~~~~~~~~~~~~~~~~~~~~~~ ``add_error(?)`` Replaces ``addError(?)`` ``add_result(?)`` Replaces ``addResult(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``should_stop`` Replaces ``shouldStop`` ``start_test(?)`` Replaces ``startTest(?)`` ``stop_test(?)`` Replaces ``stopTest(?)`` ``tests_run`` Replaces ``testsRun`` ``was_successful(?)`` Replaces ``wasSuccessful(?)`` ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, method_name='run_test')`` Replaces ``__init__(self, methodName='runTest')`` ``_test_method_doc`` Replaces ``_testMethodDoc`` ``_test_method_name`` Replaces ``_testMethodName`` ``failure_exception`` Replaces ``failureException`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``default_test_result(?)`` Replaces ``defaultTestResult(?)`` ``fail_if(?)`` Replaces ``failIf(?)`` ``fail_if_almost_equal(?)`` Replaces ``failIfAlmostEqual(?)`` ``fail_if_equal(?)`` Replaces ``failIfEqual(?)`` ``fail_unless(?)`` Replaces ``failUnless(?)`` ``fail_unless_almost_equal(?)`` Replaces ``failUnlessAlmostEqual(?)`` ``fail_unless_equal(?)`` Replaces ``failUnlessEqual(?)`` ``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)`` Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``FunctionTestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, test_func, set_up, tear_down, description)`` Replaces ``__init__(self, testFunc, setUp, tearDown, description)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``TestSuite`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~ ``add_test(?)`` Replaces ``addTest(?)`` ``add_tests(?)`` Replaces ``addTests(?)`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``TestLoader`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``sort_test_methods_using`` Replaces ``sortTestMethodsUsing`` ``suite_class`` Replaces ``suiteClass`` ``test_method_prefix`` Replaces ``testMethodPrefix`` ``get_test_case_names(self, test_case_class)`` Replaces ``getTestCaseNames(self, testCaseClass)`` ``load_tests_from_module(?)`` Replaces ``loadTestsFromModule(?)`` ``load_tests_from_name(?)`` Replaces ``loadTestsFromName(?)`` ``load_tests_from_names(?)`` Replaces ``loadTestsFromNames(?)`` ``load_tests_from_test_case(self, test_case_class)`` Replaces ``loadTestsFromTestCase(self, testCaseClass)`` ``_TextTestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``show_all`` Replaces ``showAll`` ``add_error(?)`` Replaces ``addError(?)`` ``add_failure(?)`` Replaces ``addFailure(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``get_description(?)`` Replaces ``getDescription(?)`` ``print_error_list(?)`` Replaces ``printErrorList(?)`` ``print_errors(?)`` Replaces ``printErrors(?)`` ``start_test(?)`` Replaces ``startTest(?)`` ``TextTestRunner`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``_make_result(?)`` Replaces ``_makeResult(?)`` ``TestProgram`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, module, default_test, argv, test_runner, test_loader)`` Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)`` ``create_tests(?)`` Replaces ``createTests(?)`` ``parse_args(?)`` Replaces ``parseArgs(?)`` ``run_tests(?)`` Replaces ``runTests(?)`` ``usage_exit(?)`` Replaces ``usageExit(?)`` Rationale ========= New-style classes ----------------- As a standard library module, `unittest` should not define any classic classes. Redundant names --------------- The current API, with two or in some cases three different names referencing exactly the same function, leads to an overbroad and redundant API that violates PEP 20 ("there should be one, and preferably only one, obvious way to do it"). PEP 8 names ----------- Although `unittest` (and its predecessor `PyUnit`) are intended to be familiar to users of other xUnit interfaces, there is no attempt at direct API compatibility since the only code that Python's `unittest` interfaces with is other Python code. The module is in the standard library and its names should all conform with PEP 8. Backwards Compatibility ======================= The names to be obsoleted should be deprecated and removed according to the schedule for modules in PEP 4. While deprecated, use of the deprecated attributes should raise a ``DeprecationWarning``, with a message stating which replacement name should be used. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From musiccomposition at gmail.com Tue Jul 15 00:19:25 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 14 Jul 2008 17:19:25 -0500 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module In-Reply-To: <87vdz8a983.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> Message-ID: <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote: > Use new-style classes throughout > -------------------------------- > > The following classes will inherit explicitly from the built-in > `object` type, to make all classes in the module part of the new-style > type hierarchy. > > * ``TestResult`` > * ``TestCase`` > * ``TestSuite`` > * ``TestLoader`` > * ``_WritelnDecorator`` > * ``TextTestRunner`` > * ``TestProgram`` They already do. __metaclass__ = type is found in unittest.py. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From ben+python at benfinney.id.au Tue Jul 15 00:18:54 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 08:18:54 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> Message-ID: <87iqv89kj5.fsf@benfinney.id.au> Michael Foord <fuzzyman at voidspace.org.uk> writes: > As proposed the changes don't remove or rename anything - so there > will be no code breakage, just additional test methods. Right, so I'm putting up a separate PEP just for the renaming. Should be arriving on this list soon. > However, as we're into the beta phase I don't think these changes > can make 2.6 / 3.0 anyway. Definitely agreed. -- \ ?You can be a victor without having victims.? ?Harriet Woods | `\ | _o__) | Ben Finney From bignose+hates-spam at benfinney.id.au Tue Jul 15 00:21:27 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 08:21:27 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> Message-ID: <87ej5w9kew.fsf@benfinney.id.au> "Raymond Hettinger" <python at rcn.com> writes: > Given all of the language changes in 2.6 and 3.0, I would think that > it is dangerous to make any changes at all to the unittest API. That > module is the one anchor in a sea of change. Agreed. I'm not proposing to have the unittest API change at all in Python 2.6 or 3.0. These changes, even the first deprecations, would not be suitable for anything earlier than 2.7 and 3.1. -- \ ?When in doubt tell the truth. It will confound your enemies | `\ and astound your friends.? ?Mark Twain, _Following the Equator_ | _o__) | Ben Finney From steve at pearwood.info Tue Jul 15 00:40:00 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 15 Jul 2008 08:40:00 +1000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <g5e4ja$8rg$1@ger.gmane.org> References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com> <200807140946.05034.steve@pearwood.info> <g5e4ja$8rg$1@ger.gmane.org> Message-ID: <200807150840.01490.steve@pearwood.info> On Mon, 14 Jul 2008 09:54:16 am Steve Holden wrote: > > Python may be Guido's language, and if he wants to use his > > dictatorial powers to say that tests must be written as positive > > assertions because that's the way he likes it, that's his > > prerogative. But let's not pretend that this particular bike shed > > colour has any objectively rational reason, or that the change > > won't force some people to have to change the way they think about > > tests. > > But sometimes even the wrong decision is better than no decision, and > I suspect most if us can agree that it's better if everyone thinks > about tests the same way. There's a term for what happens when everybody in a community or group thinks about a subject the same way. http://en.wikipedia.org/wiki/Groupthink -- Steven From nas at arctrix.com Tue Jul 15 00:43:43 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Mon, 14 Jul 2008 16:43:43 -0600 Subject: [Python-Dev] git repositories for trunk and py3k Message-ID: <20080714224343.GA23048@arctrix.com> Hi, In case anyone is interested, I have git repositories for both the trunk and the py3k branch of the Python source code. They are up-to-date and so using them with git-svn would be much faster than starting from scratch. If anyone is interested, I will find a place to host them. They are each about 150 MB in size. Neil From fuzzyman at voidspace.org.uk Tue Jul 15 01:08:30 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 15 Jul 2008 00:08:30 +0100 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module In-Reply-To: <87vdz8a983.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> Message-ID: <487BDC6E.2000606@voidspace.org.uk> Ben Finney wrote: > [snip..] > > Remove redundant names > ---------------------- > > The following attribute names exist only as synonyms for other names. > They are to be removed, leaving only one name for each attribute in > the API. > > ``TestCase`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~ > > * ``assertEqual`` > * ``assertEquals`` > * ``assertNotEqual`` > * ``assertNotEquals`` > * ``assertAlmostEqual`` > * ``assertAlmostEquals`` > * ``assertNotAlmostEqual`` > * ``assertNotAlmostEquals`` > * ``assertRaises`` > * ``assert_`` > * ``assertTrue`` > * ``assertFalse`` > > Although you may prefer the 'failIf' and 'failUnless' names, the consensus in the *last* discussion was that the 'assert*' names were to be preferred. I protest the removal of the assert names - and in the absence of likely consensus (and barring a dictat of course) I suggest this part of the proposal be excised. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From solipsis at pitrou.net Tue Jul 15 01:19:09 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 14 Jul 2008 23:19:09 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?PEP=3A_Consolidating_names_and_classes_in_?= =?utf-8?q?the_=60unittest=60=09module?= References: <87vdz8a983.fsf@benfinney.id.au> Message-ID: <loom.20080714T230912-310@post.gmane.org> Ben Finney <ben+python <at> benfinney.id.au> writes: > The following attribute names exist only as synonyms for other names. > They are to be removed, leaving only one name for each attribute in > the API. Just for information, here is the current distribution of the two synonym kinds: (on py3k) $ grep "self.assert" Lib/test/test_*.py | wc -l 14972 $ grep "self.fail" Lib/test/test_*.py | wc -l 1807 If no rational argument prevails, at least we have data on the past and current habits of the python-dev community. regards Antoine. From ben+python at benfinney.id.au Tue Jul 15 01:18:57 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 09:18:57 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module References: <87vdz8a983.fsf@benfinney.id.au> <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com> Message-ID: <87abgk9hr2.fsf@benfinney.id.au> "Benjamin Peterson" <musiccomposition at gmail.com> writes: > On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote: > > Use new-style classes throughout > > -------------------------------- > > > > The following classes will inherit explicitly from the built-in > > `object` type, to make all classes in the module part of the new-style > > type hierarchy. > > > > * ``TestResult`` > > * ``TestCase`` > > * ``TestSuite`` > > * ``TestLoader`` > > * ``_WritelnDecorator`` > > * ``TextTestRunner`` > > * ``TestProgram`` > > They already do. __metaclass__ = type is found in unittest.py. Not in the copy I have. Is that in 3.x only, or in 2.x also? -- \ ?I love to go down to the schoolyard and watch all the little | `\ children jump up and down and run around yelling and screaming. | _o__) They don't know I'm only using blanks.? ?Emo Philips | Ben Finney From musiccomposition at gmail.com Tue Jul 15 01:22:08 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 14 Jul 2008 18:22:08 -0500 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module In-Reply-To: <87abgk9hr2.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com> <87abgk9hr2.fsf@benfinney.id.au> Message-ID: <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com> On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > "Benjamin Peterson" <musiccomposition at gmail.com> writes: > >> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote: >> > Use new-style classes throughout >> > -------------------------------- >> > >> > The following classes will inherit explicitly from the built-in >> > `object` type, to make all classes in the module part of the new-style >> > type hierarchy. >> > >> > * ``TestResult`` >> > * ``TestCase`` >> > * ``TestSuite`` >> > * ``TestLoader`` >> > * ``_WritelnDecorator`` >> > * ``TextTestRunner`` >> > * ``TestProgram`` >> >> They already do. __metaclass__ = type is found in unittest.py. > > Not in the copy I have. Is that in 3.x only, or in 2.x also? Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import unittest >>> isinstance(unittest.TestCase, object) True -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From python at rcn.com Tue Jul 15 01:26:45 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Jul 2008 16:26:45 -0700 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> Message-ID: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> From: "Ben Finney" <ben+python at benfinney.id.au> > Right, so I'm putting up a separate PEP just for the renaming. Should > be arriving on this list soon. I would like to work with you or someone else who is interested on an alternative PEP for a separate, simpler test module using the py.test syntax. That is much simpler to learn and use. Instead of self.assertIsNot and whatnot, you write: assert a is not b No need for tons of word-by-word spellings on things we already have syntax for. Almost anyone who has used py.test can attest its syntax is much more natural, easy to learn, easy to both read and write, and is much lighter weight. I think some variant of py.test could be done that is compatable with unittest and the did not have the "magic" present in earlier versions of py.test. I wrote a recipe (somewhat rough and incomplete) that shows how easily this could be done: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194 Raymond From musiccomposition at gmail.com Tue Jul 15 01:32:21 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 14 Jul 2008 18:32:21 -0500 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> Message-ID: <1afaf6160807141632o424907aev777a8889b924ea6c@mail.gmail.com> On Mon, Jul 14, 2008 at 6:26 PM, Raymond Hettinger <python at rcn.com> wrote:. > > I would like to work with you or someone else who is interested > on an alternative PEP for a separate, simpler test module > using the py.test syntax. That is much simpler to learn and use. > Instead of self.assertIsNot and whatnot, you write: > assert a is not b > No need for tons of word-by-word spellings on things we already > have syntax for. Almost anyone who has used py.test can attest > its syntax is much more natural, easy to learn, easy to both > read and write, and is much lighter weight. I think some variant > of py.test could be done that is compatable with unittest > and the did not have the "magic" present in earlier versions of py.test. > I wrote a recipe (somewhat rough and incomplete) that shows how > easily this could be done: Bringing the total amount of test modules in the stdlib to 3. OWTDI indeed. Anyway, I don't think something like needs to be (re)written. nose[1] is already an excellent implementation of this that I would like to see in the stdlib. [1] http://www.somethingaboutorange.com/mrl/projects/nose/ -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at pearwood.info Tue Jul 15 01:40:43 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 15 Jul 2008 09:40:43 +1000 Subject: [Python-Dev] =?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?= =?iso-8859-1?q?asserts=09vs=2E=09failIf/Unlesses?= In-Reply-To: <87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <20080713234123.GA11170@mithrandi.net> <87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200807150940.44919.steve@pearwood.info> On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote: > FWIW, I meant 10 != not not 10. >>> 10 != not not 10 File "<stdin>", line 1 10 != not not 10 ^ SyntaxError: invalid syntax With respect, I think that the fact that you made an analogy with Python code that you hadn't tested, got it wrong, then corrected it, and *still* got it wrong, is telling. Its part of the pattern of this thread. People have repeatedly and consistently asserted that their particular favourite bike-shed colour is not just more attractive than any other colour, but supposedly objectively and logically better than any other colours. It's that second part that I object to. When it comes to assert* versus fail* tests, this is one case where I don't believe "one obvious way to do it" should apply. A similar, and I hope uncontroversial, case is the greater-than and less-than operators. It would be frankly silly to declare that Python code should always use x < y and never y > x on the basis that there should be "one obvious way". Sometimes a particular test is most naturally written as g-t, and sometimes as l-t, and sometimes the choice between them is just arbitrary. I believe that assert* and fail* tests are the same: while the choice is often arbitrary, sometimes a test is naturally written as an assert and sometimes as a fail. My own tests often look like this: fail_if_spam() fail_unless_ham() assert_parrot() fail_if_spanish_inquisition() because the specific parrot test is best written as an assertion rather than a fail. And frankly, I simply don't believe that this imposes an undue mental cost on people who might read my code. It's certainly true that I could invert the nature of the tests: assert_not_spam() assert_ham() assert_parrot() assert_not_spanish_inquisition() just for the sake of consistency (and that would be a good thing *why*?), but only at the cost of inverting the code inside the test, which may or may not be a simple thing to do. -- Steven From solipsis at pitrou.net Tue Jul 15 01:41:43 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 14 Jul 2008 23:41:43 +0000 (UTC) Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> Message-ID: <loom.20080714T233109-142@post.gmane.org> Hi, > I think some variant > of py.test could be done that is compatable with unittest > and the did not have the "magic" present in earlier versions of py.test. It already exists: http://www.somethingaboutorange.com/mrl/projects/nose/ Regards Antoine. From ben+python at benfinney.id.au Tue Jul 15 01:42:18 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 09:42:18 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> Message-ID: <8763r89go5.fsf@benfinney.id.au> Significant updates are to the preamble (Python-Version field), the sections "Use new-style classes throughout", "Module attributes", and a new Rationale section "Removal of ``assert*`` names". :PEP: XXX :Title: Consolidating names and classes in the `unittest` module :Version: 0.0 :Last-Modified: 2008-07-15 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: test/x-rst :Created: 2008-07-14 :Python-Version: 2.7, 3.1 :Post-History: .. contents:: Abstract ======== This PEP proposes to consolidate the names and classes that constitute the API of the standard library `unittest` module, with the goal of removing redundant names, conforming with PEP 8, and eliminating classic classes. Motivation ========== The normal use case for the `unittest` module is to subclass its classes, overriding and re-using its functios and methods. This draws constant attention to the fact that the existing implementation fails several current Python standards: * It does not use new-style classes, preventing e.g. straightforward use of ``super`` for calling the fixture set-up and tear-down methods. * It does not conform to PEP 8, requiring users to write their own non-PEP-8 conformant names when overriding methods, and encouraging extensions to further depart from PEP 8. * It has many synonyms in its API, which goes against the Zen of Python (specifically, that "there should be one, and preferably only one, obvious way to do it"). Specification ============= Use new-style classes throughout -------------------------------- The following classes are currently implemented as classic ("old-style") classes, with no metaclass. * ``TestResult`` * ``TestCase`` * ``TestSuite`` * ``TestLoader`` * ``_WritelnDecorator`` * ``TextTestRunner`` * ``TestProgram`` The `unittest` module will gain the following attribute, to set the default metaclass for classes in the module and thus make all classes in the module part of the new-style type hierarchy:: __metaclass__ = type Remove obsolete names --------------------- The following module attributes are not documented as part of the API and are marked as obsolete in the implementation. They will be removed. * ``_makeLoader`` * ``getTestCaseNames`` * ``makeSuite`` * ``findTestCases`` Remove redundant names ---------------------- The following attribute names exist only as synonyms for other names. They are to be removed, leaving only one name for each attribute in the API. ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ * ``assertEqual`` * ``assertEquals`` * ``assertNotEqual`` * ``assertNotEquals`` * ``assertAlmostEqual`` * ``assertAlmostEquals`` * ``assertNotAlmostEqual`` * ``assertNotAlmostEquals`` * ``assertRaises`` * ``assert_`` * ``assertTrue`` * ``assertFalse`` Conform API with PEP 8 ---------------------- The following names are to be introduced, each replacing an existing name, to make all names in the module conform with PEP 8. Each name is shown with the existing name that it replaces. Where function parameters are to be renamed also, they are shown. Where function parameters are not to be renamed, they are elided with the ellipse ("?") symbol. Module attributes ~~~~~~~~~~~~~~~~~ ``default_test_loader`` Replaces ``defaultTestLoader`` ``TestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``add_error(?)`` Replaces ``addError(?)`` ``add_result(?)`` Replaces ``addResult(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``should_stop`` Replaces ``shouldStop`` ``start_test(?)`` Replaces ``startTest(?)`` ``stop_test(?)`` Replaces ``stopTest(?)`` ``tests_run`` Replaces ``testsRun`` ``was_successful(?)`` Replaces ``wasSuccessful(?)`` ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, method_name='run_test')`` Replaces ``__init__(self, methodName='runTest')`` ``_test_method_doc`` Replaces ``_testMethodDoc`` ``_test_method_name`` Replaces ``_testMethodName`` ``failure_exception`` Replaces ``failureException`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``default_test_result(?)`` Replaces ``defaultTestResult(?)`` ``fail_if(?)`` Replaces ``failIf(?)`` ``fail_if_almost_equal(?)`` Replaces ``failIfAlmostEqual(?)`` ``fail_if_equal(?)`` Replaces ``failIfEqual(?)`` ``fail_unless(?)`` Replaces ``failUnless(?)`` ``fail_unless_almost_equal(?)`` Replaces ``failUnlessAlmostEqual(?)`` ``fail_unless_equal(?)`` Replaces ``failUnlessEqual(?)`` ``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)`` Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``FunctionTestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, test_func, set_up, tear_down, description)`` Replaces ``__init__(self, testFunc, setUp, tearDown, description)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``TestSuite`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~ ``add_test(?)`` Replaces ``addTest(?)`` ``add_tests(?)`` Replaces ``addTests(?)`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``TestLoader`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``sort_test_methods_using`` Replaces ``sortTestMethodsUsing`` ``suite_class`` Replaces ``suiteClass`` ``test_method_prefix`` Replaces ``testMethodPrefix`` ``get_test_case_names(self, test_case_class)`` Replaces ``getTestCaseNames(self, testCaseClass)`` ``load_tests_from_module(?)`` Replaces ``loadTestsFromModule(?)`` ``load_tests_from_name(?)`` Replaces ``loadTestsFromName(?)`` ``load_tests_from_names(?)`` Replaces ``loadTestsFromNames(?)`` ``load_tests_from_test_case(self, test_case_class)`` Replaces ``loadTestsFromTestCase(self, testCaseClass)`` ``_TextTestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``show_all`` Replaces ``showAll`` ``add_error(?)`` Replaces ``addError(?)`` ``add_failure(?)`` Replaces ``addFailure(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``get_description(?)`` Replaces ``getDescription(?)`` ``print_error_list(?)`` Replaces ``printErrorList(?)`` ``print_errors(?)`` Replaces ``printErrors(?)`` ``start_test(?)`` Replaces ``startTest(?)`` ``TextTestRunner`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``_make_result(?)`` Replaces ``_makeResult(?)`` ``TestProgram`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, module, default_test, argv, test_runner, test_loader)`` Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)`` ``create_tests(?)`` Replaces ``createTests(?)`` ``parse_args(?)`` Replaces ``parseArgs(?)`` ``run_tests(?)`` Replaces ``runTests(?)`` ``usage_exit(?)`` Replaces ``usageExit(?)`` Rationale ========= New-style classes ----------------- As a standard library module, `unittest` should not define any classic classes. Redundant names --------------- The current API, with two or in some cases three different names referencing exactly the same function, leads to an overbroad and redundant API that violates PEP 20 ("there should be one, and preferably only one, obvious way to do it"). Removal of ``assert*`` names ---------------------------- There is no overwhelming consensus on whether to remove the ``assert*`` names or the ``fail*`` names; both are in common use. This proposal argues the ``fail*`` names are slightly superior, for the following reasons: * Explicit is better than implicit: The ``fail*`` names state *what the function will do* explicitly: fail the test. With the ``assert*`` names, the action to be taken is only implicit. * Avoid false implication: The test methods do not have any necessary connection with the built-in ``assert`` statement. Even the exception raised, while it defaults to ``AssertionException``, is explicitly customisable via the documented ``failure_exception`` attribute. Choosing the ``fail*`` names avoids the false association with either of these. PEP 8 names ----------- Although `unittest` (and its predecessor `PyUnit`) are intended to be familiar to users of other xUnit interfaces, there is no attempt at direct API compatibility since the only code that Python's `unittest` interfaces with is other Python code. The module is in the standard library and its names should all conform with PEP 8. Backwards Compatibility ======================= The names to be obsoleted should be deprecated and removed according to the schedule for modules in PEP 4. While deprecated, use of the deprecated attributes should raise a ``DeprecationWarning``, with a message stating which replacement name should be used. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From fuzzyman at voidspace.org.uk Tue Jul 15 01:43:12 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 15 Jul 2008 00:43:12 +0100 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> Message-ID: <487BE490.8080406@voidspace.org.uk> Raymond Hettinger wrote: > From: "Ben Finney" <ben+python at benfinney.id.au> >> Right, so I'm putting up a separate PEP just for the renaming. Should >> be arriving on this list soon. > > I would like to work with you or someone else who is interested > on an alternative PEP for a separate, simpler test module > using the py.test syntax. That is much simpler to learn and use. > Instead of self.assertIsNot and whatnot, you write: > assert a is not b > No need for tons of word-by-word spellings on things we already > have syntax for. However, to provide readable output for errors in even simple tests (like a == b) py.test does magic with stack frames and code objects - in order to discover the objects being compared. As this relies on what are essentially implementation details of the Python interpreter it means that some implementations (specifically IronPython which doesn't have Python stack frames and only a minimal representation of frame objects) will never be able to run it. I think it would be a bad idea to move *Python testing* itself over to a framework like this. I personally find unittest pretty readable, the feature most lacking is autodiscovery of tests which nose does seem to provide very well although I haven't used it yet. Michael > Almost anyone who has used py.test can attest > its syntax is much more natural, easy to learn, easy to both > read and write, and is much lighter weight. I think some variant > of py.test could be done that is compatable with unittest > and the did not have the "magic" present in earlier versions of py.test. > I wrote a recipe (somewhat rough and incomplete) that shows how > easily this could be done: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194 > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From fuzzyman at voidspace.org.uk Tue Jul 15 01:45:18 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 15 Jul 2008 00:45:18 +0100 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> Message-ID: <487BE50E.6090200@voidspace.org.uk> Raymond Hettinger wrote: > From: "Ben Finney" <ben+python at benfinney.id.au> >> Right, so I'm putting up a separate PEP just for the renaming. Should >> be arriving on this list soon. > > I would like to work with you or someone else who is interested > on an alternative PEP for a separate, simpler test module > using the py.test syntax. That is much simpler to learn and use. > Instead of self.assertIsNot and whatnot, you write: > assert a is not b > No need for tons of word-by-word spellings on things we already > have syntax for. Almost anyone who has used py.test can attest > its syntax is much more natural, easy to learn, easy to both > read and write, and is much lighter weight. I think some variant > of py.test could be done that is compatable with unittest > and the did not have the "magic" present in earlier versions of py.test. Ah, in my haste I skipped over your comment about "magic", my apologies. But in the absence of magic how do you propose to provide a meaningful error message from the failure of: assert a == b To wrap it in a function like "assert equals(a, b)" seems to gain little over unittest. Michael > I wrote a recipe (somewhat rough and incomplete) that shows how > easily this could be done: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194 > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From jml at mumak.net Tue Jul 15 01:58:04 2008 From: jml at mumak.net (Jonathan Lange) Date: Tue, 15 Jul 2008 09:58:04 +1000 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487BE490.8080406@voidspace.org.uk> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> Message-ID: <d06a5cd30807141658t7b1d120era251efc46f7c6a5d@mail.gmail.com> On Tue, Jul 15, 2008 at 9:43 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote: > I personally find unittest pretty readable, the feature most lacking is > autodiscovery of tests which nose does seem to provide very well although I > haven't used it yet. FWIW, Twisted's 'trial' has done this since about 2003, and works with stdlib unit tests. I'd be happy to submit the discovery code to Python. jml From python at rcn.com Tue Jul 15 02:13:25 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Jul 2008 17:13:25 -0700 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> Message-ID: <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1> From: "Michael Foord" <fuzzyman at voidspace.org.uk> > However, to provide readable output for errors in even simple tests > (like a == b) py.test does magic with stack frames and code objects - in > order to discover the objects being compared. Don't have to go that route. Can use plain python assert failures with a stacktrace. Or can trigger pdb, or let the user specify a mode that calls some more advanced test runner or test reporter with introspection. This can be done without making everything hard. > I think it would be a bad idea to move *Python testing* itself over to a > framework like this. Don't want to convert the python testing. Would like to offer a lighter-weight alternative to our users. > I personally find unittest pretty readable, the feature most lacking is > autodiscovery of tests which nose does seem to provide very well > although I haven't used it yet. It takes about one day of using py.test to realize have much cleaner and more readable its syntax is. Also, writing the tests is *much* more pleasant. It has the same clean, clear joy as writing regular python code. By comparison, the code using unittest.py is javaesque. I've written tons of test with unittest.py and and find it to be joyless. I realize there is a matter of taste involved but if you talk to any regular users of py.test, they will *all* attest to the syntax being much more readable, lightweight, and pleasant to use. It encourages writing tests. That being said, I think there are less magical, much simpler ways to implement it. I think Holger is working on it as we speak. Raymond From musiccomposition at gmail.com Tue Jul 15 02:23:59 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 14 Jul 2008 19:23:59 -0500 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487BE490.8080406@voidspace.org.uk> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> Message-ID: <1afaf6160807141723j65f515c7wbe0d377f2c813ea3@mail.gmail.com> On Mon, Jul 14, 2008 at 6:43 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote: > > However, to provide readable output for errors in even simple tests (like a > == b) py.test does magic with stack frames and code objects - in order to > discover the objects being compared. Maybe what we need to do then is make the assert statement more powerful. I like the idea of having it call a builtin called __assert__ which is called by the assert statement. The AST for the node could be attached. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Tue Jul 15 02:30:42 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 14 Jul 2008 19:30:42 -0500 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <8763r89go5.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> Message-ID: <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com> On Mon, Jul 14, 2008 at 6:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > Specification > ============= > > Use new-style classes throughout > -------------------------------- > > The following classes are currently implemented as classic > ("old-style") classes, with no metaclass. > > * ``TestResult`` > * ``TestCase`` > * ``TestSuite`` > * ``TestLoader`` > * ``_WritelnDecorator`` > * ``TextTestRunner`` > * ``TestProgram`` > > The `unittest` module will gain the following attribute, to set the > default metaclass for classes in the module and thus make all classes > in the module part of the new-style type hierarchy:: > > __metaclass__ = type It's already done. Line 94-95 in unittest.py (trunk): # All classes defined herein are 'new-style' classes, allowing use of 'super()' __metaclass__ = type -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Tue Jul 15 02:32:51 2008 From: brett at python.org (Brett Cannon) Date: Mon, 14 Jul 2008 17:32:51 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <20080714224343.GA23048@arctrix.com> References: <20080714224343.GA23048@arctrix.com> Message-ID: <bbaeab100807141732r38fc8b3bi71061221d7393528@mail.gmail.com> On Mon, Jul 14, 2008 at 3:43 PM, Neil Schemenauer <nas at arctrix.com> wrote: > Hi, > > In case anyone is interested, I have git repositories for both the > trunk and the py3k branch of the Python source code. They are > up-to-date and so using them with git-svn would be much faster than > starting from scratch. > > If anyone is interested, I will find a place to host them. They are > each about 150 MB in size. > Would you mind putting the time in, Neil, to have them hosted on a python.org machine like the bzr branches and have them auto-updated? -Brett From barry at python.org Tue Jul 15 03:31:47 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Jul 2008 21:31:47 -0400 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <20080714224343.GA23048@arctrix.com> References: <20080714224343.GA23048@arctrix.com> Message-ID: <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 14, 2008, at 6:43 PM, Neil Schemenauer wrote: > In case anyone is interested, I have git repositories for both the > trunk and the py3k branch of the Python source code. They are > up-to-date and so using them with git-svn would be much faster than > starting from scratch. > > If anyone is interested, I will find a place to host them. They are > each about 150 MB in size. Neil, we should try to host them on code.python.org. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSHv+A3EjvBPtnXfVAQI0zwP9H0lZMvWxwncqg1BmI+df0WTh7+SOsxO2 RHky3TzqkY7wBXXwHPe5d7duWzflsXjB6ljH0AoR7icMs31h5ZUZhGVU/vSYouqk KhHqCHjnXlnY0qOySthblboih/Uw9ApR9akEsKAxQrzUATbZF93dS5RJg4QjpMn3 HJ06MUTt5y0= =bm3R -----END PGP SIGNATURE----- From fuzzyman at voidspace.org.uk Tue Jul 15 03:40:31 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 15 Jul 2008 02:40:31 +0100 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module In-Reply-To: <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com> <87abgk9hr2.fsf@benfinney.id.au> <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com> Message-ID: <487C000F.3050300@voidspace.org.uk> Benjamin Peterson wrote: > On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > >> "Benjamin Peterson" <musiccomposition at gmail.com> writes: >> >> >>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote: >>> >>>> Use new-style classes throughout >>>> -------------------------------- >>>> >>>> The following classes will inherit explicitly from the built-in >>>> `object` type, to make all classes in the module part of the new-style >>>> type hierarchy. >>>> >>>> * ``TestResult`` >>>> * ``TestCase`` >>>> * ``TestSuite`` >>>> * ``TestLoader`` >>>> * ``_WritelnDecorator`` >>>> * ``TextTestRunner`` >>>> * ``TestProgram`` >>>> >>> They already do. __metaclass__ = type is found in unittest.py. >>> >> Not in the copy I have. Is that in 3.x only, or in 2.x also? >> > > Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) > [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>>> import unittest >>>> isinstance(unittest.TestCase, object) >>>> > True > > > That proves nothing: Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> class x: pass ... >>> isinstance(x, object) True >>> isinstance(x, type) False >>> type(x) <type 'classobj'> -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From python at rcn.com Tue Jul 15 04:06:49 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Jul 2008 19:06:49 -0700 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> Message-ID: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> > ``set_up(?)`` > Replaces ``setUp(?)`` . . > ``tear_down(?)`` > Replaces ``tearDown(?)`` Am I the only one who finds this sort of excessive pep-8 underscoring to be horrorific? Nobody I know spells setup and teardown as two words. I dread using the module with these new names. Underscores are not fun to type. They create a weird_mental_pause when reading them. It's not going to be easy to remember where they are used (ie. isinstance is still isinstance but isset is now is_set). Go figure. > fail_if_almost_equal Another thought is that test suite code is going to get seriously crunched when the new, longer method names meet the 78/80 column pep 8 restrictions. class TestMisc(unittest.test_case): def lost_four_spaces_here_already(self): self.fail_if_almost_equal('were already on the 34th column before' 'writing anything substantive', self.testedobject.tested_method(arg1, arg2 + arg3, arg4) # Imagine typing and line wrapping dozens more tests like this Are there any ideas for some short, pithy, mnemonic names that are self-explantory and not full of underscores; something that wouldn't suck to type hundreds of times in a good test module? IMO, the current names are already too long. > * Explicit is better than implicit: Don't forgot the opposing forces: Beautiful is better than ugly. Readability counts. These are especially important for the unittest module. When I'm making tests, I have to type self.very_long_method_name so many times it isn't funny. It's easy to just stop writing tests when you get tired of it. Long api names with underscores are a disincentive (IMO). > Use new-style classes throughout This makes sense for 3.1 but of course we already get that automatically for 3.0 ;-) For 2.7, it doesn't make as much sense. Since 2.2 came out, there have been many discussions on changing standard library code to use new-style classes. There was always some concern about subtle changes in semantics and for the most part these requests were denied. I think this reason applies with even more force to the unittest module. Any risk that we subtlely break someone's test-suite is a small disaster. The 2.6 and 2.7 unittests need to be absolutely stable if they are to serve as a transition tool (providing a baseline the 3.0/3.1 is expected to match). Also, are there any expected benefits from making this change in 2.7? What will the module do differently? It seems like a risky change for zero-benefit. Raymond From fuzzyman at voidspace.org.uk Tue Jul 15 04:14:47 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 15 Jul 2008 03:14:47 +0100 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> Message-ID: <487C0817.50504@voidspace.org.uk> Raymond Hettinger wrote: >> ``set_up(?)`` >> Replaces ``setUp(?)`` > . . >> ``tear_down(?)`` >> Replaces ``tearDown(?)`` > > Am I the only one who finds this sort of excessive pep-8 underscoring > to be horrorific? > > Nobody I know spells setup and teardown as two words. I dread using > the module with these new names. Underscores are not fun to type. They > create a weird_mental_pause when reading them. > > It's not going to be easy to remember where they are used (ie. > isinstance is still isinstance but isset is now is_set). Go figure. +1 setUp and tearDown should become setup and teardown. > > >> fail_if_almost_equal > > Another thought is that test suite code is going to get seriously > crunched when the new, longer method names meet the 78/80 column pep 8 > restrictions. Well... "assert_not_equal" is slightly shorter, "assert_notequal" slightly more so. Still one char longer than the original though. > > class TestMisc(unittest.test_case): > def lost_four_spaces_here_already(self): > self.fail_if_almost_equal('were already on the 34th column before' > 'writing anything substantive', > self.testedobject.tested_method(arg1, > arg2 + > arg3, > arg4) > # Imagine typing and line wrapping dozens more tests like this > > Are there any ideas for some short, pithy, mnemonic names that are > self-explantory and not full of underscores; something that wouldn't > suck to type hundreds of times in a good test module? IMO, the > current names are already too long. > > >> * Explicit is better than implicit: > > Don't forgot the opposing forces: > > Beautiful is better than ugly. > Readability counts. > > These are especially important for the unittest module. When I'm > making tests, I have to type self.very_long_method_name so many times > it isn't funny. It's easy to just stop writing tests when you get > tired of it. Long api names with underscores are a disincentive (IMO). > > >> Use new-style classes throughout > > This makes sense for 3.1 but of course we already get that > automatically for 3.0 ;-) > > For 2.7, it doesn't make as much sense. Since 2.2 came out, there > have been many discussions on changing standard library code to use > new-style classes. There was always some concern about subtle changes > in semantics and for the most part these requests were denied. I > think this reason applies with even more force to the unittest > module. Any risk that we subtlely break someone's test-suite is a > small disaster. The 2.6 and 2.7 unittests need to be absolutely > stable if they are to serve as a transition tool (providing a baseline > the 3.0/3.1 is expected to match). > > Also, are there any expected benefits from making this change in 2.7? > What will the module do differently? It would allow you to use properties in testcases. Not sure if there is a usecase for this though. It looks like Benjamin Peterson is right, in Python 2.5 TestCase already appears to be a new style class: Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import unittest >>> type(unittest.TestCase) <type 'type'> >>> > > It seems like a risky change for zero-benefit. > Looks like that part of the PEP is unnecessary. Michael > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From jml at mumak.net Tue Jul 15 04:18:26 2008 From: jml at mumak.net (Jonathan Lange) Date: Tue, 15 Jul 2008 12:18:26 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> Message-ID: <d06a5cd30807141918w3d574c36lb9c84435e014fe36@mail.gmail.com> On Tue, Jul 15, 2008 at 12:06 PM, Raymond Hettinger <python at rcn.com> wrote: >> ``set_up(?)`` >> Replaces ``setUp(?)`` > > . . >> >> ``tear_down(?)`` >> Replaces ``tearDown(?)`` > > Am I the only one who finds this sort of excessive pep-8 underscoring to be > horrorific? > > Nobody I know spells setup and teardown as two words. I dread using the > module with these new names. > Hi, My name's Jonathan, and I spell "set up" as "set up" and "tear down" as "tear down". > It's not going to be easy to remember where they are used (ie. isinstance is > still isinstance but isset is now is_set). Go figure. > Yes, guessability via consistency is the important thing here. > >> fail_if_almost_equal > > Another thought is that test suite code is going to get seriously crunched > when the new, longer method names meet the 78/80 column pep 8 restrictions. > > class TestMisc(unittest.test_case): > def lost_four_spaces_here_already(self): > self.fail_if_almost_equal('were already on the 34th column before' > 'writing anything substantive', > self.testedobject.tested_method(arg1, arg2 + > arg3, arg4) > # Imagine typing and line wrapping dozens more tests like this > > Are there any ideas for some short, pithy, mnemonic names that are > self-explantory and not full of underscores; something that wouldn't suck to > type hundreds of times in a good test module? IMO, the current names are > already too long. > Well, "assert_" is strictly shorter than "fail_unless_". > >> * Explicit is better than implicit: > > Don't forgot the opposing forces: > > Beautiful is better than ugly. > Readability counts. > > These are especially important for the unittest module. When I'm making > tests, I have to type self.very_long_method_name so many times it isn't > funny. It's easy to just stop writing tests when you get tired of it. Long > api names with underscores are a disincentive (IMO). > I find underscores easier to read?I suspect this will vary from person to person. Typing isn't an issue since I use an auto-complete function in my editor. jml From python at rcn.com Tue Jul 15 04:40:26 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Jul 2008 19:40:26 -0700 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <487C0817.50504@voidspace.org.uk> Message-ID: <E1D388236201464C84CAE7FE74A15C01@RaymondLaptop1> > It looks like Benjamin Peterson is right, in Python 2.5 TestCase already appears to be a new style class: Yep. I stand corrected. It looks like that changed five years ago (rev 28064). Not sure how that slipped through but it doesn't seem to have caused any problems. Raymond From janzert at janzert.com Tue Jul 15 05:15:59 2008 From: janzert at janzert.com (Janzert) Date: Mon, 14 Jul 2008 23:15:59 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> Message-ID: <g5h4og$dub$1@ger.gmane.org> Raymond Hettinger wrote: >> ``set_up(?)`` >> Replaces ``setUp(?)`` > . . >> ``tear_down(?)`` >> Replaces ``tearDown(?)`` > > Am I the only one who finds this sort of excessive pep-8 underscoring to > be horrorific? > > Nobody I know spells setup and teardown as two words. I dread using the > module with these new names. Underscores are not fun to type. They > create a weird_mental_pause when reading them. > +1 And Merriam-Webster agrees, http://www.merriam-webster.com/dictionary/setup http://www.merriam-webster.com/dictionary/teardown Janzert From guido at python.org Tue Jul 15 05:22:05 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Jul 2008 20:22:05 -0700 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1> Message-ID: <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com> On Mon, Jul 14, 2008 at 5:13 PM, Raymond Hettinger <python at rcn.com> wrote: > It takes about one day of using py.test to realize have much > cleaner and more readable its syntax is. Also, writing the > tests is *much* more pleasant. It has the same clean, clear > joy as writing regular python code. By comparison, the code > using unittest.py is javaesque. I've written tons of test with > unittest.py and and find it to be joyless. I, too, have written tons of tests with unittest.py (and Google's extensions, which follow the same style), and reviewed even more. I agree that this is pretty joyless, but I'm not at all sure that the unittest API is the reason. It seems to me that a main problem with writing test code is and will always remain due to the need to use mocks, stubs and other similar techniques (e.g. dependency injection). Typical test code that I've written or reviewed spends more time setting up the input conditions for testing than it spends checking the results. Ten lines of mocking code to one self.assertEqual() call seems typical. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Tue Jul 15 05:34:18 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 15 Jul 2008 04:34:18 +0100 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1> <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com> Message-ID: <487C1ABA.4010701@voidspace.org.uk> Guido van Rossum wrote: > On Mon, Jul 14, 2008 at 5:13 PM, Raymond Hettinger <python at rcn.com> wrote: > >> It takes about one day of using py.test to realize have much >> cleaner and more readable its syntax is. Also, writing the >> tests is *much* more pleasant. It has the same clean, clear >> joy as writing regular python code. By comparison, the code >> using unittest.py is javaesque. I've written tons of test with >> unittest.py and and find it to be joyless. >> > > I, too, have written tons of tests with unittest.py (and Google's > extensions, which follow the same style), and reviewed even more. I > agree that this is pretty joyless, but I'm not at all sure that the > unittest API is the reason. It seems to me that a main problem with > writing test code is and will always remain due to the need to use > mocks, stubs and other similar techniques (e.g. dependency injection). > Typical test code that I've written or reviewed spends more time > setting up the input conditions for testing than it spends checking > the results. Ten lines of mocking code to one self.assertEqual() call > seems typical. > > Maybe Python needs a good mocking module in the standard library. There are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) It auto-creates attributes as mocks, allowing you to assert calls made to all of its children along with convenience methods like 'assert_called_with' and has a companion decorator that patches class / module level attributes just for the duration of the test. As we're changing more of our tests over to use these we're finding it reduces the volume and complexity of our test code. Michael Foord [1] Based on http://code.google.com/p/mock/ although there is some outstanding code to sync back to the project. -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From python at rcn.com Tue Jul 15 06:37:30 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Jul 2008 21:37:30 -0700 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1> <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com> <487C1ABA.4010701@voidspace.org.uk> Message-ID: <4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1> From: "Michael Foord" <fuzzyman at voidspace.org.uk> > Maybe Python needs a good mocking module in the standard library. There > are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) -1 This comes up occassionally and gets shot down. http://bugs.python.org/issue708125 Mock objects mean different things to different people. Some expect more simulated behavior and others want less. It's rare to find agreement about general purpose mock objects and frameworks. Mock libraries create their own complexities and burdens on a programmer's memory. It's often easier to create a small special case mock object than to remember how to configure a general purpose one. And, afaict, there is no fan club for some particular python mock object library -- it seems to only come up in general discussions about possibilities for growing the unittest module, and almost never comes up in the context of solving a real problem that hasn't already be addressed in some other way. Raymond From ctb at msu.edu Tue Jul 15 06:42:54 2008 From: ctb at msu.edu (C. Titus Brown) Date: Mon, 14 Jul 2008 21:42:54 -0700 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1> References: <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1> <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com> <487C1ABA.4010701@voidspace.org.uk> <4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1> Message-ID: <20080715044253.GA31517@caltech.edu> On Mon, Jul 14, 2008 at 09:37:30PM -0700, Raymond Hettinger wrote: -> From: "Michael Foord" <fuzzyman at voidspace.org.uk> -> >Maybe Python needs a good mocking module in the standard library. There -> >are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) -> -> -1 -> -> This comes up occassionally and gets shot down. -> http://bugs.python.org/issue708125 -> -> Mock objects mean different things to different people. -> Some expect more simulated behavior and others want less. -> It's rare to find agreement about general purpose mock objects and -> frameworks. -> Mock libraries create their own complexities and burdens on a programmer's -> memory. -> It's often easier to create a small special case mock object -> than to remember how to configure a general purpose one. -> And, afaict, there is no fan club for some particular python mock -> object library -- it seems to only come up in general discussions -> about possibilities for growing the unittest module, and almost -> never comes up in the context of solving a real problem that -> hasn't already be addressed in some other way. Also see: http://lists.idyll.org/pipermail/testing-in-python/2007-November/000406.html & associated thread, for those interested in the variety of mock libraries... cheers, --titus -- C. Titus Brown, ctb at msu.edu From python at rcn.com Tue Jul 15 07:11:07 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Jul 2008 22:11:07 -0700 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk><87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1><ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com><487C1ABA.4010701@voidspace.org.uk> <4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1> Message-ID: <A0D30B0412654877A54A28CB58C7DA78@RaymondLaptop1> > From: "Michael Foord" <fuzzyman at voidspace.org.uk> >> Maybe Python needs a good mocking module in the standard library. There >> are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) > > -1 > > This comes up occassionally and gets shot down. > http://bugs.python.org/issue708125 And: http://bugs.python.org/issue2156 Raymond From scott+python-dev at scottdial.com Tue Jul 15 08:17:21 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 15 Jul 2008 02:17:21 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module In-Reply-To: <487C000F.3050300@voidspace.org.uk> References: <87vdz8a983.fsf@benfinney.id.au> <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com> <87abgk9hr2.fsf@benfinney.id.au> <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com> <487C000F.3050300@voidspace.org.uk> Message-ID: <487C40F1.9000309@scottdial.com> Michael Foord wrote: > Benjamin Peterson wrote: >> On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney >> <ben+python at benfinney.id.au> wrote: >>> "Benjamin Peterson" <musiccomposition at gmail.com> writes: >>>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney >>>> <ben+python at benfinney.id.au> wrote: >>>>> Use new-style classes throughout >>>>> -------------------------------- >>>>> >>>>> The following classes will inherit explicitly from the built-in >>>>> `object` type, to make all classes in the module part of the new-style >>>>> type hierarchy. >>>>> [snip] >>>>> >>>> They already do. __metaclass__ = type is found in unittest.py. >>>> >>> Not in the copy I have. Is that in 3.x only, or in 2.x also? >>> >> >> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) >> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin >> Type "help", "copyright", "credits" or "license" for more information. >> >>>>> import unittest >>>>> isinstance(unittest.TestCase, object) >>>>> >> True >> > That proves nothing: > > Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) > [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> class x: pass > ... > >>> isinstance(x, object) > True > >>> isinstance(x, type) > False > >>> type(x) > <type 'classobj'> > While your retort is accurate, I think it's unintentionally deceptive, because you didn't finish your thought.. Benjamin is actually still correct: Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import unittest >>> isinstance(unittest.TestCase, type) True >>> type(unittest.TestCase) <type 'type'> -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From steve at holdenweb.com Tue Jul 15 08:21:01 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 15 Jul 2008 02:21:01 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module In-Reply-To: <487C000F.3050300@voidspace.org.uk> References: <87vdz8a983.fsf@benfinney.id.au> <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com> <87abgk9hr2.fsf@benfinney.id.au> <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com> <487C000F.3050300@voidspace.org.uk> Message-ID: <g5hfl6$7v3$1@ger.gmane.org> Michael Foord wrote: > Benjamin Peterson wrote: >> On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney >> <ben+python at benfinney.id.au> wrote: >> >>> "Benjamin Peterson" <musiccomposition at gmail.com> writes: >>> >>> >>>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney >>>> <ben+python at benfinney.id.au> wrote: >>>> >>>>> Use new-style classes throughout >>>>> -------------------------------- >>>>> >>>>> The following classes will inherit explicitly from the built-in >>>>> `object` type, to make all classes in the module part of the new-style >>>>> type hierarchy. >>>>> >>>>> * ``TestResult`` >>>>> * ``TestCase`` >>>>> * ``TestSuite`` >>>>> * ``TestLoader`` >>>>> * ``_WritelnDecorator`` >>>>> * ``TextTestRunner`` >>>>> * ``TestProgram`` >>>>> >>>> They already do. __metaclass__ = type is found in unittest.py. >>>> >>> Not in the copy I have. Is that in 3.x only, or in 2.x also? >>> >> >> Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) >> [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin >> Type "help", "copyright", "credits" or "license" for more information. >> >>>>> import unittest >>>>> isinstance(unittest.TestCase, object) >>>>> >> True >> >> >> > That proves nothing: > > Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) > [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> class x: pass > ... > >>> isinstance(x, object) > True > >>> isinstance(x, type) > False > >>> type(x) > <type 'classobj'> > Shouldn't isinstance(x, object) be True for any x in recent CPythons (and hopefully the other implementations too)? It's certainly so for functions and modules, for , as well as the less esoteric types and instances of classic classes. Something that often ties students' heads in knots (and this isn't from the introductory course): >>> isinstance(type, object) True >>> isinstance(object, type) True This is a classic property of general object hierarchies based on metaclasses. I remember teaching the SmallTalk-80 equivalent to M.Sc. studnets in the early 1980s, though the details are now lost in the mists of time. Eyes would always widen, and not everyone would get it. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue Jul 15 08:40:22 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 15 Jul 2008 02:40:22 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> Message-ID: <g5hgpg$an7$1@ger.gmane.org> Raymond Hettinger wrote: >> ``set_up(?)`` >> Replaces ``setUp(?)`` > . . >> ``tear_down(?)`` >> Replaces ``tearDown(?)`` > > Am I the only one who finds this sort of excessive pep-8 underscoring to > be horrorific? > Definitely not. I thin we are in danger of insisting on a foolish consistency. I'd far prefer readability (hence my preference for is_not over not_is). There's also the (possibly marginal) point that people used to Junit can actually understand Python unit tests right now. Surely there's something to be said for consistency across languages, especially when the idea was lifted from Java in the first place. > Nobody I know spells setup and teardown as two words. I dread using the > module with these new names. Underscores are not fun to type. They > create a weird_mental_pause when reading them. > > It's not going to be easy to remember where they are used (ie. > isinstance is still isinstance but isset is now is_set). Go figure. > > >> fail_if_almost_equal > > Another thought is that test suite code is going to get seriously > crunched when the new, longer method names meet the 78/80 column pep 8 > restrictions. > > class TestMisc(unittest.test_case): > def lost_four_spaces_here_already(self): > self.fail_if_almost_equal('were already on the 34th column before' > 'writing anything substantive', > self.testedobject.tested_method(arg1, > arg2 + > arg3, > arg4) > # Imagine typing and line wrapping dozens more tests like this > And the way Thunderbird has wrapped your example makes it obvious that 72 is a more satisfactory limit, though I agree transmissibility via email shouldn't necessarily be a factor. > Are there any ideas for some short, pithy, mnemonic names that are > self-explantory and not full of underscores; something that wouldn't > suck to type hundreds of times in a good test module? IMO, the current > names are already too long. > > >> * Explicit is better than implicit: > > Don't forgot the opposing forces: > > Beautiful is better than ugly. > Readability counts. > > These are especially important for the unittest module. When I'm making > tests, I have to type self.very_long_method_name so many times it isn't > funny. It's easy to just stop writing tests when you get tired of it. > Long api names with underscores are a disincentive (IMO). > > >> Use new-style classes throughout > > This makes sense for 3.1 but of course we already get that automatically > for 3.0 ;-) > > For 2.7, it doesn't make as much sense. Since 2.2 came out, there have > been many discussions on changing standard library code to use new-style > classes. There was always some concern about subtle changes in > semantics and for the most part these requests were denied. I think > this reason applies with even more force to the unittest module. Any > risk that we subtlely break someone's test-suite is a small disaster. > The 2.6 and 2.7 unittests need to be absolutely stable if they are to > serve as a transition tool (providing a baseline the 3.0/3.1 is expected > to match). > I'm not quite sure how often it has to be pointed out that since 2.5 all unittest classes *are* new-style classes. Benjamin has, I believe, already mentioned twice that unittest.py contains __metaclass__ = type before any class declarations. While this has no effect on pre-2.2 implementations, surely it means that we've been using new-style classes for some time now. Or am I missing some subtlety? 2.5.1 says: >>> isinstance(unittest.TestCase, type) True > Also, are there any expected benefits from making this change in 2.7? > What will the module do differently? > > It seems like a risky change for zero-benefit. > Better revert it, then :-) But easier to just drop that sentence from the PEP. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From thomas at thomas-lotze.de Tue Jul 15 08:55:59 2008 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Tue, 15 Jul 2008 08:55:59 +0200 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org> <87r69wa8hs.fsf@benfinney.id.au> Message-ID: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> Ben Finney wrote: > I'd count this as another (minor) point in favour of making the 'fail*' > methods canonical: the names are consistent *and* gramatically sensible: -1 I'm surprised nobody (that I've noticed) has brought up the point yet that test code is a lot easier to read if it makes positive assertions. When reading failure conditions, one has to constantly invert them in order to deduce the behaviour that is tested. failUnless and friends aren't better either IMO since while they do work with positive assertions, the method names themselves are doubly negative. assert* methods are so much more straightforward to comprehend. -- Thomas From steve at holdenweb.com Tue Jul 15 09:04:58 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 15 Jul 2008 03:04:58 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org> <87r69wa8hs.fsf@benfinney.id.au> <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> Message-ID: <g5hi76$ecr$1@ger.gmane.org> Thomas Lotze wrote: > Ben Finney wrote: > >> I'd count this as another (minor) point in favour of making the 'fail*' >> methods canonical: the names are consistent *and* gramatically sensible: > > -1 > > I'm surprised nobody (that I've noticed) has brought up the point yet that > test code is a lot easier to read if it makes positive assertions. When > reading failure conditions, one has to constantly invert them in order to > deduce the behaviour that is tested. failUnless and friends aren't better > either IMO since while they do work with positive assertions, the method > names themselves are doubly negative. assert* methods are so much more > straightforward to comprehend. > I think this is where I came in. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From stephen at xemacs.org Tue Jul 15 10:32:53 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 15 Jul 2008 17:32:53 +0900 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <200807150940.44919.steve@pearwood.info> References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <20080713234123.GA11170@mithrandi.net> <87k5fp56ar.fsf@uwakimon.sk.tsukuba.ac.jp> <200807150940.44919.steve@pearwood.info> Message-ID: <8763r7sg22.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote: > > > FWIW, I meant 10 != not not 10. > > >>> 10 != not not 10 > File "<stdin>", line 1 > 10 != not not 10 > ^ > SyntaxError: invalid syntax > > With respect, I think that the fact that you made an analogy with Python > code that you hadn't tested, got it wrong, then corrected it, and > *still* got it wrong, is telling. It's telling only if you ignore the fact that it's the ad hominem fallacy. > When it comes to assert* versus fail* tests, this is one case where I > don't believe "one obvious way to do it" should apply. In the context you were talking about, where you don't need to show anybody your tests, I don't see any reason why TOOWTDI should apply. In a debugging context, I don't see any reason why it should, either. But here we're talking about the standard unit-testing framework, which is used by Python itself. So it *is* about communication with other developers, and it *does* make sense to avoid proliferation of redundant vocabulary. Some of us have enough trouble remembering when parentheses are redundant, as you noticed. Regression test suites aspire to full coverage. If you mix assert* and fail* clauses, you are going to need to invert one or the other to determine whether there are cases that are missed. IMO those are strong indications that only one of the pair should be used in a test suite. > I believe that assert* and fail* tests are the same: In logic, they are. I just think that assert* is much more natural in several simple cases, such as the pure regression test where you use Python X.Y to compute foo(), and add "assertEqual(foo(), pyXYresult)" to the test suite. I'd be a lot more sympathetic if you'd describe a similarly plausible and generic application for fail*. From stephen at xemacs.org Tue Jul 15 10:56:57 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 15 Jul 2008 17:56:57 +0900 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> Message-ID: <874p6rsexy.fsf@uwakimon.sk.tsukuba.ac.jp> Raymond Hettinger writes: > Nobody I know spells setup and teardown as two words. I set up a house of cards. When I'm done, I'm done with setup. Similarly for "tear down" and "teardown". The two word forms are verbs, the one word forms are nouns. I don't think it's worth a column to make that distinction, though. > Another thought is that test suite code is going to get seriously crunched when the new, longer method names meet the 78/80 column > pep 8 restrictions. > > class TestMisc(unittest.test_case): > def lost_four_spaces_here_already(self): Eight spaces, actually. Make that 13 by the time you get to the "." after "self" in the next line. > Are there any ideas for some short, pithy, mnemonic names that are > self-explantory and not full of underscores; something that > wouldn't suck to type hundreds of times in a good test module? "test" or "check" instead of "assert" or "fail_unless" comes to mind to shorten the prefix. But the best I can come up with for "fail_unless_equal" is something like "equalize" which really fails EIBTI. From stephen at xemacs.org Tue Jul 15 11:18:05 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 15 Jul 2008 18:18:05 +0900 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <8763r89go5.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> Message-ID: <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > Removal of ``assert*`` names > ---------------------------- > > There is no overwhelming consensus on whether to remove the > ``assert*`` names or the ``fail*`` names; 7 to 1 is overwhelming in my book. See Message-ID: <loom.20080714T230912-310 at post.gmane.org> From: Antoine Pitrou <solipsis at pitrou.net> While people's preferences are important, I think there is a very good case to made that keeping this much continuity in the test suite as possible is more so. > * Explicit is better than implicit: The ``fail*`` names state *what > the function will do* explicitly: fail the test. With the > ``assert*`` names, the action to be taken is only implicit. EIBTI applies with the most force to "local" names, ie, specific to a particular function, class, or program. Here we propose to impose a community-wide convention. I think we can document it explicitly and expect near-instant uptake on the appropriate connotations to "assert" (especially since that connotation is pretty much universal across languages with an assert facility, anyway). > * Avoid false implication: The test methods do not have any necessary > connection with the built-in ``assert`` statement. Data point: Use of `Assert' as a test method in the XEmacs test suite has never caused any confusion with either C-level asserts, or with the Lisp function `assert'. From ncoghlan at gmail.com Tue Jul 15 11:40:51 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 15 Jul 2008 19:40:51 +1000 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <487BE50E.6090200@voidspace.org.uk> References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE50E.6090200@voidspace.org.uk> Message-ID: <487C70A3.3090808@gmail.com> Michael Foord wrote: > Raymond Hettinger wrote: >> From: "Ben Finney" <ben+python at benfinney.id.au> >>> Right, so I'm putting up a separate PEP just for the renaming. Should >>> be arriving on this list soon. >> >> I would like to work with you or someone else who is interested >> on an alternative PEP for a separate, simpler test module >> using the py.test syntax. That is much simpler to learn and use. >> Instead of self.assertIsNot and whatnot, you write: >> assert a is not b >> No need for tons of word-by-word spellings on things we already >> have syntax for. Almost anyone who has used py.test can attest >> its syntax is much more natural, easy to learn, easy to both >> read and write, and is much lighter weight. I think some variant >> of py.test could be done that is compatable with unittest >> and the did not have the "magic" present in earlier versions of py.test. > > Ah, in my haste I skipped over your comment about "magic", my apologies. > But in the absence of magic how do you propose to provide a meaningful > error message from the failure of: > > assert a == b > > To wrap it in a function like "assert equals(a, b)" seems to gain little > over unittest. Aside from the question of providing nice error messages, two questions that immediately come to mind for me are: - how do I run my tests with -O or -OO? (since the compiler will throw all the assert statements away before any Python code gets a chance to look at them) - how do I test that code raises an expected exception? - how do I explicitly fail a test case? (e.g. I'll often do this when I want to test an operation with a variety of different inputs - I'll test for all of the inputs of interest, collecting the failures in a list, then reporting a single error message at the end detailing all of the cases that failed) And I've also never had any problem whatsoever debugging unit tests with print statements - one of the effects of the -v switch is to display anything which is written to stderr/stdout on the console again. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ben+python at benfinney.id.au Tue Jul 15 12:04:33 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 20:04:33 +1000 Subject: [Python-Dev] Default metaclass in Python 3.0 modules (was: PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15)) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com> Message-ID: <87skub8nv2.fsf_-_@benfinney.id.au> "Benjamin Peterson" <musiccomposition at gmail.com> writes: > On Mon, Jul 14, 2008 at 6:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > > The `unittest` module will gain the following attribute, to set the > > default metaclass for classes in the module and thus make all classes > > in the module part of the new-style type hierarchy:: > > > > __metaclass__ = type > > It's already done. > > Line 94-95 in unittest.py (trunk): > # All classes defined herein are 'new-style' classes, allowing use of 'super()' > __metaclass__ = type Hmm, you're right; I see that in Python 2.5.2 'unittest.py'. Why is it not there in 3.0's 'unittest.py'? Is this achieved some other way? -- \ ?Pinky, are you pondering what I'm pondering?? ?I think so, | `\ Brain, but if they called them ?Sad Meals?, kids wouldn't buy | _o__) them!? ?_Pinky and The Brain_ | Ben Finney From eric+python-dev at trueblade.com Tue Jul 15 12:08:50 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 15 Jul 2008 06:08:50 -0400 Subject: [Python-Dev] Default metaclass in Python 3.0 modules In-Reply-To: <87skub8nv2.fsf_-_@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com> <87skub8nv2.fsf_-_@benfinney.id.au> Message-ID: <487C7732.4000102@trueblade.com> Ben Finney wrote: > "Benjamin Peterson" <musiccomposition at gmail.com> writes: > >> On Mon, Jul 14, 2008 at 6:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote: >>> The `unittest` module will gain the following attribute, to set the >>> default metaclass for classes in the module and thus make all classes >>> in the module part of the new-style type hierarchy:: >>> >>> __metaclass__ = type >> It's already done. >> >> Line 94-95 in unittest.py (trunk): >> # All classes defined herein are 'new-style' classes, allowing use of 'super()' >> __metaclass__ = type > > Hmm, you're right; I see that in Python 2.5.2 'unittest.py'. > > Why is it not there in 3.0's 'unittest.py'? Is this achieved some > other way? In 3.0 there are only new-style classes, so nothing needs to be done there. From bignose+hates-spam at benfinney.id.au Tue Jul 15 12:17:23 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 20:17:23 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> Message-ID: <87od4z8n9o.fsf@benfinney.id.au> "Raymond Hettinger" <python at rcn.com> writes: > > ``set_up(?)`` > > Replaces ``setUp(?)`` > . . > > ``tear_down(?)`` > > Replaces ``tearDown(?)`` > > Am I the only one who finds this sort of excessive pep-8 > underscoring to be horrorific? > > Nobody I know spells setup and teardown as two words. I spell them as two words. The existing unittest framework also spells them as two words, using camelCase. > It's not going to be easy to remember where they are used (ie. > isinstance is still isinstance but isset is now is_set). Go figure. I don't see the connection of this sentence to the existing discussion. > Another thought is that test suite code is going to get seriously > crunched when the new, longer method names meet the 78/80 column pep > 8 restrictions. > > class TestMisc(unittest.test_case): > def lost_four_spaces_here_already(self): > self.fail_if_almost_equal('were already on the 34th column before' > 'writing anything substantive', > self.testedobject.tested_method(arg1, arg2 + > arg3, arg4) > # Imagine typing and line wrapping dozens more tests like this Yikes. Why are you using such huge indents? Break the line where indents won't be so enormous. class TestMisc(unittest.test_case): def lost_four_spaces_here_already(self): self.fail_if_almost_equal( 'we know this is going to be long, so we indent before' 'writing anything substantive', self.testedobject.tested_method( arg1, arg2 + arg3, arg4) > Are there any ideas for some short, pithy, mnemonic names that are > self-explantory and not full of underscores; something that wouldn't > suck to type hundreds of times in a good test module? IMO, the > current names are already too long. I'm very much in favour of having the namessay exactly what they're testing. They need to be long because there are many of them doing slightly-different things, so need careful disambiguation. > Beautiful is better than ugly. > Readability counts. > > These are especially important for the unittest module. When I'm > making tests, I have to type self.very_long_method_name so many times > it isn't funny. It's easy to just stop writing tests when you get > tired of it. Long api names with underscores are a disincentive > (IMO). Yes, this is something that deserves consideration. -- \ ?Why was I with her? She reminds me of you. In fact, she | `\ reminds me more of you than you do!? ?Groucho Marx | _o__) | Ben Finney From bignose+hates-spam at benfinney.id.au Tue Jul 15 12:26:41 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 20:26:41 +1000 Subject: [Python-Dev] Default metaclass in Python 3.0 modules References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com> <87skub8nv2.fsf_-_@benfinney.id.au> <487C7732.4000102@trueblade.com> Message-ID: <87k5fn8mu6.fsf@benfinney.id.au> Eric Smith <eric+python-dev at trueblade.com> writes: > Ben Finney wrote: > > "Benjamin Peterson" <musiccomposition at gmail.com> writes: > >> Line 94-95 in unittest.py (trunk): > >> # All classes defined herein are 'new-style' classes, allowing use of 'super()' > >> __metaclass__ = type > > > > Hmm, you're right; I see that in Python 2.5.2 'unittest.py'. > > > > Why is it not there in 3.0's 'unittest.py'? Is this achieved some > > other way? > > In 3.0 there are only new-style classes, so nothing needs to be done > there. What makes that happen in the case where a class declares no superclass? Is there an invisible enforced "__metaclass__ = type" for every module? Where can I read about this change? -- \ ?The apparent lesson of the Inquisition is that insistence on | `\ uniformity of belief is fatal to intellectual, moral, and | _o__) spiritual health.? ?_The Uses Of The Past_, Herbert J. Muller | Ben Finney From bignose+hates-spam at benfinney.id.au Tue Jul 15 12:31:46 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 20:31:46 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87fxqb8mlp.fsf@benfinney.id.au> "Stephen J. Turnbull" <stephen at xemacs.org> writes: > Ben Finney writes: > > > Removal of ``assert*`` names > > ---------------------------- > > > > There is no overwhelming consensus on whether to remove the > > ``assert*`` names or the ``fail*`` names; > > 7 to 1 is overwhelming in my book. See > Message-ID: <loom.20080714T230912-310 at post.gmane.org> > From: Antoine Pitrou <solipsis at pitrou.net> That measured only usage of unittest *within the Python standard library*. Is that the only body of unittest-using code we need consider? > While people's preferences are important, I think there is a very > good case to made that keeping this much continuity in the test > suite as possible is more so. That's a separate argument, then. One which I don't dismiss, but it needs to be stated as such. -- \ ?A poet more than thirty years old is simply an overgrown | `\ child.? ?Henry L. Mencken | _o__) | Ben Finney From p.f.moore at gmail.com Tue Jul 15 12:41:29 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 15 Jul 2008 11:41:29 +0100 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> Message-ID: <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com> On 15/07/2008, Barry Warsaw <barry at python.org> wrote: > > In case anyone is interested, I have git repositories for both the > > trunk and the py3k branch of the Python source code. They are > > up-to-date and so using them with git-svn would be much faster than > > starting from scratch. > > > > If anyone is interested, I will find a place to host them. They are > > each about 150 MB in size. > > > > Neil, we should try to host them on code.python.org. If we're setting up a variety of DVCS systems there, I'd be willing to set up Mercurial repos (I have my own local one, just trunk at the moment but py3k would be easy enough to add). I'd need access in order to set up any sort of sync process, though. Paul. From asmodai at in-nomine.org Tue Jul 15 12:54:25 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 15 Jul 2008 12:54:25 +0200 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <87fxqb8mlp.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> Message-ID: <20080715105425.GI60130@nexus.in-nomine.org> -On [20080715 12:35], Ben Finney (bignose+hates-spam at benfinney.id.au) wrote: >That measured only usage of unittest *within the Python standard >library*. Is that the only body of unittest-using code we need >consider? Some greps on random Python projects give me a 4-10:1 ratio for assert* versus fail*. Personally I also find the assert* syntax preferable over fail*. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B If I am telling you the Truth now, do you believe it..? From p.f.moore at gmail.com Tue Jul 15 12:58:04 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 15 Jul 2008 11:58:04 +0100 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> Message-ID: <79990c6b0807150358x7ec93c8avefefa19a153aaf70@mail.gmail.com> On 15/07/2008, Raymond Hettinger <python at rcn.com> wrote: > > ``set_up(?)`` > > Replaces ``setUp(?)`` > > > . . > > ``tear_down(?)`` > > Replaces ``tearDown(?)`` > > > > Am I the only one who finds this sort of excessive pep-8 underscoring to be > horrorific? No. > Nobody I know spells setup and teardown as two words. I dread using the > module with these new names. Underscores are not fun to type. They create a > weird_mental_pause when reading them. I agree. The java-esque setUp and tearDown are (in my view) ugly, but set_up and tear_down are as bad. +1 for setup and teardown. +1 also for thinking a *lot* harder about naming, and not just going with phrases_joined_up_with_underscores, or phrasesWithNoSpacesAndFunnyCapitalisation. There are certainly issues with the simple assert expression approach, but ugliness and unreadability aren't two of them. Paul. From ncoghlan at gmail.com Tue Jul 15 12:58:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 15 Jul 2008 20:58:57 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <87od4z8n9o.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> Message-ID: <487C82F1.9010506@gmail.com> Ben Finney wrote: >> self.fail_if_almost_equal('were already on the 34th column before' >> 'writing anything substantive', >> self.testedobject.tested_method(arg1, arg2 + >> arg3, arg4) >> # Imagine typing and line wrapping dozens more tests like this > > Yikes. Why are you using such huge indents? Break the line where > indents won't be so enormous. While true, that doesn't change the fact that fail_if_almost_equal is an undesirably long method name. > I'm very much in favour of having the namessay exactly what they're > testing. They need to be long because there are many of them doing > slightly-different things, so need careful disambiguation. The "almost_equal" methods for floating point comparison are the main culprits for excessive method length, closely followed by the "fail_unless" variants. (21, 'failUnlessAlmostEqual') -> 24 with underscores (21, 'assertNotAlmostEquals') -> 25 with underscores (20, 'assertNotAlmostEqual') -> 24 with underscores (18, 'assertAlmostEquals') -> 20 with underscores (17, 'failIfAlmostEqual') -> 20 with underscores (17, 'assertAlmostEqual') -> 19 with underscores (16, 'failUnlessRaises') -> 18 with underscores (15, 'failUnlessEqual') -> 17 with underscores (15, 'assertNotEquals') -> 17 with underscores (14, 'assertNotEqual') -> 16 with underscores (12, 'assertRaises') -> 13 with underscores (12, 'assertEquals') -> 13 with underscores (11, 'failIfEqual') -> 13 with underscores (11, 'assertFalse') -> 12 with underscores (11, 'assertEqual') -> 12 with underscores (10, 'failUnless') -> 11 with underscores (10, 'assertTrue') -> 11 with underscores (7, 'assert_') -> 7 with underscores (6, 'failIf') -> 7 with underscores (4, 'fail') -> 4 with underscores One option for rationalising the API would be to merely keep the shortest version of each phrase (i.e. use assert* instead of fail_unless* for the positive tests and fail_if* instead of assert_not* for the negative tests, and always drop the trailing 's' from 'equals'). This simple rule would eliminate 12 of the 20 methods (including the four longest method names and 7 of the longest 9), leaving the following minimal set of methods: fail_if_almost_equal (20) assert_almost_equal (19) assert_raises (13) fail_if_equal (13) assert_equal (12) assert_ (7) fail_if (7) fail (4) Adding in possible positive and negative forms of the proposed new methods (using words more appropriate for a method rather than copying the infix operators): fail_if_identical (17) fail_if_contains (16) assert_identical (16) assert_contains (15) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Jul 15 13:02:16 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 15 Jul 2008 21:02:16 +1000 Subject: [Python-Dev] Default metaclass in Python 3.0 modules In-Reply-To: <87k5fn8mu6.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com> <87skub8nv2.fsf_-_@benfinney.id.au> <487C7732.4000102@trueblade.com> <87k5fn8mu6.fsf@benfinney.id.au> Message-ID: <487C83B8.2080403@gmail.com> Ben Finney wrote: > Eric Smith <eric+python-dev at trueblade.com> writes: > >> Ben Finney wrote: >>> "Benjamin Peterson" <musiccomposition at gmail.com> writes: >>>> Line 94-95 in unittest.py (trunk): >>>> # All classes defined herein are 'new-style' classes, allowing use of 'super()' >>>> __metaclass__ = type >>> Hmm, you're right; I see that in Python 2.5.2 'unittest.py'. >>> >>> Why is it not there in 3.0's 'unittest.py'? Is this achieved some >>> other way? >> In 3.0 there are only new-style classes, so nothing needs to be done >> there. > > What makes that happen in the case where a class declares no > superclass? Is there an invisible enforced "__metaclass__ = type" for > every module? Where can I read about this change? The magic is actually in 2.x, not in 3.0. In 2.x, if you don't explicit set the metaclass (or inherit explicitly from an object which sets it), then the default metaclass is 'classobj'. In 3.0, that magic goes away and the default metaclass is just 'type'. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From stephen at xemacs.org Tue Jul 15 13:52:15 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 15 Jul 2008 20:52:15 +0900 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <87fxqb8mlp.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> Message-ID: <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > > Message-ID: <loom.20080714T230912-310 at post.gmane.org> > > From: Antoine Pitrou <solipsis at pitrou.net> > > That measured only usage of unittest *within the Python standard > library*. Is that the only body of unittest-using code we need > consider? Yes, for the purposes of this PEP. We already know that many people want various different things. You want fail* /rather than/ assert*, but Steven d'Aprono wants /both/, and I prefer assert* /exclusively/. I don't see why we all shouldn't be satisfied[1], so the content of unittest should not set policy for our own projects. So there should be other modules (perhaps in the stdlib, perhaps not) to satisfy those preferences not catered to by stdlib's unittest. Thus this PEP should restrict it's concern to revising unittest to conform to PEPs and help standardize Python's own testing, without trying to impose standards on the whole community of Python users. That's my rationale. YMMV. Footnotes: [1] For myself, if the fail*-only proposal wins, I'd switch to that; fighting the tide on this isn't my idea of fun. But other assert* fans may be more faithful to their original preference. From barry at python.org Tue Jul 15 14:29:53 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 15 Jul 2008 08:29:53 -0400 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com> Message-ID: <16F705A1-663A-47C4-873D-5B90E5690AF5@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 15, 2008, at 6:41 AM, Paul Moore wrote: > On 15/07/2008, Barry Warsaw <barry at python.org> wrote: >>> In case anyone is interested, I have git repositories for both the >>> trunk and the py3k branch of the Python source code. They are >>> up-to-date and so using them with git-svn would be much faster than >>> starting from scratch. >>> >>> If anyone is interested, I will find a place to host them. They are >>> each about 150 MB in size. >>> >> >> Neil, we should try to host them on code.python.org. > > If we're setting up a variety of DVCS systems there, I'd be willing to > set up Mercurial repos (I have my own local one, just trunk at the > moment but py3k would be easy enough to add). I'd need access in order > to set up any sort of sync process, though. We need to get the python.org admins involved in the process at this point. I'm pretty sure that Thomas runs the Bazaar sync operations on one of his machines and pushes to code.python.org. I don't know if that's an option for you and/or whether we should bring this in- house. I do remember that at the time, there were issues with bzr-svn requiring svn 1.5, which hadn't been released yet. Now that that's out, and there is a more stable generic fast-import process, perhaps bringing the sync operations in will be easier. Unfortunately I will have no cycles for this in the next several weeks. When we put up the experimental bzr repos, we generalized the scripts we use to set up access control based on ssh. It would be a good thing if we can use the same arrangement to control access to git and hg. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSHyYQXEjvBPtnXfVAQKNUwP/bH9XYHqZPLipE/cWvI6MpFSrLHU7ANPd oa/VBsb0QiO3G1+dWj6Q7czJ2QvV/IE0VmXi37AXy3NSp7of80hRDRdpwNabDvnI cRKSueL/7qCb6+HG0DKoZ+4oh1g94jAok2naUNzffxdx4qFGlQ72WK3bQH1ZrNlp k5ynDPw9Xdg= =oC0s -----END PGP SIGNATURE----- From barry at python.org Tue Jul 15 14:32:37 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 15 Jul 2008 08:32:37 -0400 Subject: [Python-Dev] Reminder: beta 2's schedule for tomorrow Message-ID: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A reminder: the second betas of Python 2.6 and 3.0 are schedule for tomorrow. I will try to hang out on #python-dev today and will start looking at the trackers and buildbots. Hopefully, we're on track to get the releases out! If there is anything you need a decision on, please follow up to this thread. I'm inundated with email so I can't watch every thread on the mailing lists. Or ping me on #python-dev. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSHyY5XEjvBPtnXfVAQKvYgP+J4GYVDWOraKhErC4wRl0ylEdHcc9LypB O7yxhBjb+tLu54IImxLkj/Nzff4uUQI6zsA6lg87A4b+sC0/0ltH4+vGkZaq8z7/ xSlP0b0UOaBpOEhTR8ZJK3DJBjSk97IR8Ty3MV1DuM0cczYltorCmQVpodA5ciXj PRy/LAIalJg= =DTQM -----END PGP SIGNATURE----- From ben+python at benfinney.id.au Tue Jul 15 15:03:01 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 23:03:01 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> Message-ID: <877ibn8flm.fsf@benfinney.id.au> Nick Coghlan <ncoghlan at gmail.com> writes: > fail_if_almost_equal is an undesirably long method name. I disagree. It says what the method does, as precisely as necessary to distinguish it from other methods of the class. It is as long as it needs to be to say that while still being readable and PEP 8 compliant. > One option for rationalising the API would be to merely keep the > shortest version of each phrase (i.e. use assert* instead of > fail_unless* for the positive tests and fail_if* instead of > assert_not* for the negative tests, and always drop the trailing 's' > from 'equals'). I think clarity, consistency, and discoverability are more important criteria than saving a few characters in a method name. > Adding in possible positive and negative forms of the proposed new > methods A major point of this PEP is to *remove* redundant names in the API. -- \ ?It's dangerous to be right when the government is wrong.? | `\ ?Francois Marie Arouet Voltaire | _o__) | Ben Finney From ben+python at benfinney.id.au Tue Jul 15 15:03:52 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 23:03:52 +1000 Subject: [Python-Dev] Default metaclass in Python 3.0 modules References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com> <87skub8nv2.fsf_-_@benfinney.id.au> <487C7732.4000102@trueblade.com> <87k5fn8mu6.fsf@benfinney.id.au> <487C83B8.2080403@gmail.com> Message-ID: <873amb8fk7.fsf@benfinney.id.au> Nick Coghlan <ncoghlan at gmail.com> writes: > Ben Finney wrote: > > What makes that happen in the case where a class declares no > > superclass? Is there an invisible enforced "__metaclass__ = type" > > for every module? Where can I read about this change? > > The magic is actually in 2.x, not in 3.0. In 2.x, if you don't > explicit set the metaclass (or inherit explicitly from an object > which sets it), then the default metaclass is 'classobj'. In 3.0, > that magic goes away and the default metaclass is just 'type'. That helps. Thanks. -- \ ?When I get real bored, I like to drive downtown and get a | `\ great parking spot, then sit in my car and count how many | _o__) people ask me if I'm leaving.? ?Steven Wright | Ben Finney From andrew-pythondev at puzzling.org Tue Jul 15 15:05:33 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Tue, 15 Jul 2008 23:05:33 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <87fxqb8mlp.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> Message-ID: <20080715130533.GB17917@steerpike.home.puzzling.org> Ben Finney wrote: > "Stephen J. Turnbull" <stephen at xemacs.org> writes: > > > Ben Finney writes: > > > > > Removal of ``assert*`` names > > > ---------------------------- > > > > > > There is no overwhelming consensus on whether to remove the > > > ``assert*`` names or the ``fail*`` names; > > > > 7 to 1 is overwhelming in my book. See > > Message-ID: <loom.20080714T230912-310 at post.gmane.org> > > From: Antoine Pitrou <solipsis at pitrou.net> > > That measured only usage of unittest *within the Python standard > library*. Is that the only body of unittest-using code we need > consider? Three more data points then: bzr: 13228 assert* vs. 770 fail*. Twisted: 6149 assert* vs. 1666 fail*. paramiko: 431 assert* vs. 4 fail*. The data seems pretty overwhelmingly in favour of keeping assert*. -Andrew. From dickinsm at gmail.com Tue Jul 15 15:09:17 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 15 Jul 2008 14:09:17 +0100 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> Message-ID: <5c6f2a5d0807150609o136715f8gda5e599f1330f13c@mail.gmail.com> On Tue, Jul 15, 2008 at 1:32 PM, Barry Warsaw <barry at python.org> wrote: > If there is anything you need a decision on, please follow up to this > thread. I'm inundated with email so I can't watch every thread on the > mailing lists. Or ping me on #python-dev. Can I request permission to check in the patch attached to issue 3008 (float <-> hex) conversion? This is the revised version of the bin/oct/hex for floats patch that Raymond had to withdraw shortly after the first beta. Link to the issue: http://bugs.python.org/issue3008 and the long recent thread on python-dev http://mail.python.org/pipermail/python-dev/2008-June/080558.html Mark From solipsis at pitrou.net Tue Jul 15 15:25:16 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Jul 2008 13:25:16 +0000 (UTC) Subject: [Python-Dev] Mercurial mirrors References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com> Message-ID: <loom.20080715T131942-636@post.gmane.org> Paul Moore <p.f.moore <at> gmail.com> writes: > > If we're setting up a variety of DVCS systems there, I'd be willing to > set up Mercurial repos (I have my own local one, just trunk at the > moment but py3k would be easy enough to add). Using the convert extension or using hgsvn? I already have public mirrors using hgsvn (synced every 10 minutes): http://hg.pitrou.net/public/py3k/py3k/ http://hg.pitrou.net/public/cpython/trunk/ I don't know if I'm the only one using them... I know Ralf Schmitt has his own mirrors at http://hgpy.de/py/ , but they don't have the full history. Regards Antoine. From ncoghlan at gmail.com Tue Jul 15 15:29:34 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 15 Jul 2008 23:29:34 +1000 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> Message-ID: <487CA63E.4000207@gmail.com> Barry Warsaw wrote: > A reminder: the second betas of Python 2.6 and 3.0 are schedule for > tomorrow. I will try to hang out on #python-dev today and will start > looking at the trackers and buildbots. Hopefully, we're on track to get > the releases out! > > If there is anything you need a decision on, please follow up to this > thread. I'm inundated with email so I can't watch every thread on the > mailing lists. Or ping me on #python-dev. I'll be checking in the fix for issue 2235 shortly (the problem with __hash__ not being inherited in 2.x). A pre-review from Guido would have been nice (since monkeying with typeobject.c is always a somewhat delicate operation), but it looks like he didn't get a chance to get back to it after Europython. I'm also going to forward port the underlying implementation to Py3k (so that a non-existent hash is indicated by PyObject_HashNotImplemented in tp_hash at the C level as well as by __hash__ = None at the Python level). Cheers, Nick. _______________________________________________ Python-3000 mailing list Python-3000 at python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/ncoghlan%40gmail.com -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jnoller at gmail.com Tue Jul 15 15:36:24 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 15 Jul 2008 09:36:24 -0400 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> Message-ID: <4222a8490807150636w27d3186chd52d7266d0670b65@mail.gmail.com> On Tue, Jul 15, 2008 at 8:32 AM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > A reminder: the second betas of Python 2.6 and 3.0 are schedule for > tomorrow. I will try to hang out on #python-dev today and will start > looking at the trackers and buildbots. Hopefully, we're on track to get the > releases out! > > If there is anything you need a decision on, please follow up to this > thread. I'm inundated with email so I can't watch every thread on the > mailing lists. Or ping me on #python-dev. > > - -Barry > One fix I would like for the upcoming beta is the patch to issue874900 - this seems to have resolved a good chunk of the test_multiprocessing hangs we've been having problems with. I'm in the process of double-checking the latest patch posted by Greg. http://bugs.python.org/issue874900 From steve at pearwood.info Tue Jul 15 15:42:23 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 15 Jul 2008 23:42:23 +1000 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> References: <20080713093936.GA3623@benfinney.id.au> <87r69wa8hs.fsf@benfinney.id.au> <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> Message-ID: <200807152342.24160.steve@pearwood.info> On Tue, 15 Jul 2008 04:55:59 pm Thomas Lotze wrote: > I'm surprised nobody (that I've noticed) has brought up the point yet > that test code is a lot easier to read if it makes positive > assertions. Please don't claim that your subjective opinion is an objective fact. > When reading failure conditions, one has to constantly > invert them in order to deduce the behaviour that is tested. You might have to. Don't assume that everyone else has your difficulty. > failUnless and friends aren't better either IMO since while they do > work with positive assertions, the method names themselves are doubly > negative. assert* methods are so much more straightforward to > comprehend. Maybe for you. That's not a human universal. Please don't assume that your favourite bike-shed colour must be the favourite colour of everyone else too. -- Steven From p.f.moore at gmail.com Tue Jul 15 15:43:03 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 15 Jul 2008 14:43:03 +0100 Subject: [Python-Dev] Mercurial mirrors In-Reply-To: <loom.20080715T131942-636@post.gmane.org> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com> <loom.20080715T131942-636@post.gmane.org> Message-ID: <79990c6b0807150643s6fdcc3fes68511e737768df23@mail.gmail.com> On 15/07/2008, Antoine Pitrou <solipsis at pitrou.net> wrote: > Paul Moore <p.f.moore <at> gmail.com> writes: > > > > If we're setting up a variety of DVCS systems there, I'd be willing to > > set up Mercurial repos (I have my own local one, just trunk at the > > moment but py3k would be easy enough to add). > > Using the convert extension or using hgsvn? > I already have public mirrors using hgsvn (synced every 10 minutes): > http://hg.pitrou.net/public/py3k/py3k/ > http://hg.pitrou.net/public/cpython/trunk/ Personally, I use convert, because it's more robust on WIndows (there were a few commits with case clashes which hgsvn can't get past on a Windows box). For a central repo, I don't care - but I'd prefer it if all the options were hosted on code.python.org, just so that no one option feels more "official" than any other. If I do the work, I'd use convert, simply because I'm more familiar with it, but that's all. OTOH, if the process is just copying a local mirror up to the server, copying your repo might be better (my PC is not always on, and I don't have frequent syncs set up at the moment). Paul. From musiccomposition at gmail.com Tue Jul 15 15:43:58 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 15 Jul 2008 08:43:58 -0500 Subject: [Python-Dev] Default metaclass in Python 3.0 modules (was: PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15)) In-Reply-To: <87skub8nv2.fsf_-_@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <1afaf6160807141730u746ae370x2193d48f7b72f981@mail.gmail.com> <87skub8nv2.fsf_-_@benfinney.id.au> Message-ID: <1afaf6160807150643o2ba28d1awf993cd4594f0036f@mail.gmail.com> On Tue, Jul 15, 2008 at 5:04 AM, Ben Finney <ben+python at benfinney.id.au> wrote: > "Benjamin Peterson" <musiccomposition at gmail.com> writes: >> >> Line 94-95 in unittest.py (trunk): >> # All classes defined herein are 'new-style' classes, allowing use of 'super()' >> __metaclass__ = type > > Hmm, you're right; I see that in Python 2.5.2 'unittest.py'. > > Why is it not there in 3.0's 'unittest.py'? Is this achieved some > other way? All classes are new-style classes in 3.0! -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From ben+python at benfinney.id.au Tue Jul 15 15:48:08 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 23:48:08 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> Message-ID: <87y7436yxz.fsf@benfinney.id.au> Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > Ben Finney wrote: > > "Stephen J. Turnbull" <stephen at xemacs.org> writes: > > > Message-ID: <loom.20080714T230912-310 at post.gmane.org> > > > From: Antoine Pitrou <solipsis at pitrou.net> > > > > That measured only usage of unittest *within the Python standard > > library*. Is that the only body of unittest-using code we need > > consider? > > Three more data points then: > > bzr: 13228 assert* vs. 770 fail*. > Twisted: 6149 assert* vs. 1666 fail*. > paramiko: 431 assert* vs. 4 fail*. > > The data seems pretty overwhelmingly in favour of keeping assert*. Noted, thanks. So far I have "precedent and tradition" and "positive admonition looks better" in support of preferring the 'assert*' names. Are there any others? I believe I've stated (in the most-recent PEP revision) the strongest reasons in favour of the 'fail*' names. This all gets summarised in the Rationale section for the PEP. -- \ ?Killing the creator was the traditional method of patent | `\ protection? ?Terry Pratchett, _Small Gods_ | _o__) | Ben Finney From steve at pearwood.info Tue Jul 15 15:51:41 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 15 Jul 2008 23:51:41 +1000 Subject: [Python-Dev] =?utf-8?q?PEP=3A_Consolidating_names_and_classes_in_?= =?utf-8?q?the=09=60unittest=60_module_=28updated_2008-07-15=29?= In-Reply-To: <20080715105425.GI60130@nexus.in-nomine.org> References: <87vdz8a983.fsf@benfinney.id.au> <87fxqb8mlp.fsf@benfinney.id.au> <20080715105425.GI60130@nexus.in-nomine.org> Message-ID: <200807152351.41909.steve@pearwood.info> On Tue, 15 Jul 2008 08:54:25 pm Jeroen Ruigrok van der Werven wrote: > Some greps on random Python projects give me a 4-10:1 ratio for > assert* versus fail*. Without knowing what those "random" projects are, or what the grep was, it's impossible to interpret that statistic. For example, is it biased by the existence of the assert keyword? Do they represent programmers' free choices, or the projects' compulsory style guides? > Personally I also find the assert* syntax preferable over fail*. Thank you for acknowledging that as a personal preference rather than making another spurious claim of objective superiority. -- Steven From ben+python at benfinney.id.au Tue Jul 15 15:58:02 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 15 Jul 2008 23:58:02 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module References: <87vdz8a983.fsf@benfinney.id.au> Message-ID: <87skub6yhh.fsf@benfinney.id.au> Significant updates include removing all reference to the (already-resolved) new-style class issue, adding footnotes and references, and a Rationale summary of discussion on both sides of the divide for 'assert*' versus 'fail*' names. :PEP: XXX :Title: Consolidating names in the `unittest` module :Version: 0.2 :Last-Modified: 2008-07-15 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: test/x-rst :Created: 2008-07-14 :Python-Version: 2.7, 3.1 :Post-History: .. contents:: Abstract ======== This PEP proposes to consolidate the names that constitute the API of the standard library `unittest` module, with the goal of removing redundant names, and conforming with PEP 8. Motivation ========== The normal use case for the `unittest` module is to subclass its classes, overriding and re-using its functios and methods. This draws constant attention to the fact that the existing implementation fails several current Python standards: * It does not conform to PEP 8 [#PEP-8]_, requiring users to write their own non-PEP-8 conformant names when overriding methods, and encouraging extensions to further depart from PEP 8. * It has many synonyms in its API, which goes against the Zen of Python [#PEP-20]_ (specifically, that "there should be one, and preferably only one, obvious way to do it"). Specification ============= Remove obsolete names --------------------- The following module attributes are not documented as part of the API and are marked as obsolete in the implementation. They will be removed. * ``_makeLoader`` * ``getTestCaseNames`` * ``makeSuite`` * ``findTestCases`` Remove redundant names ---------------------- The following attribute names exist only as synonyms for other names. They are to be removed, leaving only one name for each attribute in the API. ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ * ``assertEqual`` * ``assertEquals`` * ``assertNotEqual`` * ``assertNotEquals`` * ``assertAlmostEqual`` * ``assertAlmostEquals`` * ``assertNotAlmostEqual`` * ``assertNotAlmostEquals`` * ``assertRaises`` * ``assert_`` * ``assertTrue`` * ``assertFalse`` Conform API with PEP 8 ---------------------- The following names are to be introduced, each replacing an existing name, to make all names in the module conform with PEP 8 [#PEP-8]_. Each name is shown with the existing name that it replaces. Where function parameters are to be renamed also, they are shown. Where function parameters are not to be renamed, they are elided with the ellipse ("?") symbol. Module attributes ~~~~~~~~~~~~~~~~~ ``default_test_loader`` Replaces ``defaultTestLoader`` ``TestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``add_error(?)`` Replaces ``addError(?)`` ``add_result(?)`` Replaces ``addResult(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``should_stop`` Replaces ``shouldStop`` ``start_test(?)`` Replaces ``startTest(?)`` ``stop_test(?)`` Replaces ``stopTest(?)`` ``tests_run`` Replaces ``testsRun`` ``was_successful(?)`` Replaces ``wasSuccessful(?)`` ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, method_name='run_test')`` Replaces ``__init__(self, methodName='runTest')`` ``_test_method_doc`` Replaces ``_testMethodDoc`` ``_test_method_name`` Replaces ``_testMethodName`` ``failure_exception`` Replaces ``failureException`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``default_test_result(?)`` Replaces ``defaultTestResult(?)`` ``fail_if(?)`` Replaces ``failIf(?)`` ``fail_if_almost_equal(?)`` Replaces ``failIfAlmostEqual(?)`` ``fail_if_equal(?)`` Replaces ``failIfEqual(?)`` ``fail_unless(?)`` Replaces ``failUnless(?)`` ``fail_unless_almost_equal(?)`` Replaces ``failUnlessAlmostEqual(?)`` ``fail_unless_equal(?)`` Replaces ``failUnlessEqual(?)`` ``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)`` Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``FunctionTestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, test_func, set_up, tear_down, description)`` Replaces ``__init__(self, testFunc, setUp, tearDown, description)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``TestSuite`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~ ``add_test(?)`` Replaces ``addTest(?)`` ``add_tests(?)`` Replaces ``addTests(?)`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``TestLoader`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``sort_test_methods_using`` Replaces ``sortTestMethodsUsing`` ``suite_class`` Replaces ``suiteClass`` ``test_method_prefix`` Replaces ``testMethodPrefix`` ``get_test_case_names(self, test_case_class)`` Replaces ``getTestCaseNames(self, testCaseClass)`` ``load_tests_from_module(?)`` Replaces ``loadTestsFromModule(?)`` ``load_tests_from_name(?)`` Replaces ``loadTestsFromName(?)`` ``load_tests_from_names(?)`` Replaces ``loadTestsFromNames(?)`` ``load_tests_from_test_case(self, test_case_class)`` Replaces ``loadTestsFromTestCase(self, testCaseClass)`` ``_TextTestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``show_all`` Replaces ``showAll`` ``add_error(?)`` Replaces ``addError(?)`` ``add_failure(?)`` Replaces ``addFailure(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``get_description(?)`` Replaces ``getDescription(?)`` ``print_error_list(?)`` Replaces ``printErrorList(?)`` ``print_errors(?)`` Replaces ``printErrors(?)`` ``start_test(?)`` Replaces ``startTest(?)`` ``TextTestRunner`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``_make_result(?)`` Replaces ``_makeResult(?)`` ``TestProgram`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, module, default_test, argv, test_runner, test_loader)`` Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)`` ``create_tests(?)`` Replaces ``createTests(?)`` ``parse_args(?)`` Replaces ``parseArgs(?)`` ``run_tests(?)`` Replaces ``runTests(?)`` ``usage_exit(?)`` Replaces ``usageExit(?)`` Rationale ========= Redundant names --------------- The current API, with two or in some cases three different names referencing exactly the same function, leads to an overbroad and redundant API that violates PEP 20 [#PEP-20]_ ("there should be one, and preferably only one, obvious way to do it"). Removal of ``assert*`` names ---------------------------- While there is consensus support to `remove redundant names`_ for the ``TestCase`` test methods, the issue of which set of names should be retained is controversial. Arguments in favour of retaining only the ``assert*`` names: * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference for the ``assert*`` names. * Precedent: The Python standard library currently uses the ``assert*`` names by a roughly 8:1 majority over the ``fail*`` names. (Counting unit tests in the py3k tree at 2008-07-15 [#pitrou-1]_.) An ad-hoc sampling of other projects that use `unittest` also demonstrates strong preference for use of the ``assert*`` names [#bennetts-1]_. * Positive admonition: The ``assert*`` names state the intent of how the code under test *should* behave, while the ``fail*`` names are phrased in terms of how the code *should not* behave. Arguments in favour of retaining only the ``fail*`` names: * Explicit is better than implicit: The ``fail*`` names state *what the function will do* explicitly: fail the test. With the ``assert*`` names, the action to be taken is only implicit. * Avoid false implication: The test methods do not have any necessary connection with the built-in ``assert`` statement. Even the exception raised, while it defaults to ``AssertionException``, is explicitly customisable via the documented ``failure_exception`` attribute. Choosing the ``fail*`` names avoids the false association with either of these. This is exacerbated by the plain-boolean test using a name of ``assert_`` (with a trailing underscore) to avoid a name collision with the built-in ``assert`` statement. The corresponding ``fail_if`` name has no such issue. PEP 8 names ----------- Although `unittest` (and its predecessor `PyUnit`) are intended to be familiar to users of other xUnit interfaces, there is no attempt at direct API compatibility since the only code that Python's `unittest` interfaces with is other Python code. The module is in the standard library and its names should all conform with PEP 8 [#PEP-8]_. Backwards Compatibility ======================= The names to be obsoleted should be deprecated and removed according to the schedule for modules in PEP 4 [#PEP-4]_. While deprecated, use of the deprecated attributes should raise a ``DeprecationWarning``, with a message stating which replacement name should be used. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. [#PEP-4] http://www.python.org/dev/peps/pep-0004 .. [#PEP-8] http://www.python.org/dev/peps/pep-0008 .. [#PEP-20] http://www.python.org/dev/peps/pep-0020 .. [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-April/078485.html .. [#pitrou-1] http://mail.python.org/pipermail/python-dev/2008-July/081090.html .. [#bennetts-1] http://mail.python.org/pipermail/python-dev/2008-July/081141.html .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From solipsis at pitrou.net Tue Jul 15 16:01:20 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Jul 2008 14:01:20 +0000 (UTC) Subject: [Python-Dev] Mercurial mirrors References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com> <loom.20080715T131942-636@post.gmane.org> <79990c6b0807150643s6fdcc3fes68511e737768df23@mail.gmail.com> Message-ID: <loom.20080715T135736-490@post.gmane.org> Paul Moore <p.f.moore <at> gmail.com> writes: > > Personally, I use convert, because it's more robust on WIndows (there > were a few commits with case clashes which hgsvn can't get past on a > Windows box). If the convert extension is finally usable for incremental mirroring, then it's good news. I don't want to maintain hgsvn all my life :-) > For a central repo, I don't care - but I'd prefer it if all the > options were hosted on code.python.org, just so that no one option > feels more "official" than any other. I completely agree. Regards Antoine. From R.W.Thomas.02 at cantab.net Tue Jul 15 16:00:30 2008 From: R.W.Thomas.02 at cantab.net (Richard Thomas) Date: Tue, 15 Jul 2008 15:00:30 +0100 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <87y7436yxz.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <87y7436yxz.fsf@benfinney.id.au> Message-ID: <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com> On Tue, Jul 15, 2008 at 2:48 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > So far I have "precedent and tradition" and "positive admonition looks > better" in support of preferring the 'assert*' names. Are there any > others? I've been told by a couple of non-programmers that "failUnless" is more intuitive than "assert" if only for the reason that its unclear what "assert" might do. This is similar to one of the arguments raised in the PEP, but considered from the point of view of someone new to test frameworks it could be all the more important. On another note, while I understand that consistency is a good thing is it really that important in test suites? Obviously the unittest module itself should be internally consistent but why not provide people using the all the synonyms they might want? I can see people just wrapping TestCase to add the synonyms back in. Richard From ncoghlan at gmail.com Tue Jul 15 16:16:36 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 16 Jul 2008 00:16:36 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <487C82F1.9010506@gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> Message-ID: <487CB144.1080200@gmail.com> Nick Coghlan wrote: > One option for rationalising the API would be to merely keep the > shortest version of each phrase (i.e. use assert* instead of > fail_unless* for the positive tests and fail_if* instead of > assert_not* for the negative tests, and always drop the trailing 's' > from 'equals'). Disclaimer: I'm not convinced the ideas in this message are actually a good idea myself. But I found them intriguing enough to bother posting them. To give some idea of how the different styles would look, here's an example (based on one of the tests in test_runpy) using a combination of assert* and fail_if* names: self.fail_if_contains(d1, "result") self.assert_identical(d2["initial"], initial) self.assert_equal(d2["result"], self.expected_result) self.assert_equal(d2["nested"]["x"], 1) self.assert_(d2["run_name_in_sys_modules"]) self.assert_(d2["module_in_sys_modules"]) self.assert_identical(d2["run_argv0"], file) self.assert_identical(sys.argv[0], saved_argv0) self.fail_if_contains(sys.modules, name) A somewhat odd thought that occurred to me is that the shortest possible way of writing negative assertions (i.e. asserting that something is not the case) is to treat them as denials and use the single word 'deny'. This approach would give: Test assertions: assert_almost_equal assert_identical assert_contains assert_raises assert_equal assert_ Test denials (negative assertions): deny_almost_equal (17) deny_identical (14) deny_contains (13) deny_equal (10) deny (4) Explicit failure: fail (4) Using the test_runpy example assertions: self.deny_contains(d1, "result") self.assert_identical(d2["initial"], initial) self.assert_equal(d2["result"], self.expected_result) self.assert_equal(d2["nested"]["x"], 1) self.assert_(d2["run_name_in_sys_modules"]) self.assert_(d2["module_in_sys_modules"]) self.assert_identical(d2["run_argv0"], file) self.assert_identical(sys.argv[0], saved_argv0) self.deny_contains(sys.modules, name) I actually quite like that - and it saves not only several characters, but also an underscore over the fail_if* and assert_not* variants. The second odd thought was what happens if the 'assert' is made implicit? Test assertions: are_almost_equal are_identical does_contain does_raise are_equal assert_ Test negative assertions: not_almost_equal not_identical not_contains not_equal not_ Explicit failure: fail Using the test_runpy example assertions: self.not_contains(d1, "result") self.are_identical(d2["initial"], initial) self.are_equal(d2["result"], self.expected_result) self.are_equal(d2["nested"]["x"], 1) self.assert_(d2["run_name_in_sys_modules"]) self.assert_(d2["module_in_sys_modules"]) self.are_identical(d2["run_argv0"], file) self.are_identical(sys.argv[0], saved_argv0) self.not_contains(sys.modules, name) Yecch, I think that idea can be safely consigned to the mental trash heap. Another late night API concept: create a "check" object with the LHS of the binary operation being asserted, then use methods on that check object to supply the RHS: self.check("result").not_in(d1) self.check(d2["initial"]).is_(initial) self.check(d2["result"]).equals(self.expected_result) self.check(d2["nested"]["x"]).equals(1) self.assert_(d2["run_name_in_sys_modules"]) self.assert_(d2["module_in_sys_modules"]) self.check(d2["run_argv0"]).is_(file) self.check(sys.argv[0]).is_(saved_argv0) self.check(name).not_in(sys.modules) This approach would also be handy if you needed to check multiple conditions on a single result: check = self.check(result) check.is_equal(expected_result) # Right answer check.is_not(expected_result) # Fresh object check.is_(recent_results[-1]) # Recorded properly Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ctb at msu.edu Tue Jul 15 16:23:35 2008 From: ctb at msu.edu (C. Titus Brown) Date: Tue, 15 Jul 2008 07:23:35 -0700 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <87y7436yxz.fsf@benfinney.id.au> <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com> Message-ID: <20080715142335.GG2873@caltech.edu> On Tue, Jul 15, 2008 at 03:00:30PM +0100, Richard Thomas wrote: -> On Tue, Jul 15, 2008 at 2:48 PM, Ben Finney <ben+python at benfinney.id.au> wrote: -> > So far I have "precedent and tradition" and "positive admonition looks -> > better" in support of preferring the 'assert*' names. Are there any -> > others? -> -> I've been told by a couple of non-programmers that "failUnless" is -> more intuitive than "assert" if only for the reason that its unclear -> what "assert" might do. This is similar to one of the arguments raised -> in the PEP, but considered from the point of view of someone new to -> test frameworks it could be all the more important. Maybe this is an unnecessarily hard line, but if someone doesn't know what "assert" does, then they should go look up the keyword -- it's part of Python (and present in C, C++, Perl, and PHP as well). So unless the person is new to Python AND testing, they should know it. -> On another note, while I understand that consistency is a good thing -> is it really that important in test suites? Obviously the unittest -> module itself should be internally consistent but why not provide -> people using the all the synonyms they might want? I can see people -> just wrapping TestCase to add the synonyms back in. While I don't have a strong position on the assert* vs fail*, I think consistency and simplicity in test suites is at least as important as in code: if you have to work hard to understand and debug your test suites, you've done something seriously wrong in building your tests. Tests should be as simple as possible, and no simpler. --titus -- C. Titus Brown, ctb at msu.edu From paul at rudin.co.uk Tue Jul 15 16:35:57 2008 From: paul at rudin.co.uk (Paul Rudin) Date: Tue, 15 Jul 2008 15:35:57 +0100 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> Message-ID: <87fxqb5i5u.fsf@rudin.co.uk> Nick Coghlan <ncoghlan at gmail.com> writes: > While true, that doesn't change the fact that fail_if_almost_equal is > an undesirably long method name. fail_if_near ? From p.f.moore at gmail.com Tue Jul 15 17:09:13 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 15 Jul 2008 16:09:13 +0100 Subject: [Python-Dev] Mercurial mirrors In-Reply-To: <loom.20080715T135736-490@post.gmane.org> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <79990c6b0807150341o68ed018dn185dbb706e8ca69e@mail.gmail.com> <loom.20080715T131942-636@post.gmane.org> <79990c6b0807150643s6fdcc3fes68511e737768df23@mail.gmail.com> <loom.20080715T135736-490@post.gmane.org> Message-ID: <79990c6b0807150809o1e01d5d1odc6b6ef1e83d8df8@mail.gmail.com> On 15/07/2008, Antoine Pitrou <solipsis at pitrou.net> wrote: > Paul Moore <p.f.moore <at> gmail.com> writes: > > > > Personally, I use convert, because it's more robust on WIndows (there > > were a few commits with case clashes which hgsvn can't get past on a > > Windows box). > > If the convert extension is finally usable for incremental mirroring, then it's > good news. I don't want to maintain hgsvn all my life :-) It works for me. I think a couple of very recently landed patches[1] may be needed for certain case issues on Windows, but that's all. Also, it requires the Subversion bindings, and the "official" Windows binaries don't include those - but it's easy to build your own copy that does. Paul. [1] In other words, not in 1.0 or even 1.0.1. From ncoghlan at gmail.com Tue Jul 15 17:59:17 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 16 Jul 2008 01:59:17 +1000 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <487CA63E.4000207@gmail.com> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> <487CA63E.4000207@gmail.com> Message-ID: <487CC955.1000305@gmail.com> Nick Coghlan wrote: > Barry Warsaw wrote: >> A reminder: the second betas of Python 2.6 and 3.0 are schedule for >> tomorrow. I will try to hang out on #python-dev today and will start >> looking at the trackers and buildbots. Hopefully, we're on track to >> get the releases out! >> >> If there is anything you need a decision on, please follow up to this >> thread. I'm inundated with email so I can't watch every thread on the >> mailing lists. Or ping me on #python-dev. > > I'll be checking in the fix for issue 2235 shortly (the problem with > __hash__ not being inherited in 2.x). A pre-review from Guido would have > been nice (since monkeying with typeobject.c is always a somewhat > delicate operation), but it looks like he didn't get a chance to get > back to it after Europython. > > I'm also going to forward port the underlying implementation to Py3k (so > that a non-existent hash is indicated by PyObject_HashNotImplemented in > tp_hash at the C level as well as by __hash__ = None at the Python level). This is now done. There's some documentation updates still to do, as well as adding the Py3k warning back in to the 2.6 version, but I won't have time to do that for this release - I dropped the tracker item down to deferred blocker so it gets back on the list for beta 3. Looking at the buildbots, getting the multiprocessing fixes in looks like it will be a major help in getting more of them to go green, and the import related lib2to3 tests also appear to need some attention. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Tue Jul 15 18:26:09 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jul 2008 09:26:09 -0700 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org> <87r69wa8hs.fsf@benfinney.id.au> <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> Message-ID: <ca471dc20807150926x20e4b0f5n77f9a4af355b3d17@mail.gmail.com> On Mon, Jul 14, 2008 at 11:55 PM, somebody wrote: > I'm surprised nobody (that I've noticed) has brought up the point yet that [...] Not picking on whoever wrote that specifically, but if there's anything that surprises me, it's how many people have voiced opinions already (including many of them that I hadn't heard in this group before). There doesn't seem to be an end to this debate, and it is awfully close to deteriorating to pure bikeshedding and attempted ad-hominem attacks. I really don't have time to participate in detail, since all the time I have for Python I need to spend on trying to help review the 2.6 and 3.0 beta releases. But I want to remind people that radical changes to the unittest infrastructure will inconvenience many large 3rd party projects using Python, and I urge folks to look for ways to improve the unittest APIs in other ways instead. It's not the end of the world if the unittesting API uses assertEqual() instead of assert_equal() until the end of times. It would, however, be a shame if we couldn't agree to *add* a bunch of features, for example better reporting when two lists or long strings differ. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Tue Jul 15 18:40:41 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 15 Jul 2008 12:40:41 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module In-Reply-To: <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <1afaf6160807141519g42ea554dt2b437698b4382a68@mail.gmail.com> <87abgk9hr2.fsf@benfinney.id.au> <1afaf6160807141622m140fb14dsb30c8312d1eb8af8@mail.gmail.com> Message-ID: <g5iju9$69j$1@ger.gmane.org> Benjamin Peterson wrote: > On Mon, Jul 14, 2008 at 6:18 PM, Ben Finney <ben+python at benfinney.id.au> wrote: >> "Benjamin Peterson" <musiccomposition at gmail.com> writes: >> >>> On Mon, Jul 14, 2008 at 8:25 AM, Ben Finney <ben+python at benfinney.id.au> wrote: >>>> Use new-style classes throughout >>>> -------------------------------- >>>> >>>> The following classes will inherit explicitly from the built-in >>>> `object` type, to make all classes in the module part of the new-style >>>> type hierarchy. >>>> >>>> * ``TestResult`` >>>> * ``TestCase`` >>>> * ``TestSuite`` >>>> * ``TestLoader`` >>>> * ``_WritelnDecorator`` >>>> * ``TextTestRunner`` >>>> * ``TestProgram`` >>> They already do. __metaclass__ = type is found in unittest.py. >> Not in the copy I have. Is that in 3.x only, or in 2.x also? > > Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) > [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> import unittest >>>> isinstance(unittest.TestCase, object) > True *Everything* is an instance of object. The question is whether the test classes are subclasses of object. In particular, issubclass(unittest.TestCase, object)? Either direct inheritance from object or '__metaclass__ = type' will make them so. With neither, they will not be. tjr From tjreedy at udel.edu Tue Jul 15 19:09:05 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 15 Jul 2008 13:09:05 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <g5iljh$cmt$1@ger.gmane.org> Stephen J. Turnbull wrote: > Yes, for the purposes of this PEP. We already know that many people > want various different things. You want fail* /rather than/ assert*, > but Steven d'Aprono wants /both/, and I prefer assert* /exclusively/. > I don't see why we all shouldn't be satisfied[1], so the content of > unittest should not set policy for our own projects. So there should > be other modules (perhaps in the stdlib, perhaps not) to satisfy those > preferences not catered to by stdlib's unittest. It should be trivial to write a module 'unitfail' that would 'from unittest import *' and then flip the assert names to fail names. The only question is whether it needs to be in the stdlib or merely PyPI. I word this this way because Guido has already blessed the assert forms for unittest. If he is persuaded otherwise, revise accordingly. > Thus this PEP should restrict it's concern to revising unittest to > conform to PEPs and help standardize Python's own testing, without > trying to impose standards on the whole community of Python users. For the community as a whole, all stdlib modules are suggestions and examples, not commands. tjr From tjreedy at udel.edu Tue Jul 15 19:17:47 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 15 Jul 2008 13:17:47 -0400 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <20080715044253.GA31517@caltech.edu> References: <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487BE490.8080406@voidspace.org.uk> <A7CD81E3FFBB4FCAAE71C1851F012B42@RaymondLaptop1> <ca471dc20807142022t1da3b89oc1208e2945cebfb@mail.gmail.com> <487C1ABA.4010701@voidspace.org.uk> <4A646C5A2D4C4188BA641A90E3728B8E@RaymondLaptop1> <20080715044253.GA31517@caltech.edu> Message-ID: <g5im3r$eoa$1@ger.gmane.org> > http://lists.idyll.org/pipermail/testing-in-python/2007-November/000406.html > > & associated thread, for those interested in the variety of mock > libraries... That might be a good beginning for an updateable wiki page on mock libraries. From guido at python.org Tue Jul 15 19:37:54 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jul 2008 10:37:54 -0700 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> Message-ID: <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> On Fri, Jul 11, 2008 at 1:16 PM, Brett Cannon <brett at python.org> wrote: > No, we should eat our own dog food and transition the code over. If > anything it will help with code maintenance between 2.x and 3.x. Agreed. It would be annoying to users trying to clear their own code of -3 warnings if the stdlib emitted warnings they couldn't easily suppress. It should be done with extreme caution though, and preferably on a case-by-case basis rather than in a sweeping pass. Caution should also be taken that the changes aren't merged into 3.0 (where presumably they would conflict with already 3.0-compatible code). I wonder if it might not be simpler (at least in some cases) to just disable the warnings for certain modules? I imagine in many cases fixing up the 2.6 code to suppress -3 warnings would be mere busywork -- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for such code there's no need to suppress the -3 warnings, as they serve as a warning to any user of the module that they are using something that won't survive into 3.0.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at ctypes.org Tue Jul 15 19:38:28 2008 From: theller at ctypes.org (Thomas Heller) Date: Tue, 15 Jul 2008 19:38:28 +0200 Subject: [Python-Dev] ctypes assertion failure In-Reply-To: <486E40B0.4020608@timgolden.me.uk> References: <486E40B0.4020608@timgolden.me.uk> Message-ID: <g5in9o$igt$1@ger.gmane.org> Tim Golden schrieb: > This problem was raised on the comtypes-users list > as it prevents comtypes from being imported on Python 2.6 > at the moment. > > http://bugs.python.org/issue3258 > > I'll try to find the time to step through to code to work out > what's going on, but it's inside the innards of ctypes which > I've never looked into before. > > Could someone confirm at a glance whether this should be > given a high priority, please? It results in an assertion error > in debug mode, a SystemError in non-debug referring to > a NULL return without an Exception set. > The problem is now fixed. -- Thanks, Thomas From musiccomposition at gmail.com Tue Jul 15 19:41:32 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 15 Jul 2008 12:41:32 -0500 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> Message-ID: <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> On Tue, Jul 15, 2008 at 12:37 PM, Guido van Rossum <guido at python.org> wrote: > > I wonder if it might not be simpler (at least in some cases) to just > disable the warnings for certain modules? I imagine in many cases > fixing up the 2.6 code to suppress -3 warnings would be mere busywork > -- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for > such code there's no need to suppress the -3 warnings, as they serve > as a warning to any user of the module that they are using something > that won't survive into 3.0.) Very true. It might be easiest to just throw (maybe even just temporarily) if sys.py3kwarning: warnings.filterwarnings("ignore", category=DeprecationWarning, module=__name__) at the top of the offending module. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From tseaver at palladion.com Tue Jul 15 19:45:44 2008 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 15 Jul 2008 13:45:44 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <20080715130533.GB17917@steerpike.home.puzzling.org> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> Message-ID: <487CE248.9030400@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andrew Bennetts wrote: > Ben Finney wrote: >> "Stephen J. Turnbull" <stephen at xemacs.org> writes: >> >>> Ben Finney writes: >>> >>> > Removal of ``assert*`` names >>> > ---------------------------- >>> > >>> > There is no overwhelming consensus on whether to remove the >>> > ``assert*`` names or the ``fail*`` names; >>> >>> 7 to 1 is overwhelming in my book. See >>> Message-ID: <loom.20080714T230912-310 at post.gmane.org> >>> From: Antoine Pitrou <solipsis at pitrou.net> >> That measured only usage of unittest *within the Python standard >> library*. Is that the only body of unittest-using code we need >> consider? > > Three more data points then: > > bzr: 13228 assert* vs. 770 fail*. > > Twisted: 6149 assert* vs. 1666 fail*. > > paramiko: 431 assert* vs. 4 fail*. > > The data seems pretty overwhelmingly in favour of keeping assert*. FWIW: Zope2: 16878 'assert*', 2892 'fail*'. I would keep both by preference, rather than insist on a "foolish consistency." Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIfOJI+gerLs4ltQ4RArw5AJ0fUzXiMefC4CQZDk4K0mlQFn3nAACfRLya CuhfeNQd8OUPFi5+E5aJN9M= =2Hsl -----END PGP SIGNATURE----- From brett at python.org Tue Jul 15 20:38:27 2008 From: brett at python.org (Brett Cannon) Date: Tue, 15 Jul 2008 11:38:27 -0700 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> Message-ID: <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com> On Tue, Jul 15, 2008 at 5:32 AM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > A reminder: the second betas of Python 2.6 and 3.0 are schedule for > tomorrow. I will try to hang out on #python-dev today and will start > looking at the trackers and buildbots. Hopefully, we're on track to get the > releases out! > > If there is anything you need a decision on, please follow up to this > thread. I'm inundated with email so I can't watch every thread on the > mailing lists. Or ping me on #python-dev. > The new urllib package landed between the last beta and now, but without fixers. Issue3316 has potential ones, but they have some tweaks that need to be made before they are release quality. If the beta goes out 2to3 will not be able to fix imports for urllib and urllib2. Don't know if that is enough to hold up the release or just something to put in the release notes that this will be resolved by the next beta; made it a release blocker to catch your eye, Barry. =) Oh, and fixers for test.test_support to test.support is not in either, but this is probably such a small use case outside the core that I am not sweating bullets for writing a new fixer just for this one case. -Brett From mike.klaas at gmail.com Tue Jul 15 21:16:22 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Tue, 15 Jul 2008 12:16:22 -0700 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <20080715130533.GB17917@steerpike.home.puzzling.org> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> Message-ID: <AD17D4F4-6CDD-47DF-BD6E-8F88D6E96086@gmail.com> On 15-Jul-08, at 6:05 AM, Andrew Bennetts wrote: > Ben Finney wrote: >> "Stephen J. Turnbull" <stephen at xemacs.org> writes: >> >> That measured only usage of unittest *within the Python standard >> library*. Is that the only body of unittest-using code we need >> consider? > > Three more data points then: > > bzr: 13228 assert* vs. 770 fail*. > > Twisted: 6149 assert* vs. 1666 fail*. > > paramiko: 431 assert* vs. 4 fail*. Our internal code base: $ ack self.assert. | wc -l 3232 $ ack self.fail. | wc -l 1124 -Mike From nas at arctrix.com Tue Jul 15 22:04:11 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Tue, 15 Jul 2008 14:04:11 -0600 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> Message-ID: <20080715200411.GA26998@arctrix.com> On Mon, Jul 14, 2008 at 09:31:47PM -0400, Barry Warsaw wrote: > Neil, we should try to host them on code.python.org. I was hoping to get a sense of the interest. Oh well, if you build it they might come. ;-) I've written draft instructions, temporarily at <http://python.ca/nas/tmp/git-notes.txt>. The trunk and py3k repositories are now avaliable at git://code.python.org. They currently get updated every 30 minutes. I'm definitely not a git guru so I hope the instructions work for people. The setup is a little complicated because I want updates to use the more efficient git protocol rather than hammering the SVN server. I've tried making a check-in using dcommit and everything seems okay. I'd gladly receive suggestions on improving the instructions. Neil From musiccomposition at gmail.com Tue Jul 15 22:26:24 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 15 Jul 2008 15:26:24 -0500 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <20080715200411.GA26998@arctrix.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> Message-ID: <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> On Tue, Jul 15, 2008 at 3:04 PM, Neil Schemenauer <nas at arctrix.com> wrote: > On Mon, Jul 14, 2008 at 09:31:47PM -0400, Barry Warsaw wrote: >> Neil, we should try to host them on code.python.org. > > I was hoping to get a sense of the interest. Oh well, if you build > it they might come. ;-) I've written draft instructions, temporarily > at <http://python.ca/nas/tmp/git-notes.txt>. The trunk and py3k > repositories are now avaliable at git://code.python.org. They > currently get updated every 30 minutes. Can we push branches? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Tue Jul 15 23:01:14 2008 From: brett at python.org (Brett Cannon) Date: Tue, 15 Jul 2008 14:01:14 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <20080715200411.GA26998@arctrix.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> Message-ID: <bbaeab100807151401q1439df32re0996716505c1193@mail.gmail.com> On Tue, Jul 15, 2008 at 1:04 PM, Neil Schemenauer <nas at arctrix.com> wrote: > On Mon, Jul 14, 2008 at 09:31:47PM -0400, Barry Warsaw wrote: >> Neil, we should try to host them on code.python.org. > > I was hoping to get a sense of the interest. Oh well, if you build > it they might come. ;-) I've written draft instructions, temporarily > at <http://python.ca/nas/tmp/git-notes.txt>. The trunk and py3k > repositories are now avaliable at git://code.python.org. They > currently get updated every 30 minutes. > > I'm definitely not a git guru so I hope the instructions work for > people. The setup is a little complicated because I want updates to > use the more efficient git protocol rather than hammering the SVN > server. I've tried making a check-in using dcommit and everything > seems okay. I'd gladly receive suggestions on improving the > instructions. > If you are up to keep this up and running, Neil, I (or anyone else with pydotorg access) can put your instructions online like the bzr instructions at http://www.python.org/dev/bazaar/ . -Brett From stephen at xemacs.org Tue Jul 15 23:31:51 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Jul 2008 06:31:51 +0900 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <g5iljh$cmt$1@ger.gmane.org> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> <g5iljh$cmt$1@ger.gmane.org> Message-ID: <87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp> Terry Reedy writes: > For the community as a whole, all stdlib modules are suggestions and > examples, not commands. Well, even if "standard" is too strong a word, the DeprecationWarnings and eventual removal of the methods surely constitute an imposition. From nas at arctrix.com Tue Jul 15 23:31:15 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Tue, 15 Jul 2008 21:31:15 +0000 (UTC) Subject: [Python-Dev] git repositories for trunk and py3k References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> Message-ID: <g5j4v3$6hi$1@ger.gmane.org> Benjamin Peterson <musiccomposition at gmail.com> wrote: > Can we push branches? The git-daemon is setup as read-only. If you have write access to the SVN repository then you can push back changes using git-svn. That's quite a nice way to work, IMHO and provides an easy path for people who are used to Subversion and want to dip their toe in the dvcs waters. While there is no support on code.python.org for publishing your own git branches, there's no reason why you couldn't publish a branch using some other host (it's a dvcs after all). In the short term, perhaps using private branches in combination with "git format-patch" and email is the easiest process. Regards, Neil From solipsis at pitrou.net Tue Jul 15 23:52:35 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Jul 2008 23:52:35 +0200 Subject: [Python-Dev] [Python-3000] commit access request In-Reply-To: <bbaeab100807151357p75e46a0fi4e05663368d74cb5@mail.gmail.com> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> <loom.20080715T134107-337@post.gmane.org> <1afaf6160807150647n3fa5d6c2lbaf0db42bad76fd@mail.gmail.com> <bbaeab100807151134l1ed88favcd2889571cfdd405@mail.gmail.com> <loom.20080715T200625-20@post.gmane.org> <bbaeab100807151357p75e46a0fi4e05663368d74cb5@mail.gmail.com> Message-ID: <1216158755.6020.29.camel@fsol> Le mardi 15 juillet 2008 ? 13:57 -0700, Brett Cannon a ?crit : > I can't give the privileges, Antoine, but I know that an SSH 2 public > key is going to be needed (see the dev FAQ at > http://www.python.org/dev/faq/ if you don't know how to generate one). It's attached. > They will also need to know how you want your first.last name spelled > for your user account. Please make that antoine.pitrou. > Just send that all in a separate email to python-dev and someone who > can add you will get to it at some point (probably soon). Thanks ! Antoine. -------------- next part -------------- ssh-dss AAAAB3NzaC1kc3MAAACBAJiwbeZEP4oPxn/mpbcjPIfmyqDRoyut/AAE76coOCKB3BouwYJPns4Osas6NjR3m9TErNrvcFcm2jCXYTxbrIQmnz1oLwxzmjPK36AUpXtqOy5GPdFUJiVe7iqEBslwtaPVGweXrYdv7ZaI7G6m1ZJDh03uqP6HjNIA1y3yeJFDAAAAFQCxc0R9pwTFUnB4HeHJnx0xE1qg3wAAAIAyxftjIkEAghCBeqbc7W5GRHt2AQ6nczrMCn76pc25+2SkbK750Zk5dAZtIiGkvwNBH2sDdKUzccWkKN7LcyW8ChPLLM1VNVHfgwqQEDbopjd7FkLsOwuQGSxpPoZpTPmewmfJxBJN3kNE5MRVYpzihmqCCoNRi4peMBonuevm4QAAAIEAgCRFJydgXmtk3MiGkX2MyU0DAoAmRzUtJtstYLY0ZqdUvd/0fD4JkpIyZu1N7ROYwzsVCAoU/H6gNGuUFY/NcvfKBUBPZ/yYHECTmstJIWY2X1yVSx59s4lJBZzVUgrP++KTLS+F7n6bil5rsmAaUqg/MoxoMJVLhkvdLvyVPZY= antoine.pitrou From stephen at xemacs.org Wed Jul 16 00:20:40 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Jul 2008 07:20:40 +0900 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <87skub6yhh.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> Message-ID: <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > Removal of ``assert*`` names > ---------------------------- > Arguments in favour of retaining only the ``assert*`` names: > > * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference > for the ``assert*`` names. > > * Precedent: The Python standard library currently uses the > ``assert*`` names by a roughly 8:1 majority over the ``fail*`` > names. (Counting unit tests in the py3k tree at 2008-07-15 > [#pitrou-1]_.) > > An ad-hoc sampling of other projects that use `unittest` also > demonstrates strong preference for use of the ``assert*`` names > [#bennetts-1]_. > > * Positive admonition: The ``assert*`` names state the intent of how > the code under test *should* behave, while the ``fail*`` names are > phrased in terms of how the code *should not* behave. FWIW, I think these are fairly stated. So fairly that I'm surprised you haven't been persuaded!<wink> Nitpick: the second point is not just "precedent", there's an economic reason there too. Tests in the standard distribution which use the deprecated style will need to be converted. Steven d'Aprano claims this is nontrivial (and thus error- prone) in some cases. I haven't seen that claim denied, and it seems plausible to me. > PEP 8 names > ----------- > > Although `unittest` (and its predecessor `PyUnit`) are intended to be > familiar to users of other xUnit interfaces, there is no attempt at > direct API compatibility since the only code that Python's `unittest` > interfaces with is other Python code. The module is in the standard > library and its names should all conform with PEP 8 [#PEP-8]_. You should mention Raymond's concern that PEP 8 names are quite a bit longer. From guido at python.org Wed Jul 16 00:13:22 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jul 2008 15:13:22 -0700 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <ca471dc20807151513i5fede9ccjeaac48a50a05f6b6@mail.gmail.com> On Tue, Jul 15, 2008 at 3:20 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote: > > * Positive admonition: The ``assert*`` names state the intent of how > > the code under test *should* behave, while the ``fail*`` names are > > phrased in terms of how the code *should not* behave. > > FWIW, I think these are fairly stated. So fairly that I'm surprised > you haven't been persuaded!<wink> Nitpick: the second point is not > just "precedent", there's an economic reason there too. Tests in the > standard distribution which use the deprecated style will need to be > converted. Steven d'Aprano claims this is nontrivial (and thus error- > prone) in some cases. I haven't seen that claim denied, and it seems > plausible to me. I'd like to see examples of that (this would be Steven's task if he's serious about his assertion). Since the fail and assert names are mapped to each other using aliasing I don't see how it could be nontrivial to map e.g. self.failIf(x) to self.assertFalse(x) -- these are the same function! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Wed Jul 16 00:23:11 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 15 Jul 2008 18:23:11 -0400 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> Message-ID: <g5j80e$g4d$1@ger.gmane.org> Benjamin Peterson wrote: > On Tue, Jul 15, 2008 at 12:37 PM, Guido van Rossum <guido at python.org> wrote: >> I wonder if it might not be simpler (at least in some cases) to just >> disable the warnings for certain modules? I imagine in many cases >> fixing up the 2.6 code to suppress -3 warnings would be mere busywork >> -- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for >> such code there's no need to suppress the -3 warnings, as they serve >> as a warning to any user of the module that they are using something >> that won't survive into 3.0.) > > Very true. It might be easiest to just throw (maybe even just temporarily) > > if sys.py3kwarning: > warnings.filterwarnings("ignore", category=DeprecationWarning, > module=__name__) > > at the top of the offending module. Is it possible to *first* 'raise' a DeprecationWarning("This module will disappear") before turning the rest off? Exactly 1 seems the right number to me for modules that will go. From ben+python at benfinney.id.au Wed Jul 16 00:34:00 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 08:34:00 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <87y7436yxz.fsf@benfinney.id.au> <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com> <20080715142335.GG2873@caltech.edu> Message-ID: <87d4le7p5z.fsf@benfinney.id.au> "C. Titus Brown" <ctb at msu.edu> writes: > On Tue, Jul 15, 2008 at 03:00:30PM +0100, Richard Thomas wrote: > -> I've been told by a couple of non-programmers that "failUnless" > -> is more intuitive than "assert" if only for the reason that its > -> unclear what "assert" might do. This is similar to one of the > -> arguments raised in the PEP, but considered from the point of > -> view of someone new to test frameworks it could be all the more > -> important. > > Maybe this is an unnecessarily hard line, but if someone doesn't know > what "assert" does, then they should go look up the keyword -- it's > part of Python (and present in C, C++, Perl, and PHP as well). So > unless the person is new to Python AND testing, they should know it. That's exactly the problem with the 'assert*' names: The test methods of TestCase *don't* do the same thing as the Python 'assert' statement, and aren't meant to. The association is confusing, even (especially?) if one knows what the 'assert' statement does. > While I don't have a strong position on the assert* vs fail*, I > think consistency and simplicity in test suites is at least as > important as in code: if you have to work hard to understand and > debug your test suites, you've done something seriously wrong in > building your tests. Yes. While I prefer the 'fail*' names, I would prefer to lose *either* of 'assert*' or 'fail*' than to retain redundant names in the API. -- \ ?If I had known what it would be like to have it all... I might | `\ have been willing to settle for less.? ?Jane Wagner, via Lily | _o__) Tomlin | Ben Finney From guido at python.org Wed Jul 16 00:37:37 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 15 Jul 2008 15:37:37 -0700 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <87d4le7p5z.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <87y7436yxz.fsf@benfinney.id.au> <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com> <20080715142335.GG2873@caltech.edu> <87d4le7p5z.fsf@benfinney.id.au> Message-ID: <ca471dc20807151537n2f639ef8t7e5e6ae8b4bb7445@mail.gmail.com> On Tue, Jul 15, 2008 at 3:34 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > That's exactly the problem with the 'assert*' names: The test methods > of TestCase *don't* do the same thing as the Python 'assert' > statement, and aren't meant to. The association is confusing, even > (especially?) if one knows what the 'assert' statement does. The parallel is intentional. They even raise the same exception. Too bad it doesn't work for you; get over it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Jul 16 00:38:58 2008 From: brett at python.org (Brett Cannon) Date: Tue, 15 Jul 2008 15:38:58 -0700 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <g5j80e$g4d$1@ger.gmane.org> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> <g5j80e$g4d$1@ger.gmane.org> Message-ID: <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com> On Tue, Jul 15, 2008 at 3:23 PM, Terry Reedy <tjreedy at udel.edu> wrote: > > > Benjamin Peterson wrote: >> >> On Tue, Jul 15, 2008 at 12:37 PM, Guido van Rossum <guido at python.org> >> wrote: >>> >>> I wonder if it might not be simpler (at least in some cases) to just >>> disable the warnings for certain modules? I imagine in many cases >>> fixing up the 2.6 code to suppress -3 warnings would be mere busywork >>> -- e.g. for code that gets deleted in 3.0 altogether. (Hm, in fact for >>> such code there's no need to suppress the -3 warnings, as they serve >>> as a warning to any user of the module that they are using something >>> that won't survive into 3.0.) >> >> Very true. It might be easiest to just throw (maybe even just temporarily) >> >> if sys.py3kwarning: >> warnings.filterwarnings("ignore", category=DeprecationWarning, >> module=__name__) >> >> at the top of the offending module. > > Is it possible to *first* 'raise' a DeprecationWarning("This module will > disappear") before turning the rest off? Exactly 1 seems the right number > to me for modules that will go. > Wouldn't you just add the filter after the DeprecationWarning is raised to have that behavior? -Brett From ben+python at benfinney.id.au Wed Jul 16 00:36:56 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 08:36:56 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> <g5iljh$cmt$1@ger.gmane.org> <87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <878ww27p13.fsf@benfinney.id.au> "Stephen J. Turnbull" <stephen at xemacs.org> writes: > Terry Reedy writes: > > > For the community as a whole, all stdlib modules are suggestions > > and examples, not commands. > > Well, even if "standard" is too strong a word, the DeprecationWarnings > and eventual removal of the methods surely constitute an imposition. I understood Terry's statement as meaning that there is no commandment to use the standard library 'unittest' module at all. If a user doesn't like the behaviour of a module (such as 'unittest') in the standard library, they can accept the burden of implementing one that behaves the way they like. -- \ ?I never forget a face, but in your case I'll be glad to make | `\ an exception.? ?Groucho Marx | _o__) | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 00:41:47 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 08:41:47 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> <487CB144.1080200@gmail.com> Message-ID: <87zloi6a8k.fsf@benfinney.id.au> Nick Coghlan <ncoghlan at gmail.com> writes: > A somewhat odd thought that occurred to me is that the shortest > possible way of writing negative assertions (i.e. asserting that > something is not the case) is to treat them as denials and use the > single word 'deny'. -1 This, to me, is neither intuitive nor meaningful in context. The term "deny" is strongly linked to its antonym, "permit". Whom is being denied? What have they asked to do that I am denying in my test? I think in terms of "true or false", or "pass or fail". I'm making statements about behaviour of the program, not about permitting or denying something. -- \ ?The industrial system is profoundly dependent on commercial | `\ television and could not exist in its present form without it.? | _o__) ?John Kenneth Galbraith, _The New Industrial State_, 1967 | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 00:38:59 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 08:38:59 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <487CE248.9030400@palladion.com> Message-ID: <874p6q7oxo.fsf@benfinney.id.au> Tres Seaver <tseaver at palladion.com> writes: > FWIW: Zope2: 16878 'assert*', 2892 'fail*'. Thanks. > I would keep both by preference, rather than insist on a "foolish > consistency." The "consistency" argument leads to the PEP 8 names. The removal of redundant names is not made in the name of consistency, but of API simplicity. -- \ ?Under democracy one party always devotes its chief energies to | `\ trying to prove that the other party is unfit to rule--and both | _o__) commonly succeed, and are right.? ?Henry L. Mencken | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 00:44:47 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 08:44:47 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <487A8F7B.1090901@holdenweb.com> <87d4lhba7b.fsf@benfinney.id.au> <487B18EC.6040104@gmail.com> <874p6sbx25.fsf@benfinney.id.au> <g5fhol$411$1@ger.gmane.org> <87r69wa8hs.fsf@benfinney.id.au> <pan.2008.07.15.05.35.13.848660@ID-174572.user.uni-berlin.de> <ca471dc20807150926x20e4b0f5n77f9a4af355b3d17@mail.gmail.com> Message-ID: <87vdz66a3k.fsf@benfinney.id.au> "Guido van Rossum" <guido at python.org> writes: > It would, however, be a shame if we couldn't agree to *add* a bunch > of features, for example better reporting when two lists or long > strings differ. I intend to phrase such additions in terms of PEP-8-only names, so this name consolidation seems a natural prerequisite. -- \ ?Are you pondering what I'm pondering?? ?Well, I think so | `\ Brain, but what if we stick to the seat covers?? ?_Pinky and | _o__) The Brain_ | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 00:59:08 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 08:59:08 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87r69u69fn.fsf@benfinney.id.au> "Stephen J. Turnbull" <stephen at xemacs.org> writes: > FWIW, I think these are fairly stated. So fairly that I'm surprised > you haven't been persuaded!<wink> I'm not persuaded because I find the arguments for 'fail*' more persuasive :-) I am, however, convinced that the consensus of the community is that 'assert*' is the right choice. The PEP will be revised accordingly. > Nitpick: the second point is not just "precedent", there's an > economic reason there too. [...] > You should mention Raymond's concern that PEP 8 names are quite a > bit longer. Noted both of these, thanks. -- \ ?The best way to get information on Usenet is not to ask a | `\ question, but to post the wrong information.? ?Aahz | _o__) | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 01:40:05 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 09:40:05 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module (updated 2008-07-16) References: <87vdz8a983.fsf@benfinney.id.au> Message-ID: <87k5fm67je.fsf@benfinney.id.au> Significant changes are resolving to remove the 'fail*' names, giving recommended name to use for each removed name, and further summary of related discussion. :PEP: XXX :Title: Consolidating names in the `unittest` module :Version: 0.3 :Last-Modified: 2008-07-16 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: test/x-rst :Created: 2008-07-14 :Python-Version: 2.7, 3.1 :Post-History: .. contents:: Abstract ======== This PEP proposes to consolidate the names that constitute the API of the standard library `unittest` module, with the goal of removing redundant names, and conforming with PEP 8. Motivation ========== The normal use case for the `unittest` module is to subclass its classes, overriding and re-using its functios and methods. This draws constant attention to the fact that the existing implementation fails several current Python standards: * It does not conform to PEP 8 [#PEP-8]_, requiring users to write their own non-PEP-8 conformant names when overriding methods, and encouraging extensions to further depart from PEP 8. * It has many synonyms in its API, which goes against the Zen of Python [#PEP-20]_ (specifically, that "there should be one, and preferably only one, obvious way to do it"). Specification ============= Remove obsolete names --------------------- The following module attributes are not documented as part of the API and are marked as obsolete in the implementation. They will be removed. * ``_makeLoader`` * ``getTestCaseNames`` * ``makeSuite`` * ``findTestCases`` Remove redundant names ---------------------- The following attribute names exist only as synonyms for other names. They are to be removed, leaving only one name for each attribute in the API. ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ ``assert_`` Use ``assert_true``, or an ``assert`` statement. ``assertEquals`` Use ``assert_equal``. ``assertNotEquals`` Use ``assert_not_equal``. ``assertAlmostEquals`` Use ``assert_almost_equal``. ``assertNotAlmostEquals`` Use ``assert_not_almost_equal``. ``failIf`` Use ``assert_false``. ``failUnless`` Use ``assert_true``. ``failIfAlmostEqual`` Use ``assert_not_almost_equal``. ``failIfEqual`` Use ``assert_not_equal``. ``failUnlessAlmostEqual`` Use ``assert_almost_equal``. ``failUnlessEqual`` Use ``assert_equal``. ``failUnlessRaises`` Use ``assert_raises``. Conform API with PEP 8 ---------------------- The following names are to be introduced, each replacing an existing name, to make all names in the module conform with PEP 8 [#PEP-8]_. Each name is shown with the existing name that it replaces. Where function parameters are to be renamed also, they are shown. Where function parameters are not to be renamed, they are elided with the ellipse ("?") symbol. Module attributes ~~~~~~~~~~~~~~~~~ ``default_test_loader`` Replaces ``defaultTestLoader`` ``TestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``add_error(?)`` Replaces ``addError(?)`` ``add_result(?)`` Replaces ``addResult(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``should_stop`` Replaces ``shouldStop`` ``start_test(?)`` Replaces ``startTest(?)`` ``stop_test(?)`` Replaces ``stopTest(?)`` ``tests_run`` Replaces ``testsRun`` ``was_successful(?)`` Replaces ``wasSuccessful(?)`` ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, method_name='run_test')`` Replaces ``__init__(self, methodName='runTest')`` ``_test_method_doc`` Replaces ``_testMethodDoc`` ``_test_method_name`` Replaces ``_testMethodName`` ``failure_exception`` Replaces ``failureException`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``default_test_result(?)`` Replaces ``defaultTestResult(?)`` ``assert_true(?)`` Replaces ``assertTrue(?)`` ``assert_false(?)`` Replaces ``assertFalse(?)`` ``assert_almost_equal(?)`` Replaces ``assertAlmostEqual(?)`` ``assert_equal(?)`` Replaces ``assertEqual(?)`` ``assert_not_almost_equal(?)`` Replaces ``assertNotAlmostEqual(?)`` ``assert_not_equal(?)`` Replaces ``assertNotEqual(?)`` ``assert_raises(exc_class, callable_obj, *args, **kwargs)`` Replaces ``assertRaises(excClass, callableObj, *args, **kwargs)`` ``run_test(?)`` Replaces ``runTest(?)`` ``setup(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``teardown(?)`` Replaces ``tearDown(?)`` ``FunctionTestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, test_func, set_up, tear_down, description)`` Replaces ``__init__(self, testFunc, setUp, tearDown, description)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``TestSuite`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~ ``add_test(?)`` Replaces ``addTest(?)`` ``add_tests(?)`` Replaces ``addTests(?)`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``TestLoader`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``sort_test_methods_using`` Replaces ``sortTestMethodsUsing`` ``suite_class`` Replaces ``suiteClass`` ``test_method_prefix`` Replaces ``testMethodPrefix`` ``get_test_case_names(self, test_case_class)`` Replaces ``getTestCaseNames(self, testCaseClass)`` ``load_tests_from_module(?)`` Replaces ``loadTestsFromModule(?)`` ``load_tests_from_name(?)`` Replaces ``loadTestsFromName(?)`` ``load_tests_from_names(?)`` Replaces ``loadTestsFromNames(?)`` ``load_tests_from_test_case(self, test_case_class)`` Replaces ``loadTestsFromTestCase(self, testCaseClass)`` ``_TextTestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``show_all`` Replaces ``showAll`` ``add_error(?)`` Replaces ``addError(?)`` ``add_failure(?)`` Replaces ``addFailure(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``get_description(?)`` Replaces ``getDescription(?)`` ``print_error_list(?)`` Replaces ``printErrorList(?)`` ``print_errors(?)`` Replaces ``printErrors(?)`` ``start_test(?)`` Replaces ``startTest(?)`` ``TextTestRunner`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``_make_result(?)`` Replaces ``_makeResult(?)`` ``TestProgram`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, module, default_test, argv, test_runner, test_loader)`` Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)`` ``create_tests(?)`` Replaces ``createTests(?)`` ``parse_args(?)`` Replaces ``parseArgs(?)`` ``run_tests(?)`` Replaces ``runTests(?)`` ``usage_exit(?)`` Replaces ``usageExit(?)`` Rationale ========= Redundant names --------------- The current API, with two or in some cases three different names referencing exactly the same function, leads to an overbroad and redundant API that violates PEP 20 [#PEP-20]_ ("there should be one, and preferably only one, obvious way to do it"). Removal of ``fail*`` names -------------------------- While there is consensus support to `remove redundant names`_ for the ``TestCase`` test methods, the issue of which set of names should be retained is controversial (it was several times characterised as "bike-shedding" [#bikeshed]_ by the BDFL and others). With good arguments made in favour of each of "retain ``assert*``" and "retain ``fail*``", this PEP resolves in favour of stated BDFL preference, and de facto community preference. Arguments in favour of retaining only the ``assert*`` names: * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference for the ``assert*`` names. * Precedent: The Python standard library currently uses the ``assert*`` names by a roughly 8:1 majority over the ``fail*`` names. (Counting unit tests in the py3k tree at 2008-07-15 [#pitrou-1]_.) An ad-hoc sampling of other projects that use `unittest` also demonstrates strong preference for use of the ``assert*`` names [#bennetts-1]_, [#seaver-1]_. * Economic: The above precedent indicates a simple economic argument [#turnbull-1]_: the majority of tests will not need their statements re-phrased, so updating in favour of them will involve less disruption and risk of error. * Positive admonition: The ``assert*`` names state the intent of how the code under test *should* behave, while the ``fail*`` names are phrased in terms of how the code *should not* behave. Arguments in favour of retaining only the ``fail*`` names: * Explicit is better than implicit: The ``fail*`` names state *what the function will do* explicitly: fail the test. With the ``assert*`` names, the action to be taken is only implicit. * Avoid false implication: The test methods do not have any necessary connection with the built-in ``assert`` statement. Even the exception raised, while it defaults to ``AssertionException``, is explicitly customisable via the documented ``failure_exception`` attribute. Choosing the ``fail*`` names avoids the false association with either of these. It has been argued [#thomas-1]_ that newcomers to `unittest` who already know what ``assert`` does can be confused by the behaviour of the ``assert*`` names. This confusion does not exist for the ``fail*`` names, which don't conflict with existing concepts. * Avoid name collision: The above confusion with ``assert`` is exacerbated by the plain-boolean test using a name of ``assert_`` (with a trailing underscore) to avoid a name collision with the built-in ``assert`` statement. The corresponding ``fail_if`` name has no such issue. PEP 8 names ----------- Although `unittest` (and its predecessor `PyUnit`) are intended to be familiar to users of other xUnit interfaces, there is no attempt at direct API compatibility since the only code that Python's `unittest` interfaces with is other Python code. The module is in the standard library and its names should all conform with PEP 8 [#PEP-8]_. While argument was raised [#hettinger-1]_ over the length of names resulting from a simple conversion of names to multi_word_with_underscores, this PEP chooses names that are consistent in wording with the existing names. An exception was made in the case of ``setup`` and ``teardown``. The two-word form of these names was judged [#hettinger-1]_ too cumbersome compared to the new single-word form. Backwards Compatibility ======================= The names to be obsoleted should be deprecated and removed according to the schedule for modules in PEP 4 [#PEP-4]_. While deprecated, use of the deprecated attributes should raise a ``DeprecationWarning``, with a message stating which replacement name should be used. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. [#PEP-4] http://www.python.org/dev/peps/pep-0004 .. [#PEP-8] http://www.python.org/dev/peps/pep-0008 .. [#PEP-20] http://www.python.org/dev/peps/pep-0020 .. [#bikeshed] http://www.catb.org/~esr/jargon/html/B/bikeshedding.html .. [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-April/078485.html .. [#pitrou-1] http://mail.python.org/pipermail/python-dev/2008-July/081090.html .. [#bennetts-1] http://mail.python.org/pipermail/python-dev/2008-July/081141.html .. [#seaver-1] http://mail.python.org/pipermail/python-dev/2008-July/081166.html .. [#thomas-1] http://mail.python.org/pipermail/python-dev/2008-July/081153.html .. [#hettinger-1] http://mail.python.org/pipermail/python-dev/2008-July/081107.html .. [#turnbull-1] http://mail.python.org/pipermail/python-dev/2008-July/081175.html .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From barry at python.org Wed Jul 16 01:42:17 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 15 Jul 2008 19:42:17 -0400 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com> Message-ID: <9CD62042-4AA1-421E-BB63-129DEC30175A@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I realized that the previous schedule really called for a release today 7/15 because I'm a bit busy tomorrow night. In any event, let's try to stick to doing it on 7/16. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSH012XEjvBPtnXfVAQLJiQQAiP+/S1KVuv9ZgzyS7XdQNW2USTcBGkCh LIQs3Z+ueuZ9MORdwuDnuZmJWpxataYe26UiPKxt8BFJO81x1i7hy4/uvrO4aE+z doksz9eJtclNWQNAcvgRX2bKM0OBO57egJrqxYFUgqlziELxOT/U2//josdl3jR1 ovOb/C0nWVU= =LBea -----END PGP SIGNATURE----- From barry at python.org Wed Jul 16 01:45:42 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 15 Jul 2008 19:45:42 -0400 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com> Message-ID: <AD5D714F-3513-4BF5-A72C-2ADF6C62181E@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 15, 2008, at 2:38 PM, Brett Cannon wrote: > On Tue, Jul 15, 2008 at 5:32 AM, Barry Warsaw <barry at python.org> > wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> A reminder: the second betas of Python 2.6 and 3.0 are schedule for >> tomorrow. I will try to hang out on #python-dev today and will start >> looking at the trackers and buildbots. Hopefully, we're on track >> to get the >> releases out! >> >> If there is anything you need a decision on, please follow up to this >> thread. I'm inundated with email so I can't watch every thread on >> the >> mailing lists. Or ping me on #python-dev. >> > > The new urllib package landed between the last beta and now, but > without fixers. Issue3316 has potential ones, but they have some > tweaks that need to be made before they are release quality. If the > beta goes out 2to3 will not be able to fix imports for urllib and > urllib2. Don't know if that is enough to hold up the release or just > something to put in the release notes that this will be resolved by > the next beta; made it a release blocker to catch your eye, Barry. =) I knocked this down to a deferred blocker, so I won't let it hold up beta2, though I'd /really/ like to get this cleaned up and committed in time. If I get enough deferred blockers though, I might hold things up after all. > Oh, and fixers for test.test_support to test.support is not in either, > but this is probably such a small use case outside the core that I am > not sweating bullets for writing a new fixer just for this one case. Ok. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSH02pnEjvBPtnXfVAQL1qgP/a2yf6fagOS7Hvbwd08ZhYkaB+GDTcFMy mo/bJtvrdCRwPm+60q5mZYuMBIz/I7IBSQmSI09IOdEi0RPoXH6Los3LMrt+Aa6v XgPu7nYcO1vNa/b+6vNvsBAEPfEuaMJsSRpHNJfAWsjF82BsiNEpJ0D5906CAhyW MrjaPUb47bs= =/qoQ -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Wed Jul 16 01:46:02 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Jul 2008 11:46:02 +1200 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <d06a5cd30807141918w3d574c36lb9c84435e014fe36@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <d06a5cd30807141918w3d574c36lb9c84435e014fe36@mail.gmail.com> Message-ID: <487D36BA.7060807@canterbury.ac.nz> Jonathan Lange wrote: > My name's Jonathan, and I spell "set up" as "set up" and "tear down" > as "tear down". In English, it depends on how they're being used. As nouns they're single words, as verbs they're two words. As function names, they could be read either way, so it comes down to readability. To my eyes there is no loss of readability when omitting the underscores, so simplicity argues for leaving them out. -- Greg From steve at pearwood.info Wed Jul 16 01:54:56 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 16 Jul 2008 09:54:56 +1000 Subject: [Python-Dev] =?iso-8859-1?q?unittest=27s_redundant_assertions=3A_?= =?iso-8859-1?q?asserts=09vs=2E=09failIf/Unlesses?= In-Reply-To: <8763r7sg22.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> <200807150940.44919.steve@pearwood.info> <8763r7sg22.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200807160954.56875.steve@pearwood.info> On Tue, 15 Jul 2008 06:32:53 pm Stephen J. Turnbull wrote: > Steven D'Aprano writes: > > On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote: > > > FWIW, I meant 10 != not not 10. > > > > > >>> 10 != not not 10 > > > > File "<stdin>", line 1 > > 10 != not not 10 > > ^ > > SyntaxError: invalid syntax > > > > With respect, I think that the fact that you made an analogy with > > Python code that you hadn't tested, got it wrong, then corrected > > it, and *still* got it wrong, is telling. > > It's telling only if you ignore the fact that it's the ad hominem > fallacy. The ad hominem fallacy does not mean "somebody pointed out I made a silly mistake". It is not a fallacy to reject a person's argument because it is hastily or carelessly made. However, fair is fair -- I too have to accept a slice of humble pie. In *my* haste to make my point, I neglected to actually make the point I intended to. I went on to write "It's part of the pattern of this thread" but didn't explain why. Namely that with so many people trying to push their personal preference, and signs of frustration on both sides, people are making unsupported statements of opinion as if they were facts, and generally getting sloppy. Ironic, yes? Having communicated poorly, and by doing so giving you offence, I will accept an appropriate fish-slapping. > > When it comes to assert* versus fail* tests, this is one case > > where I don't believe "one obvious way to do it" should apply. > > In the context you were talking about, where you don't need to show > anybody your tests, I don't see any reason why TOOWTDI should apply. > In a debugging context, I don't see any reason why it should, either. > > But here we're talking about the standard unit-testing framework, > which is used by Python itself. But I'm using that same framework, so decisions made regarding that framework are going to impact me. Terry Reedy says that "stdlib modules are suggestions and examples, not commands", which is strictly true but I think he discounts just how strong the stdlib's influence is. Perhaps large projects, and tiny ones, are free to go their own way, but in my experience (such that it is), small projects tend to follow the stdlib. > So it *is* about communication with > other developers, and it *does* make sense to avoid proliferation of > redundant vocabulary. Certainly redundant vocabulary has a cost, but it also has a benefit. At least for those people (including myself) who frequently think of a unit test as a sequence of potential failures rather than expected successes. Only if the code "fails to fail" does it pass the test, and so I often find fail* tests more suitable. YMMV. -- Steven From solipsis at pitrou.net Tue Jul 15 19:55:00 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Jul 2008 19:55:00 +0200 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <87y7436yxz.fsf@benfinney.id.au> Message-ID: <1216144500.6020.5.camel@fsol> > So far I have "precedent and tradition" and "positive admonition looks > better" in support of preferring the 'assert*' names. Are there any > others? Avoiding double negatives. (and don't tell me "fail" is not a negative word, because you just used the phrase "positive admonition" to refer to the assert* variants, which implies quite clearly that the fail* variants are on the negative side ;-)) Regards Antoine. From brett at python.org Wed Jul 16 02:05:26 2008 From: brett at python.org (Brett Cannon) Date: Tue, 15 Jul 2008 17:05:26 -0700 Subject: [Python-Dev] [Python-3000] Reminder: beta 2's schedule for tomorrow In-Reply-To: <AD5D714F-3513-4BF5-A72C-2ADF6C62181E@python.org> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> <bbaeab100807151138w1d61ed83k863016187708b25c@mail.gmail.com> <AD5D714F-3513-4BF5-A72C-2ADF6C62181E@python.org> Message-ID: <bbaeab100807151705q56cade4cpc9b6417d8f70d34c@mail.gmail.com> On Tue, Jul 15, 2008 at 4:45 PM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jul 15, 2008, at 2:38 PM, Brett Cannon wrote: > >> On Tue, Jul 15, 2008 at 5:32 AM, Barry Warsaw <barry at python.org> wrote: >>> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> A reminder: the second betas of Python 2.6 and 3.0 are schedule for >>> tomorrow. I will try to hang out on #python-dev today and will start >>> looking at the trackers and buildbots. Hopefully, we're on track to get >>> the >>> releases out! >>> >>> If there is anything you need a decision on, please follow up to this >>> thread. I'm inundated with email so I can't watch every thread on the >>> mailing lists. Or ping me on #python-dev. >>> >> >> The new urllib package landed between the last beta and now, but >> without fixers. Issue3316 has potential ones, but they have some >> tweaks that need to be made before they are release quality. If the >> beta goes out 2to3 will not be able to fix imports for urllib and >> urllib2. Don't know if that is enough to hold up the release or just >> something to put in the release notes that this will be resolved by >> the next beta; made it a release blocker to catch your eye, Barry. =) > > I knocked this down to a deferred blocker, so I won't let it hold up beta2, > though I'd /really/ like to get this cleaned up and committed in time. If I > get enough deferred blockers though, I might hold things up after all. > Nick is actively working on it, so it might make it in b2. -Brett From steve at pearwood.info Wed Jul 16 02:10:57 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 16 Jul 2008 10:10:57 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <ca471dc20807151513i5fede9ccjeaac48a50a05f6b6@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp> <ca471dc20807151513i5fede9ccjeaac48a50a05f6b6@mail.gmail.com> Message-ID: <200807161010.57836.steve@pearwood.info> On Wed, 16 Jul 2008 08:13:22 am Guido van Rossum wrote: > > Tests in the standard distribution which use the deprecated > > style will need to be converted. Steven d'Aprano claims this is > > nontrivial (and thus error- prone) in some cases. I haven't seen > > that claim denied, and it seems plausible to me. > > I'd like to see examples of that (this would be Steven's task if he's > serious about his assertion). Since the fail and assert names are > mapped to each other using aliasing I don't see how it could be > nontrivial to map e.g. self.failIf(x) to self.assertFalse(x) -- these > are the same function! I have not knowingly claimed that mechanically swapping fail* to assert* tests was difficult. The difficulty I refer to is about readability and understanding of the code. I often think about tests as sequences of possible failures, and as such my unit tests are most naturally written as fail*. It is that mental effort of reversing the sense of the tests when reading and writing assert* tests that I refer to. If you want to declare that fail* must go, I'll be disappointed but life will go on. But despite the claims of those who have asserted (pun intended) that fail* tests are always more difficult to understand, that's not the case for everyone. -- Steven From grflanagan at gmail.com Wed Jul 16 02:12:02 2008 From: grflanagan at gmail.com (Gerard flanagan) Date: Wed, 16 Jul 2008 02:12:02 +0200 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <877ibn8flm.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> <877ibn8flm.fsf@benfinney.id.au> Message-ID: <g5jecg$191$1@ger.gmane.org> Ben Finney wrote: > Nick Coghlan <ncoghlan at gmail.com> writes: >> One option for rationalising the API would be to merely keep the >> shortest version of each phrase (i.e. use assert* instead of >> fail_unless* for the positive tests and fail_if* instead of >> assert_not* for the negative tests, and always drop the trailing 's' >> from 'equals'). > > I think clarity, consistency, and discoverability are more important > criteria than saving a few characters in a method name. > >> Adding in possible positive and negative forms of the proposed new >> methods > > A major point of this PEP is to *remove* redundant names in the API. > def it_is_a_truth_universally_recognised_that(...): def we_are_all_feckin_doomed_unless(...): should be quite sufficient ;-) entia-non-sunt-multiplicanda-ly y'rs G. From collinw at gmail.com Wed Jul 16 02:20:30 2008 From: collinw at gmail.com (Collin Winter) Date: Tue, 15 Jul 2008 17:20:30 -0700 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <87skub6yhh.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> Message-ID: <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote: > Significant updates include removing all reference to the > (already-resolved) new-style class issue, adding footnotes and > references, and a Rationale summary of discussion on both sides of the > divide for 'assert*' versus 'fail*' names. > > > :PEP: XXX > :Title: Consolidating names in the `unittest` module > :Version: 0.2 > :Last-Modified: 2008-07-15 > :Author: Ben Finney <ben+python at benfinney.id.au> > :Status: Draft > :Type: Standards Track > :Content-Type: test/x-rst > :Created: 2008-07-14 > :Python-Version: 2.7, 3.1 > :Post-History: > > > .. contents:: > > > Abstract > ======== > > This PEP proposes to consolidate the names that constitute the API of > the standard library `unittest` module, with the goal of removing > redundant names, and conforming with PEP 8. > > > Motivation > ========== > > The normal use case for the `unittest` module is to subclass its > classes, overriding and re-using its functios and methods. This draws > constant attention to the fact that the existing implementation fails > several current Python standards: > > * It does not conform to PEP 8 [#PEP-8]_, requiring users to write > their own non-PEP-8 conformant names when overriding methods, and > encouraging extensions to further depart from PEP 8. > > * It has many synonyms in its API, which goes against the Zen of > Python [#PEP-20]_ (specifically, that "there should be one, and > preferably only one, obvious way to do it"). > > > Specification > ============= > > Remove obsolete names > --------------------- > > The following module attributes are not documented as part of the API > and are marked as obsolete in the implementation. They will be > removed. > > * ``_makeLoader`` > * ``getTestCaseNames`` > * ``makeSuite`` > * ``findTestCases`` > > Remove redundant names > ---------------------- > > The following attribute names exist only as synonyms for other names. > They are to be removed, leaving only one name for each attribute in > the API. > > ``TestCase`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~ > > * ``assertEqual`` > * ``assertEquals`` > * ``assertNotEqual`` > * ``assertNotEquals`` > * ``assertAlmostEqual`` > * ``assertAlmostEquals`` > * ``assertNotAlmostEqual`` > * ``assertNotAlmostEquals`` > * ``assertRaises`` > * ``assert_`` > * ``assertTrue`` > * ``assertFalse`` > > Conform API with PEP 8 > ---------------------- > > The following names are to be introduced, each replacing an existing > name, to make all names in the module conform with PEP 8 [#PEP-8]_. > Each name is shown with the existing name that it replaces. > > Where function parameters are to be renamed also, they are shown. > Where function parameters are not to be renamed, they are elided with > the ellipse ("?") symbol. > > Module attributes > ~~~~~~~~~~~~~~~~~ > > ``default_test_loader`` > Replaces ``defaultTestLoader`` > > ``TestResult`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~~~ > > ``add_error(?)`` > Replaces ``addError(?)`` > > ``add_result(?)`` > Replaces ``addResult(?)`` > > ``add_success(?)`` > Replaces ``addSuccess(?)`` > > ``should_stop`` > Replaces ``shouldStop`` > > ``start_test(?)`` > Replaces ``startTest(?)`` > > ``stop_test(?)`` > Replaces ``stopTest(?)`` > > ``tests_run`` > Replaces ``testsRun`` > > ``was_successful(?)`` > Replaces ``wasSuccessful(?)`` > > ``TestCase`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~ > > ``__init__(self, method_name='run_test')`` > Replaces ``__init__(self, methodName='runTest')`` > > ``_test_method_doc`` > Replaces ``_testMethodDoc`` > > ``_test_method_name`` > Replaces ``_testMethodName`` > > ``failure_exception`` > Replaces ``failureException`` > > ``count_test_cases(?)`` > Replaces ``countTestCases(?)`` > > ``default_test_result(?)`` > Replaces ``defaultTestResult(?)`` > > ``fail_if(?)`` > Replaces ``failIf(?)`` > > ``fail_if_almost_equal(?)`` > Replaces ``failIfAlmostEqual(?)`` > > ``fail_if_equal(?)`` > Replaces ``failIfEqual(?)`` > > ``fail_unless(?)`` > Replaces ``failUnless(?)`` > > ``fail_unless_almost_equal(?)`` > Replaces ``failUnlessAlmostEqual(?)`` > > ``fail_unless_equal(?)`` > Replaces ``failUnlessEqual(?)`` > > ``fail_unless_raises(exc_class, callable_obj, *args, **kwargs)`` > Replaces ``failUnlessRaises(excClass, callableObj, *args, **kwargs)`` > > ``run_test(?)`` > Replaces ``runTest(?)`` > > ``set_up(?)`` > Replaces ``setUp(?)`` > > ``short_description(?)`` > Replaces ``shortDescription(?)`` > > ``tear_down(?)`` > Replaces ``tearDown(?)`` > > ``FunctionTestCase`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > ``__init__(self, test_func, set_up, tear_down, description)`` > Replaces ``__init__(self, testFunc, setUp, tearDown, description)`` > > ``run_test(?)`` > Replaces ``runTest(?)`` > > ``set_up(?)`` > Replaces ``setUp(?)`` > > ``short_description(?)`` > Replaces ``shortDescription(?)`` > > ``tear_down(?)`` > Replaces ``tearDown(?)`` > > ``TestSuite`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~~ > > ``add_test(?)`` > Replaces ``addTest(?)`` > > ``add_tests(?)`` > Replaces ``addTests(?)`` > > ``count_test_cases(?)`` > Replaces ``countTestCases(?)`` > > ``TestLoader`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~~~ > > ``sort_test_methods_using`` > Replaces ``sortTestMethodsUsing`` > > ``suite_class`` > Replaces ``suiteClass`` > > ``test_method_prefix`` > Replaces ``testMethodPrefix`` > > ``get_test_case_names(self, test_case_class)`` > Replaces ``getTestCaseNames(self, testCaseClass)`` > > ``load_tests_from_module(?)`` > Replaces ``loadTestsFromModule(?)`` > > ``load_tests_from_name(?)`` > Replaces ``loadTestsFromName(?)`` > > ``load_tests_from_names(?)`` > Replaces ``loadTestsFromNames(?)`` > > ``load_tests_from_test_case(self, test_case_class)`` > Replaces ``loadTestsFromTestCase(self, testCaseClass)`` > > ``_TextTestResult`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > ``show_all`` > Replaces ``showAll`` > > ``add_error(?)`` > Replaces ``addError(?)`` > > ``add_failure(?)`` > Replaces ``addFailure(?)`` > > ``add_success(?)`` > Replaces ``addSuccess(?)`` > > ``get_description(?)`` > Replaces ``getDescription(?)`` > > ``print_error_list(?)`` > Replaces ``printErrorList(?)`` > > ``print_errors(?)`` > Replaces ``printErrors(?)`` > > ``start_test(?)`` > Replaces ``startTest(?)`` > > ``TextTestRunner`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > ``_make_result(?)`` > Replaces ``_makeResult(?)`` > > ``TestProgram`` attributes > ~~~~~~~~~~~~~~~~~~~~~~~~~~ > > ``__init__(self, module, default_test, argv, test_runner, test_loader)`` > Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)`` > > ``create_tests(?)`` > Replaces ``createTests(?)`` > > ``parse_args(?)`` > Replaces ``parseArgs(?)`` > > ``run_tests(?)`` > Replaces ``runTests(?)`` > > ``usage_exit(?)`` > Replaces ``usageExit(?)`` > > > Rationale > ========= > > Redundant names > --------------- > > The current API, with two or in some cases three different names > referencing exactly the same function, leads to an overbroad and > redundant API that violates PEP 20 [#PEP-20]_ ("there should be one, > and preferably only one, obvious way to do it"). > > Removal of ``assert*`` names > ---------------------------- > > While there is consensus support to `remove redundant names`_ for the > ``TestCase`` test methods, the issue of which set of names should be > retained is controversial. > > Arguments in favour of retaining only the ``assert*`` names: > > * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference > for the ``assert*`` names. > > * Precedent: The Python standard library currently uses the > ``assert*`` names by a roughly 8:1 majority over the ``fail*`` > names. (Counting unit tests in the py3k tree at 2008-07-15 > [#pitrou-1]_.) > > An ad-hoc sampling of other projects that use `unittest` also > demonstrates strong preference for use of the ``assert*`` names > [#bennetts-1]_. > > * Positive admonition: The ``assert*`` names state the intent of how > the code under test *should* behave, while the ``fail*`` names are > phrased in terms of how the code *should not* behave. > > Arguments in favour of retaining only the ``fail*`` names: > > * Explicit is better than implicit: The ``fail*`` names state *what > the function will do* explicitly: fail the test. With the > ``assert*`` names, the action to be taken is only implicit. > > * Avoid false implication: The test methods do not have any necessary > connection with the built-in ``assert`` statement. Even the > exception raised, while it defaults to ``AssertionException``, is > explicitly customisable via the documented ``failure_exception`` > attribute. Choosing the ``fail*`` names avoids the false association > with either of these. > > This is exacerbated by the plain-boolean test using a name of > ``assert_`` (with a trailing underscore) to avoid a name collision > with the built-in ``assert`` statement. The corresponding > ``fail_if`` name has no such issue. > > PEP 8 names > ----------- > > Although `unittest` (and its predecessor `PyUnit`) are intended to be > familiar to users of other xUnit interfaces, there is no attempt at > direct API compatibility since the only code that Python's `unittest` > interfaces with is other Python code. The module is in the standard > library and its names should all conform with PEP 8 [#PEP-8]_. > > > Backwards Compatibility > ======================= > > The names to be obsoleted should be deprecated and removed according > to the schedule for modules in PEP 4 [#PEP-4]_. > > While deprecated, use of the deprecated attributes should raise a > ``DeprecationWarning``, with a message stating which replacement name > should be used. Is any provision being made for a 2to3 fixer/otherwise-automated transition for the changes you propose here? Collin Winter From barry at python.org Wed Jul 16 02:22:09 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 15 Jul 2008 20:22:09 -0400 Subject: [Python-Dev] Bug 3139 Message-ID: <585A47BF-246A-416D-B83C-E74933B0EB0E@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 But 3139 appears important enough to hold up beta 2. http://bugs.python.org/issue3139 bytearrays are not thread safe Can we get this fixed by tomorrow? Does anybody disagree that we should hold up the release for this one? We don't have much time left to fix things like this. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSH0/MXEjvBPtnXfVAQJoHwP8CFYbafuSTbnVdBBydZJ5k8Fb1L4YYVG/ 0ee81N4AiQZ5uGQlk4ywAVBycPa+DMWE+cdrkYiGXkPaarhcOd55dfdvL1Ww6OgO tor2fh2IxT0ee7gO/vgbyv4ybWsxuPH/9W5O+W2hZkHd60NkJCqpiHlMKGz6caOX +rwbgqaQOU4= =8lO1 -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Wed Jul 16 02:22:45 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Jul 2008 12:22:45 +1200 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <87y7436yxz.fsf@benfinney.id.au> <1b54228d0807150700k1985321eob37fd22bf143e1a5@mail.gmail.com> Message-ID: <487D3F55.5010309@canterbury.ac.nz> Richard Thomas wrote: > I've been told by a couple of non-programmers that "failUnless" is > more intuitive than "assert" if only for the reason that its unclear > what "assert" might do. But test frameworks are for use by programmers, not non-programmers. Given that it's a test framework, would a programmer really find it hard to guess what these mean? -- Greg From greg.ewing at canterbury.ac.nz Wed Jul 16 02:53:54 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Jul 2008 12:53:54 +1200 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <87zloi6a8k.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> <487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au> Message-ID: <487D46A2.4050207@canterbury.ac.nz> Ben Finney wrote: > Nick Coghlan <ncoghlan at gmail.com> writes: > > > the shortest > > possible way of writing negative assertions (i.e. asserting that > > something is not the case) is to treat them as denials and use the > > single word 'deny'. > > This, to me, is neither intuitive nor meaningful in context. The term > "deny" is strongly linked to its antonym, "permit". "Deny" also has the meaning of claiming that something is not true (as in "deny an allegation"). When used that way, it's not an antonym of "permit". However, that meaning doesn't quite seem to fit here, as we don't just want to claim that the condition is false, but *ensure* that it's false. I can't think of a single word offhand that means that. -- Greg From fuzzyman at voidspace.org.uk Wed Jul 16 03:03:53 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 02:03:53 +0100 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> Message-ID: <487D48F9.3000903@voidspace.org.uk> Collin Winter wrote: > On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote: >> >> Backwards Compatibility >> ======================= >> >> The names to be obsoleted should be deprecated and removed according >> to the schedule for modules in PEP 4 [#PEP-4]_. >> >> While deprecated, use of the deprecated attributes should raise a >> ``DeprecationWarning``, with a message stating which replacement name >> should be used. >> > > Is any provision being made for a 2to3 fixer/otherwise-automated > transition for the changes you propose here? > As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed? Michael > Collin Winter > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From ben+python at benfinney.id.au Wed Jul 16 03:19:44 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 11:19:44 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module Message-ID: <87fxqa62xb.fsf@benfinney.id.au> :PEP: XXX :Title: Frequently-requested additional features for the `unittest` module :Version: 0.3 :Last-Modified: 2008-07-16 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: test/x-rst :Requires: PEP XXX (Consolidating names in the `unittest` module) :Created: 2008-07-14 :Post-History: .. contents:: Abstract ======== This PEP proposes frequently-requested additions to the standard library `unittest` module that are natural extensions of its existing functionality. Motivation ========== The `unittest` module is functionally complete. Nevertheless, many users request and devise extensions to perform functions that are so common they deserve to be in the standard module. Specification ============= New condition tests ------------------- The following test methods will be added to the ``TestCase`` class. The function signature is part of this specification. The body is to be treated as a reference implementation only; any functionally identical implementation also meets this specification. :: import operator import re class TestCase(object): # ? def assert_compare_true(op, first, second, msg=None): if msg is None: msg = "%(first)r %(op)r %(second)" % vars() if not op(first, second): raise self.failure_exception(msg) def assert_in(container, member, msg=None): op = operator.__contains__ self.assert_compare_true(op, container, member, msg) def assert_is(first, second, msg=None): op = operator.is_ self.assert_compare_true(op, first, second, msg) def assert_less_than(first, second, msg=None): op = operator.lt self.assert_compare_true(op, first, second, msg) def assert_greater_than(first, second, msg=None): op = operator.gt self.assert_compare_true(op, first, second, msg) def assert_less_than_or_equal(first, second, msg=None): op = operator.le self.assert_compare_true(op, first, second, msg) def assert_greater_than_or_equal(first, second, msg=None): op = operator.ge self.assert_compare_true(op, first, second, msg) def assert_members_equal(first, second, msg=None): self.assert_equal(set(first), set(second), msg) def assert_sequence_equal(first, second, msg=None): self.assert_equal(list(first), list(second), msg) def assert_raises_with_message_regex( exc_class, message_regex, callable_obj, *args, **kwargs): exc_name = exc_class.__name__ message_pattern = re.compile(message_regex) try: callable_obj(*args, **kwargs) except exc_class, exc: exc_message = str(exc) if not message_pattern.match(exc_message): msg = ( "%(exc_name)s raised" " without message matching %(message_regex)r" " (got message %(exc_message)r)" ) % vars() raise self.failure_exception(msg) else: msg = "%(exc_name)s not raised" % vars() raise self.failure_exception(msg) The following test methods are also added. Their implementation in each case is simply the logical inverse of a corresponding method above. :: def assert_compare_false(op, first, second, msg=None): # Logical inverse of assert_compare_true def assert_not_in(container, member, msg=None): # Logical inverse of assert_in def assert_is_not(first, second, msg=None) # Logical inverse of assert_is def assert_not_less_than(first, second, msg=None) # Logical inverse of assert_less_than def assert_not_greater_than(first, second, msg=None) # Logical inverse of assert_greater_than def assert_not_less_than_or_equal(first, second, msg=None) # Logical inverse of assert_less_than_or_equal def assert_not_greater_than_or_equal(first, second, msg=None) # Logical inverse of assert_greater_than_or_equal def assert_members_not_equal(first, second, msg=None) # Logical inverse of assert_members_equal def assert_sequence_not_equal(first, second, msg=None) # Logical inverse of assert_sequence_equal Enhanced failure message for equality tests ------------------------------------------- The equality tests will change their behaviour such that the message always, even if overridden with a specific message when called, includes extra information: * For both ``assert_equal`` and ``assert_not_equal``: the ``repr()`` output of the objects that were compared. * For ``assert_equal`` comparisons of ``basestring`` instances that are multi-line text: the output of ``diff`` comparing the two texts. * For membership comparisons with ``assert_*_equal``: the ``repr()`` output of the members that were not equal in each collection. (This change is not done for the corresponding ``assert_*_not_equal`` tests, which only fail if the collection members are equal.) Simple invocation of test collection ------------------------------------ The following new functionality will be added to the `unittest` module. The function signature for ``run_tests`` is part of this specification; the implementation is to be considered a reference implementation only. :: def run_tests( tests, loader_class=TestLoader, suite_class=TestSuite, runner_class=TextTestRunner): """ Run a collection of tests with a test runner :param tests: A sequence of objects that can contain test cases: modules, `TestSuite` instances, or `TestCase` subclasses. :param loader_class: The type of test loader to use for collecting tests from the `tests` collection. :param suite_class: The type of test suite to use for accumulating the collected test cases. :param runner_class: The type of test runner to instantiate for running the collected test cases. :return: None. """ def iter_testcases_recursively(collection, loader): # Flatten and iterate over collection, generating # instances of TestCase loader = loader_class() suite = suite_class() for testcase in iter_testcases_recursively(tests, loader): suite.add_tests(testcase) runner = runner_class() runner.run(suite) Rationale ========= Names for logical-inverse tests ------------------------------- The simple pattern established by ``assert_foo`` having a logical inverse named ``assert_not_foo`` sometimes results in gramatically awkward names. The following names were chosen in exception of this pattern, in the interest of the API names being more intuitive: * ``assert_is_not`` * ``assert_members_not_equal`` * ``assert_sequence_not_equal`` Order of method parameters -------------------------- The methods ``assert_in``, ``assert_not_in`` have the container as the first parameter. This makes the grammatical object of the function name come immediately after the function name: "Assert in `container`". This matches the convention set by the existing ``assert_raises(exception, callable_obj, ?)`` "(Assert the code raises `exception`"). Backwards Compatibility ======================= This PEP proposes only additional features. There are no backward-incompatible changes. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From scott+python-dev at scottdial.com Wed Jul 16 03:41:54 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 15 Jul 2008 21:41:54 -0400 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module In-Reply-To: <87fxqa62xb.fsf@benfinney.id.au> References: <87fxqa62xb.fsf@benfinney.id.au> Message-ID: <487D51E2.8060908@scottdial.com> Ben Finney wrote: > New condition tests > ------------------- > > def assert_less_than(first, second, msg=None): > op = operator.lt > self.assert_compare_true(op, first, second, msg) > > def assert_greater_than(first, second, msg=None): > op = operator.gt > self.assert_compare_true(op, first, second, msg) > > def assert_less_than_or_equal(first, second, msg=None): > op = operator.le > self.assert_compare_true(op, first, second, msg) > > def assert_greater_than_or_equal(first, second, msg=None): > op = operator.ge > self.assert_compare_true(op, first, second, msg) > > The following test methods are also added. Their implementation in > each case is simply the logical inverse of a corresponding method > above. > > def assert_not_less_than(first, second, msg=None) > # Logical inverse of assert_less_than > > def assert_not_greater_than(first, second, msg=None) > # Logical inverse of assert_greater_than > > def assert_not_less_than_or_equal(first, second, msg=None) > # Logical inverse of assert_less_than_or_equal > > def assert_not_greater_than_or_equal(first, second, msg=None) > # Logical inverse of assert_greater_than_or_equal Why?! Your other PEP is about removing redundant names, and here you have introduced 4 of them: assert_not_less_than = assert_greater_than_or_equal assert_not_greater_than = assert_less_than_or_equal assert_not_less_than_or_equal = assert_greater_than assert_not_greater_than_or_equal = assert_less_than Besides, ``assert_not_greater_than_or_equal`` is god-awful-long, along with the complaints about PEP-8-ifying. I wonder if it would be better to abbreviate these names with the *same name* that was used for the attribute in the operator module. Let's not reinvent the wheel here.. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From ben+python at benfinney.id.au Wed Jul 16 04:05:53 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 12:05:53 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com> Message-ID: <87bq0y60se.fsf@benfinney.id.au> Scott Dial <scott+python-dev at scottdial.com> writes: > Why [introduce redundant test names]? > > assert_not_less_than = assert_greater_than_or_equal > assert_not_greater_than = assert_less_than_or_equal > assert_not_less_than_or_equal = assert_greater_than > assert_not_greater_than_or_equal = assert_less_than To answer the question: The above tests are logically equivalent, but the failure message would be different, reporting failure in terms of what the caller wanted to test. I se your point though. I'd like to see what others think on this issue. > Besides, ``assert_not_greater_than_or_equal`` is god-awful-long, > along with the complaints about PEP-8-ifying. I wonder if it would > be better to abbreviate these names with the *same name* that was > used for the attribute in the operator module. Let's not reinvent > the wheel here.. Interesting. So you advocate collapsing the above eight tests into the following four: assert_lt assert_gt assert_le assert_ge -- \ ?I always had a repulsive need to be something more than | `\ human.? ?David Bowie | _o__) | Ben Finney From brett at python.org Wed Jul 16 04:29:26 2008 From: brett at python.org (Brett Cannon) Date: Tue, 15 Jul 2008 19:29:26 -0700 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module In-Reply-To: <87bq0y60se.fsf@benfinney.id.au> References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com> <87bq0y60se.fsf@benfinney.id.au> Message-ID: <bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com> On Tue, Jul 15, 2008 at 7:05 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > Scott Dial <scott+python-dev at scottdial.com> writes: > >> Why [introduce redundant test names]? >> >> assert_not_less_than = assert_greater_than_or_equal >> assert_not_greater_than = assert_less_than_or_equal >> assert_not_less_than_or_equal = assert_greater_than >> assert_not_greater_than_or_equal = assert_less_than > > To answer the question: The above tests are logically equivalent, but > the failure message would be different, reporting failure in terms of > what the caller wanted to test. > > I se your point though. I'd like to see what others think on this > issue. > >> Besides, ``assert_not_greater_than_or_equal`` is god-awful-long, >> along with the complaints about PEP-8-ifying. I wonder if it would >> be better to abbreviate these names with the *same name* that was >> used for the attribute in the operator module. Let's not reinvent >> the wheel here.. > > Interesting. So you advocate collapsing the above eight tests into the > following four: > > assert_lt > assert_gt > assert_le > assert_ge Is any of this really necessary? Isn't this the equivalent of ``assert_(a < b)``? It seems like the only thing you get out of this is a nicer error message, but ``assert_(a < b, 'not %r <= %s' % (a, b))`` is not that complex. And do these cases really come up that often? I would want to see some numbers showing that these are really necessary (in both usage and people even specifying an error message in the first place). -Brett From scott at scottdial.com Wed Jul 16 04:27:59 2008 From: scott at scottdial.com (Scott Dial) Date: Tue, 15 Jul 2008 22:27:59 -0400 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module In-Reply-To: <87bq0y60se.fsf@benfinney.id.au> References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com> <87bq0y60se.fsf@benfinney.id.au> Message-ID: <487D5CAF.60009@scottdial.com> Ben Finney wrote: > Scott Dial <scott+python-dev at scottdial.com> writes: > >> Why [introduce redundant test names]? > > To answer the question: The above tests are logically equivalent, but > the failure message would be different, reporting failure in terms of > what the caller wanted to test. I can see how this argument makes sense, and is distinct from the fail* vs. assert* discussion. As you say, I'm interested what other think about this. >> Besides, ``assert_not_greater_than_or_equal`` is god-awful-long, >> along with the complaints about PEP-8-ifying. I wonder if it would >> be better to abbreviate these names with the *same name* that was >> used for the attribute in the operator module. Let's not reinvent >> the wheel here.. > > Interesting. So you advocate collapsing the above eight tests into the > following four: > > assert_lt > assert_gt > assert_le > assert_ge I would argue to go even further: assertEqual = assert_eq assertAlmostEqual = assert_almost_eq assertNotEqual = assert_ne assertNotAlmostEqual = assert_almost_ne I'm not sure if there are others, but using the same abbreviations from operator is consistent and readable and short, in my opinion. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From ben+python at benfinney.id.au Wed Jul 16 05:04:12 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 13:04:12 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> Message-ID: <877ibm5y37.fsf_-_@benfinney.id.au> Michael Foord <fuzzyman at voidspace.org.uk> writes: > The full list of changes proposed [?] and not shot down was > something like: [?] > assertLessThan > assertGreaterThan > assertLessThanOrEquals > assertGreaterThanOrEquals [?] "Brett Cannon" <brett at python.org> writes: > Is any of this really necessary? Isn't this the equivalent of > ``assert_(a < b)``? It seems like the only thing you get out of this > is a nicer error message, but ``assert_(a < b, 'not %r <= %s' % (a, > b))`` is not that complex. And do these cases really come up that > often? I would want to see some numbers showing that these are > really necessary (in both usage and people even specifying an error > message in the first place). Though I'm the champion of this PEP, I'll have to refer to Michael Foord for his reasoning (or reference to others' reasoning) on this. -- \ ?The process by which banks create money is so simple that the | `\ mind is repelled.? ?John Kenneth Galbraith, _Money: Whence It | _o__) Came, Where It Went_, 1975 | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 05:06:29 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 13:06:29 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol> Message-ID: <873ama5xze.fsf@benfinney.id.au> Antoine Pitrou <solipsis at pitrou.net> writes: > (and don't tell me "fail" is not a negative word, because you just > used the phrase "positive admonition" to refer to the assert* > variants, which implies quite clearly that the fail* variants are on > the negative side ;-)) This "fail is a negative word" has already been rebutted, by native speakers of English. If you don't want to hear "fail is not a negative word", I can't help you. This doesn't seem to introduce anything new, so I'll leave the existing summary of this argument as is in the PEP. -- \ ?Holy unrefillable prescriptions, Batman!? ?Robin | `\ | _o__) | Ben Finney From ncoghlan at gmail.com Wed Jul 16 07:25:12 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 16 Jul 2008 15:25:12 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <487D46A2.4050207@canterbury.ac.nz> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> <487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au> <487D46A2.4050207@canterbury.ac.nz> Message-ID: <487D8638.9060807@gmail.com> Greg Ewing wrote: > Ben Finney wrote: >> Nick Coghlan <ncoghlan at gmail.com> writes: >> >> > the shortest >> > possible way of writing negative assertions (i.e. asserting that >> > something is not the case) is to treat them as denials and use the >> > single word 'deny'. >> >> This, to me, is neither intuitive nor meaningful in context. The term >> "deny" is strongly linked to its antonym, "permit". > > "Deny" also has the meaning of claiming that something is > not true (as in "deny an allegation"). When used that way, > it's not an antonym of "permit". > > However, that meaning doesn't quite seem to fit here, as > we don't just want to claim that the condition is false, > but *ensure* that it's false. I can't think of a single > word offhand that means that. That was the meaning I was going for, but I agree that it doesn't quite fit well enough to make it a good idea. There's a reason I put that disclaimer at the top of the message :) What did you think of the "check" idea at the end of the email? Test assertions: check(x).almost_equal(y) check(x).is_(y) check(x).in_(y) check(x).equals(y) Test negative assertions: check(x).not_almost_equal(y) check(x).is_not(y) check(x).not_in(y) check(x).not_equal(y) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Jul 16 07:33:33 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 16 Jul 2008 15:33:33 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <87wsjmx007.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <487D882D.4070108@gmail.com> Stephen J. Turnbull wrote: > Ben Finney writes: > > > Removal of ``assert*`` names > > ---------------------------- > > > Arguments in favour of retaining only the ``assert*`` names: > > > > * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference > > for the ``assert*`` names. > > > > * Precedent: The Python standard library currently uses the > > ``assert*`` names by a roughly 8:1 majority over the ``fail*`` > > names. (Counting unit tests in the py3k tree at 2008-07-15 > > [#pitrou-1]_.) > > > > An ad-hoc sampling of other projects that use `unittest` also > > demonstrates strong preference for use of the ``assert*`` names > > [#bennetts-1]_. > > > > * Positive admonition: The ``assert*`` names state the intent of how > > the code under test *should* behave, while the ``fail*`` names are > > phrased in terms of how the code *should not* behave. > > FWIW, I think these are fairly stated. So fairly that I'm surprised > you haven't been persuaded!<wink> Nitpick: the second point is not > just "precedent", there's an economic reason there too. Tests in the > standard distribution which use the deprecated style will need to be > converted. Steven d'Aprano claims this is nontrivial (and thus error- > prone) in some cases. I haven't seen that claim denied, and it seems > plausible to me. I disagree with SdA's statement there: failUnless* maps directly to assert* and failIf* maps directly to assertNot*. There are no semantic changes whatsoever involved in that remapping, just a change in the method names. Note that if the API is to be rationalised at all, my personal preference is to keep assert* and failIf* and get rid of their longer counterparts (failUnless* and assertNot*). Alternatively, ditch the binary assertion methods entirely, and use a check object to state those assertions: check(x).almost_equal(y) instead assertAlmostEqual(x, y) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ben+python at benfinney.id.au Wed Jul 16 07:48:09 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 15:48:09 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module References: <87fxqa62xb.fsf@benfinney.id.au> Message-ID: <87abgi4bxi.fsf@benfinney.id.au> Significant changes: Add a new 'TestLoader.load_tests_from_collection' method, with full reference implementation. This makes the 'run_tests' reference implementation straightforward. :PEP: XXX :Title: Frequently-requested additional features for the `unittest` module :Version: 0.4 :Last-Modified: 2008-07-16 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: test/x-rst :Requires: PEP XXX (Consolidating names in the `unittest` module) :Created: 2008-07-14 :Post-History: .. contents:: Abstract ======== This PEP proposes frequently-requested additions to the standard library `unittest` module that are natural extensions of its existing functionality. Motivation ========== The `unittest` module is functionally complete. Nevertheless, many users request and devise extensions to perform functions that are so common they deserve to have a standard implementation. Specification ============= New condition tests ------------------- The following test methods will be added to the ``TestCase`` class. The function signature is part of this specification. The body is to be treated as a reference implementation only; any functionally identical implementation also meets this specification. :: import operator import re class TestCase(object): # ? def assert_compare_true(op, first, second, msg=None): if msg is None: msg = "%(first)r %(op)r %(second)" % vars() if not op(first, second): raise self.failure_exception(msg) def assert_in(container, member, msg=None): op = operator.__contains__ self.assert_compare_true(op, container, member, msg) def assert_is(first, second, msg=None): op = operator.is_ self.assert_compare_true(op, first, second, msg) def assert_less_than(first, second, msg=None): op = operator.lt self.assert_compare_true(op, first, second, msg) def assert_greater_than(first, second, msg=None): op = operator.gt self.assert_compare_true(op, first, second, msg) def assert_less_than_or_equal(first, second, msg=None): op = operator.le self.assert_compare_true(op, first, second, msg) def assert_greater_than_or_equal(first, second, msg=None): op = operator.ge self.assert_compare_true(op, first, second, msg) def assert_members_equal(first, second, msg=None): self.assert_equal(set(first), set(second), msg) def assert_sequence_equal(first, second, msg=None): self.assert_equal(list(first), list(second), msg) def assert_raises_with_message_regex( exc_class, message_regex, callable_obj, *args, **kwargs): exc_name = exc_class.__name__ message_pattern = re.compile(message_regex) try: callable_obj(*args, **kwargs) except exc_class, exc: exc_message = str(exc) if not message_pattern.match(exc_message): msg = ( "%(exc_name)s raised" " without message matching %(message_regex)r" " (got message %(exc_message)r)" ) % vars() raise self.failure_exception(msg) else: msg = "%(exc_name)s not raised" % vars() raise self.failure_exception(msg) The following test methods are also added. Their implementation in each case is simply the logical inverse of a corresponding method above. :: def assert_compare_false(op, first, second, msg=None): # Logical inverse of assert_compare_true def assert_not_in(container, member, msg=None): # Logical inverse of assert_in def assert_is_not(first, second, msg=None) # Logical inverse of assert_is def assert_not_less_than(first, second, msg=None) # Logical inverse of assert_less_than def assert_not_greater_than(first, second, msg=None) # Logical inverse of assert_greater_than def assert_not_less_than_or_equal(first, second, msg=None) # Logical inverse of assert_less_than_or_equal def assert_not_greater_than_or_equal(first, second, msg=None) # Logical inverse of assert_greater_than_or_equal def assert_members_not_equal(first, second, msg=None) # Logical inverse of assert_members_equal def assert_sequence_not_equal(first, second, msg=None) # Logical inverse of assert_sequence_equal Enhanced failure message for equality tests ------------------------------------------- The equality tests will change their behaviour such that the message always, even if overridden with a specific message when called, includes extra information: * For both ``assert_equal`` and ``assert_not_equal``: the ``repr()`` output of the objects that were compared. * For ``assert_equal`` comparisons of ``basestring`` instances that are multi-line text: the output of ``diff`` comparing the two texts. * For membership comparisons with ``assert_*_equal``: the ``repr()`` output of the members that were not equal in each collection. (This change is not done for the corresponding ``assert_*_not_equal`` tests, which only fail if the collection members are equal.) Simple invocation of test collection ------------------------------------ To allow simpler loading and running of test cases from an arbitrary collection, the following new functionality will be added to the `unittest` module. The function signatures are part of this specification; the implementation is to be considered a reference implementation only. :: class TestLoader(object): # ? def load_tests_from_collection(self, collection): """ Return a suite of all test cases found in `collection` :param collection: One of the following: * a `TestSuite` instance * a `TestCase` subclass * a module * an iterable producing any of these types :return: A `TestSuite` instance containing all the test cases found recursively within `collection`. """ suite = None if isinstance(collection, TestSuite): suite = collection elif isinstance(collection, type): if issubclass(collection, TestCase): suite = self.load_tests_from_test_case(collection) elif isinstance(collection, types.ModuleType): suite = self.load_tests_from_module(collection) elif hasattr(collection, '__iter__'): suite = self.suite_class() for item in collection: suite.add_test(self.load_tests_from_collection(item)) else: msg = "not a test collection: %(collection)r" % vars() raise TypeError(msg) return suite def run_tests( tests, loader_class=TestLoader, runner_class=TextTestRunner): """ Run a collection of tests with a test runner :param tests: A test collection as defined by the `loader_class` method `load_tests_from_collection`. :param loader_class: The type of test loader to use for collecting tests from the `tests` collection. :param runner_class: The type of test runner to instantiate for running the collected test suite. :return: None. """ loader = loader_class() suite = loader.load_tests_from_collection(tests) runner = runner_class() runner.run(suite) Rationale ========= Names for logical-inverse tests ------------------------------- The simple pattern established by ``assert_foo`` having a logical inverse named ``assert_not_foo`` sometimes results in gramatically awkward names. The following names were chosen in exception of this pattern, in the interest of the API names being more intuitive: * ``assert_is_not`` * ``assert_members_not_equal`` * ``assert_sequence_not_equal`` Order of method parameters -------------------------- The methods ``assert_in``, ``assert_not_in`` have the container as the first parameter. This makes the grammatical object of the function name come immediately after the function name: "Assert in `container`". This matches the convention set by the existing ``assert_raises(exception, callable_obj, ?)`` "(Assert the code raises `exception`"). Backwards Compatibility ======================= This PEP proposes only additional features. There are no backward-incompatible changes. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From tjreedy at udel.edu Wed Jul 16 08:10:54 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 16 Jul 2008 02:10:54 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <878ww27p13.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> <g5iljh$cmt$1@ger.gmane.org> <87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp> <878ww27p13.fsf@benfinney.id.au> Message-ID: <g5k3dc$hgk$1@ger.gmane.org> Ben Finney wrote: > "Stephen J. Turnbull" <stephen at xemacs.org> writes: > >> Terry Reedy writes: >> >> > For the community as a whole, all stdlib modules are suggestions >> > and examples, not commands. >> >> Well, even if "standard" is too strong a word, the DeprecationWarnings >> and eventual removal of the methods surely constitute an imposition. > > I understood Terry's statement as meaning that there is no commandment > to use the standard library 'unittest' module at all. If a user > doesn't like the behaviour of a module (such as 'unittest') in the > standard library, they can accept the burden of implementing one that > behaves the way they like. I have written the few specialized test functions that I need my current project. I did this after considering (and taking ideas from) py.test. I will also use doctest as a supplement. I also meant that one who does use something is free to rename it. I commonly import xskjfl as x;-) From tjreedy at udel.edu Wed Jul 16 08:17:44 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 16 Jul 2008 02:17:44 -0400 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <487D48F9.3000903@voidspace.org.uk> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487D48F9.3000903@voidspace.org.uk> Message-ID: <g5k3q7$ig7$1@ger.gmane.org> Michael Foord wrote: > Collin Winter wrote: >> Is any provision being made for a 2to3 fixer/otherwise-automated >> transition for the changes you propose here? >> > > As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed? A fixer will only be needed when it actually is needed, but when it is, it should be a unittest-name fixer since previous 3.x code will also need fixing. Since the duplicates are multiples names for the same objects, the fixer should be a trivial name substitution. From ben+python at benfinney.id.au Wed Jul 16 08:25:06 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 16:25:06 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> <487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au> <487D46A2.4050207@canterbury.ac.nz> <487D8638.9060807@gmail.com> Message-ID: <8763r64a7x.fsf@benfinney.id.au> Nick Coghlan <ncoghlan at gmail.com> writes: > What did you think of the "check" idea at the end of the email? > > Test assertions: > check(x).almost_equal(y) > check(x).is_(y) > check(x).in_(y) > check(x).equals(y) > > Test negative assertions: > check(x).not_almost_equal(y) > check(x).is_not(y) > check(x).not_in(y) > check(x).not_equal(y) -1 'check' is even less explicit about what will happen than 'assert'. At least the latter has existing programming-language connotations of "fail immediately if not true", which 'check' lacks. -- \ ?I used to work in a fire hydrant factory. You couldn't park | `\ anywhere near the place.? ?Steven Wright | _o__) | Ben Finney From grubert at users.sourceforge.net Wed Jul 16 08:50:02 2008 From: grubert at users.sourceforge.net (engelbert gruber) Date: Wed, 16 Jul 2008 08:50:02 +0200 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> <g5j80e$g4d$1@ger.gmane.org> <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com> Message-ID: <ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com> I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes ``DeprecationWarning: callable() not supported in 3.x;`` . and if someone with commit rights would take i would volunteer to fix these two on case by case basis, even if it is not my case. all the best From andrew-pythondev at puzzling.org Wed Jul 16 09:00:56 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Wed, 16 Jul 2008 17:00:56 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) In-Reply-To: <487D8638.9060807@gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> <487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au> <487D46A2.4050207@canterbury.ac.nz> <487D8638.9060807@gmail.com> Message-ID: <20080716070056.GD26808@steerpike.home.puzzling.org> Nick Coghlan wrote: [...] > > What did you think of the "check" idea at the end of the email? > > Test assertions: > check(x).almost_equal(y) > check(x).is_(y) > check(x).in_(y) > check(x).equals(y) > > Test negative assertions: > check(x).not_almost_equal(y) > check(x).is_not(y) > check(x).not_in(y) > check(x).not_equal(y) Wow. This is a textbook example of a bikeshed discussion. The names (and now syntax!) of assertions are the most cosmetic issue there is with the unittest module, yet I see over *200* messages about it. Deeper, but important issues, such as how to raise structured exceptions that a GUI test runner could usefully display and introspect, are ignored. I can list lots of others. For assertions, just flip a coin already (or a roll a d20, perhaps...). Can we perhaps devote this considerable energy to significant improvements, please? -Andrew. From ben+python at benfinney.id.au Wed Jul 16 09:01:30 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 17:01:30 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com> <87bq0y60se.fsf@benfinney.id.au> <487D5CAF.60009@scottdial.com> Message-ID: <871w1u48j9.fsf@benfinney.id.au> Scott Dial <scott at scottdial.com> writes: > I would argue to go even further: > > assertEqual = assert_eq > assertAlmostEqual = assert_almost_eq > assertNotEqual = assert_ne > assertNotAlmostEqual = assert_almost_ne > > I'm not sure if there are others, but using the same abbreviations > from operator is consistent and readable and short, in my opinion. This doesn't seem to be a good fit for either of "Consolidate names in unittest API" (which seeks to make the renames meet standards, not find better names), nor "Frequently-requested additions to unittest" (which seeks *only* to add functionality, not change existing names). -- \ ?An expert is a man who has made all the mistakes which can be | `\ made in a very narrow field.? ?Niels Bohr | _o__) | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 09:53:22 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 17:53:22 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest`module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <0AAE04B8D8904B90BD9E8B613DEF0B2E@RaymondLaptop1> <87od4z8n9o.fsf@benfinney.id.au> <487C82F1.9010506@gmail.com> <487CB144.1080200@gmail.com> <87zloi6a8k.fsf@benfinney.id.au> <487D46A2.4050207@canterbury.ac.nz> <487D8638.9060807@gmail.com> <20080716070056.GD26808@steerpike.home.puzzling.org> Message-ID: <87skua2rkd.fsf@benfinney.id.au> Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > This is a textbook example of a bikeshed discussion. The names (and > now syntax!) of assertions are the most cosmetic issue there is with > the unittest module, yet I see over *200* messages about it. I've found it much more insightful than you seem to. I'm sorry it hasn't been of more interest to you, but please don't discount the value of this discussion for the rest of us. > Deeper, but important issues, such as how to raise structured > exceptions that a GUI test runner could usefully display and > introspect, are ignored. I can list lots of others. I look forward to you writing, promoting, and championing the PEP document(s) for these issues that you consider important. -- \ ?Some forms of reality are so horrible we refuse to face them, | `\ unless we are trapped into it by comedy. To label any subject | _o__) unsuitable for comedy is to admit defeat.? ?Peter Sellers | Ben Finney From mal at egenix.com Wed Jul 16 10:00:32 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 16 Jul 2008 10:00:32 +0200 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> Message-ID: <487DAAA0.1040604@egenix.com> On 2008-07-16 02:20, Collin Winter wrote: > On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote: >> Significant updates include removing all reference to the >> (already-resolved) new-style class issue, adding footnotes and >> references, and a Rationale summary of discussion on both sides of the >> divide for 'assert*' versus 'fail*' names. >> >> >> :PEP: XXX >> :Title: Consolidating names in the `unittest` module >> :Version: 0.2 >> :Last-Modified: 2008-07-15 >> :Author: Ben Finney <ben+python at benfinney.id.au> >> :Status: Draft >> :Type: Standards Track >> :Content-Type: test/x-rst >> :Created: 2008-07-14 >> :Python-Version: 2.7, 3.1 +1 for doing this in 3.1. -1 for Python 2.7. The main reason is that there's just too much 2.x code out there using the API names you are suggesting to change and/or remove from the module. Since this is a major change in the unit test API, I'd also like to suggest that you use a new module name. This is both a precaution to prevent tests failing due to not having been upgraded and a way for old code to continue working by adding the old unittest module on sys.path. Please note that the required renaming of the methods in existing tests is not going to be as straight forward as you may think, since you may well rename method calls into the tested application rather than just the unit test class method calls if you're not careful. >> Abstract >> ======== >> >> This PEP proposes to consolidate the names that constitute the API of >> the standard library `unittest` module, with the goal of removing >> redundant names, and conforming with PEP 8. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ben+python at benfinney.id.au Wed Jul 16 10:14:37 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 18:14:37 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> Message-ID: <87od4y2qky.fsf@benfinney.id.au> "M.-A. Lemburg" <mal at egenix.com> writes: > Since this is a major change in the unit test API, I'd also like > to suggest that you use a new module name. > > This is both a precaution to prevent tests failing due to not having > been upgraded and a way for old code to continue working by adding > the old unittest module on sys.path. Do you have a specific argument against the provisions already stated in the PEP for backward compatibility? They seem to address your concerns already. -- \ ?If you don't know what your program is supposed to do, you'd | `\ better not start writing it.? ?Edsger W. Dijkstra | _o__) | Ben Finney From stephen at xemacs.org Wed Jul 16 12:21:21 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Jul 2008 19:21:21 +0900 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <873ama5xze.fsf@benfinney.id.au> References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol> <873ama5xze.fsf@benfinney.id.au> Message-ID: <87od4yw2n2.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > This "fail is a negative word" has already been rebutted, by native > speakers of English. Not successfully, it hasn't. Steven d'Aprano describes one style of testing as "the test passes if it fails to fail in each of a sequence of cases." That is perfectly good English, which makes no sense if "fail" completely lacks the semantics of negation. The intuition that "fail" is a negative word is thus well-founded in standard usage. By the way, a native speaker is a person who has no need to understand how his language works; he just uses it. Being a native speaker doesn't qualify one as an authority on her language. From solipsis at pitrou.net Wed Jul 16 12:16:26 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 16 Jul 2008 10:16:26 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?PEP=3A_Consolidating_names_and_classes_in_?= =?utf-8?q?the=09=60unittest=60_module_=28updated_2008-07-15=29?= References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol> <873ama5xze.fsf@benfinney.id.au> Message-ID: <loom.20080716T100023-565@post.gmane.org> Ben Finney <ben+python <at> benfinney.id.au> writes: > > This "fail is a negative word" has already been rebutted, by native > speakers of English. Well, Stephen's and Greg's own answers notwithstanding, if you really want an authoritative answer, the best would be to open a dictionary and contrast the given definitions... * http://www.thefreedictionary.com/fail "To prove deficient or lacking; perform ineffectively or inadequately; To be unsuccessful" * http://www.thefreedictionary.com/assert "To state or express positively" But perhaps there is enough of this fail/assert discussion now :-) Regards Antoine. From mal at egenix.com Wed Jul 16 13:20:19 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 16 Jul 2008 13:20:19 +0200 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <87od4y2qky.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au> Message-ID: <487DD973.7010007@egenix.com> On 2008-07-16 10:14, Ben Finney wrote: > "M.-A. Lemburg" <mal at egenix.com> writes: > >> Since this is a major change in the unit test API, I'd also like >> to suggest that you use a new module name. >> >> This is both a precaution to prevent tests failing due to not having >> been upgraded and a way for old code to continue working by adding >> the old unittest module on sys.path. > > Do you have a specific argument against the provisions already stated > in the PEP for backward compatibility? They seem to address your > concerns already. The PEP doesn't mention changing the module name and deprecating the old one. Instead it wants to deprecate all the old names (and cites PEP 4 for this), but keeping the module name. Note that PEP 4 targets deprecating use of whole modules, not single APIs, or - like in your case - more or less the complete existing API of a module. Given the scope of the changes, you are really creating a completely new API and as a result should also get a new module name. You can then deprecate use of the old "unittest" module name and point users to the new one. Developers who don't feel like changing 10000+ tests can then continue to use the old module and start using the new module for new projects. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From fuzzyman at voidspace.org.uk Wed Jul 16 14:02:38 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 13:02:38 +0100 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <487DD973.7010007@egenix.com> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au> <487DD973.7010007@egenix.com> Message-ID: <487DE35E.2070501@voidspace.org.uk> M.-A. Lemburg wrote: > On 2008-07-16 10:14, Ben Finney wrote: >> "M.-A. Lemburg" <mal at egenix.com> writes: >> >>> Since this is a major change in the unit test API, I'd also like >>> to suggest that you use a new module name. >>> >>> This is both a precaution to prevent tests failing due to not having >>> been upgraded and a way for old code to continue working by adding >>> the old unittest module on sys.path. >> >> Do you have a specific argument against the provisions already stated >> in the PEP for backward compatibility? They seem to address your >> concerns already. > > The PEP doesn't mention changing the module name and deprecating > the old one. Instead it wants to deprecate all the old names (and cites > PEP 4 for this), but keeping the module name. > > Note that PEP 4 targets deprecating use of whole modules, not single > APIs, or - like in your case - more or less the complete existing API > of a module. Which PEP is usually referenced for the deprecation of individual APIs? > > Given the scope of the changes, you are really creating a completely > new API and as a result should also get a new module name. You can then > deprecate use of the old "unittest" module name and point users to the > new one. You propose that we duplicate the entire module with a new name, maintaining both in parallel but with different method names? That doesn't sound wise to me. > > Developers who don't feel like changing 10000+ tests can then continue > to use the old module and start using the new module for new projects. > So we shouldn't bring the API inline with PEP 8 because it is widely used? Even if it causes some pain (and the methods won't be removed for several years if we follow the normal deprecation schedule), the fact that the API is widely used would seem to be an argument in favor of making it follow the Python style guidelines. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From rrr at ronadam.com Wed Jul 16 14:13:20 2008 From: rrr at ronadam.com (Ron Adam) Date: Wed, 16 Jul 2008 07:13:20 -0500 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> Message-ID: <487DE5E0.4060406@ronadam.com> Raymond Hettinger wrote: > From: "Ben Finney" <ben+python at benfinney.id.au> >> Right, so I'm putting up a separate PEP just for the renaming. Should >> be arriving on this list soon. > > I would like to work with you or someone else who is interested > on an alternative PEP for a separate, simpler test module > using the py.test syntax. That is much simpler to learn and use. > Instead of self.assertIsNot and whatnot, you write: > assert a is not b > No need for tons of word-by-word spellings on things we already > have syntax for. Almost anyone who has used py.test can attest > its syntax is much more natural, easy to learn, easy to both > read and write, and is much lighter weight. I think some variant > of py.test could be done that is compatable with unittest > and the did not have the "magic" present in earlier versions of py.test. > I wrote a recipe (somewhat rough and incomplete) that shows how > easily this could be done: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194 > > Raymond +1 for a simpler testing module. Just letting you know there is interest in a lighter weight testing suite. Looking at the unittest discussions, it doesn't look like it is getting easier to use, but more complex. Py.test looks very interesting, especially the test discovery parts. I also agree we don't need special methods for every variation of assertedness. I've been thinking that a few decorators may go a long way to making writing tests easy. It also reduces the level of indentation needed. Ron From rrr at ronadam.com Wed Jul 16 14:13:20 2008 From: rrr at ronadam.com (Ron Adam) Date: Wed, 16 Jul 2008 07:13:20 -0500 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au><487A8700.8000200@voidspace.org.uk><87mykka8el.fsf@benfinney.id.au><ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1><487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> Message-ID: <487DE5E0.4060406@ronadam.com> Raymond Hettinger wrote: > From: "Ben Finney" <ben+python at benfinney.id.au> >> Right, so I'm putting up a separate PEP just for the renaming. Should >> be arriving on this list soon. > > I would like to work with you or someone else who is interested > on an alternative PEP for a separate, simpler test module > using the py.test syntax. That is much simpler to learn and use. > Instead of self.assertIsNot and whatnot, you write: > assert a is not b > No need for tons of word-by-word spellings on things we already > have syntax for. Almost anyone who has used py.test can attest > its syntax is much more natural, easy to learn, easy to both > read and write, and is much lighter weight. I think some variant > of py.test could be done that is compatable with unittest > and the did not have the "magic" present in earlier versions of py.test. > I wrote a recipe (somewhat rough and incomplete) that shows how > easily this could be done: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194 > > Raymond +1 for a simpler testing module. Just letting you know there is interest in a lighter weight testing suite. Looking at the unittest discussions, it doesn't look like it is getting easier to use, but more complex. Py.test looks very interesting, especially the test discovery parts. I also agree we don't need special methods for every variation of assertedness. I've been thinking that a few decorators may go a long way to making writing tests easy. It also reduces the level of indentation needed. Ron From fuzzyman at voidspace.org.uk Wed Jul 16 14:21:44 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 13:21:44 +0100 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <g5k3q7$ig7$1@ger.gmane.org> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487D48F9.3000903@voidspace.org.uk> <g5k3q7$ig7$1@ger.gmane.org> Message-ID: <487DE7D8.6050308@voidspace.org.uk> Terry Reedy wrote: > > > Michael Foord wrote: >> Collin Winter wrote: > >>> Is any provision being made for a 2to3 fixer/otherwise-automated >>> transition for the changes you propose here? >>> >> >> As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed? > > A fixer will only be needed when it actually is needed, but when it > is, it should be a unittest-name fixer since previous 3.x code will > also need fixing. Since the duplicates are multiples names for the > same objects, the fixer should be a trivial name substitution. Can 2to3 fixers be used for 2to2 and 3to3 translation then? Michael > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From ben+python at benfinney.id.au Wed Jul 16 14:22:48 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 22:22:48 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol> <873ama5xze.fsf@benfinney.id.au> <87od4yw2n2.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <877ibm2f3b.fsf@benfinney.id.au> "Stephen J. Turnbull" <stephen at xemacs.org> writes: > The intuition that "fail" is a negative word is thus well-founded in > standard usage. That's not the same thing as "fail" being a negative word in the sense meant by "double negative". That is, "not fail" is not a double negative; nor is "fail if X is not inside Y" a double negative. The use of "fail" in those phrases is a action, a verb, not a "negative". So, this issue of avoiding "fail" in order to "avoid double negatives" has no basis in the use of "fail" in unittest. -- \ ?It was half way to Rivendell when the drugs began to take | `\ hold? ?Hunter S. Tolkien, _Fear and Loathing in Barad-D?r_ | _o__) | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 14:24:29 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 22:24:29 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol> <873ama5xze.fsf@benfinney.id.au> <loom.20080716T100023-565@post.gmane.org> Message-ID: <873ama2f0i.fsf@benfinney.id.au> Antoine Pitrou <solipsis at pitrou.net> writes: > * http://www.thefreedictionary.com/fail > "To prove deficient or lacking; perform ineffectively or inadequately; To be > unsuccessful" Yes. It's a verb, not a negative modifer. To use it with a negative like "not" is not creating a "double negative". -- \ ?It's dangerous to be right when the government is wrong.? | `\ ?Francois Marie Arouet Voltaire | _o__) | Ben Finney From fuzzyman at voidspace.org.uk Wed Jul 16 14:30:58 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 13:30:58 +0100 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module In-Reply-To: <bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com> References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com> <87bq0y60se.fsf@benfinney.id.au> <bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com> Message-ID: <487DEA02.2070508@voidspace.org.uk> Brett Cannon wrote: > On Tue, Jul 15, 2008 at 7:05 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > >> Scott Dial <scott+python-dev at scottdial.com> writes: >> >> >>> Why [introduce redundant test names]? >>> >>> assert_not_less_than = assert_greater_than_or_equal >>> assert_not_greater_than = assert_less_than_or_equal >>> assert_not_less_than_or_equal = assert_greater_than >>> assert_not_greater_than_or_equal = assert_less_than >>> >> To answer the question: The above tests are logically equivalent, but >> the failure message would be different, reporting failure in terms of >> what the caller wanted to test. >> >> I se your point though. I'd like to see what others think on this >> issue. >> >> >>> Besides, ``assert_not_greater_than_or_equal`` is god-awful-long, >>> along with the complaints about PEP-8-ifying. I wonder if it would >>> be better to abbreviate these names with the *same name* that was >>> used for the attribute in the operator module. Let's not reinvent >>> the wheel here.. >>> >> Interesting. So you advocate collapsing the above eight tests into the >> following four: >> >> assert_lt >> assert_gt >> assert_le >> assert_ge >> > > Is any of this really necessary? Isn't this the equivalent of > ``assert_(a < b)``? It seems like the only thing you get out of this > is a nicer error message, but ``assert_(a < b, 'not %r <= %s' % (a, > b))`` is not that complex. And do these cases really come up that > often? I would want to see some numbers showing that these are really > necessary (in both usage and people even specifying an error message > in the first place). > I'm inclined to agree. It was part of a set of additions suggested by Guido. From here I think (as part of the unittest extensions that google maintains): http://mail.python.org/pipermail/python-dev/2008-April/078702.html I've used tests like that when implementing numeric objects, which has been several times - but only a tiny proportion of the tests I've written. I wouldn't be sorry not to include them. Michael > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From mal at egenix.com Wed Jul 16 14:36:45 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 16 Jul 2008 14:36:45 +0200 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <487DE35E.2070501@voidspace.org.uk> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au> <487DD973.7010007@egenix.com> <487DE35E.2070501@voidspace.org.uk> Message-ID: <487DEB5D.7010507@egenix.com> On 2008-07-16 14:02, Michael Foord wrote: > M.-A. Lemburg wrote: >> On 2008-07-16 10:14, Ben Finney wrote: >>> "M.-A. Lemburg" <mal at egenix.com> writes: >>> >>>> Since this is a major change in the unit test API, I'd also like >>>> to suggest that you use a new module name. >>>> >>>> This is both a precaution to prevent tests failing due to not having >>>> been upgraded and a way for old code to continue working by adding >>>> the old unittest module on sys.path. >>> >>> Do you have a specific argument against the provisions already stated >>> in the PEP for backward compatibility? They seem to address your >>> concerns already. >> >> The PEP doesn't mention changing the module name and deprecating >> the old one. Instead it wants to deprecate all the old names (and cites >> PEP 4 for this), but keeping the module name. >> >> Note that PEP 4 targets deprecating use of whole modules, not single >> APIs, or - like in your case - more or less the complete existing API >> of a module. > > Which PEP is usually referenced for the deprecation of individual APIs? PEP 5 could be used for that. Adding several 10s of deprecation warnings to the unittest module is not going to make life easier for anyone. Adding just a single one on import and following PEP 4 is. If you do want to apply major changes to a module without changing the name, then this could be done as part of the 2.x -> 3.x transition. The 2.x branch is not the right place for such breakage. >> Given the scope of the changes, you are really creating a completely >> new API and as a result should also get a new module name. You can then >> deprecate use of the old "unittest" module name and point users to the >> new one. > > You propose that we duplicate the entire module with a new name, > maintaining both in parallel but with different method names? No, I'm proposing to apply all the name changes to a new module and deprecate the old one. unittest will then go unmaintained until it is removed. > That doesn't sound wise to me. > >> >> Developers who don't feel like changing 10000+ tests can then continue >> to use the old module and start using the new module for new projects. >> > > So we shouldn't bring the API inline with PEP 8 because it is widely used? I didn't say that. However, if it's not required, then breaking a complete module API isn't necessary - "practicality beats purity". Instead add a new module with all the changes and have developers gradually migrate to the new code. > Even if it causes some pain (and the methods won't be removed for > several years if we follow the normal deprecation schedule), the fact > that the API is widely used would seem to be an argument in favor of > making it follow the Python style guidelines. So you're saying that because many people use the code, we should be more inclined to make their life harder. That's an interesting argument :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ben+python at benfinney.id.au Wed Jul 16 14:37:31 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 22:37:31 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au> <487DD973.7010007@egenix.com> Message-ID: <87y7420zuc.fsf@benfinney.id.au> "M.-A. Lemburg" <mal at egenix.com> writes: > The PEP doesn't mention changing the module name and deprecating > the old one. Right. The intention is to have a PEP-8-conformant 'unittest' module, not an entirely new module. > Instead it wants to deprecate all the old names (and cites PEP 4 for > this), but keeping the module name. Yes, because all the existing documented API is retained as is, just with new names that conform with PEP 8. > Note that PEP 4 targets deprecating use of whole modules, not single > APIs, or - like in your case - more or less the complete existing > API of a module. This is true, I tried to be specific about what was to be done to deprecate individual attributes, and could only find PEP 4. Is there a better reference for deprecation of Python features? > Given the scope of the changes, you are really creating a completely > new API and as a result should also get a new module name. I disagree. The API is not "completely new"; it has exactly the same functionality and expected behaviour, just with a different set of names. > Developers who don't feel like changing 10000+ tests can then > continue to use the old module and start using the new module for > new projects. I agree with Michael Foord that an extensively-used standard library module is an argument in favour of re-working that module to be in line with the standard library guidelines. The change is, as has been pointed out elsewhere, a replacement of one set of names with another, retaining exactly the same expected behaviour. It's not on the scale of deprecating usage of an entire module. -- \ ?I went to the museum where they had all the heads and arms | `\ from the statues that are in all the other museums.? ?Steven | _o__) Wright | Ben Finney From solipsis at pitrou.net Wed Jul 16 14:40:22 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 16 Jul 2008 12:40:22 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?PEP=3A_Consolidating_names_and_classes_in?= =?utf-8?q?=09the=09=60unittest=60_module_=28updated_2008-07-15=29?= References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol> <873ama5xze.fsf@benfinney.id.au> <loom.20080716T100023-565@post.gmane.org> <873ama2f0i.fsf@benfinney.id.au> Message-ID: <loom.20080716T123508-851@post.gmane.org> Ben Finney <ben+python <at> benfinney.id.au> writes: > > * http://www.thefreedictionary.com/fail > > "To prove deficient or lacking; perform ineffectively or inadequately; To be > > unsuccessful" > > Yes. It's a verb, not a negative modifer. To use it with a negative > like "not" is not creating a "double negative". Ok, let's stop it there. I'm tired of trying to point the obvious. From ben+python at benfinney.id.au Wed Jul 16 15:00:51 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 23:00:51 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487DE5E0.4060406@ronadam.com> Message-ID: <87prpe0yrg.fsf@benfinney.id.au> Ron Adam <rrr at ronadam.com> writes: > +1 for a simpler testing module. I've no objection. > Just letting you know there is interest in a lighter weight testing > suite. 'doctest' is a very simple testing module, that is a very useful tool. > Looking at the unittest discussions, it doesn't look like it is > getting easier to use, but more complex. How so? One PEP proposed this week specifies to do nothing but conform 'unittest' with the standard library guidelines, and remove redundant names. That surely makes it simpler to use. Another PEP specifies to add helper methods that simplify a number of common cases. What is it you see making unittest "more complex to use"? > Py.test looks very interesting, especially the test discovery parts. > I also agree we don't need special methods for every variation of > assertedness. My main complaint about 'py.test' is that it's yet-another-framework. We already have 'doctest' and 'unittest', and they play together reasonably well. 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks better for consideration, especially since it integrates well with 'unittest'. > I've been thinking that a few decorators may go a long way to making > writing tests easy. It also reduces the level of indentation needed. There are a number of these already in 'nose'. -- \ ?I fly Air Bizarre. You buy a combination one-way round-trip | `\ ticket. Leave any Monday, and they bring you back the previous | _o__) Friday. That way you still have the weekend.? ?Steven Wright | Ben Finney From Scott.Daniels at Acm.Org Wed Jul 16 15:13:53 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Wed, 16 Jul 2008 06:13:53 -0700 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module In-Reply-To: <87abgi4bxi.fsf@benfinney.id.au> References: <87fxqa62xb.fsf@benfinney.id.au> <87abgi4bxi.fsf@benfinney.id.au> Message-ID: <g5krih$4f8$1@ger.gmane.org> Ben Finney wrote: ... > def assert_compare_true(op, first, second, msg=None): > if msg is None: > msg = "%(first)r %(op)r %(second)" % vars() > if not op(first, second): > raise self.failure_exception(msg) I would rather something more like: def assert_compare_true(op, first, second, msg=None): if op(first, second): return raise self.failure_exception(msg) if msg is None: self.failure_exception("%(first)r %(op)r %(second)" % vars()) self.failure_exception("%(first)r %(op)r %(second): %(msg)" % vars()) --Scott David Daniels Scott.Daniels at Acm.Org From ben+python at benfinney.id.au Wed Jul 16 15:12:03 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 23:12:03 +1000 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au> <487DD973.7010007@egenix.com> <487DE35E.2070501@voidspace.org.uk> <487DEB5D.7010507@egenix.com> Message-ID: <87lk020y8s.fsf@benfinney.id.au> "M.-A. Lemburg" <mal at egenix.com> writes: > On 2008-07-16 14:02, Michael Foord wrote: > > M.-A. Lemburg wrote: > >> Note that PEP 4 targets deprecating use of whole modules, not > >> single APIs, or - like in your case - more or less the complete > >> existing API of a module. > > > > Which PEP is usually referenced for the deprecation of individual > > APIs? > > PEP 5 could be used for that. That seems an even worse fit; it speaks of changing language features, not library modules. At least PEP 4 talks about when to raise DeprecationWarning. > Adding several 10s of deprecation warnings to the unittest module is > not going to make life easier for anyone. Adding just a single one > on import and following PEP 4 is. I don't see how the first "is not going to make life easier" if the second somehow is. Is a programmer going to be helpless in the face of some DeprecationWarnings but not others? > If you do want to apply major changes to a module without changing > the name, then this could be done as part of the 2.x -> 3.x > transition. This has already been rejected <URL:http://mail.python.org/pipermail/python-dev/2008-April/078485.html>. I'm inclined to agree that it's not right for 2.x. I'll revise the PEP accordingly. -- \ ?First they came for the verbs, and I said nothing, for verbing | `\ weirds language. Then, they arrival for the nouns and I speech | _o__) nothing, for I no verbs.? ?Peter Ellis | Ben Finney From barry at python.org Wed Jul 16 15:18:19 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 16 Jul 2008 09:18:19 -0400 Subject: [Python-Dev] Reminder: beta 2's schedule for tomorrow In-Reply-To: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> References: <294E40E5-06DF-43B8-8895-96AB2C319038@python.org> Message-ID: <53E31F7D-875C-466C-B5F6-2D3D274847D4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 15, 2008, at 8:32 AM, Barry Warsaw wrote: > If there is anything you need a decision on, please follow up to > this thread. I'm inundated with email so I can't watch every thread > on the mailing lists. Or ping me on #python-dev. I'm not currently optimistic that we can release today. As I see it we have several problems serious enough to delay a release. First, we have no green 2.6 or 3.0 buildbots http://www.python.org/dev/buildbot/stable/ It looks like few of the buildbots for 2.6 are actually online, but still, the 3.0 buildbots are all failing. We still have, and have had for a long time now, serious problems with the multiprocessing module: http://bugs.python.org/issue?@columns=title,id,activity,versions,status&@sort=activity&@filter=priority,status&@pagesize=50&@startwith=0&priority=1&status=1&@dispname=Showstoppers I noticed today that _multiprocessing.so has build problems on OS X 10.5 and Ubuntu 8.04. I opened a separate issue about that so as not to get lost in the other bug about test_multiprocessing hanging. As evidenced by the thread for issue 3088, lots of people have put a lot of work into this module, but unfortunately we're still not there. These seem serious enough to me to hold up the release, especially since we have only one more beta left. Speaking of which, we have a number of deferred blockers: http://bugs.python.org/issue?@columns=title,id,activity,status&@sort=activity&@group=priority&@filter=priority,status&@pagesize=50&@startwith=0&priority=6&status=1&@dispname=Deferred%20Blockers These will not block beta 2, but they /will/ block beta 3, so they really need some attention. Please everyone, if you have only a little bit of time to work on Python, I hope you will attack the release critical and deferred blocker issues, and work on turning the buildbots green. These are the top priorities in order to get 2.6 and 3.0 out on time. And just as added incentive, our October 1st goal is being noted by downstream vendors. I've been told that "Apple is willing to take the new Python if it is GM on schedule by Oct 1st, but may not if it is delayed" though you should not infer anything about Apple's schedule from this. Still, we won't sacrifice quality in order to hit the October 1st goal. After beta 2, I start getting mean. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSH31HHEjvBPtnXfVAQIH+AQAtkRJo0uhvZ300jq0YSR6ezUU0tMHJwmd pwLWwZWDqPyh3/ohKwrajdGm5hYS8gOnTo+k2XEEJjQJdLMTblzZ3ArsGfhXHqbV GPeKK/6BYT6SOD75ubINq+jSsu6Pvjsadbk/cp/x53WwHL8kX40D0YQBhp3KQRrz zgFmfVFPcys= =3SDN -----END PGP SIGNATURE----- From ben+python at benfinney.id.au Wed Jul 16 15:26:12 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 23:26:12 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module References: <87fxqa62xb.fsf@benfinney.id.au> <487D51E2.8060908@scottdial.com> <87bq0y60se.fsf@benfinney.id.au> <bbaeab100807151929y5c7d37d7w982ace9e730a211a@mail.gmail.com> <487DEA02.2070508@voidspace.org.uk> Message-ID: <87hcaq0xl7.fsf@benfinney.id.au> Michael Foord <fuzzyman at voidspace.org.uk> writes: > I'm inclined to agree. It was part of a set of additions suggested > by Guido. From here I think (as part of the unittest extensions that > google maintains): > > http://mail.python.org/pipermail/python-dev/2008-April/078702.html > > I've used tests like that when implementing numeric objects, which > has been several times - but only a tiny proportion of the tests > I've written. > > I wouldn't be sorry not to include them. Okay. I'll revise the PEP to remove all these lt, gt, le, ge comparison tests, and note that they are supported entirely by ``assert_compare_true`` and ``assert_compare_false`` with a specific comparison operator. -- \ ?Many are stubborn in pursuit of the path they have chosen, few | `\ in pursuit of the goal.? ?Friedrich Nietzsche | _o__) | Ben Finney From ben+python at benfinney.id.au Wed Jul 16 15:36:32 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 23:36:32 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module References: <87fxqa62xb.fsf@benfinney.id.au> <87abgi4bxi.fsf@benfinney.id.au> <g5krih$4f8$1@ger.gmane.org> Message-ID: <87d4le0x3z.fsf@benfinney.id.au> Scott David Daniels <Scott.Daniels at Acm.Org> writes: > I would rather something more like: > > def assert_compare_true(op, first, second, msg=None): > if op(first, second): > return > raise self.failure_exception(msg) > if msg is None: > self.failure_exception("%(first)r %(op)r %(second)" > % vars()) > self.failure_exception("%(first)r %(op)r %(second): %(msg)" > % vars()) I'm confused. It appears to me that your version never gets past the first 'raise' statement, which is unconditional; and the rest seems to do nothing but instantiate exceptions without using them. Do you perhaps mean something like this:: def assert_compare_true(op, first, second, msg=None): fail_detail = "%(first)r %(op)r %(second)r" % vars() if msg is None: msg = fail_detail else: msg = "%(fail_detail)s: %(msg)s" % vars() if not op(first, second): raise self.failure_exception(msg) If so, that sems to be in line with the "Enhanced failure message" principle exercised elsewhere in the same PEP, i.e. that the failure message should *always* contain certain information, even if a message is specified by the caller. One downside I can see is that, in optimising for this common case, it makes the function useless to someone who wants to specify their own failure message exactly, without the specific extra information in that specific format. What do others think of this? -- \ ?Holy contributing to the delinquency of minors, Batman!? ?Robin | `\ | _o__) | Ben Finney From rbp at isnomore.net Wed Jul 16 15:35:39 2008 From: rbp at isnomore.net (Rodrigo Bernardo Pimentel) Date: Wed, 16 Jul 2008 10:35:39 -0300 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <874p6q7oxo.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <487CE248.9030400@palladion.com> <874p6q7oxo.fsf@benfinney.id.au> Message-ID: <20080716133539.GA16835@isnomore.net> On Tue, Jul 15 2008 at 07:38:59PM BRT, Ben Finney <ben+python at benfinney.id.au> wrote: > Tres Seaver <tseaver at palladion.com> writes: > > > I would keep both by preference, rather than insist on a "foolish > > consistency." +1 > The "consistency" argument leads to the PEP 8 names. The removal of > redundant names is not made in the name of consistency, but of API > simplicity. Isn't this *enourmous* discussion a clear sign that a lot of people *won't* find a trimmed-down API simpler? If assert* to fail* mapping is consistent, then voil?, simple, everyone can remember method names (+1 for PEP-8 renaming), everyone is free to think of each test as they please. rbp -- Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978> From ben+python at benfinney.id.au Wed Jul 16 15:54:26 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 16 Jul 2008 23:54:26 +1000 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <487CE248.9030400@palladion.com> <874p6q7oxo.fsf@benfinney.id.au> <20080716133539.GA16835@isnomore.net> Message-ID: <878ww20wa5.fsf@benfinney.id.au> Rodrigo Bernardo Pimentel <rbp at isnomore.net> writes: > On Tue, Jul 15 2008 at 07:38:59PM BRT, Ben Finney <ben+python at benfinney.id.au> wrote: > > The "consistency" argument leads to the PEP 8 names. The removal > > of redundant names is not made in the name of consistency, but of > > API simplicity. > > Isn't this *enourmous* discussion a clear sign that a lot of people > *won't* find a trimmed-down API simpler? The majority of this discussion hasn't been about whether or not to trim the API, so I'd have to answer no. Instead, it shows that people have different ideas of how it should be done. > If assert* to fail* mapping is consistent, then voil?, simple, > everyone can remember method names (+1 for PEP-8 renaming), everyone > is free to think of each test as they please. Thanks for the feedback. -- \ ?Pity the meek, for they shall inherit the earth.? ?Donald | `\ Robert Perry Marquis | _o__) | Ben Finney From rbp at isnomore.net Wed Jul 16 16:26:30 2008 From: rbp at isnomore.net (Rodrigo Bernardo Pimentel) Date: Wed, 16 Jul 2008 11:26:30 -0300 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <878ww20wa5.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <20080715130533.GB17917@steerpike.home.puzzling.org> <487CE248.9030400@palladion.com> <874p6q7oxo.fsf@benfinney.id.au> <20080716133539.GA16835@isnomore.net> <878ww20wa5.fsf@benfinney.id.au> Message-ID: <20080716142629.GA19934@isnomore.net> On Wed, Jul 16 2008 at 10:54:26AM BRT, Ben Finney <ben+python at benfinney.id.au> wrote: > Rodrigo Bernardo Pimentel <rbp at isnomore.net> writes: > > > On Tue, Jul 15 2008 at 07:38:59PM BRT, Ben Finney <ben+python at benfinney.id.au> wrote: > > > The "consistency" argument leads to the PEP 8 names. The removal > > > of redundant names is not made in the name of consistency, but of > > > API simplicity. > > > > Isn't this *enourmous* discussion a clear sign that a lot of people > > *won't* find a trimmed-down API simpler? > > The majority of this discussion hasn't been about whether or not to > trim the API, so I'd have to answer no. Instead, it shows that people > have different ideas of how it should be done. Ok, I've oversimplified my sentence. In more words, what I meant was that I don't think the benefits of getting rid of assert* or fail* methods are very clear (thus my "+1" on Tres's suggestion of leaving both), and it doesn't seem to me like we're progressing in the direction of reaching an agreement (though there has been great exposition of arguments). I mean, the benefits of removing fail* seem clear for some people, but are strongly opposed by others, and even the benefits of removing assert* seem clear for some (IIRC). All this lead to my previous sentence, which should read: isn't this *enourmous* discussion a clear sign that a lot of people *won't* find an API without fail* (or assert*) simpler? I agree there's trimming to be done, of course. I'm just saying that there's clearly no agreement on this particular point of whether to remove fail* completely, so maybe we should take it as as indication that it might not actually simplify the API (in the sense of how comfortable people are with using it). > > If assert* to fail* mapping is consistent, then voil?, simple, > > everyone can remember method names (+1 for PEP-8 renaming), everyone > > is free to think of each test as they please. > > Thanks for the feedback. No problem! It's great to be in a community where these things are so openly discussed :) rbp -- Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978> From ncoghlan at gmail.com Wed Jul 16 16:29:45 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 17 Jul 2008 00:29:45 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module In-Reply-To: <87d4le0x3z.fsf@benfinney.id.au> References: <87fxqa62xb.fsf@benfinney.id.au> <87abgi4bxi.fsf@benfinney.id.au> <g5krih$4f8$1@ger.gmane.org> <87d4le0x3z.fsf@benfinney.id.au> Message-ID: <487E05D9.4050600@gmail.com> Ben Finney wrote: > Do you perhaps mean something like this:: > > def assert_compare_true(op, first, second, msg=None): > fail_detail = "%(first)r %(op)r %(second)r" % vars() > if msg is None: > msg = fail_detail > else: > msg = "%(fail_detail)s: %(msg)s" % vars() > if not op(first, second): > raise self.failure_exception(msg) > > If so, that sems to be in line with the "Enhanced failure message" > principle exercised elsewhere in the same PEP, i.e. that the failure > message should *always* contain certain information, even if a message > is specified by the caller. I was going to suggest the same thing, so this sounds like a good idea to me. The other point I was going to make is that repr(operator.lt) is actually "<built-in function lt>". It may be better to just make a general method for asserting that an operation gives an expected result and gives the name of the operation and the details of the arguments and expected result if anything goes wrong: def assert_op(op, *args, msg=None, expected=True): fail_detail = "%r%r != %r" % (op.__name__, arg, expect) if msg is None: msg = fail_detail else: msg = "%s: %s" % (msg, fail_detail) if op(*args) != expected: raise self.failure_exception(msg) > One downside I can see is that, in optimising for this common case, it > makes the function useless to someone who wants to specify their own > failure message exactly, without the specific extra information in > that specific format. If you don't want the extra details in the error messages then you can just use the basic self.assert_ method - the only advantage to use the other assertion methods is they provide additional details in the assertion error. (except assert_raises, where it provides the try/catch block as well) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From eric+python-dev at trueblade.com Wed Jul 16 16:35:16 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 16 Jul 2008 10:35:16 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' Message-ID: <487E0724.6020002@trueblade.com> Does anyone know why 'F' is the same as 'f'? Wouldn't it make more sense to either drop it, or make it convert the exponent to upper case (like 'E' and 'G')? Compatibility with %-formatting is the only reason I can think of to keep up, but I get the sense we've given up on an automatic conversion from %-formatting to str.format(). Plus, I can find no uses of '%F' in the standard library. And really, if you could write something to automatically convert %-formatting to str.format(), you'd be able to make 'F' to 'f', anyway. I know it's a nit, but I'm reviewing the documentation and it sticks out. From dickinsm at gmail.com Wed Jul 16 17:07:03 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 16 Jul 2008 16:07:03 +0100 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <487E0724.6020002@trueblade.com> References: <487E0724.6020002@trueblade.com> Message-ID: <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> On Wed, Jul 16, 2008 at 3:35 PM, Eric Smith <eric+python-dev at trueblade.com> wrote: > Does anyone know why 'F' is the same as 'f'? Wouldn't it make more sense to > either drop it, or make it convert the exponent to upper case What exponent? Isn't the point of 'f' formatting that there is no exponent? In C, the only difference seems to be that a NaN or infinity formatted with '%F' is turned into "NAN" or "INF" instead of "nan" or "inf". Mark From daniel at stutzbachenterprises.com Wed Jul 16 17:14:23 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 16 Jul 2008 10:14:23 -0500 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> Message-ID: <eae285400807160814p16d3bab3w82dff27d69c9a80@mail.gmail.com> On Wed, Jul 16, 2008 at 10:07 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > On Wed, Jul 16, 2008 at 3:35 PM, Eric Smith > <eric+python-dev at trueblade.com> wrote: >> Does anyone know why 'F' is the same as 'f'? Wouldn't it make more sense to >> either drop it, or make it convert the exponent to upper case > > What exponent? Isn't the point of 'f' formatting that there is no exponent? There's no exponent for small-magnitude numbers, but still an exponent for large-magnitude numbers: >>> '%f' % (10**100) '1e+100' -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From collinw at gmail.com Wed Jul 16 17:15:13 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 16 Jul 2008 08:15:13 -0700 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <487D48F9.3000903@voidspace.org.uk> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487D48F9.3000903@voidspace.org.uk> Message-ID: <43aa6ff70807160815m21fc967ah976af182f9402e95@mail.gmail.com> On Tue, Jul 15, 2008 at 6:03 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote: > Collin Winter wrote: >> >> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> >> wrote: >>> >>> Backwards Compatibility >>> ======================= >>> >>> The names to be obsoleted should be deprecated and removed according >>> to the schedule for modules in PEP 4 [#PEP-4]_. >>> >>> While deprecated, use of the deprecated attributes should raise a >>> ``DeprecationWarning``, with a message stating which replacement name >>> should be used. >>> >> >> Is any provision being made for a 2to3 fixer/otherwise-automated >> transition for the changes you propose here? >> > > As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed? IMO some kind of automated transition tool is needed -- anyone who has the time to convert their codebase by hand (for some definition of "by hand" that involves sed) doesn't have enough to do. Collin From eric+python-dev at trueblade.com Wed Jul 16 17:15:40 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 16 Jul 2008 11:15:40 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> Message-ID: <487E109C.7050806@trueblade.com> Mark Dickinson wrote: > On Wed, Jul 16, 2008 at 3:35 PM, Eric Smith > <eric+python-dev at trueblade.com> wrote: >> Does anyone know why 'F' is the same as 'f'? Wouldn't it make more sense to >> either drop it, or make it convert the exponent to upper case > > What exponent? Isn't the point of 'f' formatting that there is no exponent? There's no exponent until the number gets large. I haven't looked up how big the number has to get. On my Mac, it's somewhere between 1e50 and 1e60. $ ./python.exe Python 3.0b1+ (py3k:64984:64985, Jul 15 2008, 20:17:06) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> '%f' % 1e100 '1e+100' >>> '%F' % 1e100 '1e+100' >>> '%f' % 1.2 '1.200000' str.format() works the same. > In C, the only difference seems to be that a NaN or infinity formatted with '%F' > is turned into "NAN" or "INF" instead of "nan" or "inf". Not so in Python. >>> '%f' % float('nan') 'nan' >>> '%F' % float('nan') 'nan' But it is the case with 'e': >>> '%e' % float('nan') 'nan' >>> '%E' % float('nan') 'NAN' From collinw at gmail.com Wed Jul 16 17:16:38 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 16 Jul 2008 08:16:38 -0700 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <487DE7D8.6050308@voidspace.org.uk> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487D48F9.3000903@voidspace.org.uk> <g5k3q7$ig7$1@ger.gmane.org> <487DE7D8.6050308@voidspace.org.uk> Message-ID: <43aa6ff70807160816p87ee58dlca68168692053dc8@mail.gmail.com> On Wed, Jul 16, 2008 at 5:21 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote: > Terry Reedy wrote: >> >> >> Michael Foord wrote: >>> >>> Collin Winter wrote: >> >>>> Is any provision being made for a 2to3 fixer/otherwise-automated >>>> transition for the changes you propose here? >>>> >>> >>> As the deprecation is intended for 2.X and 3.X - is 2to3 fixer needed? >> >> A fixer will only be needed when it actually is needed, but when it is, it >> should be a unittest-name fixer since previous 3.x code will also need >> fixing. Since the duplicates are multiples names for the same objects, the >> fixer should be a trivial name substitution. > > Can 2to3 fixers be used for 2to2 and 3to3 translation then? The intention is for the infrastructure behind 2to3 to be generally reusable for other Python source-to-source translation tools, be that 2to2 or 3to3. That hasn't fully materialized yet, but it's getting there. Collin From dickinsm at gmail.com Wed Jul 16 17:17:26 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 16 Jul 2008 16:17:26 +0100 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <eae285400807160814p16d3bab3w82dff27d69c9a80@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> <eae285400807160814p16d3bab3w82dff27d69c9a80@mail.gmail.com> Message-ID: <5c6f2a5d0807160817u1126b13cm542b42000fddbf47@mail.gmail.com> On Wed, Jul 16, 2008 at 4:14 PM, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote: > There's no exponent for small-magnitude numbers, but still an exponent > for large-magnitude numbers: > >>>> '%f' % (10**100) > '1e+100' So there is! Thanks for the correction. Mark From dickinsm at gmail.com Wed Jul 16 17:51:39 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 16 Jul 2008 16:51:39 +0100 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <487E109C.7050806@trueblade.com> References: <487E0724.6020002@trueblade.com> <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> <487E109C.7050806@trueblade.com> Message-ID: <5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com> On Wed, Jul 16, 2008 at 4:15 PM, Eric Smith <eric+python-dev at trueblade.com> wrote: > There's no exponent until the number gets large. I haven't looked up how > big the number has to get. On my Mac, it's somewhere between 1e50 and 1e60. I think it's around 1e50, courtesy of the rather oddly-phrased line in unicodeobject.c (this is in py3k) that looks like: if (type == 'f' && (fabs(x) / 1e25) >= 1e25) In any case, I agree that the current 'F' is strange. Even after having read the relevant line of PEP 3101 several times in the past, part of my brain still believes that something formatted with 'F' should have all letters appearing in upper case. Mark From mal at egenix.com Wed Jul 16 18:36:58 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 16 Jul 2008 18:36:58 +0200 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <87lk020y8s.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> <87od4y2qky.fsf@benfinney.id.au> <487DD973.7010007@egenix.com> <487DE35E.2070501@voidspace.org.uk> <487DEB5D.7010507@egenix.com> <87lk020y8s.fsf@benfinney.id.au> Message-ID: <487E23AA.9010400@egenix.com> On 2008-07-16 15:12, Ben Finney wrote: > "M.-A. Lemburg" <mal at egenix.com> writes: > >> On 2008-07-16 14:02, Michael Foord wrote: >>> M.-A. Lemburg wrote: >>>> Note that PEP 4 targets deprecating use of whole modules, not >>>> single APIs, or - like in your case - more or less the complete >>>> existing API of a module. >>> Which PEP is usually referenced for the deprecation of individual >>> APIs? >> PEP 5 could be used for that. > > That seems an even worse fit; it speaks of changing language features, > not library modules. At least PEP 4 talks about when to raise > DeprecationWarning. Right and the methods described there are usually also applied to language changes and API changes. I just wanted to make clear that your "...see PEP 4 for how to handle backwards compatibility..." statement doesn't apply to the changes described in your PEP. However, it does point at a possible compromise which would make the transition easier on everyone. >> Adding several 10s of deprecation warnings to the unittest module is >> not going to make life easier for anyone. Adding just a single one >> on import and following PEP 4 is. > > I don't see how the first "is not going to make life easier" if the > second somehow is. Is a programmer going to be helpless in the face of > some DeprecationWarnings but not others? Using the first method (changing the API names), you force developers to change existing code, which results in testing the test code. Lots of work with no real benefit. With the second method, they can use the new names with new test code (which then imports the new module). They don't have to test their existing tests for obscure search&replace errors. >> If you do want to apply major changes to a module without changing >> the name, then this could be done as part of the 2.x -> 3.x >> transition. > > This has already been rejected > <URL:http://mail.python.org/pipermail/python-dev/2008-April/078485.html> I wasn't suggesting to apply to the change to 3.0, but instead suggesting that if you want to implement such a major API change, this should be done only on the 3.x branch and be dealt with in the 2to3 tool. > I'm inclined to agree that it's not right for 2.x. I'll revise the PEP > accordingly. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From rrr at ronadam.com Wed Jul 16 18:56:33 2008 From: rrr at ronadam.com (Ron Adam) Date: Wed, 16 Jul 2008 11:56:33 -0500 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <87prpe0yrg.fsf@benfinney.id.au> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487DE5E0.4060406@ronadam.com> <87prpe0yrg.fsf@benfinney.id.au> Message-ID: <487E2841.1050101@ronadam.com> Ben Finney wrote: > Ron Adam <rrr at ronadam.com> writes: > >> +1 for a simpler testing module. > > I've no objection. > >> Just letting you know there is interest in a lighter weight testing >> suite. > > 'doctest' is a very simple testing module, that is a very useful tool. > >> Looking at the unittest discussions, it doesn't look like it is >> getting easier to use, but more complex. > > How so? > > One PEP proposed this week specifies to do nothing but conform > 'unittest' with the standard library guidelines, and remove redundant > names. That surely makes it simpler to use. No complaint here. It's a good place to start. > Another PEP specifies to add helper methods that simplify a number of > common cases. In my opinion adding 19 more methods makes it more complex. I'd rather see more focus on handling test failures in general without the need for so many special helper functions. > What is it you see making unittest "more complex to use"? More methods and method signatures to learn and remember. assert_true(?) assert_false(?) assert_almost_equal(?) assert_equal(?) assert_not_almost_equal(?) assert_not_equal(?) assert_raises(exc_class, callable_obj, *args, **kwargs) assert_compare_true(op, first, second, msg=None) assert_in(container, member, msg=None) assert_is(first, second, msg=None) assert_less_than(first, second, msg=None) assert_greater_than(first, second, msg=None) assert_less_than_or_equal(first, second, msg=None) assert_greater_than_or_equal(first, second, msg=None) assert_members_equal(first, second, msg=None) assert_sequence_equal(first, second, msg=None) assert_raises_with_message_regex(exc_class, message_regex, callable_obj, *args, **kwargs) assert_compare_false(op, first, second, msg=None) assert_not_in(container, member, msg=None) assert_is_not(first, second, msg=None) assert_not_less_than(first, second, msg=None) assert_not_greater_than(first, second, msg=None) assert_not_less_than_or_equal(first, second, msg=None) assert_not_greater_than_or_equal(first, second, msg=None) assert_members_not_equal(first, second, msg=None) assert_sequence_not_equal(first, second, msg=None) That comes to 26 variations of assert. There are really only a small set of basic conditions to test for. correct values incorrect values unexpected exceptions correct exceptions incorrect exceptions missing exceptions I think the unittest module could better handle testing of exceptions and distinguishing exception produced by test code from the code being tested. That was painfully clear (to me) when we where fixing all the Python 3000 tests. >> Py.test looks very interesting, especially the test discovery parts. >> I also agree we don't need special methods for every variation of >> assertedness. > > My main complaint about 'py.test' is that it's yet-another-framework. > We already have 'doctest' and 'unittest', and they play together > reasonably well. I love doctest. Because it is very different in how it works, I don't think it competes with more formal testing methods. > 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks > better for consideration, especially since it integrates well with > 'unittest'. Thanks for the link! I'll give 'nose' a try next time I write tests. >> I've been thinking that a few decorators may go a long way to making >> writing tests easy. It also reduces the level of indentation needed. > > There are a number of these already in 'nose'. Yes, I looked at the decorator and think it is very nice and simple to use. If something like "nose" was a submodule of unittest, unittest may be able to use some of it's parts. Maybe gradually the more java like unittest classes and methods could be depreciated? Cheers, Ron From rrr at ronadam.com Wed Jul 16 18:56:33 2008 From: rrr at ronadam.com (Ron Adam) Date: Wed, 16 Jul 2008 11:56:33 -0500 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <87prpe0yrg.fsf@benfinney.id.au> References: <20080713093936.GA3623@benfinney.id.au> <487A8700.8000200@voidspace.org.uk> <87mykka8el.fsf@benfinney.id.au> <ECFFE68DE5E84390B9A3B98320E08994@RaymondLaptop1> <487B60FB.5010602@voidspace.org.uk> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <487DE5E0.4060406@ronadam.com> <87prpe0yrg.fsf@benfinney.id.au> Message-ID: <487E2841.1050101@ronadam.com> Ben Finney wrote: > Ron Adam <rrr at ronadam.com> writes: > >> +1 for a simpler testing module. > > I've no objection. > >> Just letting you know there is interest in a lighter weight testing >> suite. > > 'doctest' is a very simple testing module, that is a very useful tool. > >> Looking at the unittest discussions, it doesn't look like it is >> getting easier to use, but more complex. > > How so? > > One PEP proposed this week specifies to do nothing but conform > 'unittest' with the standard library guidelines, and remove redundant > names. That surely makes it simpler to use. No complaint here. It's a good place to start. > Another PEP specifies to add helper methods that simplify a number of > common cases. In my opinion adding 19 more methods makes it more complex. I'd rather see more focus on handling test failures in general without the need for so many special helper functions. > What is it you see making unittest "more complex to use"? More methods and method signatures to learn and remember. assert_true(?) assert_false(?) assert_almost_equal(?) assert_equal(?) assert_not_almost_equal(?) assert_not_equal(?) assert_raises(exc_class, callable_obj, *args, **kwargs) assert_compare_true(op, first, second, msg=None) assert_in(container, member, msg=None) assert_is(first, second, msg=None) assert_less_than(first, second, msg=None) assert_greater_than(first, second, msg=None) assert_less_than_or_equal(first, second, msg=None) assert_greater_than_or_equal(first, second, msg=None) assert_members_equal(first, second, msg=None) assert_sequence_equal(first, second, msg=None) assert_raises_with_message_regex(exc_class, message_regex, callable_obj, *args, **kwargs) assert_compare_false(op, first, second, msg=None) assert_not_in(container, member, msg=None) assert_is_not(first, second, msg=None) assert_not_less_than(first, second, msg=None) assert_not_greater_than(first, second, msg=None) assert_not_less_than_or_equal(first, second, msg=None) assert_not_greater_than_or_equal(first, second, msg=None) assert_members_not_equal(first, second, msg=None) assert_sequence_not_equal(first, second, msg=None) That comes to 26 variations of assert. There are really only a small set of basic conditions to test for. correct values incorrect values unexpected exceptions correct exceptions incorrect exceptions missing exceptions I think the unittest module could better handle testing of exceptions and distinguishing exception produced by test code from the code being tested. That was painfully clear (to me) when we where fixing all the Python 3000 tests. >> Py.test looks very interesting, especially the test discovery parts. >> I also agree we don't need special methods for every variation of >> assertedness. > > My main complaint about 'py.test' is that it's yet-another-framework. > We already have 'doctest' and 'unittest', and they play together > reasonably well. I love doctest. Because it is very different in how it works, I don't think it competes with more formal testing methods. > 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks > better for consideration, especially since it integrates well with > 'unittest'. Thanks for the link! I'll give 'nose' a try next time I write tests. >> I've been thinking that a few decorators may go a long way to making >> writing tests easy. It also reduces the level of indentation needed. > > There are a number of these already in 'nose'. Yes, I looked at the decorator and think it is very nice and simple to use. If something like "nose" was a submodule of unittest, unittest may be able to use some of it's parts. Maybe gradually the more java like unittest classes and methods could be depreciated? Cheers, Ron From guido at python.org Wed Jul 16 19:02:35 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 10:02:35 -0700 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <1afaf6160807110519w16b97efbt8a7b13b1276df135@mail.gmail.com> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> <g5j80e$g4d$1@ger.gmane.org> <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com> <ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com> Message-ID: <ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com> On Tue, Jul 15, 2008 at 11:50 PM, engelbert gruber <grubert at users.sourceforge.net> wrote: > I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes > ``DeprecationWarning: callable() not supported in 3.x;`` . Good catch, Engelbert. But why would has_key() need a warning when 2to3 will fix it just fine? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 16 19:25:29 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 10:25:29 -0700 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <487E0724.6020002@trueblade.com> References: <487E0724.6020002@trueblade.com> Message-ID: <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> On Wed, Jul 16, 2008 at 7:35 AM, Eric Smith <eric+python-dev at trueblade.com> wrote: > Does anyone know why 'F' is the same as 'f'? Wouldn't it make more sense to > either drop it, or make it convert the exponent to upper case (like 'E' and > 'G')? Compatibility with %-formatting is the only reason I can think of to > keep up, but I get the sense we've given up on an automatic conversion from > %-formatting to str.format(). Plus, I can find no uses of '%F' in the > standard library. My best guess as to why 'F' is the same as 'f' is that somebody (could've been me :-) thought, like several others in this thread, that %f never prints an exponent. I agree that making it emit an 'E' when an exponent is used is the right thing to do. Do it now! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Wed Jul 16 19:30:24 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 16 Jul 2008 13:30:24 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> Message-ID: <487E3030.5010203@trueblade.com> Guido van Rossum wrote: > On Wed, Jul 16, 2008 at 7:35 AM, Eric Smith > <eric+python-dev at trueblade.com> wrote: >> Does anyone know why 'F' is the same as 'f'? Wouldn't it make more sense to >> either drop it, or make it convert the exponent to upper case (like 'E' and >> 'G')? Compatibility with %-formatting is the only reason I can think of to >> keep up, but I get the sense we've given up on an automatic conversion from >> %-formatting to str.format(). Plus, I can find no uses of '%F' in the >> standard library. > > My best guess as to why 'F' is the same as 'f' is that somebody > (could've been me :-) thought, like several others in this thread, > that %f never prints an exponent. I agree that making it emit an 'E' > when an exponent is used is the right thing to do. Do it now! > It shares code with %-formatting. Change that, too? I couldn't find any occurrences of %F in the stdlib. Not that that's the entire universe, of course. The change is slightly less elegant if I don't change %-formatting, but still doable, especially if the betas don't get cut today. From guido at python.org Wed Jul 16 19:38:24 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 10:38:24 -0700 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <487E3030.5010203@trueblade.com> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> <487E3030.5010203@trueblade.com> Message-ID: <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com> On Wed, Jul 16, 2008 at 10:30 AM, Eric Smith <eric+python-dev at trueblade.com> wrote: > Guido van Rossum wrote: >> >> On Wed, Jul 16, 2008 at 7:35 AM, Eric Smith >> <eric+python-dev at trueblade.com> wrote: >>> >>> Does anyone know why 'F' is the same as 'f'? Wouldn't it make more sense >>> to >>> either drop it, or make it convert the exponent to upper case (like 'E' >>> and >>> 'G')? Compatibility with %-formatting is the only reason I can think of >>> to >>> keep up, but I get the sense we've given up on an automatic conversion >>> from >>> %-formatting to str.format(). Plus, I can find no uses of '%F' in the >>> standard library. >> >> My best guess as to why 'F' is the same as 'f' is that somebody >> (could've been me :-) thought, like several others in this thread, >> that %f never prints an exponent. I agree that making it emit an 'E' >> when an exponent is used is the right thing to do. Do it now! >> > > It shares code with %-formatting. Change that, too? I couldn't find any > occurrences of %F in the stdlib. Not that that's the entire universe, of > course. > > The change is slightly less elegant if I don't change %-formatting, but > still doable, especially if the betas don't get cut today. Fine with me to change %F as well -- after all that's where the misunderstanding comes from. (More and more I'm beginning to think it was my mistake. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From grubert at users.sourceforge.net Wed Jul 16 20:18:59 2008 From: grubert at users.sourceforge.net (engelbert gruber) Date: Wed, 16 Jul 2008 20:18:59 +0200 Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> <g5j80e$g4d$1@ger.gmane.org> <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com> <ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com> <ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com> Message-ID: <ac12bf3a0807161118v252bca49ic62a14332d5132f7@mail.gmail.com> On Wed, Jul 16, 2008 at 7:02 PM, Guido van Rossum <guido at python.org> wrote: > On Tue, Jul 15, 2008 at 11:50 PM, engelbert gruber > <grubert at users.sourceforge.net> wrote: >> I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes >> ``DeprecationWarning: callable() not supported in 3.x;`` . > > Good catch, Engelbert. > > But why would has_key() need a warning when 2to3 will fix it just fine? for example, if has_key is the only problem in my module, it is possible to support py22 to py3 with one source which i think would be quite handy. and for this the warning is great. From tim.peters at gmail.com Wed Jul 16 20:28:04 2008 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 16 Jul 2008 14:28:04 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> Message-ID: <1f7befae0807161128k7629e2a7x28fd1b2f5bf1a139@mail.gmail.com> [Guido] > My best guess as to why 'F' is the same as 'f' is that somebody > (could've been me :-) thought, like several others in this thread, > that %f never prints an exponent. I agree that making it emit an 'E' > when an exponent is used is the right thing to do. Do it now! The C standard doesn't allow for %f (or %F) to produce an exponent. That appears to be a Python innovation. People should try their examples under their native C compiler instead (best I can tell, the idea that %f/%F can produce an exponent came only from running Python examples, never from running C examples). For example, """ #include <stdio.h> int main() { printf("%f\n", 1e300); } """ Under the Cygwin gcc, that displays (the admittedly atrocious, but that's why people shouldn't use %f inappropriately to begin with ;-)): 100000000000000005250476025520442024870446900000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000.000000 As far as C is concerned, the only difference between %f and %F is: The F conversion specifier produces INF, INFINITY, or NAN instead of inf, infinity, or nan, respectively From eric+python-dev at trueblade.com Wed Jul 16 20:30:51 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 16 Jul 2008 14:30:51 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> <487E3030.5010203@trueblade.com> <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com> Message-ID: <487E3E5B.5070700@trueblade.com> Guido van Rossum wrote: >> It shares code with %-formatting. Change that, too? I couldn't find any >> occurrences of %F in the stdlib. Not that that's the entire universe, of >> course. >> >> The change is slightly less elegant if I don't change %-formatting, but >> still doable, especially if the betas don't get cut today. > > Fine with me to change %F as well -- after all that's where the > misunderstanding comes from. (More and more I'm beginning to think it > was my mistake. :-) I added this as http://mail.python.org/pipermail/python-dev/2008-July/081242.html I should be able to get it checked in today. Eric. From tseaver at palladion.com Wed Jul 16 21:36:32 2008 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 16 Jul 2008 15:36:32 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <878ww27p13.fsf@benfinney.id.au> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> <g5iljh$cmt$1@ger.gmane.org> <87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp> <878ww27p13.fsf@benfinney.id.au> Message-ID: <487E4DC0.7030602@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ben Finney wrote: > "Stephen J. Turnbull" <stephen at xemacs.org> writes: > >> Terry Reedy writes: >> >> > For the community as a whole, all stdlib modules are suggestions >> > and examples, not commands. >> >> Well, even if "standard" is too strong a word, the DeprecationWarnings >> and eventual removal of the methods surely constitute an imposition. > > I understood Terry's statement as meaning that there is no commandment > to use the standard library 'unittest' module at all. If a user > doesn't like the behaviour of a module (such as 'unittest') in the > standard library, they can accept the burden of implementing one that > behaves the way they like. One might argue that making gratutiously backward-incompatible changes to the existing 'unittest' module should fall under that heading: stability in that package is going to be crucial for projects which have any hope of making it across the 2-to-3 gulf, and they won't all get there at once. If camelCase / duplicated names are such a pain, write a *new* module, 'unittest2', and port Python's tests to use it, thereby leaving the much larger volume of not-in-Python tests still working. One might then remove the 'unittest' module in a later release, packaging it as a standalone distibution for the projects which still need it. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIfk2/+gerLs4ltQ4RAjd7AJ4iRGnOp+Udw9Jth/VtdMJhJWL50QCePUEY O1fn7KSSIfIGr0HbIX2xl74= =s0m/ -----END PGP SIGNATURE----- From tseaver at palladion.com Wed Jul 16 21:44:23 2008 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 16 Jul 2008 15:44:23 -0400 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <487DAAA0.1040604@egenix.com> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> Message-ID: <487E4F97.7030407@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 M.-A. Lemburg wrote: > On 2008-07-16 02:20, Collin Winter wrote: >> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote: >>> Significant updates include removing all reference to the >>> (already-resolved) new-style class issue, adding footnotes and >>> references, and a Rationale summary of discussion on both sides of the >>> divide for 'assert*' versus 'fail*' names. >>> >>> >>> :PEP: XXX >>> :Title: Consolidating names in the `unittest` module >>> :Version: 0.2 >>> :Last-Modified: 2008-07-15 >>> :Author: Ben Finney <ben+python at benfinney.id.au> >>> :Status: Draft >>> :Type: Standards Track >>> :Content-Type: test/x-rst >>> :Created: 2008-07-14 >>> :Python-Version: 2.7, 3.1 > > +1 for doing this in 3.1. > > -1 for Python 2.7. > > The main reason is that there's just too much 2.x code out there > using the API names you are suggesting to change and/or remove > from the module. > > Since this is a major change in the unit test API, I'd also like > to suggest that you use a new module name. > > This is both a precaution to prevent tests failing due to not having > been upgraded and a way for old code to continue working by adding > the old unittest module on sys.path. > > Please note that the required renaming of the methods in existing > tests is not going to be as straight forward as you may think, > since you may well rename method calls into the tested application > rather than just the unit test class method calls if you're not > careful. +1. I had just groped my way to that counter-proposal myself, for exactly your reasons. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIfk+W+gerLs4ltQ4RApZlAJ47NVKXxbL/oaYyVZEUgRnnvajm+wCgyOO2 4GbVo2D1eWYcJvpx1yf8bLs= =2HV6 -----END PGP SIGNATURE----- From fuzzyman at voidspace.org.uk Wed Jul 16 21:47:00 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 20:47:00 +0100 Subject: [Python-Dev] PEP: Consolidating names in the `unittest` module In-Reply-To: <487E4F97.7030407@palladion.com> References: <87vdz8a983.fsf@benfinney.id.au> <87skub6yhh.fsf@benfinney.id.au> <43aa6ff70807151720w1b84d2f1k2b9c5d18124076b8@mail.gmail.com> <487DAAA0.1040604@egenix.com> <487E4F97.7030407@palladion.com> Message-ID: <487E5034.30008@voidspace.org.uk> Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > M.-A. Lemburg wrote: > >> On 2008-07-16 02:20, Collin Winter wrote: >> >>> On Tue, Jul 15, 2008 at 6:58 AM, Ben Finney <ben+python at benfinney.id.au> wrote: >>> >>>> Significant updates include removing all reference to the >>>> (already-resolved) new-style class issue, adding footnotes and >>>> references, and a Rationale summary of discussion on both sides of the >>>> divide for 'assert*' versus 'fail*' names. >>>> >>>> >>>> :PEP: XXX >>>> :Title: Consolidating names in the `unittest` module >>>> :Version: 0.2 >>>> :Last-Modified: 2008-07-15 >>>> :Author: Ben Finney <ben+python at benfinney.id.au> >>>> :Status: Draft >>>> :Type: Standards Track >>>> :Content-Type: test/x-rst >>>> :Created: 2008-07-14 >>>> :Python-Version: 2.7, 3.1 >>>> >> +1 for doing this in 3.1. >> >> -1 for Python 2.7. >> >> The main reason is that there's just too much 2.x code out there >> using the API names you are suggesting to change and/or remove >> from the module. >> >> Since this is a major change in the unit test API, I'd also like >> to suggest that you use a new module name. >> >> This is both a precaution to prevent tests failing due to not having >> been upgraded and a way for old code to continue working by adding >> the old unittest module on sys.path. >> >> Please note that the required renaming of the methods in existing >> tests is not going to be as straight forward as you may think, >> since you may well rename method calls into the tested application >> rather than just the unit test class method calls if you're not >> careful. >> > > Do you have production code methods called 'assertEquals' and the like? It sounds pretty unlikely to me. Michael > +1. I had just groped my way to that counter-proposal myself, for > exactly your reasons. > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFIfk+W+gerLs4ltQ4RApZlAJ47NVKXxbL/oaYyVZEUgRnnvajm+wCgyOO2 > 4GbVo2D1eWYcJvpx1yf8bLs= > =2HV6 > -----END PGP SIGNATURE----- > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From guido at python.org Wed Jul 16 21:57:47 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 12:57:47 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) Message-ID: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> Having skimmed much material about proposed changes to the venerable unitest module, I'd like to set some boundaries. PEPs that don't follow the following rules are very unlikely to be accepted. 1. The API is not going to be renamed to PEP-8 conformance. This notwithstanding the purported outcome of earlier discussions. The renaming will cause too much grief for 3rd party developers; tracking down why unittests fail is hard enough without also having to consider changes to the unittest infrastructure itself. It's not the end of the world if some standard API doesn't follow the style guide. 2. Radical changes to the API are off the table. If a radically different API is to be accepted, the road to such acceptance is not a design-by-committee PEP, but adoption of a 3rd party module with a multi-year track record. If you have radically different ideas about how to do unittesting, by all means implement them and try them out and try to get a large audience to use them. When you are successful in all that, *then* we'll talk about adoption into the standard library. 3. I like assertEqual better than failUnlessEqual (and similar for all assert* versions in favor of their fail* alias), and I don't like that there is both assertEqual and assertEquals. But rule #1 means we have to live with the aliases. At best we can discourage the undesirables by documenting them out of existence. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Wed Jul 16 22:01:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 16 Jul 2008 15:01:37 -0500 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> Message-ID: <1afaf6160807161301t43568775y46611d70cb58003d@mail.gmail.com> On Wed, Jul 16, 2008 at 2:57 PM, Guido van Rossum <guido at python.org> wrote: > Having skimmed much material about proposed changes to the venerable > unitest module, I'd like to set some boundaries. PEPs that don't > follow the following rules are very unlikely to be accepted. So basically, discussion is open for additions and improvements? Great, I love it! -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From fuzzyman at voidspace.org.uk Wed Jul 16 22:03:18 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 21:03:18 +0100 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> Message-ID: <487E5406.1000802@voidspace.org.uk> Guido van Rossum wrote: > Having skimmed much material about proposed changes to the venerable > unitest module, I'd like to set some boundaries. PEPs that don't > follow the following rules are very unlikely to be accepted. > > 1. The API is not going to be renamed to PEP-8 conformance. This > notwithstanding the purported outcome of earlier discussions. The > renaming will cause too much grief for 3rd party developers; tracking > down why unittests fail is hard enough without also having to consider > changes to the unittest infrastructure itself. It's not the end of the > world if some standard API doesn't follow the style guide. > I'm glad to see a pronouncement. > 2. Radical changes to the API are off the table. If a radically > different API is to be accepted, the road to such acceptance is not a > design-by-committee PEP, but adoption of a 3rd party module with a > multi-year track record. If you have radically different ideas about > how to do unittesting, by all means implement them and try them out > and try to get a large audience to use them. When you are successful > in all that, *then* we'll talk about adoption into the standard > library. > I assume this doesn't rule out the addition of [some of..] the new convenience test methods? > 3. I like assertEqual better than failUnlessEqual (and similar for all > assert* versions in favor of their fail* alias), and I don't like that > there is both assertEqual and assertEquals. But rule #1 means we have > to live with the aliases. At best we can discourage the undesirables > by documenting them out of existence. > > Presumably new methods should *not* follow PEP8 but be internally consistent with the existing API? Does this mean that new methods should be added with *both* assert* and fail* names? Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From python at rcn.com Wed Jul 16 22:37:16 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 16 Jul 2008 13:37:16 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> Message-ID: <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> From: "Michael Foord" <fuzzyman at voidspace.org.uk> > I assume this doesn't rule out the addition of [some of..] the new > convenience test methods? In Kent Beck's book on Test Driven Development, he complains that most unittest implementations spawned from his original work have grown far too complicated and would be better served by sticking to a simple framework for writing and running tests. Accordlingly, if a new test method gets added, it needs to add some signficant new capability. IMO, little "convenience" methods like self.assertLessThan() do not meet the criterion. Polluting the module with more methods makes it harder to fit into your head and does little to simplify the task of writing mountains of tests. If some people want to proceed down the path of "useful additions", I challenge them to think bigger. Give me some test methods that improve my life. Don't give me thirty ways to spell something I can already do. Raymond From fuzzyman at voidspace.org.uk Wed Jul 16 22:48:54 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 21:48:54 +0100 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> Message-ID: <487E5EB6.8070306@voidspace.org.uk> Raymond Hettinger wrote: > From: "Michael Foord" <fuzzyman at voidspace.org.uk> >> I assume this doesn't rule out the addition of [some of..] the new >> convenience test methods? > > In Kent Beck's book on Test Driven Development, he complains that most > unittest implementations spawned from his original work have grown far > too complicated and would be better served by sticking to a simple > framework for writing and running tests. > > Accordlingly, if a new test method gets added, it needs to add some > signficant new capability. IMO, little "convenience" methods like > self.assertLessThan() do not meet the criterion. Polluting the module > with more methods makes it harder to fit into your head and does > little to simplify the task of writing mountains of tests. > > If some people want to proceed down the path of "useful additions", > I challenge them to think bigger. Give me some test methods that > improve my life. Don't give me thirty ways to spell something I can > already do. > I assert that... the following changes do meet those conditions: assertRaisesWithMessage - for testing the error messages from library functions, where the error message is part of the API under test (I'm less convinced with the need for a regex matching version myself.) Changes to assertEquals so that the failure messages are more useful (telling you which member failed the comparison for collections and showing a diff for long strings). This improves rather than adds. assertIn / assertNotIn I use very regularly for collection membership tests (although personally I find member, container to be a more memorable order for the arguments - assert this is in that. The comparison with assertRaises in the PEP is wrong - those parameters are ordered so that the arbitrary number of arguments immediately follow the callable they belong to.) The run_tests function for running collections of tests. Almost every project I've worked on has had an ad-hoc imnplementation of this, collecting test modules and turning them into a suitable collection for use with unittest. These are the most important changes in my opinion. assertIs / assertIsNot also sounds good, but is not something I would miss if they weren't added. Michael > > Raymond -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From python at rcn.com Wed Jul 16 23:03:29 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 16 Jul 2008 14:03:29 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> Message-ID: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> >> If some people want to proceed down the path of "useful additions", >> I challenge them to think bigger. Give me some test methods that >> improve my life. Don't give me thirty ways to spell something I can >> already do. From: "Michael Foord" <fuzzyman at voidspace.org.uk> > I assert that... the following changes do meet those conditions: > > assertRaisesWithMessage . . . > Changes to assertEquals so that the failure messages are more useful ... > assertIn / assertNotIn I use very regularly for collection membership - self.assert_(func(x) in result_set) + self.assertIn(func(x), result_set) Yawn. The gain is zero. Actually, it's negative because the second doesn't read as nicely as the pure python expression. Think bigger! No fat APIs. Do something cool! Checkout the dynamic test creation in test_decimal to see if it can be generalized. Give me some cool test runners. Maybe find a way to automatically launch pdb or to dump the locals variables at the time of failure. Maybe move the "test_*.py" search into the unittest module. We want *small* and powerful. The api for TestCase instances is already way too fat. See an old discussion on the subject at: http://bugs.python.org/issue2578 > The run_tests function for running collections of tests. Almost every > project I've worked on has had an ad-hoc imnplementation of this, > collecting test modules and turning them into a suitable collection for > use with unittest. Now, that's more like it. Propose more cool stuff like this and the module really will be improved. > assertIs / assertIsNot also sounds good, but is not something I would > miss if they weren't added. Doh! We're back to replacing clean expressions using pure python syntax with a method name equivalent. That's a step backwards. Raymond From guido at python.org Wed Jul 16 23:12:58 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 14:12:58 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <487E5406.1000802@voidspace.org.uk> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> Message-ID: <ca471dc20807161412s57bd0b34k412539d2122c87c1@mail.gmail.com> On Wed, Jul 16, 2008 at 1:03 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote: > Guido van Rossum wrote: >> 2. Radical changes to the API are off the table. If a radically >> different API is to be accepted, the road to such acceptance is not a >> design-by-committee PEP, but adoption of a 3rd party module with a >> multi-year track record. If you have radically different ideas about >> how to do unittesting, by all means implement them and try them out >> and try to get a large audience to use them. When you are successful >> in all that, *then* we'll talk about adoption into the standard >> library. > I assume this doesn't rule out the addition of [some of..] the new > convenience test methods? Correct. >> 3. I like assertEqual better than failUnlessEqual (and similar for all >> assert* versions in favor of their fail* alias), and I don't like that >> there is both assertEqual and assertEquals. But rule #1 means we have >> to live with the aliases. At best we can discourage the undesirables >> by documenting them out of existence. > Presumably new methods should *not* follow PEP8 but be internally consistent > with the existing API? Right again. > Does this mean that new methods should be added with *both* assert* and > fail* names? No, I don't see any reason to perpetuate that particular design snafu. I said "live with the aliases", not "add more of them". -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ctb at msu.edu Wed Jul 16 23:15:29 2008 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 16 Jul 2008 14:15:29 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> Message-ID: <20080716211529.GA30109@caltech.edu> On Wed, Jul 16, 2008 at 02:03:29PM -0700, Raymond Hettinger wrote: -> - self.assert_(func(x) in result_set) -> + self.assertIn(func(x), result_set) -> -> Yawn. The gain is zero. Actually, it's negative because the second -> doesn't read as nicely as the pure python expression. People are proposing these craptastic methods because 'assert' doesn't do good reporting by default. Maybe we can fix that instead?? -> >The run_tests function for running collections of tests. Almost every -> >project I've worked on has had an ad-hoc imnplementation of this, -> >collecting test modules and turning them into a suitable collection for -> >use with unittest. -> -> Now, that's more like it. Propose more cool stuff like this and -> the module really will be improved. At this point I might suggest taking a look at the nose and py.test discovery rules and writing a simple test discovery system to find & wrap 'test_' functions/classes and doctests in a unittest wrapper. Many people use nose and py.test (which use remarkably similar test discovery procedures, note) and the basic algorithm is pretty well worked out. And, since nose wraps such tests in unittests anyway, it can be made entirely compatible with pre-existing TestRunner derivatives. This steers a middle ground between trying to co-opt py.test and nose entirely (which no one would be happy with) and leaving the existing (and hideous, IMO) unittest as the only "standard" option. It also helps out people who want to take advantage of some of the niftier py.test/nosetests features during development but *also* want clean test code that uses a standard library module. Let me know if I should expand on this proposal... Paranthetically, wrt unittest, the world seems to be divided into two kinds of people : those who find the current API uninspiring but ok, and those who absolutely hate it. Has anyone said that they *love* the current unittest API with all of its boilerplate? If not, then I think that says something, no? --titus -- C. Titus Brown, ctb at msu.edu From fuzzyman at voidspace.org.uk Wed Jul 16 23:18:20 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 22:18:20 +0100 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <20080716211529.GA30109@caltech.edu> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> Message-ID: <487E659C.2000505@voidspace.org.uk> C. Titus Brown wrote: > [snip..] > Paranthetically, wrt unittest, the world seems to be divided into two > kinds of people : those who find the current API uninspiring but ok, and > those who absolutely hate it. Has anyone said that they *love* the > current unittest API with all of its boilerplate? If not, then I think > that says something, no? > > --titus > Collecting testcases from the filesystem is a pain. But actually writing tests (including custom TestCases) using the unittest API is fine. I find unittest straightforward and readable, I like it. I don't understand a lot of the criticism comes in for. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From ctb at msu.edu Wed Jul 16 23:20:07 2008 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 16 Jul 2008 14:20:07 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <20080716211529.GA30109@caltech.edu> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> Message-ID: <20080716212007.GB30109@caltech.edu> On Wed, Jul 16, 2008 at 02:15:29PM -0700, C. Titus Brown wrote: -> At this point I might suggest taking a look at the nose and py.test -> discovery rules and writing a simple test discovery system to find & -> wrap 'test_' functions/classes and doctests in a unittest wrapper. -> -> Many people use nose and py.test (which use remarkably similar test -> discovery procedures, note) and the basic algorithm is pretty well -> worked out. And, since nose wraps such tests in unittests anyway, it -> can be made entirely compatible with pre-existing TestRunner -> derivatives. Sorry for the second message, but... let's compare: test_sort.py: #! /usr/bin/env python import unittest class Test(unittest.TestCase): def test_me(self): seq = [ 5, 4, 1, 3, 2 ] seq.sort() self.assertEqual(seq, [1, 2, 3, 4, 5]) if __name__ == '__main__': unittest.main() with test_sort2.py : def test_me(): seq = [ 5, 4, 1, 3 2 ] seq.sort() assert seq == [1, 2, 3, 4, 5] The *only value* that unittest adds here is in the 'assertEqual' statement, which (I think) returns a richer error message than 'assert'. If I could run the second program by doing unittest.discover_tests('test_sort2.py') I would be a very happy man... right now it requires installing nose or py.test. cheers, --titus -- C. Titus Brown, ctb at msu.edu From guido at python.org Wed Jul 16 23:24:37 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 14:24:37 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> Message-ID: <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com> On Wed, Jul 16, 2008 at 2:03 PM, Raymond Hettinger <python at rcn.com> wrote: > From: "Michael Foord" <fuzzyman at voidspace.org.uk> >> assertIn / assertNotIn I use very regularly for collection membership > > - self.assert_(func(x) in result_set) > + self.assertIn(func(x), result_set) > > Yawn. The gain is zero. Actually, it's negative because the second > doesn't read as nicely as the pure python expression. I disagree. The reason why we have assertEquals(x, y) is that the error message can show the values of x and y, whereas assert x == y can't show those. Showing the values can be tremendously useful to debugging the failure. (Doing an intelligent comparison, e.g. a string or list "diff" instead of showing the two values, can be even more useful, and I'd be in favor of that rather than adding new methods like assertListsEqual.) (Titus asks if the assert statement could be adjusted to do better reporting. But that's not going to happen -- it would require a tremendous amount of compiler support that would have to be implemented in every Python implementation (last I counted there were at least five). In addition, python -O removes all asserts from your code -- that's why we use assertXxx functions in the first place.) > Think bigger! No fat APIs. Do something cool! Checkout the > dynamic test creation in test_decimal to see if it can be generalized. > Give me some cool test runners. Maybe find a way to automatically > launch pdb or to dump the locals variables at the time of failure. > Maybe move the "test_*.py" search into the unittest module. Those ideas are cool too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From stephen at xemacs.org Wed Jul 16 23:45:36 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 17 Jul 2008 06:45:36 +0900 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <877ibm2f3b.fsf@benfinney.id.au> References: <87y7436yxz.fsf@benfinney.id.au> <1216144500.6020.5.camel@fsol> <873ama5xze.fsf@benfinney.id.au> <87od4yw2n2.fsf@uwakimon.sk.tsukuba.ac.jp> <877ibm2f3b.fsf@benfinney.id.au> Message-ID: <87fxq9wlj3.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > "Stephen J. Turnbull" <stephen at xemacs.org> writes: > > > The intuition that "fail" is a negative word is thus well-founded in > > standard usage. > > That's not the same thing as "fail" being a negative word in the sense > meant by "double negative". So what? This whole exercise is about human psychology. If it were just a matter of defining things and we're done, it wouldn't matter how many identifiers we use or how they're spelled, right? You should treat those perceptions with respect when writing this kind of PEP, not deny them outright. From fuzzyman at voidspace.org.uk Wed Jul 16 23:37:46 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Jul 2008 22:37:46 +0100 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <20080716212007.GB30109@caltech.edu> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> <20080716212007.GB30109@caltech.edu> Message-ID: <487E6A2A.9050107@voidspace.org.uk> C. Titus Brown wrote: > On Wed, Jul 16, 2008 at 02:15:29PM -0700, C. Titus Brown wrote: > -> At this point I might suggest taking a look at the nose and py.test > -> discovery rules and writing a simple test discovery system to find & > -> wrap 'test_' functions/classes and doctests in a unittest wrapper. > -> > -> Many people use nose and py.test (which use remarkably similar test > -> discovery procedures, note) and the basic algorithm is pretty well > -> worked out. And, since nose wraps such tests in unittests anyway, it > -> can be made entirely compatible with pre-existing TestRunner > -> derivatives. > > Sorry for the second message, but... let's compare: > > test_sort.py: > #! /usr/bin/env python > import unittest > class Test(unittest.TestCase): > def test_me(self): > seq = [ 5, 4, 1, 3, 2 ] > seq.sort() > self.assertEqual(seq, [1, 2, 3, 4, 5]) > > if __name__ == '__main__': > unittest.main() > > with > > test_sort2.py : > > def test_me(): > seq = [ 5, 4, 1, 3 2 ] > seq.sort() > assert seq == [1, 2, 3, 4, 5] > > The *only value* that unittest adds here is in the 'assertEqual' > statement, which (I think) returns a richer error message than 'assert'. > > But if you exclude the import and class definition (two lines), then the test methods themselves are identical - except with unittest you have the advantage of the 'assertEquals' error collecting and reporting. The boilerplate at the end is useful for running the test file in isolation, but you don't include anything comparable in the second example. > If I could run the second program by doing > > unittest.discover_tests('test_sort2.py') > > I would be a very happy man... right now it requires installing nose or > py.test. > > What about if you could run all tests in a project (of the first kind) with: tests = unittest.discover_tests('path/', filter='*test.py') unittest.run_tests(tests) (or even just the first line). With 'discover_tests' recursively globbing the path provided and collecting test files as modules. Michael > cheers, > --titus > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From g.brandl at gmx.net Wed Jul 16 23:46:46 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 16 Jul 2008 23:46:46 +0200 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <20080716212007.GB30109@caltech.edu> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> <20080716212007.GB30109@caltech.edu> Message-ID: <g5lq87$pca$1@ger.gmane.org> C. Titus Brown schrieb: > Sorry for the second message, but... let's compare: > > test_sort.py: > #! /usr/bin/env python > import unittest > class Test(unittest.TestCase): > def test_me(self): > seq = [ 5, 4, 1, 3, 2 ] > seq.sort() > self.assertEqual(seq, [1, 2, 3, 4, 5]) > > if __name__ == '__main__': > unittest.main() > > with > > test_sort2.py : > > def test_me(): > seq = [ 5, 4, 1, 3 2 ] > seq.sort() > assert seq == [1, 2, 3, 4, 5] > > The *only value* that unittest adds here is in the 'assertEqual' > statement, which (I think) returns a richer error message than 'assert'. If you use py.test, it does some magic to find out your test is an equality comparison and displays both operands' repr(). Don't know about nose. Georg From g.brandl at gmx.net Wed Jul 16 23:48:52 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 16 Jul 2008 23:48:52 +0200 Subject: [Python-Dev] accepted bytearray items -- integers or numbers? Message-ID: <g5lqc5$pca$2@ger.gmane.org> Currently, most mutating bytearray methods only accept integers as items (in 3k, in 2.6 they also accept single-char strings, for a reason I can't remember). Single-index assignment accepts anything compatible with operator.index(). This should be made consistent, but in which direction? Georg From guido at python.org Thu Jul 17 00:08:31 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 15:08:31 -0700 Subject: [Python-Dev] accepted bytearray items -- integers or numbers? In-Reply-To: <g5lqc5$pca$2@ger.gmane.org> References: <g5lqc5$pca$2@ger.gmane.org> Message-ID: <ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com> On Wed, Jul 16, 2008 at 2:48 PM, Georg Brandl <g.brandl at gmx.net> wrote: > Currently, most mutating bytearray methods only accept integers > as items (in 3k, in 2.6 they also accept single-char strings, for > a reason I can't remember). > > Single-index assignment accepts anything compatible with > operator.index(). This should be made consistent, but in which > direction? I think they should all made to go through operator.index(). BTW, looking at the 3.0 code, I was initially a little confused. While the term 'item' generally refers to the value of a list or sequence, several functions in bytearrayobject.c seem to be using it for an argument that can be either an index or a slice. Notably bytes_subscript() and bytes_ass-subscript() have this confusing terminology. This is unrelated but you might want to fix this too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From santagada at gmail.com Thu Jul 17 00:09:03 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 16 Jul 2008 19:09:03 -0300 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com> Message-ID: <6C103619-2556-4A55-88A6-CABEFDC9F4F2@gmail.com> On 16/07/2008, at 18:24, Guido van Rossum wrote: >> >> Think bigger! No fat APIs. Do something cool! Checkout the >> dynamic test creation in test_decimal to see if it can be >> generalized. >> Give me some cool test runners. Maybe find a way to automatically >> launch pdb or to dump the locals variables at the time of failure. >> Maybe move the "test_*.py" search into the unittest module. > > Those ideas are cool too. all those features are already implemented in py.test and most probably also on nose. Why not just port those to unittest? is this even being considered? -- Leonardo Santagada From ctb at msu.edu Thu Jul 17 00:11:54 2008 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 16 Jul 2008 15:11:54 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <487E6A2A.9050107@voidspace.org.uk> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> <20080716212007.GB30109@caltech.edu> <487E6A2A.9050107@voidspace.org.uk> Message-ID: <20080716221154.GA7494@caltech.edu> On Wed, Jul 16, 2008 at 10:37:46PM +0100, Michael Foord wrote: -> >test_sort2.py : -> > -> > def test_me(): -> > seq = [ 5, 4, 1, 3 2 ] -> > seq.sort() -> > assert seq == [1, 2, 3, 4, 5] -> > -> >The *only value* that unittest adds here is in the 'assertEqual' -> >statement, which (I think) returns a richer error message than 'assert'. -> -> But if you exclude the import and class definition (two lines), then the -> test methods themselves are identical - except with unittest you have -> the advantage of the 'assertEquals' error collecting and reporting. -> -> The boilerplate at the end is useful for running the test file in -> isolation, but you don't include anything comparable in the second example. You omit import and class definition (two lines), as well as 'self' in every function definition. And don't forget the multiple levels of indentation -- that's a fair amount of gratuitous typing & boilerplate, IMO. With nose and py.test you always have a test runner and you can run the tests with nosetests test_sort2.py which to my mind is better than having it be the default __main__ function, anyway. -> >If I could run the second program by doing -> > -> > unittest.discover_tests('test_sort2.py') -> > -> >I would be a very happy man... right now it requires installing nose or -> >py.test. -> > -> What about if you could run all tests in a project (of the first kind) with: -> -> tests = unittest.discover_tests('path/', filter='*test.py') -> unittest.run_tests(tests) -> -> (or even just the first line). I would very much like that, although I think some sensible defaults would let you omit the filter and path in obvious cases (test_*.py and cwd, basically). I think the exact test discovery behavior is solved in similar ways by the py.test and nose folks, and the GCD of these solutions would provide a good baseline for unittest. Having *one* Python-general way to name your test files and test functions/classes that is also compatible across nose, py.test, and unittest would be a real gain for Python, IMO. You could even set the default unittest __main__ to run the discover_tests function, e.g. python -m unittest [<directory> [<filter>]] or some such... cheers, --titus -- C. Titus Brown, ctb at msu.edu From collinw at gmail.com Thu Jul 17 00:16:38 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 16 Jul 2008 15:16:38 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> Message-ID: <43aa6ff70807161516k547749ddn9e92520aa9316428@mail.gmail.com> On Wed, Jul 16, 2008 at 2:03 PM, Raymond Hettinger <python at rcn.com> wrote: >>> If some people want to proceed down the path of "useful additions", >>> I challenge them to think bigger. Give me some test methods that >>> improve my life. Don't give me thirty ways to spell something I can >>> already do. > > From: "Michael Foord" <fuzzyman at voidspace.org.uk> >> >> I assert that... the following changes do meet those conditions: >> >> assertRaisesWithMessage > > . . . >> >> Changes to assertEquals so that the failure messages are more useful > > ... >> >> assertIn / assertNotIn I use very regularly for collection membership > > - self.assert_(func(x) in result_set) > + self.assertIn(func(x), result_set) > > Yawn. The gain is zero. Actually, it's negative because the second > doesn't read as nicely as the pure python expression. It's only negative if the method doesn't do anything special. For example, an assertListEqual() method can tell you *how* the lists differ, which the pure Python expression can't -- all the Python expression can say is "yes" or "no". We have methods like this at work and they're very useful. That said, I see no reason why these things have to be methods. The self. method boilerplate is cluttering line-noise in this case. I can easily imagine a module of nothing but comparison functions. Collin Winter > Think bigger! No fat APIs. Do something cool! Checkout the > dynamic test creation in test_decimal to see if it can be generalized. > Give me some cool test runners. Maybe find a way to automatically > launch pdb or to dump the locals variables at the time of failure. > Maybe move the "test_*.py" search into the unittest module. > > We want *small* and powerful. The api for TestCase instances is > already way too fat. See an old discussion on the subject at: > http://bugs.python.org/issue2578 > > >> The run_tests function for running collections of tests. Almost every >> project I've worked on has had an ad-hoc imnplementation of this, collecting >> test modules and turning them into a suitable collection for use with >> unittest. > > Now, that's more like it. Propose more cool stuff like this and > the module really will be improved. > > >> assertIs / assertIsNot also sounds good, but is not something I would miss >> if they weren't added. > > Doh! We're back to replacing clean expressions using pure python syntax > with a method name equivalent. That's a step backwards. > > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/collinw%40gmail.com > From tjreedy at udel.edu Thu Jul 17 00:46:59 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 16 Jul 2008 18:46:59 -0400 Subject: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15) In-Reply-To: <487E4DC0.7030602@palladion.com> References: <87vdz8a983.fsf@benfinney.id.au> <8763r89go5.fsf@benfinney.id.au> <873ambsdyq.fsf@uwakimon.sk.tsukuba.ac.jp> <87fxqb8mlp.fsf@benfinney.id.au> <87wsjnqs9c.fsf@uwakimon.sk.tsukuba.ac.jp> <g5iljh$cmt$1@ger.gmane.org> <87y742x29k.fsf@uwakimon.sk.tsukuba.ac.jp> <878ww27p13.fsf@benfinney.id.au> <487E4DC0.7030602@palladion.com> Message-ID: <g5ltp3$5ju$1@ger.gmane.org> Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > If camelCase / duplicated names are such a pain, write a *new* module, > 'unittest2', and port Python's tests to use it, thereby leaving the much > larger volume of not-in-Python tests still working. One might then > remove the 'unittest' module in a later release, packaging it as a > standalone distibution for the projects which still need it. There was, at one time at least, some degree of consensus for dropping the Fail names and keeping the Assert names, which appear to comprise 75%-80% of usage 'in the wild'. There seems to less consensus on also changing the Assert names from CamelCase to under_score versions. So I was also thinking about a 'unittest2', recommended for new projects and gradual changeover of the Python test suite. 'Unittest' could be left unchanged but gradually disrecommended, deprecated, and removed (perhaps in 4.0 if not before). From ben+python at benfinney.id.au Thu Jul 17 00:49:13 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 08:49:13 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> Message-ID: <874p6p1m3a.fsf@benfinney.id.au> "Guido van Rossum" <guido at python.org> writes: > Having skimmed much material about proposed changes to the venerable > unitest module, I'd like to set some boundaries. PEPs that don't > follow the following rules are very unlikely to be accepted. Thanks for giving the attention to this topic and producing a pronouncement. > 1. The API is not going to be renamed to PEP-8 conformance. [...] > 3. I like assertEqual better than failUnlessEqual (and similar for > all assert* versions in favor of their fail* alias), and I don't > like that there is both assertEqual and assertEquals. But rule #1 > means we have to live with the aliases. At best we can discourage > the undesirables by documenting them out of existence. These two together kill any interest I have in being PEP champion for unittest changes, or on putting effort into the changes. Thanks, everyone, for the lively discussion. -- \ ?The way to build large Python applications is to componentize | `\ and loosely-couple the hell out of everything.? ?Aahz | _o__) | Ben Finney From ben+python at benfinney.id.au Thu Jul 17 00:52:41 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 08:52:41 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> <487E659C.2000505@voidspace.org.uk> Message-ID: <87zlohzbk6.fsf@benfinney.id.au> Michael Foord <fuzzyman at voidspace.org.uk> writes: > Collecting testcases from the filesystem is a pain. But actually > writing tests (including custom TestCases) using the unittest API is > fine. I find unittest straightforward and readable, I like it. > > I don't understand a lot of the criticism comes in for. For my part, I wanted the redundancies removed and the PEP 8 conformance fixed as a precondition too *any* addition to the unittest API. Without those two (and the BDFL's pronouncement at the head of this thread means they're not an option), I can't see the unittest API getting anything except increasingly hideous and painful to use. -- \ ?Never do anything against conscience even if the state demands | `\ it.? ?Albert Einstein | _o__) | Ben Finney From ben+python at benfinney.id.au Thu Jul 17 01:01:08 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 09:01:08 +1000 Subject: [Python-Dev] PEP: Consolidating names and in the `unittest` module (version 0.4) - REJECTED References: <87vdz8a983.fsf@benfinney.id.au> Message-ID: <87vdz5zb63.fsf@benfinney.id.au> Significant changes: now targets only Python 3.1, and recording the new status (and rationale) as rejected by BDFL pronouncement. Feel free to mine for ideas. :PEP: XXX :Title: Consolidating names in the `unittest` module :Version: 0.4 :Last-Modified: 2008-07-17 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Rejected :Type: Standards Track :Content-Type: test/x-rst :Created: 2008-07-14 :Python-Version: 3.1 :Post-History: .. contents:: Abstract ======== This PEP proposes to consolidate the names that constitute the API of the standard library `unittest` module, with the goal of removing redundant names, and conforming with PEP 8. Motivation ========== The normal use case for the `unittest` module is to subclass its classes, overriding and re-using its functions and methods. This draws constant attention to the fact that the existing implementation fails several current Python standards: * It does not conform to `PEP 8`_, requiring users to write their own non-PEP-8-conformant names when overriding methods, and encouraging extensions to further depart from PEP 8. * It has many synonyms in its API, which goes against `PEP 20`_. Specification ============= Remove obsolete names --------------------- The following module attributes are not documented as part of the API and are marked as obsolete in the implementation. They will be removed. * ``_makeLoader`` * ``getTestCaseNames`` * ``makeSuite`` * ``findTestCases`` Remove redundant names ---------------------- The following attribute names exist only as synonyms for other names. They are to be removed, leaving only one name for each attribute in the API. ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ ``assert_`` Use ``assert_true``, or an ``assert`` statement. ``assertEquals`` Use ``assert_equal``. ``assertNotEquals`` Use ``assert_not_equal``. ``assertAlmostEquals`` Use ``assert_almost_equal``. ``assertNotAlmostEquals`` Use ``assert_not_almost_equal``. ``failIf`` Use ``assert_false``. ``failUnless`` Use ``assert_true``. ``failIfAlmostEqual`` Use ``assert_not_almost_equal``. ``failIfEqual`` Use ``assert_not_equal``. ``failUnlessAlmostEqual`` Use ``assert_almost_equal``. ``failUnlessEqual`` Use ``assert_equal``. ``failUnlessRaises`` Use ``assert_raises``. Conform API with PEP 8 ---------------------- The following names are to be introduced, each replacing an existing name, to make all names in the module conform with `PEP 8`_. Each name is shown with the existing name that it replaces. Where function parameters are to be renamed also, they are shown. Where function parameters are not to be renamed, they are elided with the ellipse ("?") symbol. Module attributes ~~~~~~~~~~~~~~~~~ ``default_test_loader`` Replaces ``defaultTestLoader`` ``TestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``add_error(?)`` Replaces ``addError(?)`` ``add_result(?)`` Replaces ``addResult(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``should_stop`` Replaces ``shouldStop`` ``start_test(?)`` Replaces ``startTest(?)`` ``stop_test(?)`` Replaces ``stopTest(?)`` ``tests_run`` Replaces ``testsRun`` ``was_successful(?)`` Replaces ``wasSuccessful(?)`` ``TestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, method_name='run_test')`` Replaces ``__init__(self, methodName='runTest')`` ``_test_method_doc`` Replaces ``_testMethodDoc`` ``_test_method_name`` Replaces ``_testMethodName`` ``failure_exception`` Replaces ``failureException`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``default_test_result(?)`` Replaces ``defaultTestResult(?)`` ``assert_true(?)`` Replaces ``assertTrue(?)`` ``assert_false(?)`` Replaces ``assertFalse(?)`` ``assert_almost_equal(?)`` Replaces ``assertAlmostEqual(?)`` ``assert_equal(?)`` Replaces ``assertEqual(?)`` ``assert_not_almost_equal(?)`` Replaces ``assertNotAlmostEqual(?)`` ``assert_not_equal(?)`` Replaces ``assertNotEqual(?)`` ``assert_raises(exc_class, callable_obj, *args, **kwargs)`` Replaces ``assertRaises(excClass, callableObj, *args, **kwargs)`` ``run_test(?)`` Replaces ``runTest(?)`` ``setup(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``teardown(?)`` Replaces ``tearDown(?)`` ``FunctionTestCase`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, test_func, set_up, tear_down, description)`` Replaces ``__init__(self, testFunc, setUp, tearDown, description)`` ``run_test(?)`` Replaces ``runTest(?)`` ``set_up(?)`` Replaces ``setUp(?)`` ``short_description(?)`` Replaces ``shortDescription(?)`` ``tear_down(?)`` Replaces ``tearDown(?)`` ``TestSuite`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~ ``add_test(?)`` Replaces ``addTest(?)`` ``add_tests(?)`` Replaces ``addTests(?)`` ``count_test_cases(?)`` Replaces ``countTestCases(?)`` ``TestLoader`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ ``sort_test_methods_using`` Replaces ``sortTestMethodsUsing`` ``suite_class`` Replaces ``suiteClass`` ``test_method_prefix`` Replaces ``testMethodPrefix`` ``get_test_case_names(self, test_case_class)`` Replaces ``getTestCaseNames(self, testCaseClass)`` ``load_tests_from_module(?)`` Replaces ``loadTestsFromModule(?)`` ``load_tests_from_name(?)`` Replaces ``loadTestsFromName(?)`` ``load_tests_from_names(?)`` Replaces ``loadTestsFromNames(?)`` ``load_tests_from_test_case(self, test_case_class)`` Replaces ``loadTestsFromTestCase(self, testCaseClass)`` ``_TextTestResult`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``show_all`` Replaces ``showAll`` ``add_error(?)`` Replaces ``addError(?)`` ``add_failure(?)`` Replaces ``addFailure(?)`` ``add_success(?)`` Replaces ``addSuccess(?)`` ``get_description(?)`` Replaces ``getDescription(?)`` ``print_error_list(?)`` Replaces ``printErrorList(?)`` ``print_errors(?)`` Replaces ``printErrors(?)`` ``start_test(?)`` Replaces ``startTest(?)`` ``TextTestRunner`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``_make_result(?)`` Replaces ``_makeResult(?)`` ``TestProgram`` attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~ ``__init__(self, module, default_test, argv, test_runner, test_loader)`` Replaces ``__init__(self, module, defaultTest, argv, testRunner, testLoader)`` ``create_tests(?)`` Replaces ``createTests(?)`` ``parse_args(?)`` Replaces ``parseArgs(?)`` ``run_tests(?)`` Replaces ``runTests(?)`` ``usage_exit(?)`` Replaces ``usageExit(?)`` Rationale ========= BDFL pronouncement ------------------ The BDFL has pronounced [#vanrossum-2]_ that no API renaming is to occur in the `unittest` module. This PEP is therefore rejected. Introduction after migration to Python 3.0 ------------------------------------------ The proposal to introduce these changes along with the migration to Python 3.0 is specifically not approved by the BDFL [#vanrossum-1]_, has many detractors, and has little definite support. This PEP therefore aims for introduction no earlier than Python 3.1. Redundant names --------------- The current API, with two or in some cases three different names referencing exactly the same function, leads to an overbroad and redundant API that violates `PEP 20`_ ("there should be one, and preferably only one, obvious way to do it"). Removal of ``fail*`` names -------------------------- While there is consensus support to `remove redundant names`_ for the ``TestCase`` test methods, the issue of which set of names should be retained is controversial (it was several times characterised as "bike-shedding" [#bikeshed]_ by the BDFL and others). With good arguments made in favour of each of "retain ``assert*``" and "retain ``fail*``", this PEP resolves in favour of stated BDFL preference, and de facto community preference. Arguments in favour of retaining only the ``assert*`` names: * BDFL preference: The BDFL has stated [#vanrossum-1]_ a preference for the ``assert*`` names. * Precedent: The Python standard library currently uses the ``assert*`` names by a roughly 8:1 majority over the ``fail*`` names. (Counting unit tests in the py3k tree at 2008-07-15 [#pitrou-1]_.) An ad-hoc sampling of other projects that use `unittest` also demonstrates strong preference for use of the ``assert*`` names [#bennetts-1]_, [#seaver-1]_. * Economic: The above precedent indicates a simple economic argument [#turnbull-1]_: the majority of tests will not need their statements re-phrased, so updating in favour of them will involve less disruption and risk of error. * Positive admonition: The ``assert*`` names state the intent of how the code under test *should* behave, while the ``fail*`` names are phrased in terms of how the code *should not* behave. Arguments in favour of retaining only the ``fail*`` names: * Explicit is better than implicit: The ``fail*`` names state *what the function will do* explicitly: fail the test. With the ``assert*`` names, the action to be taken is only implicit. * Avoid false implication: The test methods do not have any necessary connection with the built-in ``assert`` statement. Even the exception raised, while it defaults to ``AssertionException``, is explicitly customisable via the documented ``failure_exception`` attribute. Choosing the ``fail*`` names avoids the false association with either of these. It has been argued [#thomas-1]_ that newcomers to `unittest` who already know what ``assert`` does can be confused by the behaviour of the ``assert*`` names. This confusion does not exist for the ``fail*`` names, which don't conflict with existing concepts. * Avoid name collision: The above confusion with ``assert`` is exacerbated by the plain-boolean test using a name of ``assert_`` (with a trailing underscore) to avoid a name collision with the built-in ``assert`` statement. The corresponding ``fail_if`` name has no such issue. PEP 8 names ----------- Although `unittest` (and its predecessor `PyUnit`) are intended to be familiar to users of other xUnit interfaces, there is no attempt at direct API compatibility since the only code that Python's `unittest` interfaces with is other Python code. The module is in the standard library and its names should all conform with `PEP 8`_. While argument was raised [#hettinger-1]_ over the length of names resulting from a simple conversion of names to multi_word_with_underscores, this PEP chooses names that are consistent in wording with the existing names. An exception was made in the case of ``setup`` and ``teardown``. The two-word form of these names was judged [#hettinger-1]_ too cumbersome compared to the new single-word form. Backwards Compatibility ======================= The names to be obsoleted should be deprecated and removed according to the schedule for modules in `PEP 4`_. While deprecated, use of the deprecated attributes should raise a ``DeprecationWarning``, with a message stating which replacement name should be used. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. _PEP 4: http://www.python.org/dev/peps/pep-0004 .. _PEP 8: http://www.python.org/dev/peps/pep-0008 .. _PEP 20: http://www.python.org/dev/peps/pep-0020 .. [#bikeshed] http://www.catb.org/~esr/jargon/html/B/bikeshedding.html .. [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-April/078485.html .. [#vanrossum-2] http://mail.python.org/pipermail/python-dev/2008-July/081263.html .. [#pitrou-1] http://mail.python.org/pipermail/python-dev/2008-July/081090.html .. [#bennetts-1] http://mail.python.org/pipermail/python-dev/2008-July/081141.html .. [#seaver-1] http://mail.python.org/pipermail/python-dev/2008-July/081166.html .. [#thomas-1] http://mail.python.org/pipermail/python-dev/2008-July/081153.html .. [#hettinger-1] http://mail.python.org/pipermail/python-dev/2008-July/081107.html .. [#turnbull-1] http://mail.python.org/pipermail/python-dev/2008-July/081175.html .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From ben+python at benfinney.id.au Thu Jul 17 01:14:48 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 09:14:48 +1000 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module (version 0.5) References: <87fxqa62xb.fsf@benfinney.id.au> Message-ID: <87r69tzajb.fsf@benfinney.id.au> Significant changes: targeting Python 3.1, removal of separate {lt,gt,le,ge} comparison tests, implementation of enhanced-information failure message, reference to BDFL pronouncement. I won't be working on this further; someone else should feel free to champion this further if they wish. :PEP: XXX :Title: Frequently-requested additional features for the `unittest` module :Version: 0.5 :Last-Modified: 2008-07-16 :Author: Ben Finney <ben+python at benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: test/x-rst :Requires: PEP XXX (Consolidating names in the `unittest` module) :Created: 2008-07-14 :Python-Version: 3.1 :Post-History: .. contents:: Abstract ======== This PEP proposes frequently-requested additions to the standard library `unittest` module that are natural extensions of its existing functionality. Motivation ========== The `unittest` module is functionally complete. Nevertheless, many users request and devise extensions to perform functions that are so common they deserve to have a standard implementation. Specification ============= New condition tests ------------------- The following test methods will be added to the ``TestCase`` class. The function signature is part of this specification. The body is to be treated as a reference implementation only; any functionally identical implementation also meets this specification. :: import operator import re class TestCase(object): # ? def assert_compare_true(op, first, second, msg=None): fail_detail = "%(first)r %(op)r %(second)r" % vars() if msg is None: msg = fail_detail else: msg = "%(fail_detail)s: %(msg)s" % vars() if not op(first, second): raise self.failure_exception(msg) def assert_in(container, member, msg=None): op = operator.__contains__ self.assert_compare_true(op, container, member, msg) def assert_is(first, second, msg=None): op = operator.is_ self.assert_compare_true(op, first, second, msg) def assert_members_equal(first, second, msg=None): op = operator.eq self.assert_compare_true(op, set(first), set(second), msg) def assert_sequence_equal(first, second, msg=None): op = operator.eq self.assert_compare_true(op, list(first), list(second), msg) def assert_raises_with_message_regex( exc_class, message_regex, callable_obj, *args, **kwargs): exc_name = exc_class.__name__ message_pattern = re.compile(message_regex) try: callable_obj(*args, **kwargs) except exc_class, exc: exc_message = str(exc) if not message_pattern.match(exc_message): msg = ( "%(exc_name)s raised" " without message matching %(message_regex)r" " (got message %(exc_message)r)" ) % vars() raise self.failure_exception(msg) else: msg = "%(exc_name)s not raised" % vars() raise self.failure_exception(msg) The following test methods are also added. Their implementation in each case is simply the logical inverse of a corresponding method above. :: def assert_compare_false(op, first, second, msg=None): # Logical inverse of assert_compare_true def assert_not_in(container, member, msg=None): # Logical inverse of assert_in def assert_is_not(first, second, msg=None) # Logical inverse of assert_is def assert_members_not_equal(first, second, msg=None) # Logical inverse of assert_members_equal def assert_sequence_not_equal(first, second, msg=None) # Logical inverse of assert_sequence_equal Enhanced failure message for comparison tests --------------------------------------------- The comparison tests will change their behaviour such that the message always, even if overridden with a specific message when called, includes extra information: * For both ``assert_equal`` and ``assert_not_equal``: the ``repr()`` output of the objects that were compared. This matches the behaviour of ``assert_compare_true`` and ``assert_compare_false``, above. * For ``assert_equal`` comparisons of ``basestring`` instances that are multi-line text: the output of ``diff`` comparing the two texts. * For membership comparisons with ``assert_members_equal`` and ``assert_sequence_equal``: the ``repr()`` output of the members that were not equal in each collection. (This change is not done for the corresponding ``assert_*_not_equal`` tests, which only fail if the collection members are equal.) Simple invocation of test collection ------------------------------------ To allow simpler loading and running of test cases from an arbitrary collection, the following new functionality will be added to the `unittest` module. The function signatures are part of this specification; the implementation is to be considered a reference implementation only. :: class TestLoader(object): # ? def load_tests_from_collection(self, collection): """ Return a suite of all test cases found in `collection` :param collection: One of the following: * a `TestSuite` instance * a `TestCase` subclass * a module * an iterable producing any of these types :return: A `TestSuite` instance containing all the test cases found recursively within `collection`. """ suite = None if isinstance(collection, TestSuite): suite = collection elif isinstance(collection, type): if issubclass(collection, TestCase): suite = self.load_tests_from_test_case(collection) elif isinstance(collection, types.ModuleType): suite = self.load_tests_from_module(collection) elif hasattr(collection, '__iter__'): suite = self.suite_class() for item in collection: suite.add_test(self.load_tests_from_collection(item)) else: msg = "not a test collection: %(collection)r" % vars() raise TypeError(msg) return suite def run_tests( tests, loader_class=TestLoader, runner_class=TextTestRunner): """ Run a collection of tests with a test runner :param tests: A test collection as defined by the `loader_class` method `load_tests_from_collection`. :param loader_class: The type of test loader to use for collecting tests from the `tests` collection. :param runner_class: The type of test runner to instantiate for running the collected test suite. :return: None. """ loader = loader_class() suite = loader.load_tests_from_collection(tests) runner = runner_class() runner.run(suite) Rationale ========= BDFL pronouncement ------------------ The BDFL pronounced [#vanrossum-1]_ a set of boundaries within which changes to the `unittest` module need to operate. This PEP may currently violate some of those, making it currently unacceptable. Names for logical-inverse tests ------------------------------- The simple pattern established by ``assert_foo`` having a logical inverse named ``assert_not_foo`` sometimes results in gramatically awkward names. The following names were chosen in exception of this pattern, in the interest of the API names being more intuitive: * ``assert_is_not`` * ``assert_members_not_equal`` * ``assert_sequence_not_equal`` Order of method parameters -------------------------- The methods ``assert_in``, ``assert_not_in`` have the container as the first parameter. This makes the grammatical object of the function name come immediately after the function name: "Assert in `container`". This matches the convention set by the existing ``assert_raises(exception, callable_obj, ?)`` "(Assert the code raises `exception`"). Set of additional tests ----------------------- Test methods that were considered but failed to gain significant support include: * ``assert_less_than``, ``assert_greater_than``, ``assert_less_than_or_equal``, ``assert_greater_than_or_equal``, and logical inverses. These, and other less-used boolean comparison tests, can all be covered adequately with the new ``assert_compare_true`` and ``assert_compare_false`` methods with an appropriate comparison operator function. Backwards Compatibility ======================= This PEP proposes only additional features. There are no backward-incompatible changes. Reference Implementation ======================== None yet. Copyright ========= This document is hereby placed in the public domain by its author. .. [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-July/081263.html .. Local Variables: mode: rst coding: utf-8 End: vim: filetype=rst : From g.brandl at gmx.net Thu Jul 17 01:20:46 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 17 Jul 2008 01:20:46 +0200 Subject: [Python-Dev] accepted bytearray items -- integers or numbers? In-Reply-To: <ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com> References: <g5lqc5$pca$2@ger.gmane.org> <ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com> Message-ID: <g5lvof$bag$1@ger.gmane.org> Guido van Rossum schrieb: > On Wed, Jul 16, 2008 at 2:48 PM, Georg Brandl <g.brandl at gmx.net> wrote: >> Currently, most mutating bytearray methods only accept integers >> as items (in 3k, in 2.6 they also accept single-char strings, for >> a reason I can't remember). >> >> Single-index assignment accepts anything compatible with >> operator.index(). This should be made consistent, but in which >> direction? > > I think they should all made to go through operator.index(). > > BTW, looking at the 3.0 code, I was initially a little confused. While > the term 'item' generally refers to the value of a list or sequence, > several functions in bytearrayobject.c seem to be using it for an > argument that can be either an index or a slice. Notably > bytes_subscript() and bytes_ass-subscript() have this confusing > terminology. This is unrelated but you might want to fix this too. OK, fixed in trunk and 3k. One consequence is that bytearray("xyz") is no error in 2.6 -- I hope this is what was intended in the first place. Georg From g.brandl at gmx.net Thu Jul 17 01:34:06 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 17 Jul 2008 01:34:06 +0200 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module (version 0.5) In-Reply-To: <87r69tzajb.fsf@benfinney.id.au> References: <87fxqa62xb.fsf@benfinney.id.au> <87r69tzajb.fsf@benfinney.id.au> Message-ID: <g5m0he$dhd$1@ger.gmane.org> Ben Finney schrieb: > Significant changes: targeting Python 3.1, removal of separate > {lt,gt,le,ge} comparison tests, implementation of enhanced-information > failure message, reference to BDFL pronouncement. > > I won't be working on this further; someone else should feel free to > champion this further if they wish. Note that even if it is abandoned, it should be submitted to peps at python.org at some point to become an official PEP. Georg From guido at python.org Thu Jul 17 01:40:57 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 16:40:57 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <87zlohzbk6.fsf@benfinney.id.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> <487E659C.2000505@voidspace.org.uk> <87zlohzbk6.fsf@benfinney.id.au> Message-ID: <ca471dc20807161640w393db6caj45bf3caddbc5d8e4@mail.gmail.com> On Wed, Jul 16, 2008 at 3:52 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > For my part, I wanted the redundancies removed and the PEP 8 > conformance fixed as a precondition too *any* addition to the unittest > API. That seems an unproductive attitude towards backwards incompatibility. I'm glad you see the incompatibility with the way we do business here though. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jul 17 01:42:16 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 16:42:16 -0700 Subject: [Python-Dev] accepted bytearray items -- integers or numbers? In-Reply-To: <g5lvof$bag$1@ger.gmane.org> References: <g5lqc5$pca$2@ger.gmane.org> <ca471dc20807161508v7e149eaeh61e9704ff1fa2806@mail.gmail.com> <g5lvof$bag$1@ger.gmane.org> Message-ID: <ca471dc20807161642o17ba7988s21502ab589296ca5@mail.gmail.com> On Wed, Jul 16, 2008 at 4:20 PM, Georg Brandl <g.brandl at gmx.net> wrote: > Guido van Rossum schrieb: >> >> On Wed, Jul 16, 2008 at 2:48 PM, Georg Brandl <g.brandl at gmx.net> wrote: >>> >>> Currently, most mutating bytearray methods only accept integers >>> as items (in 3k, in 2.6 they also accept single-char strings, for >>> a reason I can't remember). >>> >>> Single-index assignment accepts anything compatible with >>> operator.index(). This should be made consistent, but in which >>> direction? >> >> I think they should all made to go through operator.index(). >> >> BTW, looking at the 3.0 code, I was initially a little confused. While >> the term 'item' generally refers to the value of a list or sequence, >> several functions in bytearrayobject.c seem to be using it for an >> argument that can be either an index or a slice. Notably >> bytes_subscript() and bytes_ass-subscript() have this confusing >> terminology. This is unrelated but you might want to fix this too. > > OK, fixed in trunk and 3k. One consequence is that bytearray("xyz") > is no error in 2.6 -- I hope this is what was intended in the first place. Yes, since this is how you spell bytearray(b"xyz") in 2.6. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From robert.kern at gmail.com Thu Jul 17 01:45:45 2008 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 16 Jul 2008 18:45:45 -0500 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <ca471dc20807161424n6b3f13cft91c5e201ea2a59ff@mail.gmail.com> Message-ID: <g5m179$fb3$1@ger.gmane.org> Guido van Rossum wrote: > On Wed, Jul 16, 2008 at 2:03 PM, Raymond Hettinger <python at rcn.com> wrote: >> From: "Michael Foord" <fuzzyman at voidspace.org.uk> >>> assertIn / assertNotIn I use very regularly for collection membership >> - self.assert_(func(x) in result_set) >> + self.assertIn(func(x), result_set) >> >> Yawn. The gain is zero. Actually, it's negative because the second >> doesn't read as nicely as the pure python expression. > > I disagree. The reason why we have assertEquals(x, y) is that the > error message can show the values of x and y, whereas assert x == y > can't show those. Showing the values can be tremendously useful to > debugging the failure. (Doing an intelligent comparison, e.g. a string > or list "diff" instead of showing the two values, can be even more > useful, and I'd be in favor of that rather than adding new methods > like assertListsEqual.) > > (Titus asks if the assert statement could be adjusted to do better > reporting. But that's not going to happen -- it would require a > tremendous amount of compiler support that would have to be > implemented in every Python implementation (last I counted there were > at least five). In addition, python -O removes all asserts from your > code -- that's why we use assertXxx functions in the first place.) The assert statement itself does not have to be modified. nose and py.test are already capable of picking out the variables used in a failing assert expression and print them out. For example: [~]$ cat test_nicefail.py def test_nicefail(): x = 12 assert x == 10 [~]$ nosetests --detailed-errors test_nicefail.py F ====================================================================== FAIL: test_nicefail.test_nicefail ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/nose-0.10.3-py2.5.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/Users/rkern/test_nicefail.py", line 3, in test_nicefail assert x == 10 AssertionError: 12 = 12 >> assert 12 == 10 ---------------------------------------------------------------------- Ran 1 test in 0.044s FAILED (failures=1) [~]$ py.test test_nicefail.py ====================== test process starts ======================= executable: /Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python (2.5.1-final-0) using py lib: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/py <rev unknown> test_nicefail.py[1] F __________________________________________________________________ ___________________ entrypoint: test_nicefail ____________________ def test_nicefail(): x = 12 E assert x == 10 > assert 12 == 10 [/Users/rkern/test_nicefail.py:3] __________________________________________________________________ ============ tests finished: 1 failed in 0.09 seconds ============ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From kristjan at ccpgames.com Thu Jul 17 00:46:39 2008 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 16 Jul 2008 22:46:39 +0000 Subject: [Python-Dev] defects submitted by me. Message-ID: <4E9372E6B2234D4F859320D896059A950FFDE04E4C@exchis.ccp.ad.local> Recently, I have submitted a number of defects (user krisvale) I do have checkin privileges but since I have been lurking for so long, I didn't want to actualhange anything. Is this the way to proceed? I notic e that a couple of the defects (3367, 3368, 3369) have received attention and are presumabely being worked on, but 3327, 3377 and 3378 have not. Not that for defect 3377 I have no patch, since I am unclear on where the logical error lies. Also, some of these problem Cheers, Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080716/fb5a8398/attachment.htm> From andrew-pythondev at puzzling.org Thu Jul 17 02:22:52 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Thu, 17 Jul 2008 10:22:52 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <487E5EB6.8070306@voidspace.org.uk> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> Message-ID: <20080717002252.GG26808@steerpike.home.puzzling.org> Michael Foord wrote: > Raymond Hettinger wrote: [...] >> >> If some people want to proceed down the path of "useful additions", >> I challenge them to think bigger. Give me some test methods that >> improve my life. Don't give me thirty ways to spell something I can >> already do. >> > > I assert that... the following changes do meet those conditions: > > assertRaisesWithMessage - for testing the error messages from library > functions, where the error message is part of the API under test (I'm > less convinced with the need for a regex matching version myself.) This one is easily solved by making assertRaises return the exception it caught. e.g.: exc = self.assertRaises(AttributeError, getattr, foo, 'bar') self.assertEqual("'Foo' object has no attribute 'bar'", exc.message) At least Twisted and bzr have already made this exact change. It requires no new methods, and is more flexible than the proposed assertRaisesWithMessage. -Andrew. From guido at python.org Thu Jul 17 02:28:34 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 17:28:34 -0700 Subject: [Python-Dev] defects submitted by me. In-Reply-To: <4E9372E6B2234D4F859320D896059A950FFDE04E4C@exchis.ccp.ad.local> References: <4E9372E6B2234D4F859320D896059A950FFDE04E4C@exchis.ccp.ad.local> Message-ID: <ca471dc20807161728v2b2a23e4l3ad44c09b6f27ade@mail.gmail.com> Hi Kristjan, You are doing the right thing. In general, we prefer to have changes reviewed by a senior committer first. Unfortunately, there is no guarantee that bugs will get the attention they deserve, since the senior developers are all pretty busy. Sending email to python-dev is good, although to really pique folks' interest it would probably be better to add a 1-2 line summary of each of the items for which you are hoping to get someone's attention. (And ad the Python version[s] affected.) --Guido On Wed, Jul 16, 2008 at 3:46 PM, Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote: > Recently, I have submitted a number of defects (user krisvale) > > I do have checkin privileges but since I have been lurking for so long, I > didn't want to actualhange anything. > > Is this the way to proceed? I notic e that a couple of the defects (3367, > 3368, 3369) have received attention and are presumabely being worked on, but > 3327, 3377 and 3378 have not. > > > > Not that for defect 3377 I have no patch, since I am unclear on where the > logical error lies. Also, some of these problem > > > > Cheers, > > > > Kristj?n > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ben+python at benfinney.id.au Thu Jul 17 02:43:30 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 10:43:30 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> Message-ID: <87ej5tz6fh.fsf@benfinney.id.au> Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > This one is easily solved by making assertRaises return the > exception it caught. That breaks one simple feature of the unittest API: that all the test methods will either raise a failure asertion, or return None. -- \ ?In case you haven't noticed, [the USA] are now almost as | `\ feared and hated all over the world as the Nazis were.? ?Kurt | _o__) Vonnegut, 2004 | Ben Finney From andrew-pythondev at puzzling.org Thu Jul 17 03:07:30 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Thu, 17 Jul 2008 11:07:30 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <87ej5tz6fh.fsf@benfinney.id.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> Message-ID: <20080717010730.GA29299@steerpike.home.puzzling.org> Ben Finney wrote: > Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > > > This one is easily solved by making assertRaises return the > > exception it caught. > > That breaks one simple feature of the unittest API: that all the test > methods will either raise a failure asertion, or return None. How is returning None a feature? I've never seen code that somehow depends on assertRaises returning None. This ?feature? is not documented as being significant in the unittest module documentation anywhere. It is not mentioned anywhere in the *eight* pages of the xUnit Test Patterns book[1] dedicated to Assertion Methods in their general form. Where did you get the notion that it is a feature? Further, I have lots of evidence that in practice returning the exception instance from assertRaises is not a problem, and is in fact quite useful. I'd quote ?Practicality beats purity?, but I'm not even sure if it is purity that you have in mind. Demanding that assertion methods return None seems like foolish consistency and dogma. -Andrew. [1] _xUnit Test Patterns: Refactoring Test Code_, by Gerard Meszaros. <http://xunitpatterns.com/>. From ben+python at benfinney.id.au Thu Jul 17 03:28:17 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 11:28:17 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> Message-ID: <87abghz4cu.fsf@benfinney.id.au> Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > Ben Finney wrote: > > Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > > > > > This one is easily solved by making assertRaises return the > > > exception it caught. > > > > That breaks one simple feature of the unittest API: that all the > > test methods will either raise a failure asertion, or return None. > > How is returning None a feature? A test method having exactly one meaning is a feature. If it's consistent across the API, the API retains a level of simplicity. > I've never seen code that somehow depends on assertRaises returning > None. I hope never to see code depending on methods named "assert*" returning something, instead of *only* asserting a condition. > Further, I have lots of evidence that in practice returning the > exception instance from assertRaises is not a problem, and is in > fact quite useful. I'm sure it's useful. I don't think it belongs in the return value of such a method. > I'd quote ?Practicality beats purity?, but I'm not even sure if it > is purity that you have in mind. Close: I'm interested in keeping camel's noses out of tents. -- \ ?Laugh and the world laughs with you; snore and you sleep | `\ alone.? ?anonymous | _o__) | Ben Finney From andrew-pythondev at puzzling.org Thu Jul 17 03:45:56 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Thu, 17 Jul 2008 11:45:56 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <87abghz4cu.fsf@benfinney.id.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> Message-ID: <20080717014556.GB29299@steerpike.home.puzzling.org> Ben Finney wrote: > Andrew Bennetts <andrew-pythondev at puzzling.org> writes: [...] > > How is returning None a feature? > > A test method having exactly one meaning is a feature. If it's > consistent across the API, the API retains a level of simplicity. Your reply makes no sense to me. I am proposing that it should have exactly one meaning. Callers will be free to ignore the return value if they don't need it, and will see zero difference in behaviour. It seems you have a different meaning of ?meaning? than I do, which suggests that this conversation is doomed. I hope I don't sound hostile, I'm just bewildered. Your argument isn't making any sense to me. I can of course keep using TestCase subclasses that override assertRaises to be more useful. And I will, because there are no actual downsides that I've seen any evidence of. I'd be even happier if the standard library adopted this improvment, though. -Andrew. From fdrake at acm.org Thu Jul 17 03:56:46 2008 From: fdrake at acm.org (Fred Drake) Date: Wed, 16 Jul 2008 21:56:46 -0400 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <20080717014556.GB29299@steerpike.home.puzzling.org> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> Message-ID: <A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org> On Jul 16, 2008, at 9:45 PM, Andrew Bennetts wrote: > I am proposing that it should have exactly one meaning. Callers > will be free to > ignore the return value if they don't need it, and will see zero > difference in > behaviour. Sounds like adding a new method, catchException(...), that returns the exception it catches, would be a reasonable compromise. I can't think of any reason that the method that catches-and-returns needs to be the existing API, which does something different. OTOH, I really don't have a problem with doing a try/except/else myself if that's what I need. -Fred -- Fred Drake <fdrake at acm.org> From python at rcn.com Thu Jul 17 04:09:24 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 16 Jul 2008 19:09:24 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com><487E5406.1000802@voidspace.org.uk><B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1><487E5EB6.8070306@voidspace.org.uk><20080717002252.GG26808@steerpike.home.puzzling.org><87ej5tz6fh.fsf@benfinney.id.au><20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> Message-ID: <FF963E56D28441879E89763BF5F07546@RaymondLaptop1> >> I'd quote ?Practicality beats purity?, but I'm not even sure if it >> is purity that you have in mind. From: "Ben Finney" <ben+python at benfinney.id.au> > Close: I'm interested in keeping camel's noses out of tents. I have no idea what you mean or are trying to accomplish (unless the camel's nose refers to camelcase). You could always just fork the unittest module and see if the masses flock to your version. That way at least people will have a choice about whether to accept these new design proclivities. Raymond From ben+python at benfinney.id.au Thu Jul 17 04:18:26 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 12:18:26 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> Message-ID: <871w1tz219.fsf@benfinney.id.au> Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > Ben Finney wrote: > > Andrew Bennetts <andrew-pythondev at puzzling.org> writes: > [...] > > > How is returning None a feature? > > > > A test method having exactly one meaning is a feature. If it's > > consistent across the API, the API retains a level of simplicity. > > Your reply makes no sense to me. > > I am proposing that it should have exactly one meaning. You're proposing to give "assertRaises" a *new* meaning, without changing its name to "assertRaisesAndReturnExceptionIfRaises". The function name should say *all* that the function does, from the perspective of the caller. If you're proposing to have the function do extra things that aren't part of what the name says it will do, then that deserves either a rename or a new function. > It seems you have a different meaning of ?meaning? than I do, > which suggests that this conversation is doomed. I hope I don't > sound hostile, I'm just bewildered. Your argument isn't making any > sense to me. I hope that clarifies it. The name of a thing, in Python especially, is very important; in an API, even more so. If the behaviour of the function isn't matched by the name, it's a poorly chosen name, a poorly designed function, or both. > I can of course keep using TestCase subclasses that override > assertRaises to be more useful. Why would you override that function to do something not described by the name, instead of simply adding a new method that performs this extra task you want performed? -- \ ?Pinky, are you pondering what I'm pondering?? ?I think so, | `\ Brain, but isn't a cucumber that small called a gherkin?? | _o__) ?_Pinky and The Brain_ | Ben Finney From steve at holdenweb.com Thu Jul 17 04:22:06 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 16 Jul 2008 22:22:06 -0400 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <87zlohzbk6.fsf@benfinney.id.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> <487E659C.2000505@voidspace.org.uk> <87zlohzbk6.fsf@benfinney.id.au> Message-ID: <g5macs$11o$1@ger.gmane.org> Ben Finney wrote: > Michael Foord <fuzzyman at voidspace.org.uk> writes: > >> Collecting testcases from the filesystem is a pain. But actually >> writing tests (including custom TestCases) using the unittest API is >> fine. I find unittest straightforward and readable, I like it. >> >> I don't understand a lot of the criticism comes in for. > > For my part, I wanted the redundancies removed and the PEP 8 > conformance fixed as a precondition too *any* addition to the unittest > API. > > Without those two (and the BDFL's pronouncement at the head of this > thread means they're not an option), I can't see the unittest API > getting anything except increasingly hideous and painful to use. > Yes, but unless I misunderstand you, you don't regard a mass renaming of the module's functionality and removal of existing aliases as a change to the API. As far as I'm concerned, if I have to alter my code to use the updated module you have changed the API. Test code is particularly sensitive to such changes. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From andrew-pythondev at puzzling.org Thu Jul 17 04:23:49 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Thu, 17 Jul 2008 12:23:49 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <871w1tz219.fsf@benfinney.id.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> Message-ID: <20080717022349.GB28341@steerpike.home.puzzling.org> Ben Finney wrote: [...] > > I hope that clarifies it. The name of a thing, in Python especially, > is very important; in an API, even more so. If the behaviour of the > function isn't matched by the name, it's a poorly chosen name, a > poorly designed function, or both. It doesn't really clarify it for me. We're both just repeating ourselves, though. -Andrew. From ben+python at benfinney.id.au Thu Jul 17 04:36:13 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 12:36:13 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <308761DACA7843C7B5132CF61F34C334@RaymondLaptop1> <20080716211529.GA30109@caltech.edu> <487E659C.2000505@voidspace.org.uk> <87zlohzbk6.fsf@benfinney.id.au> <g5macs$11o$1@ger.gmane.org> Message-ID: <87wsjlxmn6.fsf@benfinney.id.au> Steve Holden <steve at holdenweb.com> writes: > Yes, but unless I misunderstand you, you don't regard a mass > renaming of the module's functionality and removal of existing > aliases as a change to the API. You slightly misunderstand me. The above changes *are* a change to the API, by definition. My position is that those changes, which *only* (and deliberately) change names without changing behaviour, are far below the threshold that might justify a new module. > As far as I'm concerned, if I have to alter my code to use the > updated module you have changed the API. Test code is particularly > sensitive to such changes. I agree entirely with this. -- \ ?Smoking cures weight problems. Eventually.? ?Steven Wright | `\ | _o__) | Ben Finney From ben+python at benfinney.id.au Thu Jul 17 04:42:28 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 12:42:28 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> Message-ID: <87sku9xmcr.fsf@benfinney.id.au> Ben Finney <ben+python at benfinney.id.au> writes: > You're proposing to give "assertRaises" a *new* meaning, without > changing its name to "assertRaisesAndReturnExceptionIfRaises". This might be misunderstood, so I'll make it clearer. The name "assert raises" has a strong (and, per Guido, deliberate) association with the existing 'assert' statement. So the name makes a strong promise of what the function will do: raise an exception if the condition is false, and *have no effect* otherwise. If the behaviour desired is, instead, to return a value, then that is an important part of what the function will do and needs to be reflected in the name. "catch exception" was one suggestion for a name, another might be "get exception raised". If the idea is to have a *single* function that does both disparate things, then the name should definitely state that. That it would result in an overly long name is a strong indicator that the function is doing too much and should be split. The result I'm trying to avoid by this is that of having the externally visible behaviour of functions drift from the promise made by their names. Either rename the function, or create a new one for the new functionality. -- \ ?Consider the daffodil. And while you're doing that, I'll be | `\ over here, looking through your stuff.? ?Jack Handey | _o__) | Ben Finney From eric+python-dev at trueblade.com Thu Jul 17 04:52:54 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 16 Jul 2008 22:52:54 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> <487E109C.7050806@trueblade.com> <5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com> Message-ID: <487EB406.1000904@trueblade.com> Mark Dickinson wrote: > On Wed, Jul 16, 2008 at 4:15 PM, Eric Smith > <eric+python-dev at trueblade.com> wrote: >> There's no exponent until the number gets large. I haven't looked up how >> big the number has to get. On my Mac, it's somewhere between 1e50 and 1e60. > > I think it's around 1e50, courtesy of the rather oddly-phrased line in > unicodeobject.c > (this is in py3k) that looks like: > > if (type == 'f' && (fabs(x) / 1e25) >= 1e25) I don't know why that test exists. It comes from Guido in r3417 (1993-03-16!). It's from the very first incarnation of formatfloat(). I'd like to remove it, but there's no telling what it would break. Surely something written in the last 15+ years depends on it. But if it were removed, then (as Tim points out) only INF and NAN would be in uppercase for 'F'. regrtest.py still works with it removed, except for string_tests.py, which is explicitly looking for this behavior, and has this comment: # The formatfloat() code in stringobject.c and # unicodeobject.c uses a 120 byte buffer and switches from # 'f' formatting to 'g' at precision 50, so we expect # OverflowErrors for the ranges x < 50 and prec >= 67. The fixed size buffer could be dealt with, but it doesn't seem worth it given the potential breakage. For now, I'll just make 'F' convert to uppercase, and leave the 1e50 issue for another release. Eric. From guido at python.org Thu Jul 17 05:20:04 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 20:20:04 -0700 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <487EB406.1000904@trueblade.com> References: <487E0724.6020002@trueblade.com> <5c6f2a5d0807160807j6bb2ed34p19a6c5a93e5e0b9c@mail.gmail.com> <487E109C.7050806@trueblade.com> <5c6f2a5d0807160851m304781f5w7f5bc37c9ab054e7@mail.gmail.com> <487EB406.1000904@trueblade.com> Message-ID: <ca471dc20807162020p7b3c4bces2a579d804530f59f@mail.gmail.com> On Wed, Jul 16, 2008 at 7:52 PM, Eric Smith <eric+python-dev at trueblade.com> wrote: > Mark Dickinson wrote: >> >> On Wed, Jul 16, 2008 at 4:15 PM, Eric Smith >> <eric+python-dev at trueblade.com> wrote: >>> >>> There's no exponent until the number gets large. I haven't looked up how >>> big the number has to get. On my Mac, it's somewhere between 1e50 and >>> 1e60. >> >> I think it's around 1e50, courtesy of the rather oddly-phrased line in >> unicodeobject.c >> (this is in py3k) that looks like: >> >> if (type == 'f' && (fabs(x) / 1e25) >= 1e25) > > I don't know why that test exists. It comes from Guido in r3417 > (1993-03-16!). It's from the very first incarnation of formatfloat(). > > I'd like to remove it, but there's no telling what it would break. Surely > something written in the last 15+ years depends on it. But if it were > removed, then (as Tim points out) only INF and NAN would be in uppercase for > 'F'. > > regrtest.py still works with it removed, except for string_tests.py, which > is explicitly looking for this behavior, and has this comment: > # The formatfloat() code in stringobject.c and > # unicodeobject.c uses a 120 byte buffer and switches from > # 'f' formatting to 'g' at precision 50, so we expect > # OverflowErrors for the ranges x < 50 and prec >= 67. > > The fixed size buffer could be dealt with, but it doesn't seem worth it > given the potential breakage. > > For now, I'll just make 'F' convert to uppercase, and leave the 1e50 issue > for another release. It was an attempt to prevent ridiculously long output, with lots of nonsensical digits suggesting precision that isn't there. Since nobody in their right mind should use %f for such large numbers anyway, removing it shouldn't hurt anything, but it might startle users who try things at the interactive prompt, or who simply have a bug in their code causing oversized numbers to be produced. So my recommendation would be to leave it in. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jul 17 05:23:03 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Jul 2008 20:23:03 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <87sku9xmcr.fsf@benfinney.id.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <87sku9xmcr.fsf@benfinney.id.au> Message-ID: <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com> On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > The result I'm trying to avoid by this is that of having the > externally visible behaviour of functions drift from the promise made > by their names. Either rename the function, or create a new one for > the new functionality. Oh get over it. This is getting ridiculous. Where did you *get* these design "principles" you are so fond of? Read the zen of Python and come back after you've memorized it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ben+python at benfinney.id.au Thu Jul 17 05:30:26 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 13:30:26 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <87sku9xmcr.fsf@benfinney.id.au> <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com> Message-ID: <87y741yyp9.fsf@benfinney.id.au> "Guido van Rossum" <guido at python.org> writes: > On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote: > > The result I'm trying to avoid by this is that of having the > > externally visible behaviour of functions drift from the promise > > made by their names. Either rename the function, or create a new > > one for the new functionality. > > Oh get over it. This is getting ridiculous. Indeed. > Where did you *get* these design "principles" you are so fond of? The Zen of Python. > Read the zen of Python and come back after you've memorized it. The wonderful thing about the Zen of Python is that I can find support in it for *all* the contradictory views expresed in these threads. The Zen needs interpretation by mortals with human judgement in order to apply its principles. -- \ "Those who will not reason, are bigots, those who cannot, are | `\ fools, and those who dare not, are slaves." ??Lord? George | _o__) Gordon Noel Byron | Ben Finney From steve at pearwood.info Thu Jul 17 06:09:29 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 17 Jul 2008 14:09:29 +1000 Subject: [Python-Dev] Proposed unittest changes In-Reply-To: <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> References: <20080713093936.GA3623@benfinney.id.au> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> Message-ID: <200807171409.30085.steve@pearwood.info> On Tue, 15 Jul 2008 09:26:45 am Raymond Hettinger wrote: > From: "Ben Finney" <ben+python at benfinney.id.au> > > > Right, so I'm putting up a separate PEP just for the renaming. > > Should be arriving on this list soon. > > I would like to work with you or someone else who is interested > on an alternative PEP for a separate, simpler test module > using the py.test syntax. I am interested in this suggestion. I didn't know about py.test. I admit to dissatisfaction with unittest (too Java-ish and heavyweight for my tastes). I would love a test suite midway in weight between doctests and unittest, so I will check it out. -- Steven From barry at python.org Thu Jul 17 06:30:51 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 17 Jul 2008 00:30:51 -0400 Subject: [Python-Dev] No beta2 tonight Message-ID: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We have green buildbots, yay! Thanks everyone for that. However, we still have three release blocker issues that I am not comfortable deferring. 3088 test_multiprocessing hangs intermittently on POSIX platforms 3375 _multiprocessing.so build problems 874900 threading module can deadlock after fork My biggest concern is 3375 and 3088, the latter has a dependency on 874900. 3375 is very strange since a subsequent 'make' completes just fine. This issue happens for me on both OS X 10.5 and Ubuntu 8.04 for py3k. 3088 does not appear to be happening now on OS X or Ubuntu, but the issue (and its dependent issue) are not closed, so I'm not sure exactly what's going on here. So we'll give these releases another 24 hours. Please, if you have time see what you can do about resolving these three blockers. Chat with me on #python-dev if you think we should release anyway. I will try to look into 3375, but I'd like to know if anybody else is seeing these build failures. I'll note that I plan to hold the beta3 releases until all release blocker and deferred blockers are resolved. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSH7K+3EjvBPtnXfVAQJPXgQAo+uHlgEEpOqlSYINJuXhTHNRWIQEByAu RCJZw4jNABKR4Pyero9z2LR31bEWS0Osg8a9DdoeD7bvVmPYyIJCG9bfA3BUwpD8 LtZ1tx9ou/XVGkDD7vQLuTt3hJXSaUvovx4ieeA7OQMiI0Uf3klIny+s2mBFH9IA COd48C7B/S4= =tcGt -----END PGP SIGNATURE----- From Scott.Daniels at Acm.Org Thu Jul 17 06:48:50 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Wed, 16 Jul 2008 21:48:50 -0700 Subject: [Python-Dev] PEP: Frequently-requested additional features for the `unittest` module In-Reply-To: <87d4le0x3z.fsf@benfinney.id.au> References: <87fxqa62xb.fsf@benfinney.id.au> <87abgi4bxi.fsf@benfinney.id.au> <g5krih$4f8$1@ger.gmane.org> <87d4le0x3z.fsf@benfinney.id.au> Message-ID: <g5mibh$kbe$1@ger.gmane.org> Ben Finney wrote: > Scott David Daniels <Scott.Daniels at Acm.Org> writes:> >> I would rather something more like: >> >> def assert_compare_true(op, first, second, msg=None): >> if op(first, second): >> return >> raise self.failure_exception(msg) >> if msg is None: >> self.failure_exception("%(first)r %(op)r %(second)" >> % vars()) >> self.failure_exception("%(first)r %(op)r %(second): %(msg)" >> % vars()) > I'm confused. It appears to me that your version never gets past the > first 'raise' statement, which is unconditional; and the rest seems to > do nothing but instantiate exceptions without using them. Sorry, I was too hasty last time (had to jet out of the house) and sent out the unfinished version. This is what I meant: def assert_compare_true(op, first, second, msg=None): if op(first, second): return if msg is None: raise self.failure_exception( "%(first)r %(op)r %(second)" % vars()) raise self.failure_exception( "%(first)r %(op)r %(second): %(msg)" % vars()) (1) Displaying args is the whole point, otherwise just use assert_. This form fosters tests that say what is wrong, and not simply _that_ something has gone wrong. The point is a readable test, reducing boilerplate at the call location. Something like: ... self.assert_le(sum(source) // len(source), 5, "Mean OK") (2) No point to doing string conversion except on failure; slow __repr__ methods are fine to use if the result is not discarded. --Scott David Daniels Scott.Daniels at Acm.Org From ben+python at benfinney.id.au Thu Jul 17 07:10:38 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 15:10:38 +1000 Subject: [Python-Dev] Proposed unittest changes References: <20080713093936.GA3623@benfinney.id.au> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <200807171409.30085.steve@pearwood.info> Message-ID: <87tzepyu29.fsf@benfinney.id.au> "Steven D'Aprano" <steve at pearwood.info> writes: > On Tue, 15 Jul 2008 09:26:45 am Raymond Hettinger wrote: > > From: "Ben Finney" <ben+python at benfinney.id.au> > > > > > Right, so I'm putting up a separate PEP just for the renaming. > > > Should be arriving on this list soon. > > > > I would like to work with you or someone else who is interested > > on an alternative PEP for a separate, simpler test module > > using the py.test syntax. > > I am interested in this suggestion. I didn't know about py.test. > > I admit to dissatisfaction with unittest (too Java-ish and heavyweight > for my tastes). I would love a test suite midway in weight between > doctests and unittest, so I will check it out. I still think 'nose' is a better candidate for this: it appears to offer what people say they want from 'py.test', yet (unlike 'py.test') is integrated well with 'unittest'. -- \ ?Pinky, are you pondering what I'm pondering?? ?I think so, | `\ Brain, but what kind of rides do they have in Fabioland?? | _o__) ?_Pinky and The Brain_ | Ben Finney From steve at holdenweb.com Thu Jul 17 07:50:48 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 17 Jul 2008 01:50:48 -0400 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <87sku9xmcr.fsf@benfinney.id.au> <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com> Message-ID: <487EDDB8.1030803@holdenweb.com> Guido van Rossum wrote: > On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote: >> The result I'm trying to avoid by this is that of having the >> externally visible behaviour of functions drift from the promise made >> by their names. Either rename the function, or create a new one for >> the new functionality. > > Oh get over it. This is getting ridiculous. Where did you *get* these > design "principles" you are so fond of? Read the zen of Python and > come back after you've memorized it. > Thank you. I was about to suggest that len() should be renamed length_or_raise_exception_if_not_a_container(). But then I did remark (before I stopped adding to this thread's interminable length, some aeons ago) that this was getting silly. Someone would surely have replied that the proposal had negative connotations and the correct name was really length_as_long_as_a_container. Or something equally trivial. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From grubert at users.sourceforge.net Thu Jul 17 08:52:14 2008 From: grubert at users.sourceforge.net (grubert at users.sourceforge.net) Date: Thu, 17 Jul 2008 08:52:14 +0200 (CEST) Subject: [Python-Dev] Running Py2.6 with the -3 option In-Reply-To: <ac12bf3a0807161118v252bca49ic62a14332d5132f7@mail.gmail.com> References: <731F1430F8054EC1A183F121F84B4504@RaymondLaptop1> <g57ll0$kqn$1@ger.gmane.org> <aac2c7cb0807111226k6cbbc9eai9580abf473c67ea3@mail.gmail.com> <bbaeab100807111316n3d07aaffq8d11bbc44358f92d@mail.gmail.com> <ca471dc20807151037m759f684cv543d6d925a64c9b3@mail.gmail.com> <1afaf6160807151041vbdb3351wd740f66351ffb066@mail.gmail.com> <g5j80e$g4d$1@ger.gmane.org> <bbaeab100807151538w21724d08jfd1942152333ac4a@mail.gmail.com> <ac12bf3a0807152350y166c00e9m577e85ca4f872f12@mail.gmail.com> <ca471dc20807161002h68bcd7aag4a3551548fae0543@mail.gmail.com> <ac12bf3a0807161118v252bca49ic62a14332d5132f7@mail.gmail.com> Message-ID: <Pine.LNX.4.64.0807170846030.8174@lx5.local> On Wed, 16 Jul 2008, engelbert gruber wrote: > On Wed, Jul 16, 2008 at 7:02 PM, Guido van Rossum <guido at python.org> wrote: > >> On Tue, Jul 15, 2008 at 11:50 PM, engelbert gruber >> <grubert at users.sourceforge.net> wrote: >>> I see mostly ``dict.has_key() not supported in 3.x;`` and sometimes >>> ``DeprecationWarning: callable() not supported in 3.x;`` . >> >> Good catch, Engelbert. isnt it that i throw and need someone who catches ? >> But why would has_key() need a warning when 2to3 will fix it just fine? > > for example, if has_key is the only problem in my module, it is > possible to support py22 to py3 with one source which i think would be > quite handy. rather naive assumption of me. anyway i'll file patches on case base. all the best -- From solipsis at pitrou.net Thu Jul 17 11:10:33 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Jul 2008 09:10:33 +0000 (UTC) Subject: [Python-Dev] light-weight testing References: <20080713093936.GA3623@benfinney.id.au> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <200807171409.30085.steve@pearwood.info> Message-ID: <loom.20080717T090441-407@post.gmane.org> Steven D'Aprano <steve <at> pearwood.info> writes: > > I am interested in this suggestion. I didn't know about py.test. > > I admit to dissatisfaction with unittest (too Java-ish and heavyweight > for my tastes). I would love a test suite midway in weight between > doctests and unittest, so I will check it out. > For what it's worth, I've been using nose for quite a long time and the first reason I did so is, like you, because I wanted to write tests in a light way (without having to declare classes). Then after writing some dozens of tests I switched back to wrapping tests in classes, just because it makes tests more readable and better organized (especially when you come to have setup/teardown functions shared by several tests). (but nose is still very nice) From solipsis at pitrou.net Thu Jul 17 11:12:59 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Jul 2008 09:12:59 +0000 (UTC) Subject: [Python-Dev] assertRaises References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org> Message-ID: <loom.20080717T091112-655@post.gmane.org> Fred Drake <fdrake <at> acm.org> writes: > > Sounds like adding a new method, catchException(...), that returns the > exception it catches, would be a reasonable compromise. I can't think > of any reason that the method that catches-and-returns needs to be the > existing API, which does something different. So you'd have a method that just catches (assertRaises), and another one that catches-and-returns (catchException)? It doesn't seem very practical to have two different methods based on such a small and trivial difference. Let's just make assertRaises return the exception instance, it seems like it feels the need correctly. From solipsis at pitrou.net Thu Jul 17 11:54:46 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Jul 2008 09:54:46 +0000 (UTC) Subject: [Python-Dev] assertRaises References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org> <loom.20080717T091112-655@post.gmane.org> Message-ID: <loom.20080717T095323-385@post.gmane.org> I said: > Let's just make assertRaises return the exception instance, it seems like it > feels the need correctly. and I meant "fills", not "feels", obviously... From ncoghlan at gmail.com Thu Jul 17 12:18:55 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 17 Jul 2008 20:18:55 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <871w1tz219.fsf@benfinney.id.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> Message-ID: <487F1C8F.7080405@gmail.com> Ben Finney wrote: > The function name should say *all* that the function does, from the > perspective of the caller. I have to disagree with that (and I think you'll find plenty of other folks here will disagree as well). A good function names needs to have a few characters: - serve as a mnemonic reminder as to what the function does for a programmer already familiar with the API - allow a less familiar user to easily look it up in the API documentation - be easy to remember to make the transition to familiarity with the API as rapid as possible Wanting to say *everything* a function does in its name is an utterly unreasonable goal that leads to ridiculously long function names that nobody can remember. Taking an existing function such as assertRaises and going "hey, we aren't using the return value from this, wouldn't it be really convenient if it told us the exact exception it actually caught?" doesn't cause any problems for existing code, and makes it much easier to write tests that need to check additional details about the exception that is raised (e.g. to check that the correct error code is captured and reported for things like OSError). The essence of the function remains unchanged - you're still asserting that a particular exception is raised. Returning the actual exception object that was caught is merely a convenience that makes a lot of sense. Try to think of it as being like adding a new optional argument to a function - just because the function now provides more flexibility is no reason to change it's name (e.g. when the key and reversed parameters were added to list.sort, the method name stayed the same). Taking an unused return value and making it meaningful is a similar kind of change. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jul 17 12:34:34 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 17 Jul 2008 20:34:34 +1000 Subject: [Python-Dev] light-weight testing In-Reply-To: <loom.20080717T090441-407@post.gmane.org> References: <20080713093936.GA3623@benfinney.id.au> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <200807171409.30085.steve@pearwood.info> <loom.20080717T090441-407@post.gmane.org> Message-ID: <487F203A.6010107@gmail.com> Antoine Pitrou wrote: > (especially when you come to have setup/teardown functions shared by several > tests). These days, I tend to just write a context manager for common setup/teardown code rather than using the setUp/tearDown hooks (at least for Python's own test suite, where I have the luxury of assuming 2.5+ as the Python version). Where I find unittest.TestCase quite convenient is when I want to run the same set of tests with a few different settings - for those, putting the settings into class attributes and then inheriting from the basic test case and using different values is very convenient. This trick is particularly useful for testing that a class supports inheritance properly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ben+python at benfinney.id.au Thu Jul 17 14:00:00 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 17 Jul 2008 22:00:00 +1000 Subject: [Python-Dev] light-weight testing References: <20080713093936.GA3623@benfinney.id.au> <87iqv89kj5.fsf@benfinney.id.au> <93F397279D1144DAB5767950F3A07868@RaymondLaptop1> <200807171409.30085.steve@pearwood.info> <loom.20080717T090441-407@post.gmane.org> Message-ID: <87prpczpof.fsf@benfinney.id.au> Antoine Pitrou <solipsis at pitrou.net> writes: > For what it's worth, I've been using nose for quite a long time and > the first reason I did so is, like you, because I wanted to write > tests in a light way (without having to declare classes). > > Then after writing some dozens of tests I switched back to wrapping > tests in classes, just because it makes tests more readable and > better organized (especially when you come to have setup/teardown > functions shared by several tests). > > (but nose is still very nice) It's also entirely compatible with wrapping one's tests in classes. The test discovery and collection in 'nose' is one of the attractions: it discovers them at package, module, class, and plain-function level, whether doctest or not, whether unittest or not, and collects them all to run. -- \ ?Well, my brother says Hello. So, hooray for speech therapy.? | `\ ?Emo Philips | _o__) | Ben Finney From jnoller at gmail.com Thu Jul 17 15:16:37 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 17 Jul 2008 09:16:37 -0400 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> Message-ID: <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com> On Thu, Jul 17, 2008 at 12:30 AM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > We have green buildbots, yay! Thanks everyone for that. > > However, we still have three release blocker issues that I am not > comfortable deferring. > > 3088 test_multiprocessing hangs intermittently on POSIX platforms > 3375 _multiprocessing.so build problems > 874900 threading module can deadlock after fork > > My biggest concern is 3375 and 3088, the latter has a dependency on 874900. > > 3375 is very strange since a subsequent 'make' completes just fine. This > issue happens for me on both OS X 10.5 and Ubuntu 8.04 for py3k. > > 3088 does not appear to be happening now on OS X or Ubuntu, but the issue > (and its dependent issue) are not closed, so I'm not sure exactly what's > going on here. > > So we'll give these releases another 24 hours. Please, if you have time see > what you can do about resolving these three blockers. Chat with me on > #python-dev if you think we should release anyway. I will try to look into > 3375, but I'd like to know if anybody else is seeing these build failures. > > I'll note that I plan to hold the beta3 releases until all release blocker > and deferred blockers are resolved. > > - -Barry > Quick status update, since I'm the gating factor. 3088: I didn't want to close this until 874900's patch was submitted, ported to py3k and I saw green build bots. By the time I went to sleep last night, we were still seeing 2to3 test timeouts on the build bots. I will port the patch to 3k today and lower the priority of 874900 and close 3088 shortly. 3375: Guido (thanks guido) looked into this, and while I banged my head on it a lot yesterday - guido's identified the issue, and now I need to figure out a fix - help is welcome on this one. I apologize for not getting these resolved last night -jesse From steve at holdenweb.com Thu Jul 17 15:57:02 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 17 Jul 2008 09:57:02 -0400 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> Message-ID: <487F4FAE.3050600@holdenweb.com> Barry Warsaw wrote: [...] > > I'll note that I plan to hold the beta3 releases until all release > blocker and deferred blockers are resolved. > Ian Ozsvald of ShowMeDo.com noticed a blog post of mine about this beta, and responded as follows: > Re. http://holdenweb.blogspot.com/2008/07/cpython-getting-serious-about-quality.html > If someone on the dev side wanted to make a 5-10 minute 'cast on how to check out the python base, build it, test it, check stuff back in, we'd be honoured to host it. Obviously the entire process deserves more attention but a short overview pointing out the key steps would be: > 1) simple for the author to create > 2) easy for possible contributors to watch and receive inspiration > 3) powerful enough to attract another set of developers into the fold > so that might be a useful, low-cost, quick win to help with the current effort? I'd do this myself but a) I'm not the obvious candidate and b) I'm too busy to give it my attention now. If someone wanted to tackle this, it might help getting people testing for the next beta, and also to recruit more testers in the long-term. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu Jul 17 15:57:02 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 17 Jul 2008 09:57:02 -0400 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> Message-ID: <487F4FAE.3050600@holdenweb.com> Barry Warsaw wrote: [...] > > I'll note that I plan to hold the beta3 releases until all release > blocker and deferred blockers are resolved. > Ian Ozsvald of ShowMeDo.com noticed a blog post of mine about this beta, and responded as follows: > Re. http://holdenweb.blogspot.com/2008/07/cpython-getting-serious-about-quality.html > If someone on the dev side wanted to make a 5-10 minute 'cast on how to check out the python base, build it, test it, check stuff back in, we'd be honoured to host it. Obviously the entire process deserves more attention but a short overview pointing out the key steps would be: > 1) simple for the author to create > 2) easy for possible contributors to watch and receive inspiration > 3) powerful enough to attract another set of developers into the fold > so that might be a useful, low-cost, quick win to help with the current effort? I'd do this myself but a) I'm not the obvious candidate and b) I'm too busy to give it my attention now. If someone wanted to tackle this, it might help getting people testing for the next beta, and also to recruit more testers in the long-term. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Thu Jul 17 16:07:09 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 07:07:09 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com> Message-ID: <ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com> On Thu, Jul 17, 2008 at 6:16 AM, Jesse Noller <jnoller at gmail.com> wrote: > 3375: Guido (thanks guido) looked into this, and while I banged my > head on it a lot yesterday - guido's identified the issue, and now I > need to figure out a fix - help is welcome on this one. You're welcome. I would have never found this if I hadn't had a heightened awareness of the sys.path_importer_cache variable recently, due to implementing a zipimport.py for Google App Engine. :-) I can try looking for a fix later today (in a few hours). A very crude fix would be to just remove all keys that have NullImporter values from that variable just before attempting to import the module that was just built. I'm hoping for something subtler though; I wonder if there's an identifyable point where the lib directory got created. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aleaxit at gmail.com Thu Jul 17 17:07:09 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Thu, 17 Jul 2008 08:07:09 -0700 Subject: [Python-Dev] assertRaises In-Reply-To: <loom.20080717T095323-385@post.gmane.org> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org> <loom.20080717T091112-655@post.gmane.org> <loom.20080717T095323-385@post.gmane.org> Message-ID: <e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com> On Thu, Jul 17, 2008 at 2:54 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > > I said: >> Let's just make assertRaises return the exception instance, it seems like it >> feels the need correctly. > > and I meant "fills", not "feels", obviously... +1 : enriching the existing method in a way that's perfectly transparent and innocuous to all existing uses _feels_ right, because it's more practical. Alex From jnoller at gmail.com Thu Jul 17 17:36:34 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 17 Jul 2008 11:36:34 -0400 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com> <ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com> Message-ID: <4222a8490807170836o1875cef4u8b42d3e7edb1160b@mail.gmail.com> On Thu, Jul 17, 2008 at 10:07 AM, Guido van Rossum <guido at python.org> wrote: > On Thu, Jul 17, 2008 at 6:16 AM, Jesse Noller <jnoller at gmail.com> wrote: >> 3375: Guido (thanks guido) looked into this, and while I banged my >> head on it a lot yesterday - guido's identified the issue, and now I >> need to figure out a fix - help is welcome on this one. > > You're welcome. I would have never found this if I hadn't had a > heightened awareness of the sys.path_importer_cache variable recently, > due to implementing a zipimport.py for Google App Engine. :-) > > I can try looking for a fix later today (in a few hours). A very crude > fix would be to just remove all keys that have NullImporter values > from that variable just before attempting to import the module that > was just built. I'm hoping for something subtler though; I wonder if > there's an identifyable point where the lib directory got created. > An additional note for anyone else - this is only under py3k - trunk is perfectly fine. From musiccomposition at gmail.com Thu Jul 17 17:41:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 17 Jul 2008 10:41:37 -0500 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> Message-ID: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> On Wed, Jul 16, 2008 at 11:30 PM, Barry Warsaw <barry at python.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > We have green buildbots, yay! Thanks everyone for that. The Windows buildbots are not very happy, though. test_ssl and test_bsddb and constantly failing on both the trunk and py3k. I don't know much about either of these items (or Windows for that matter), so any help would be greatly appreciated. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From eric+python-dev at trueblade.com Thu Jul 17 17:38:41 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 17 Jul 2008 11:38:41 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <487E3E5B.5070700@trueblade.com> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> <487E3030.5010203@trueblade.com> <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com> <487E3E5B.5070700@trueblade.com> Message-ID: <487F6781.1000206@trueblade.com> Eric Smith wrote: > Guido van Rossum wrote: > >>> It shares code with %-formatting. Change that, too? I couldn't find >>> any >>> occurrences of %F in the stdlib. Not that that's the entire >>> universe, of >>> course. >>> >>> The change is slightly less elegant if I don't change %-formatting, but >>> still doable, especially if the betas don't get cut today. >> >> Fine with me to change %F as well -- after all that's where the >> misunderstanding comes from. (More and more I'm beginning to think it >> was my mistake. :-) > > I added this as > http://mail.python.org/pipermail/python-dev/2008-July/081242.html > > I should be able to get it checked in today. I have this ready for checkin (with docs and tests). I'd like to get it in for this beta, since it does involved changed behavior, no matter how small ('1e+100' becomes '1E+100' with '%F'). But it relies on the platform's vsnprintf to do the right thing with 'F', so I'm worried about breakages on platforms I don't have access to. Resolving those issues might take a few days. Any advice on checking this in now, or waiting until after this beta is released? On a related note, I've wanted to clean up float formatting for ages. The test: if ((type == 'f' || type == 'F') && (fabs(x) / 1e25) >= 1e25) is in the code in 3 places, for example. I'm hoping I can get to that project, but since it won't change functionality I can do it later. Eric. From rrr at ronadam.com Thu Jul 17 18:19:57 2008 From: rrr at ronadam.com (Ron Adam) Date: Thu, 17 Jul 2008 11:19:57 -0500 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <487F1C8F.7080405@gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <487F1C8F.7080405@gmail.com> Message-ID: <487F712D.6050908@ronadam.com> Nick Coghlan wrote: > Taking an existing function such as assertRaises and going "hey, we > aren't using the return value from this, wouldn't it be really > convenient if it told us the exact exception it actually caught?" > doesn't cause any problems for existing code, and makes it much easier > to write tests that need to check additional details about the exception > that is raised (e.g. to check that the correct error code is captured > and reported for things like OSError). > > The essence of the function remains unchanged - you're still asserting > that a particular exception is raised. Returning the actual exception > object that was caught is merely a convenience that makes a lot of sense. I'm not sure I understand... If "a particular exception is raised", every thing is good and there is no error to report. ie... the code being tested did the right thing. If it does not raise the particular desired exception, isn't either a failureException raised or someother exception, which is caught by the unittest test runner. In that case the assertRaises method never gets a chance to return anything. If this is correct, then the exception needs to be caught and passed out via the failureException, and not the returned value. Ron From rrr at ronadam.com Thu Jul 17 18:19:57 2008 From: rrr at ronadam.com (Ron Adam) Date: Thu, 17 Jul 2008 11:19:57 -0500 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <487F1C8F.7080405@gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <487F1C8F.7080405@gmail.com> Message-ID: <487F712D.6050908@ronadam.com> Nick Coghlan wrote: > Taking an existing function such as assertRaises and going "hey, we > aren't using the return value from this, wouldn't it be really > convenient if it told us the exact exception it actually caught?" > doesn't cause any problems for existing code, and makes it much easier > to write tests that need to check additional details about the exception > that is raised (e.g. to check that the correct error code is captured > and reported for things like OSError). > > The essence of the function remains unchanged - you're still asserting > that a particular exception is raised. Returning the actual exception > object that was caught is merely a convenience that makes a lot of sense. I'm not sure I understand... If "a particular exception is raised", every thing is good and there is no error to report. ie... the code being tested did the right thing. If it does not raise the particular desired exception, isn't either a failureException raised or someother exception, which is caught by the unittest test runner. In that case the assertRaises method never gets a chance to return anything. If this is correct, then the exception needs to be caught and passed out via the failureException, and not the returned value. Ron From python at rcn.com Thu Jul 17 18:25:37 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 17 Jul 2008 09:25:37 -0700 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> <487E3030.5010203@trueblade.com> <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com><487E3E5B.5070700@trueblade.com> <487F6781.1000206@trueblade.com> Message-ID: <E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1> From: "Eric Smith" <eric+python-dev at trueblade.com> > I have this ready for checkin (with docs and tests). I'd like to get it > in for this beta, since it does involved changed behavior, no matter how > small ('1e+100' becomes '1E+100' with '%F'). But it relies on the > platform's vsnprintf to do the right thing with 'F', so I'm worried > about breakages on platforms I don't have access to. Resolving those > issues might take a few days. > > Any advice on checking this in now, or waiting until after this beta is > released? I recommend doing it after the release. It's unlikely to be exercised much by the beta users so there's no real advantage. If you wait until afterwards, then there is time to let the buildbots have a go at it and reveal any cross-platform issues. Besides, Barry said something about getting meaner. Would hate to find out what he meant the hard way ;-) Raymond From guido at python.org Thu Jul 17 18:27:58 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 09:27:58 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <4222a8490807170836o1875cef4u8b42d3e7edb1160b@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com> <ca471dc20807170707y2f40e5e3lf4af011e8219b096@mail.gmail.com> <4222a8490807170836o1875cef4u8b42d3e7edb1160b@mail.gmail.com> Message-ID: <ca471dc20807170927g1b7c810r1cad09bf1e1a93dd@mail.gmail.com> On Thu, Jul 17, 2008 at 8:36 AM, Jesse Noller <jnoller at gmail.com> wrote: > On Thu, Jul 17, 2008 at 10:07 AM, Guido van Rossum <guido at python.org> wrote: >> On Thu, Jul 17, 2008 at 6:16 AM, Jesse Noller <jnoller at gmail.com> wrote: >>> 3375: Guido (thanks guido) looked into this, and while I banged my >>> head on it a lot yesterday - guido's identified the issue, and now I >>> need to figure out a fix - help is welcome on this one. >> >> You're welcome. I would have never found this if I hadn't had a >> heightened awareness of the sys.path_importer_cache variable recently, >> due to implementing a zipimport.py for Google App Engine. :-) >> >> I can try looking for a fix later today (in a few hours). A very crude >> fix would be to just remove all keys that have NullImporter values >> from that variable just before attempting to import the module that >> was just built. I'm hoping for something subtler though; I wonder if >> there's an identifyable point where the lib directory got created. I submitted a fix along these lines -- other things I tried did not work out. The issue is fixed. > An additional note for anyone else - this is only under py3k - trunk > is perfectly fine. Right. I'm thinking that something changed in the massaging of the default search path, but I can't be bothered to investigate further. I do know that right after startup there are more non-trivial entries in sys.path_importer_cache in 3.0 than there are in 2.6. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jul 17 18:29:56 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 09:29:56 -0700 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> <487E3030.5010203@trueblade.com> <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com> <487E3E5B.5070700@trueblade.com> <487F6781.1000206@trueblade.com> <E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1> Message-ID: <ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com> On Thu, Jul 17, 2008 at 9:25 AM, Raymond Hettinger <python at rcn.com> wrote: > From: "Eric Smith" <eric+python-dev at trueblade.com> >> >> I have this ready for checkin (with docs and tests). I'd like to get it >> in for this beta, since it does involved changed behavior, no matter how >> small ('1e+100' becomes '1E+100' with '%F'). But it relies on the >> platform's vsnprintf to do the right thing with 'F', so I'm worried >> about breakages on platforms I don't have access to. Resolving those >> issues might take a few days. >> >> Any advice on checking this in now, or waiting until after this beta is >> released? > > I recommend doing it after the release. It's unlikely to be exercised > much by the beta users so there's no real advantage. If you wait > until afterwards, then there is time to let the buildbots have a go > at it and reveal any cross-platform issues. > > Besides, Barry said something about getting meaner. > Would hate to find out what he meant the hard way ;-) I'd advise the opposite -- check it in now. It's not going to break anything, and if it is, let's find out sooner rather than later. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Thu Jul 17 18:57:21 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 17 Jul 2008 09:57:21 PDT Subject: [Python-Dev] No beta2 tonight In-Reply-To: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> Message-ID: <08Jul17.095729pdt."58698"@synergy1.parc.xerox.com> > test_ssl ... constantly failing on both the trunk and py3k. I'll take a closer look at this. It's the new test added in lately. Seems to be working on non-Windows platforms, so I'm guessing it's some Windows oddity, which I'm not very good at diagnosing. Worst comes to worst, we can take out that test. Bill From tseaver at palladion.com Thu Jul 17 19:21:24 2008 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 17 Jul 2008 13:21:24 -0400 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <487F712D.6050908@ronadam.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <487F1C8F.7080405@gmail.com> <487F712D.6050908@ronadam.com> Message-ID: <487F7F94.9080201@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ron Adam wrote: > > Nick Coghlan wrote: >> The essence of the function remains unchanged - you're still asserting >> that a particular exception is raised. Returning the actual exception >> object that was caught is merely a convenience that makes a lot of sense. > > I'm not sure I understand... > > If "a particular exception is raised", every thing is good and there is no > error to report. ie... the code being tested did the right thing. I *think* that in this case the propoent wants to be able to make further assertions about the raised exception (e.g., to test its message). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIf3+T+gerLs4ltQ4RAk/VAKCK8hnX2C7DE57RSxA88THttT5tSwCg2taQ zzRkJ/SEPwZvgfcluo2DTvw= =KTGx -----END PGP SIGNATURE----- From guido at python.org Thu Jul 17 19:22:27 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 10:22:27 -0700 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> Message-ID: <ca471dc20807171022u70b76960vf6e47b3d22efbce8@mail.gmail.com> Please move all discussions of unittest frameworks to python-ideas at python.org. It is an interesting topic -- so interesting, in fact, that exploring all the different ideas under discussion is overwhelming the primary purpose of python-dev, which at this point is to get the 2.6 and 3.0 releases into shape. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Thu Jul 17 19:41:32 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 17 Jul 2008 10:41:32 PDT Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <08Jul17.095729pdt."58698"@synergy1.parc.xerox.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <08Jul17.095729pdt."58698"@synergy1.parc.xerox.com> Message-ID: <08Jul17.104140pdt."58698"@synergy1.parc.xerox.com> > > test_ssl ... constantly failing on both the trunk and py3k. > > I'll take a closer look at this. It's the new test added in lately. > Seems to be working on non-Windows platforms, so I'm guessing it's > some Windows oddity, which I'm not very good at diagnosing. Worst > comes to worst, we can take out that test. It looks like the handshake code in _ssl.c is returning this Windows error (I'm looking at the trunk): ERROR: testWrongCert (test.test_ssl.ThreadedTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\test\test_ssl.py", line 807, in testWrongCert "wrongcert.pem")) File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\test\test_ssl.py", line 597, in badCertTest s.connect((HOST, server.port)) File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\ssl.py", line 272, in connect self.do_handshake() File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\ssl.py", line 256, in do_handshake self._sslobj.do_handshake() error: [Errno 10054] An existing connection was forcibly closed by the remote host Error 10054 is WSAECONNRESET, according to Google. Arguably, it's an error in this scrap of _ssl.c: } else if (ret == -1) { /* underlying BIO reported an I/O error */ return obj->Socket->errorhandler(); } I should be checking for EOF error returns like this one from Socket, and returning an PY_SSL_ERROR_EOF exception. Is there a list somewhere of what EOF errors are signalled by Socket? Bill From jnoller at gmail.com Thu Jul 17 19:45:44 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 17 Jul 2008 13:45:44 -0400 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <4222a8490807170616x5ca600fdw3e6d0bced885c943@mail.gmail.com> Message-ID: <4222a8490807171045y17b7252sfc99e1fe67e4f1d6@mail.gmail.com> On Thu, Jul 17, 2008 at 9:16 AM, Jesse Noller <jnoller at gmail.com> wrote: > On Thu, Jul 17, 2008 at 12:30 AM, Barry Warsaw <barry at python.org> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> We have green buildbots, yay! Thanks everyone for that. >> >> However, we still have three release blocker issues that I am not >> comfortable deferring. >> >> 3088 test_multiprocessing hangs intermittently on POSIX platforms >> 3375 _multiprocessing.so build problems >> 874900 threading module can deadlock after fork >> >> My biggest concern is 3375 and 3088, the latter has a dependency on 874900. >> >> 3375 is very strange since a subsequent 'make' completes just fine. This >> issue happens for me on both OS X 10.5 and Ubuntu 8.04 for py3k. >> >> 3088 does not appear to be happening now on OS X or Ubuntu, but the issue >> (and its dependent issue) are not closed, so I'm not sure exactly what's >> going on here. >> >> So we'll give these releases another 24 hours. Please, if you have time see >> what you can do about resolving these three blockers. Chat with me on >> #python-dev if you think we should release anyway. I will try to look into >> 3375, but I'd like to know if anybody else is seeing these build failures. >> >> I'll note that I plan to hold the beta3 releases until all release blocker >> and deferred blockers are resolved. >> >> - -Barry >> > > Quick status update, since I'm the gating factor. > > 3088: I didn't want to close this until 874900's patch was submitted, > ported to py3k and I saw green build bots. By the time I went to sleep > last night, we were still seeing 2to3 test timeouts on the build bots. > I will port the patch to 3k today and lower the priority of 874900 and > close 3088 shortly. > > 3375: Guido (thanks guido) looked into this, and while I banged my > head on it a lot yesterday - guido's identified the issue, and now I > need to figure out a fix - help is welcome on this one. > > I apologize for not getting these resolved last night > > -jesse > Just to close the loop - 3088 is closed now, and 874900 is on py3k, therefore lowered from a release blocker for now (it needs to remain open to finish resolving a few issues) Guido fixed 3375 (thanks again) -jesse From ranjithkannikara at gmail.com Thu Jul 17 19:54:38 2008 From: ranjithkannikara at gmail.com (ranjith kannikara) Date: Thu, 17 Jul 2008 23:24:38 +0530 Subject: [Python-Dev] Implementing restricted Python in Zope2 Message-ID: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> I have taken the gsoc 08 project of porting zope2 to python2.5. Through my way to the successful completion of the project I have to implement Restricted python in Zope2. I could only get the information that the python AST has not changed on moving from python2.4 to 2.5 but Restricted Python is not well documented enough for a stident to test the Zope2 's Restricted Python implentation. As a student I am not familiar with Restricted Python and python AST implementation.And in need of help to start the Restricted Python implementation. Ranjith. From janssen at parc.com Thu Jul 17 20:18:34 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 17 Jul 2008 11:18:34 PDT Subject: [Python-Dev] No beta2 tonight In-Reply-To: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> Message-ID: <08Jul17.111840pdt."58698"@synergy1.parc.xerox.com> > The Windows buildbots are not very happy, though. test_ssl ... > constantly failing on both the trunk and py3k. I've checked in patches for test_ssl on both branches. Let's see how the Windows buildbots do. Bill From brett at python.org Thu Jul 17 20:27:42 2008 From: brett at python.org (Brett Cannon) Date: Thu, 17 Jul 2008 11:27:42 -0700 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> Message-ID: <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara <ranjithkannikara at gmail.com> wrote: > I have taken the gsoc 08 project of porting zope2 to python2.5. > Through my way to the successful completion of the project I have to > implement Restricted python in Zope2. I could only get the information > that the python AST has not changed on moving from python2.4 to 2.5 > but Restricted Python is not well documented enough for a stident to > test the Zope2 's Restricted Python implentation. > > As a student I am not familiar with Restricted Python and python AST > implementation.And in need of help to start the Restricted Python > implementation. > What do you mean, "Restricted Python"? If you mean rexec and Bastion, they are no longer supported, and that began before 2.5. -Brett From guido at python.org Thu Jul 17 20:47:26 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 11:47:26 -0700 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> Message-ID: <ca471dc20807171147u26105c05ue63e6dc227b78052@mail.gmail.com> On Thu, Jul 17, 2008 at 11:27 AM, Brett Cannon <brett at python.org> wrote: > On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara > <ranjithkannikara at gmail.com> wrote: >> I have taken the gsoc 08 project of porting zope2 to python2.5. >> Through my way to the successful completion of the project I have to >> implement Restricted python in Zope2. I could only get the information >> that the python AST has not changed on moving from python2.4 to 2.5 >> but Restricted Python is not well documented enough for a stident to >> test the Zope2 's Restricted Python implentation. >> >> As a student I am not familiar with Restricted Python and python AST >> implementation.And in need of help to start the Restricted Python >> implementation. > What do you mean, "Restricted Python"? If you mean rexec and Bastion, > they are no longer supported, and that began before 2.5. Maybe he's talking about PyPy's RPython, which is notoriously (and it seems intentionally) underspecified? In that case, please contact the PyPy team. Or maybe there is something in Zope 2 called Restricted Python -- in that case please contact the Zope lists. There's nothing in the collective memory of python-dev called Restricted Python that we can help you with. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at ctypes.org Thu Jul 17 21:02:21 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 17 Jul 2008 21:02:21 +0200 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> <487E3030.5010203@trueblade.com> <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com> <487E3E5B.5070700@trueblade.com> <487F6781.1000206@trueblade.com> <E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1> <ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com> Message-ID: <g5o4uu$tpf$1@ger.gmane.org> Guido van Rossum schrieb: > On Thu, Jul 17, 2008 at 9:25 AM, Raymond Hettinger <python at rcn.com> wrote: >> From: "Eric Smith" <eric+python-dev at trueblade.com> >>> >>> I have this ready for checkin (with docs and tests). I'd like to get it >>> in for this beta, since it does involved changed behavior, no matter how >>> small ('1e+100' becomes '1E+100' with '%F'). But it relies on the >>> platform's vsnprintf to do the right thing with 'F', so I'm worried >>> about breakages on platforms I don't have access to. Resolving those >>> issues might take a few days. >>> >>> Any advice on checking this in now, or waiting until after this beta is >>> released? >> >> I recommend doing it after the release. It's unlikely to be exercised >> much by the beta users so there's no real advantage. If you wait >> until afterwards, then there is time to let the buildbots have a go >> at it and reveal any cross-platform issues. >> >> Besides, Barry said something about getting meaner. >> Would hate to find out what he meant the hard way ;-) > > I'd advise the opposite -- check it in now. It's not going to break > anything, and if it is, let's find out sooner rather than later. > Before we get old waiting for the windows buildbots, here how test_format fails now (on XP SP2 x86, trunk rev 65072: c:\svn\trunk\PCbuild>.\\python_d -E -tt ../lib/test/regrtest.py test_format test_format test test_format produced unexpected output: ********************************************************************** '%F' % (1.0,) == '' != '1.000000' u'%F' % (1.0,) == u'' != '1.000000' '%f' % (nan,) == '-1.#IND00' != 'nan' u'%f' % (nan,) == u'-1.#IND00' != 'nan' '%F' % (nan,) == '' != 'NAN' u'%F' % (nan,) == u'' != 'NAN' '%f' % (inf,) == '1.#INF' != 'inf' u'%f' % (inf,) == u'1.#INF' != 'inf' '%F' % (inf,) == '1.#INF' != 'INF' u'%F' % (inf,) == u'1.#INF' != 'INF' ********************************************************************** 1 test failed: test_format [23521 refs] c:\svn\trunk\PCbuild> Windows vsnprintf does not support '%F' at all, and '%f' does not behave as expected with nan and inf. -- Thanks, Thomas From eric+python-dev at trueblade.com Thu Jul 17 21:10:56 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 17 Jul 2008 15:10:56 -0400 Subject: [Python-Dev] PEP 3101: floats format 'f' and 'F' In-Reply-To: <g5o4uu$tpf$1@ger.gmane.org> References: <487E0724.6020002@trueblade.com> <ca471dc20807161025u4e227574g47169bbf0af8ac1c@mail.gmail.com> <487E3030.5010203@trueblade.com> <ca471dc20807161038l57140083t14f6375bae4f6723@mail.gmail.com> <487E3E5B.5070700@trueblade.com> <487F6781.1000206@trueblade.com> <E761C23B2EF84CF8B16B2CF1B28DBA17@RaymondLaptop1> <ca471dc20807170929x2fd919dcvf1680a9365daef41@mail.gmail.com> <g5o4uu$tpf$1@ger.gmane.org> Message-ID: <487F9940.3020208@trueblade.com> Thomas Heller wrote: > Guido van Rossum schrieb: >> On Thu, Jul 17, 2008 at 9:25 AM, Raymond Hettinger <python at rcn.com> wrote: >>> From: "Eric Smith" <eric+python-dev at trueblade.com> >>>> I have this ready for checkin (with docs and tests). I'd like to get it >>>> in for this beta, since it does involved changed behavior, no matter how >>>> small ('1e+100' becomes '1E+100' with '%F'). But it relies on the >>>> platform's vsnprintf to do the right thing with 'F', so I'm worried >>>> about breakages on platforms I don't have access to. Resolving those >>>> issues might take a few days. >>>> >>>> Any advice on checking this in now, or waiting until after this beta is >>>> released? >>> I recommend doing it after the release. It's unlikely to be exercised >>> much by the beta users so there's no real advantage. If you wait >>> until afterwards, then there is time to let the buildbots have a go >>> at it and reveal any cross-platform issues. >>> >>> Besides, Barry said something about getting meaner. >>> Would hate to find out what he meant the hard way ;-) >> I'd advise the opposite -- check it in now. It's not going to break >> anything, and if it is, let's find out sooner rather than later. >> > > Before we get old waiting for the windows buildbots, here how test_format > fails now (on XP SP2 x86, trunk rev 65072: > > c:\svn\trunk\PCbuild>.\\python_d -E -tt ../lib/test/regrtest.py test_format > test_format > test test_format produced unexpected output: > ********************************************************************** > '%F' % (1.0,) == '' != '1.000000' > u'%F' % (1.0,) == u'' != '1.000000' > '%f' % (nan,) == '-1.#IND00' != 'nan' > u'%f' % (nan,) == u'-1.#IND00' != 'nan' > '%F' % (nan,) == '' != 'NAN' > u'%F' % (nan,) == u'' != 'NAN' > '%f' % (inf,) == '1.#INF' != 'inf' > u'%f' % (inf,) == u'1.#INF' != 'inf' > '%F' % (inf,) == '1.#INF' != 'INF' > u'%F' % (inf,) == u'1.#INF' != 'INF' > > ********************************************************************** > 1 test failed: > test_format > [23521 refs] > > c:\svn\trunk\PCbuild> > > Windows vsnprintf does not support '%F' at all, and '%f' does not > behave as expected with nan and inf. > I was afraid of that. I'll back these out until I can dig up a Windows box to test on. Thanks for checking this. Eric. From shigin at rambler-co.ru Thu Jul 17 21:41:19 2008 From: shigin at rambler-co.ru (Alexander Shigin) Date: Thu, 17 Jul 2008 23:41:19 +0400 Subject: [Python-Dev] Issue2944: need a review Message-ID: <1216323679.4911.6.camel@jenner> Can anyone look at the patch for Issue2944? I hope the issue can be fixed before the release of python 2.6. From pje at telecommunity.com Thu Jul 17 22:26:12 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 17 Jul 2008 16:26:12 -0400 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.co m> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> Message-ID: <20080717202531.DA16A3A4080@sparrow.telecommunity.com> At 11:27 AM 7/17/2008 -0700, Brett Cannon wrote: >On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara ><ranjithkannikara at gmail.com> wrote: > > I have taken the gsoc 08 project of porting zope2 to python2.5. > > Through my way to the successful completion of the project I have to > > implement Restricted python in Zope2. I could only get the information > > that the python AST has not changed on moving from python2.4 to 2.5 > > but Restricted Python is not well documented enough for a stident to > > test the Zope2 's Restricted Python implentation. > > > > As a student I am not familiar with Restricted Python and python AST > > implementation.And in need of help to start the Restricted Python > > implementation. > > > >What do you mean, "Restricted Python"? If you mean rexec and Bastion, >they are no longer supported, and that began before 2.5. No, he means the restricted Python compiler and capability-proxy system used by Zope. You know, the one I always bring up whenever anybody says they want to implement capabilities in Python? ;-) Zope's restricted Python is basically a combination of a special compiler, __builtin__ replacements, and a proxy type. Instead of using LOAD_ATTR opcodes, the compiler generates code that calls a special getattr() function instead, and most objects other than relatively-safe builtin types are wrapped in proxies that control what attributes can be accessed and what operations can be performed. The restricted Python framework itself doesn't impose any particular security policy; proxies delegate checks to "checker" objects that are essentially capabilities. Mostly, it focuses on creating a safe sandbox that can be expanded. There are two parts to the implication; one is called RestrictedPython and lives at: http://svn.zope.org/RestrictedPython/trunk The other part is "zope.security.untrustedpython", and it's part of the zope.security distribution; see: http://svn.zope.org/zope.security/trunk/src/zope/security/untrustedpython/ for its specific code and docs. Both packages appear to have automated tests. From guido at python.org Thu Jul 17 23:03:44 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 14:03:44 -0700 Subject: [Python-Dev] Issue2944: need a review In-Reply-To: <1216323679.4911.6.camel@jenner> References: <1216323679.4911.6.camel@jenner> Message-ID: <ca471dc20807171403t638fb207h218fa11fb97caa5f@mail.gmail.com> Suggestion for people asking developers to look into issues: indicate more than the issue number in the email. Show the issue summary, other relevant metadata, and what needs to be done in some detail. This will pique the interest of those who *can* help (and allow people who can't help anyway to skip the message more efficiently). On Thu, Jul 17, 2008 at 12:41 PM, Alexander Shigin <shigin at rambler-co.ru> wrote: > Can anyone look at the patch for Issue2944? > > I hope the issue can be fixed before the release of python 2.6. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jul 17 23:04:34 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 14:04:34 -0700 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <20080717202531.DA16A3A4080@sparrow.telecommunity.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> <20080717202531.DA16A3A4080@sparrow.telecommunity.com> Message-ID: <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com> Thanks. Then python-dev is *definitely* the wrong forum. :-) On Thu, Jul 17, 2008 at 1:26 PM, Phillip J. Eby <pje at telecommunity.com> wrote: > At 11:27 AM 7/17/2008 -0700, Brett Cannon wrote: >> >> On Thu, Jul 17, 2008 at 10:54 AM, ranjith kannikara >> <ranjithkannikara at gmail.com> wrote: >> > I have taken the gsoc 08 project of porting zope2 to python2.5. >> > Through my way to the successful completion of the project I have to >> > implement Restricted python in Zope2. I could only get the information >> > that the python AST has not changed on moving from python2.4 to 2.5 >> > but Restricted Python is not well documented enough for a stident to >> > test the Zope2 's Restricted Python implentation. >> > >> > As a student I am not familiar with Restricted Python and python AST >> > implementation.And in need of help to start the Restricted Python >> > implementation. >> > >> >> What do you mean, "Restricted Python"? If you mean rexec and Bastion, >> they are no longer supported, and that began before 2.5. > > No, he means the restricted Python compiler and capability-proxy system used > by Zope. You know, the one I always bring up whenever anybody says they > want to implement capabilities in Python? ;-) > > Zope's restricted Python is basically a combination of a special compiler, > __builtin__ replacements, and a proxy type. Instead of using LOAD_ATTR > opcodes, the compiler generates code that calls a special getattr() function > instead, and most objects other than relatively-safe builtin types are > wrapped in proxies that control what attributes can be accessed and what > operations can be performed. > > The restricted Python framework itself doesn't impose any particular > security policy; proxies delegate checks to "checker" objects that are > essentially capabilities. Mostly, it focuses on creating a safe sandbox > that can be expanded. > > There are two parts to the implication; one is called RestrictedPython and > lives at: > > http://svn.zope.org/RestrictedPython/trunk > > The other part is "zope.security.untrustedpython", and it's part of the > zope.security distribution; see: > > http://svn.zope.org/zope.security/trunk/src/zope/security/untrustedpython/ > > for its specific code and docs. > > Both packages appear to have automated tests. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Thu Jul 17 23:04:40 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 17 Jul 2008 14:04:40 PDT Subject: [Python-Dev] Issue2944: need a review In-Reply-To: <1216323679.4911.6.camel@jenner> References: <1216323679.4911.6.camel@jenner> Message-ID: <08Jul17.140446pdt."58698"@synergy1.parc.xerox.com> These requests always have a higher probability of being addressed if you summarize the issue in the request. Bill > Can anyone look at the patch for Issue2944? > > I hope the issue can be fixed before the release of python 2.6. From sidnei at enfoldsystems.com Thu Jul 17 23:10:03 2008 From: sidnei at enfoldsystems.com (Sidnei da Silva) Date: Thu, 17 Jul 2008 18:10:03 -0300 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> <20080717202531.DA16A3A4080@sparrow.telecommunity.com> <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com> Message-ID: <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com> On Thu, Jul 17, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote: > Thanks. Then python-dev is *definitely* the wrong forum. :-) Definitely wrong forum for RestrictedPython-specifc questions, but not if you consider those two paragraphs from Shane's email, which clarify that Ranjith is seeking for information on possible AST changes that might have occurred in Python 2.5: """ The safety of RestrictedPython has been validated in a somewhat formal process with Python 2.4. Ranjith is working to validate it with Python 2.5. He is first working to discover all changes between Python 2.4 and 2.5 that might have affected the safety of a RestrictedPython sandbox. Any changes to the AST, builtin functions, methods of builtin types, etc., need to be evaluated for safety. So, in general, he is looking for detailed lists of changes between Python 2.4 and 2.5--more than the "What's New" doc. """ -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 From musiccomposition at gmail.com Thu Jul 17 23:13:04 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 17 Jul 2008 16:13:04 -0500 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> <20080717202531.DA16A3A4080@sparrow.telecommunity.com> <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com> <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com> Message-ID: <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com> On Thu, Jul 17, 2008 at 4:10 PM, Sidnei da Silva <sidnei at enfoldsystems.com> wrote: > On Thu, Jul 17, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote: >> Thanks. Then python-dev is *definitely* the wrong forum. :-) > > Definitely wrong forum for RestrictedPython-specifc questions, but not > if you consider those two paragraphs from Shane's email, which clarify > that Ranjith is seeking for information on possible AST changes that > might have occurred in Python 2.5: > > """ > The safety of RestrictedPython has been validated in a somewhat formal > process with Python 2.4. Ranjith is working to validate it with > Python 2.5. He is first working to discover all changes between > Python 2.4 and 2.5 that might have affected the safety of a > RestrictedPython sandbox. Any changes to the AST, builtin functions, > methods of builtin types, etc., need to be evaluated for safety. > > So, in general, he is looking for detailed lists of changes between > Python 2.4 and 2.5--more than the "What's New" doc. > """ Then I would recommend he look at Misc/NEWS. If he needs more information, *specific* questions will bring the best results. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From tjreedy at udel.edu Thu Jul 17 23:38:44 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 17 Jul 2008 17:38:44 -0400 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> <20080717202531.DA16A3A4080@sparrow.telecommunity.com> <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com> <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com> <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com> Message-ID: <g5oe52$u64$2@ger.gmane.org> Benjamin Peterson wrote: > On Thu, Jul 17, 2008 at 4:10 PM, Sidnei da Silva > <sidnei at enfoldsystems.com> wrote: > >> """ >> The safety of RestrictedPython has been validated in a somewhat formal >> process with Python 2.4. Ranjith is working to validate it with >> Python 2.5. He is first working to discover all changes between >> Python 2.4 and 2.5 that might have affected the safety of a >> RestrictedPython sandbox. Any changes to the AST, builtin functions, >> methods of builtin types, etc., need to be evaluated for safety. >> >> So, in general, he is looking for detailed lists of changes between >> Python 2.4 and 2.5--more than the "What's New" doc. >> """ > > Then I would recommend he look at Misc/NEWS. If he needs more > information, *specific* questions will bring the best results. Ultimately, he can do a diff of the corresponding C files, and read the checkin messages that hopefully explain some of the changes. From ncoghlan at gmail.com Fri Jul 18 00:41:12 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 18 Jul 2008 08:41:12 +1000 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <487F7F94.9080201@palladion.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5406.1000802@voidspace.org.uk> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <487F1C8F.7080405@gmail.com> <487F712D.6050908@ronadam.com> <487F7F94.9080201@palladion.com> Message-ID: <487FCA88.4000405@gmail.com> Tres Seaver wrote: > Ron Adam wrote: >> Nick Coghlan wrote: > >>> The essence of the function remains unchanged - you're still asserting >>> that a particular exception is raised. Returning the actual exception >>> object that was caught is merely a convenience that makes a lot of sense. >> I'm not sure I understand... > >> If "a particular exception is raised", every thing is good and there is no >> error to report. ie... the code being tested did the right thing. > > I *think* that in this case the propoent wants to be able to make > further assertions about the raised exception (e.g., to test its message). Exactly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jul 18 00:51:11 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 18 Jul 2008 08:51:11 +1000 Subject: [Python-Dev] Issue2944: need a review In-Reply-To: <ca471dc20807171403t638fb207h218fa11fb97caa5f@mail.gmail.com> References: <1216323679.4911.6.camel@jenner> <ca471dc20807171403t638fb207h218fa11fb97caa5f@mail.gmail.com> Message-ID: <487FCCDF.1070605@gmail.com> Guido van Rossum wrote: > Suggestion for people asking developers to look into issues: indicate > more than the issue number in the email. Show the issue summary, other > relevant metadata, and what needs to be done in some detail. This will > pique the interest of those who *can* help (and allow people who can't > help anyway to skip the message more efficiently). Links of the form http://bugs.python.org/issue2944 don't hurt either :) That particular issue is an asyncore problem, so I cc'ed Josiah directly on this message. Cheers, Nick. > > On Thu, Jul 17, 2008 at 12:41 PM, Alexander Shigin <shigin at rambler-co.ru> wrote: >> Can anyone look at the patch for Issue2944? >> >> I hope the issue can be fixed before the release of python 2.6. > -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mhammond at skippinet.com.au Fri Jul 18 00:51:45 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri, 18 Jul 2008 08:51:45 +1000 Subject: [Python-Dev] assertRaises In-Reply-To: <e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org> <loom.20080717T091112-655@post.gmane.org> <loom.20080717T095323-385@post.gmane.org> <e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com> Message-ID: <03e501c8e85f$b07b8000$11728000$@com.au> > >> Let's just make assertRaises return the exception instance, it seems > >> like it feels the need correctly. > > > > and I meant "fills", not "feels", obviously... > > +1 : enriching the existing method in a way that's perfectly > transparent and innocuous to all existing uses _feels_ right, because > it's more practical. For the record, and primarily to give the champion of this proposal a little more resolve the run the python-dev gauntlet on this issue given the recent - err - spirited discussions, I'm also strongly +1 on this idea. Cheers, Mark From martin at v.loewis.de Fri Jul 18 01:27:08 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 18 Jul 2008 01:27:08 +0200 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> Message-ID: <487FD54C.2040308@v.loewis.de> > The Windows buildbots are not very happy, though. test_ssl and > test_bsddb and constantly failing on both the trunk and py3k. I don't > know much about either of these items (or Windows for that matter), so > any help would be greatly appreciated. bsddb is in a very bad shape, as the 2.6 code hasn't been merged into 3k. I somewhat doubt that this gets resolved before the release, so bsddb users might need to skip 3.0. Regards, Martin From musiccomposition at gmail.com Fri Jul 18 02:06:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 17 Jul 2008 19:06:37 -0500 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <487FD54C.2040308@v.loewis.de> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> Message-ID: <1afaf6160807171706p8c2b226wc7bd8d0a072840fa@mail.gmail.com> On Thu, Jul 17, 2008 at 6:27 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote: >> The Windows buildbots are not very happy, though. test_ssl and >> test_bsddb and constantly failing on both the trunk and py3k. I don't >> know much about either of these items (or Windows for that matter), so >> any help would be greatly appreciated. > > bsddb is in a very bad shape, as the 2.6 code hasn't been merged into > 3k. I somewhat doubt that this gets resolved before the release, so > bsddb users might need to skip 3.0. Ok. ssl has stopped failing now (Thanks Bill). test_wsgiref is failing on the trunk, but as it is an externally maintained module, there isn't much I can do about it. (see issue #3401). Overall, though, I think we've done what we can here, so these shouldn't hold up the release. > > Regards, > Martin > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From barry at python.org Fri Jul 18 04:13:33 2008 From: barry at python.org (barry at python.org) Date: Thu, 17 Jul 2008 22:13:33 -0400 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <487FD54C.2040308@v.loewis.de> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> Message-ID: <20080717221333.1d61bfe1@mission.wooz.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 18, 2008, at 01:27 AM, Martin v. L?wis wrote: >> The Windows buildbots are not very happy, though. test_ssl and >> test_bsddb and constantly failing on both the trunk and py3k. I don't >> know much about either of these items (or Windows for that matter), so >> any help would be greatly appreciated. > >bsddb is in a very bad shape, as the 2.6 code hasn't been merged into >3k. I somewhat doubt that this gets resolved before the release, so >bsddb users might need to skip 3.0. We have no showstoppers and several green buildbots, so I'm going to make the releases tonight. Please, NO CHECKINS until I say so, or ping me on #python-dev. As for bsddb, we'll make a determination after beta3. If it's terminally busted for Python 3.0, so be it. Thanks everyone for working so hard at getting beta2 ready. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIf/xT2YZpQepbvXERAtmoAKCyUkejAFxFG10Bsn/CJjZfGy0B9QCeMO0z momJmXRLCdmxs84j2hXGrTY= =Y3wS -----END PGP SIGNATURE----- From barry at python.org Fri Jul 18 04:16:38 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 17 Jul 2008 22:16:38 -0400 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <487F4FAE.3050600@holdenweb.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <487F4FAE.3050600@holdenweb.com> Message-ID: <20080717221638.226ae4ee@mission.wooz.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 17, 2008, at 09:57 AM, Steve Holden wrote: >Barry Warsaw wrote: >[...] >> >> I'll note that I plan to hold the beta3 releases until all release >> blocker and deferred blockers are resolved. >> >Ian Ozsvald of ShowMeDo.com noticed a blog post of mine about this beta, >and responded as follows: > >> Re. http://holdenweb.blogspot.com/2008/07/cpython-getting-serious-about-quality.html >> If someone on the dev side wanted to make a 5-10 minute 'cast on how to check out the python base, build it, test it, check stuff back in, we'd be honoured to host it. Obviously the entire process deserves more attention but a short overview pointing out the key steps would be: >> 1) simple for the author to create >> 2) easy for possible contributors to watch and receive inspiration >> 3) powerful enough to attract another set of developers into the fold >> so that might be a useful, low-cost, quick win to help with the current effort? > >I'd do this myself but a) I'm not the obvious candidate and b) I'm too >busy to give it my attention now. > >If someone wanted to tackle this, it might help getting people testing >for the next beta, and also to recruit more testers in the long-term. It's a great idea, but unfortunately I also don't have time to do this right now. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIf/0G2YZpQepbvXERAtNzAJ9Os117wAD8KCkhkt6x+jhKuJ4snwCfbjXS VtkFGzUUTzg+ersqU3+TObg= =jdUf -----END PGP SIGNATURE----- From fdrake at acm.org Fri Jul 18 04:30:38 2008 From: fdrake at acm.org (Fred Drake) Date: Thu, 17 Jul 2008 22:30:38 -0400 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <487FD54C.2040308@v.loewis.de> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> Message-ID: <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: > bsddb is in a very bad shape, as the 2.6 code hasn't been merged into > 3k. I somewhat doubt that this gets resolved before the release, so > bsddb users might need to skip 3.0. In fact, bsddb as packages in core Python has rarely been in good shape. Has anyone actually considered that bsddb might do better if maintained strictly as an external package? That would make it easier for the any maintainers to release updates, and removes a source of frustration for users who either don't need it or would rather wait for a happier version. I think this is worth considering. I vaguely recall that the bsddb module was maintained before it was incorporated into the core Python release. -Fred -- Fred Drake <fdrake at acm.org> From guido at python.org Fri Jul 18 04:37:32 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 19:37:32 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> Message-ID: <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: > On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: >> >> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into >> 3k. I somewhat doubt that this gets resolved before the release, so >> bsddb users might need to skip 3.0. > > > In fact, bsddb as packages in core Python has rarely been in good shape. > > Has anyone actually considered that bsddb might do better if maintained > strictly as an external package? That would make it easier for the any > maintainers to release updates, and removes a source of frustration for > users who either don't need it or would rather wait for a happier version. > > I think this is worth considering. I vaguely recall that the bsddb module > was maintained before it was incorporated into the core Python release. +1. In my recollection maintaining bsddb has been nothing but trouble right from the start when we were all sitting together at "Zope Corp North" in a rented office in McLean... We can remove it from 3.0. We can't really remove it from 2.6, but we can certainly start end-of-lifing it in 2.6. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ranjithkannikara at gmail.com Fri Jul 18 04:40:58 2008 From: ranjithkannikara at gmail.com (ranjith kannikara) Date: Fri, 18 Jul 2008 08:10:58 +0530 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> <20080717202531.DA16A3A4080@sparrow.telecommunity.com> <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com> <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com> <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com> Message-ID: <20aa8c370807171940w1710ae44ga986f6f5f8a8bd85@mail.gmail.com> Hi , I am really sorry that I couldn,t make clear in the first mail that What I actually need. But Shane and my Gsoc mentor Sidnei have already made it clear I think. I would like to know how much the AST have been changed on moving from python2.4 to python 2.5 so that I can bring the corresponding changes in Zope2 to get it adapted to those changes in the AST. Also the changes that have come to the builtin function and so which should have affected the safety of a RestrictedPython sandbox. As Shane have told a detailed list of such changes can help me than the "Whats New" doc... Ranjith. On Fri, Jul 18, 2008 at 2:43 AM, Benjamin Peterson <musiccomposition at gmail.com> wrote: > On Thu, Jul 17, 2008 at 4:10 PM, Sidnei da Silva > <sidnei at enfoldsystems.com> wrote: >> On Thu, Jul 17, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote: >>> Thanks. Then python-dev is *definitely* the wrong forum. :-) >> >> Definitely wrong forum for RestrictedPython-specifc questions, but not >> if you consider those two paragraphs from Shane's email, which clarify >> that Ranjith is seeking for information on possible AST changes that >> might have occurred in Python 2.5: >> >> """ >> The safety of RestrictedPython has been validated in a somewhat formal >> process with Python 2.4. Ranjith is working to validate it with >> Python 2.5. He is first working to discover all changes between >> Python 2.4 and 2.5 that might have affected the safety of a >> RestrictedPython sandbox. Any changes to the AST, builtin functions, >> methods of builtin types, etc., need to be evaluated for safety. >> >> So, in general, he is looking for detailed lists of changes between >> Python 2.4 and 2.5--more than the "What's New" doc. >> """ > > Then I would recommend he look at Misc/NEWS. If he needs more > information, *specific* questions will bring the best results. > > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > From barry at python.org Fri Jul 18 04:51:06 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 17 Jul 2008 22:51:06 -0400 Subject: [Python-Dev] NO CHECKINS - BETA 2 COMING Message-ID: <8914AB44-1B5D-45B9-9EB9-DDC728B1B032@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Please, no checkins on the 3.0 or 2.6 branches until further notice. We're a go with the releases tonight. Email is not the quickest way to get my attention. For that, use irc on freenode, #python-dev. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSIAFGnEjvBPtnXfVAQIBmwP/VzIYu58DzSND0CNM0NKfsWVS9yTnBVi8 EijaxenUv0rRVloCawJbK8cMOyI0JmEa4GE5bMrCf+o9niRgFeJxrYEaw1jN4fWW dsXH8Fuw/nHpoUnFDbu5ePaBdvcOSuWKTS/iGq2i9DzfYXdx+FaXqZ/mhx43bY3o eCF7I+y/fPg= =iTgT -----END PGP SIGNATURE----- From barry at python.org Fri Jul 18 04:52:40 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 17 Jul 2008 22:52:40 -0400 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> Message-ID: <EA2CA734-E0D1-49AB-8E14-1AE30082F971@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 17, 2008, at 10:37 PM, Guido van Rossum wrote: > On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: >> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: >>> >>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged >>> into >>> 3k. I somewhat doubt that this gets resolved before the release, so >>> bsddb users might need to skip 3.0. >> >> >> In fact, bsddb as packages in core Python has rarely been in good >> shape. >> >> Has anyone actually considered that bsddb might do better if >> maintained >> strictly as an external package? That would make it easier for the >> any >> maintainers to release updates, and removes a source of frustration >> for >> users who either don't need it or would rather wait for a happier >> version. >> >> I think this is worth considering. I vaguely recall that the bsddb >> module >> was maintained before it was incorporated into the core Python >> release. > > +1. In my recollection maintaining bsddb has been nothing but trouble > right from the start when we were all sitting together at "Zope Corp > North" in a rented office in McLean... We can remove it from 3.0. We > can't really remove it from 2.6, but we can certainly start > end-of-lifing it in 2.6. +1, but please, after I make tonight's releases. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSIAFenEjvBPtnXfVAQKEewP+OWCBAH437X4+EptdcuIFFYrCCVXqbrV4 F2dZMyv/RU0jYgd6YTLspklEIzuEcH1sxYsnh2q4aWNfFlfL50LPf1/TKurZpO2C 9CrjEZpTcK0k5z5FCxAOLdVFLQDC4w7FGYop3uR27g5u9KCLdQvTrPs/mZllv263 /g2YdwhZFEE= =NAOe -----END PGP SIGNATURE----- From barry at python.org Fri Jul 18 05:42:31 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 17 Jul 2008 23:42:31 -0400 Subject: [Python-Dev] RELEASED Python 2.6b2 and 3.0b2 Message-ID: <E9E66405-B49F-479C-A9E4-47F48FBF1AA5@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I am happy to announce the second beta releases of Python 2.6 and Python 3.0. Please note that these are beta releases, and as such are not suitable for production environments. We continue to strive for a high degree of quality, and these releases are intended to freeze the feature set for Python 2.6 and 3.0. From now until the planned final releases in October 2008, we will be fixing known problems and stabilizing these new Python versions. You can help by downloading and testing them, providing feedback and hopefully helping to fix bugs. You can also use these releases to determine how changes in 2.6 and 3.0 might impact you. ONLY ONE MORE BETA RELEASE IS PLANNED, so now is a great time to download the releases and try them with your code. If you find things broken or incorrect, please submit bug reports at http://bugs.python.org For more information and downloadable distributions, see the Python 2.6 website: http://www.python.org/download/releases/2.6/ and the Python 3.0 web site: http://www.python.org/download/releases/3.0/ See PEP 361 for release schedule details: http://www.python.org/dev/peps/pep-0361/ Enjoy, - -Barry Barry Warsaw barry at python.org Python 2.6/3.0 Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSIARKHEjvBPtnXfVAQL7AQQAjPSbfKz2Oh/au/hPzS4x2IR5/R6FVe+g o9UYrONNRKJ14UHpbZRzvIvw/4G3PdpzzGxjYFIhVGEesEGJnMzT3YdkMHt4NW9d HOZxL3hseGbTdpUJPCsIkNG+4hX7iuY3NSV81Z75LGAL4tqbooGqwwUslXMT5f8s lRrZUcBRKZ0= =ju6s -----END PGP SIGNATURE----- From barry at python.org Fri Jul 18 05:50:24 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 17 Jul 2008 23:50:24 -0400 Subject: [Python-Dev] SVN IS OPEN Message-ID: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The releases have been made, so both the 3.0 branch and the trunk (2.6) are now open for commits. Remember, there's only one more planned beta, and we /really/ want to try to hit the October 1st deadline. Let's do everything we can to stabilize these releases, address the blockers, and turn those buildbots green. Thanks everyone. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSIATAHEjvBPtnXfVAQIWLAP+OVUXh3H5j8WN4cm3fHcLkgd+E4FuWWwb zKr7JPYUHhgvwpYB5iZjQuJe/62qKXh70wNWyDcxLi9/TEbx8NvhQ+nMbHAJkfE2 1/HkP9bbgiwE/JYJsN0a0DK/MSpwvxfxxakQrQV9H8SiL5aDqlVCFdzLu31SkOhx iQ+iTCqwIdY= =UOPo -----END PGP SIGNATURE----- From guido at python.org Fri Jul 18 06:05:35 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 17 Jul 2008 21:05:35 -0700 Subject: [Python-Dev] SVN IS OPEN In-Reply-To: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org> References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org> Message-ID: <ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com> On Thu, Jul 17, 2008 at 8:50 PM, Barry Warsaw <barry at python.org> wrote: > The releases have been made, so both the 3.0 branch and the trunk (2.6) are > now open for commits. Remember, there's only one more planned beta, and we > /really/ want to try to hit the October 1st deadline. Let's do everything > we can to stabilize these releases, address the blockers, and turn those > buildbots green. Thanks Barry! Indeed, I want everyone to focus on stabilization and release blockers (and the occasional speed-up). Be extra careful with what you submit between now and October, ask for a code review unless you're really sure. Also, remember, NO NEW FEATURES! > Thanks everyone. > - -Barry And thanks Barry for doing the release!!! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Fri Jul 18 06:24:12 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 18 Jul 2008 06:24:12 +0200 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <20aa8c370807171940w1710ae44ga986f6f5f8a8bd85@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <bbaeab100807171127q140019fdj32101528a5ce11e0@mail.gmail.com> <20080717202531.DA16A3A4080@sparrow.telecommunity.com> <ca471dc20807171404t39e4c0b3wc21d65594026d2bf@mail.gmail.com> <a7a2b76b0807171410i45416d8fseb628ddc7d453387@mail.gmail.com> <1afaf6160807171413r28e15fbdt4649286bc80f22d8@mail.gmail.com> <20aa8c370807171940w1710ae44ga986f6f5f8a8bd85@mail.gmail.com> Message-ID: <48801AEC.3090606@v.loewis.de> > I would like to know how much the AST have been changed on moving from > python2.4 to python 2.5 so that I can bring the corresponding changes > in Zope2 to get it adapted to those changes in the AST. Notice that Parser/Python.asdl is new in Python 2.5, so (by today's terminology) Python 2.4 did not really have any AST. The relevant NEWS entry is - A new AST parser implementation was completed. The abstract syntax tree is available for read-only (non-compile) access to Python code; an _ast module was added. Python's prior notion of abstract syntax (which is rather a CST) directly reflects from the grammar; do svn log Grammar/Grammar to find out what exactly was changed. Regards, Martin From brett at python.org Fri Jul 18 07:43:15 2008 From: brett at python.org (Brett Cannon) Date: Thu, 17 Jul 2008 22:43:15 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> Message-ID: <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote: > On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: >> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: >>> >>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into >>> 3k. I somewhat doubt that this gets resolved before the release, so >>> bsddb users might need to skip 3.0. >> >> >> In fact, bsddb as packages in core Python has rarely been in good shape. >> >> Has anyone actually considered that bsddb might do better if maintained >> strictly as an external package? That would make it easier for the any >> maintainers to release updates, and removes a source of frustration for >> users who either don't need it or would rather wait for a happier version. >> >> I think this is worth considering. I vaguely recall that the bsddb module >> was maintained before it was incorporated into the core Python release. > > +1. In my recollection maintaining bsddb has been nothing but trouble > right from the start when we were all sitting together at "Zope Corp > North" in a rented office in McLean... We can remove it from 3.0. We > can't really remove it from 2.6, but we can certainly start > end-of-lifing it in 2.6. > Unless I hear otherwise, I will add it to PEP 3108. -Brett From jml at mumak.net Fri Jul 18 07:50:06 2008 From: jml at mumak.net (Jonathan Lange) Date: Fri, 18 Jul 2008 15:50:06 +1000 Subject: [Python-Dev] assertRaises In-Reply-To: <03e501c8e85f$b07b8000$11728000$@com.au> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <A8D00EB1-8888-4A90-B4A5-016B82AB2AD5@acm.org> <loom.20080717T091112-655@post.gmane.org> <loom.20080717T095323-385@post.gmane.org> <e8a0972d0807170807u5f6e5872x734c1b7b7289d27f@mail.gmail.com> <03e501c8e85f$b07b8000$11728000$@com.au> Message-ID: <d06a5cd30807172250s20b94221tf4fc2cec2a98e27f@mail.gmail.com> On Fri, Jul 18, 2008 at 8:51 AM, Mark Hammond <mhammond at skippinet.com.au> wrote: >> >> Let's just make assertRaises return the exception instance, it seems >> >> like it feels the need correctly. >> > >> > and I meant "fills", not "feels", obviously... >> >> +1 : enriching the existing method in a way that's perfectly >> transparent and innocuous to all existing uses _feels_ right, because >> it's more practical. > > For the record, and primarily to give the champion of this proposal a little > more resolve the run the python-dev gauntlet on this issue given the recent > - err - spirited discussions, I'm also strongly +1 on this idea. > I've got code and tests for this. I'll file a patch on the tracker as soon as I get the chance -- Internet is dicey at the moment. jml From brett at python.org Fri Jul 18 07:52:06 2008 From: brett at python.org (Brett Cannon) Date: Thu, 17 Jul 2008 22:52:06 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <g5j4v3$6hi$1@ger.gmane.org> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> Message-ID: <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> On Tue, Jul 15, 2008 at 2:31 PM, Neil Schemenauer <nas at arctrix.com> wrote: > Benjamin Peterson <musiccomposition at gmail.com> wrote: >> Can we push branches? > > The git-daemon is setup as read-only. If you have write access to > the SVN repository then you can push back changes using git-svn. > That's quite a nice way to work, IMHO and provides an easy path for > people who are used to Subversion and want to dip their toe in the > dvcs waters. > > While there is no support on code.python.org for publishing your own > git branches, there's no reason why you couldn't publish a branch > using some other host (it's a dvcs after all). In the short term, > perhaps using private branches in combination with "git > format-patch" and email is the easiest process. > So I have a change that I have in git. I would like to just push it to svn through git. I ran ``git svn dcommit``, but then a ton of stuff came streaming by on my terminal. Is that normal? Also, how does one refer to revisions for things like ``git diff``? Is it really the commit commit number (which I think is something like a SHA-1)? -Brett From nas at arctrix.com Fri Jul 18 08:54:30 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Fri, 18 Jul 2008 00:54:30 -0600 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> Message-ID: <20080718065430.GA6991@arctrix.com> [back on the list] On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote: > Turned out to be a rebuild:: > > .... > r65077 = 82d954e8c20c91562c4c660859d17756cba10992 > r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a > r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536 > Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771 > > How do I know what is going to be sent? ``git log`` seems to suggest > something by not listing a git-svn-id for my last commit, but is that > really the best I got? The command git log git-svn.. will show you changes in your HEAD (by default "master") tree that are not in the remote tree (git-svn). > Is there some other way to see what will be pushed? I like running "gitk" before I push something. > And how do I diff easily between commits? It depends on what you want, exactly. Maybe you can describe some use cases. A DVCS can't use identify revisions like SVN does. Generally I find myself using heads or tags to identify versions in combination with the ^ operator. For example, git diff HEAD^ would show the difference between the current working tree and the commit before the head of the stored tree. If you want the patch for a single commit, use "git show <object>". For example, "git show" will display the last commit. To see amk's typo fix: git show 6cadb9c1b7e30a8b66cdba01cd79aa6397a07080 You can also abbreviate the commit id, eg. git show 6cadb9 As I say in my guide, "git format-patch" and "git am" are very handy when slinging patches around (e.g. to and from a bug tracker or mailing list). HTH, Neil From theller at ctypes.org Fri Jul 18 09:14:35 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 18 Jul 2008 09:14:35 +0200 Subject: [Python-Dev] Windows buildbot trick Message-ID: <g5pfrp$d9e$1@ger.gmane.org> The most annoying thing on the windows buildbots is when Python crashes hard (with a general protection fault or stack overflow, for example). Usually the system pops up a dialog in this case which allows to attach a debugger to the process. This dialog will stay open until the maintainer manually closes it, and will prevent the next builds. On my boxes I have inserted these two lines into the PythonXY/Scripts/buildbot script, right at the top: import win32api; win32api.SetErrorMode(7) print "SetErrorMode(7) called." These lines prevent the dialog box to be displayed for critical errors. For example, on Windows AMD64 the test_cpickle test in the trunk currently fails with a stack overflow; instead of hanging with the mentioned dialog box open the test now terminates: http://www.python.org/dev/buildbot/trunk/amd64%20XP%20trunk/builds/40/step-test/0 -- Thanks, Thomas From solipsis at pitrou.net Fri Jul 18 11:46:03 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 18 Jul 2008 09:46:03 +0000 (UTC) Subject: [Python-Dev] SVN IS OPEN References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org> <ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com> Message-ID: <loom.20080718T094343-311@post.gmane.org> Guido van Rossum <guido <at> python.org> writes: > > Thanks Barry! Indeed, I want everyone to focus on stabilization and > release blockers (and the occasional speed-up). Be extra careful with > what you submit between now and October, ask for a code review unless > you're really sure. Also, remember, NO NEW FEATURES! What should be done about http://bugs.python.org/issue2834? Should it make it into 3.0 or be delayed until 3.1? (code-wise there isn't left to be done, take a decision about a potential "(? a)" modifier and then implement it) From ncoghlan at gmail.com Fri Jul 18 12:24:56 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 18 Jul 2008 20:24:56 +1000 Subject: [Python-Dev] SVN IS OPEN In-Reply-To: <loom.20080718T094343-311@post.gmane.org> References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org> <ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com> <loom.20080718T094343-311@post.gmane.org> Message-ID: <48806F78.4060209@gmail.com> Antoine Pitrou wrote: > Guido van Rossum <guido <at> python.org> writes: >> Thanks Barry! Indeed, I want everyone to focus on stabilization and >> release blockers (and the occasional speed-up). Be extra careful with >> what you submit between now and October, ask for a code review unless >> you're really sure. Also, remember, NO NEW FEATURES! > > What should be done about http://bugs.python.org/issue2834? > Should it make it into 3.0 or be delayed until 3.1? > > (code-wise there isn't left to be done, take a decision about a potential "(? > a)" modifier and then implement it) Before beta 3, I think if we need minor features (such as re.ASCII) to fix fairly major bugs (re.IGNORECASE not working properly by default in Py3k) then they should probably go in (case by case permission still needed from Barry or Guido though, as with any post-beta1 feature addition). Once we're past beta 3 and heading for rc1 then the bar for approving feature additions to fix bugs will move even higher though. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From solipsis at pitrou.net Fri Jul 18 12:39:57 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 18 Jul 2008 10:39:57 +0000 (UTC) Subject: [Python-Dev] SVN IS OPEN References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org> <ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com> <loom.20080718T094343-311@post.gmane.org> <48806F78.4060209@gmail.com> Message-ID: <loom.20080718T103811-83@post.gmane.org> Nick Coghlan <ncoghlan <at> gmail.com> writes: > > Before beta 3, I think if we need minor features (such as re.ASCII) to > fix fairly major bugs (re.IGNORECASE not working properly by default in > Py3k) then they should probably go in (case by case permission still > needed from Barry or Guido though, as with any post-beta1 feature addition). Ok I'll tackle the remaining of it before beta 3 then. Thanks Antoine. From guido at python.org Fri Jul 18 16:13:14 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Jul 2008 07:13:14 -0700 Subject: [Python-Dev] SVN IS OPEN In-Reply-To: <48806F78.4060209@gmail.com> References: <B34131B7-30CE-44B9-9B45-7BB71F9562E2@python.org> <ca471dc20807172105u50b686aya2c7be22679b9690@mail.gmail.com> <loom.20080718T094343-311@post.gmane.org> <48806F78.4060209@gmail.com> Message-ID: <ca471dc20807180713h4177eacese3b8b9821164ce7a@mail.gmail.com> On Fri, Jul 18, 2008 at 3:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: > Antoine Pitrou wrote: >> >> Guido van Rossum <guido <at> python.org> writes: >>> >>> Thanks Barry! Indeed, I want everyone to focus on stabilization and >>> release blockers (and the occasional speed-up). Be extra careful with >>> what you submit between now and October, ask for a code review unless >>> you're really sure. Also, remember, NO NEW FEATURES! >> >> What should be done about http://bugs.python.org/issue2834? >> Should it make it into 3.0 or be delayed until 3.1? >> >> (code-wise there isn't left to be done, take a decision about a potential >> "(? >> a)" modifier and then implement it) > > Before beta 3, I think if we need minor features (such as re.ASCII) to fix > fairly major bugs (re.IGNORECASE not working properly by default in Py3k) > then they should probably go in (case by case permission still needed from > Barry or Guido though, as with any post-beta1 feature addition). > > Once we're past beta 3 and heading for rc1 then the bar for approving > feature additions to fix bugs will move even higher though. Right. Once b3 is out the bar will be *really* high. Once rc1 is out the bar will be raised even for bug fixes! So do this now. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jul 18 16:21:45 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Jul 2008 07:21:45 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> Message-ID: <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote: > On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote: >> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: >>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: >>>> >>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into >>>> 3k. I somewhat doubt that this gets resolved before the release, so >>>> bsddb users might need to skip 3.0. >>> >>> >>> In fact, bsddb as packages in core Python has rarely been in good shape. >>> >>> Has anyone actually considered that bsddb might do better if maintained >>> strictly as an external package? That would make it easier for the any >>> maintainers to release updates, and removes a source of frustration for >>> users who either don't need it or would rather wait for a happier version. >>> >>> I think this is worth considering. I vaguely recall that the bsddb module >>> was maintained before it was incorporated into the core Python release. >> >> +1. In my recollection maintaining bsddb has been nothing but trouble >> right from the start when we were all sitting together at "Zope Corp >> North" in a rented office in McLean... We can remove it from 3.0. We >> can't really remove it from 2.6, but we can certainly start >> end-of-lifing it in 2.6. > Unless I hear otherwise, I will add it to PEP 3108. Please do! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From josiah.carlson at gmail.com Fri Jul 18 16:57:01 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 18 Jul 2008 07:57:01 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> Message-ID: <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> On Fri, Jul 18, 2008 at 7:21 AM, Guido van Rossum <guido at python.org> wrote: > On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote: >> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote: >>> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: >>>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: >>>>> >>>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into >>>>> 3k. I somewhat doubt that this gets resolved before the release, so >>>>> bsddb users might need to skip 3.0. >>>> >>>> >>>> In fact, bsddb as packages in core Python has rarely been in good shape. >>>> >>>> Has anyone actually considered that bsddb might do better if maintained >>>> strictly as an external package? That would make it easier for the any >>>> maintainers to release updates, and removes a source of frustration for >>>> users who either don't need it or would rather wait for a happier version. >>>> >>>> I think this is worth considering. I vaguely recall that the bsddb module >>>> was maintained before it was incorporated into the core Python release. >>> >>> +1. In my recollection maintaining bsddb has been nothing but trouble >>> right from the start when we were all sitting together at "Zope Corp >>> North" in a rented office in McLean... We can remove it from 3.0. We >>> can't really remove it from 2.6, but we can certainly start >>> end-of-lifing it in 2.6. > >> Unless I hear otherwise, I will add it to PEP 3108. > > Please do! Invariably, when someone goes and removes a module, someone else is going to complain, "but I used feature X, not having feature X will break my code." We, as maintainers can then say, "if you cared, maintain it." But I'm not sure that is the greatest thing to tell people. I suspect that we may have to include some sort of "work-alike" for 2.7 and if not 3.0, 3.1 . If I were to vote for a work-alike, it would be based on sqlite. For one of the most common use-cases (bsddb.btree), simple sqlite code can be written to do the right thing. Recno is a little more tricky, but can also be done. The bsddb hash may not be possible, because sqlite doesn't support hashed indices :/. Just an idea. - Josiah From guido at python.org Fri Jul 18 17:11:06 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Jul 2008 08:11:06 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> Message-ID: <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> On Fri, Jul 18, 2008 at 7:57 AM, Josiah Carlson <josiah.carlson at gmail.com> wrote: > Invariably, when someone goes and removes a module, someone else is > going to complain, "but I used feature X, not having feature X will > break my code." We, as maintainers can then say, "if you cared, > maintain it." But I'm not sure that is the greatest thing to tell > people. I suspect that we may have to include some sort of > "work-alike" for 2.7 and if not 3.0, 3.1 . If I were to vote for a > work-alike, it would be based on sqlite. For one of the most common > use-cases (bsddb.btree), simple sqlite code can be written to do the > right thing. Recno is a little more tricky, but can also be done. > The bsddb hash may not be possible, because sqlite doesn't support > hashed indices :/. In my mind, BSDDB is pretty much the most heavy-weight extension we're maintaining. I think it's an illusion that a sqlite-based look-alike is going to fool anyone. The correct solution is to take support for bsddb to a separate project where those who care about it can maintain it together. That also makes it a lot easier to track the versions of Berkeley DB as they come out. Of course, you're free to try writing the work-alike you're proposing. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From sidnei at enfoldsystems.com Fri Jul 18 17:42:11 2008 From: sidnei at enfoldsystems.com (Sidnei da Silva) Date: Fri, 18 Jul 2008 12:42:11 -0300 Subject: [Python-Dev] Windows buildbot trick In-Reply-To: <g5pfrp$d9e$1@ger.gmane.org> References: <g5pfrp$d9e$1@ger.gmane.org> Message-ID: <a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com> On Fri, Jul 18, 2008 at 4:14 AM, Thomas Heller <theller at ctypes.org> wrote: > The most annoying thing on the windows buildbots is when Python crashes hard > (with a general protection fault or stack overflow, for example). > Usually the system pops up a dialog in this case which allows to > attach a debugger to the process. This dialog will stay open until > the maintainer manually closes it, and will prevent the next builds. > > On my boxes I have inserted these two lines into the PythonXY/Scripts/buildbot > script, right at the top: > > import win32api; win32api.SetErrorMode(7) > print "SetErrorMode(7) called." > > These lines prevent the dialog box to be displayed for critical errors. > > For example, on Windows AMD64 the test_cpickle test in the trunk currently fails with > a stack overflow; instead of hanging with the mentioned dialog box open the test now terminates: > > http://www.python.org/dev/buildbot/trunk/amd64%20XP%20trunk/builds/40/step-test/0 That's a great trick! I seem to remember that there is a way to turn that off globally though, but not sure where. Maybe running drwtsn32.exe and un-checking 'Visual Notification' does the trick? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 From sidnei at enfoldsystems.com Fri Jul 18 17:53:08 2008 From: sidnei at enfoldsystems.com (Sidnei da Silva) Date: Fri, 18 Jul 2008 12:53:08 -0300 Subject: [Python-Dev] Windows buildbot trick In-Reply-To: <a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com> References: <g5pfrp$d9e$1@ger.gmane.org> <a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com> Message-ID: <a7a2b76b0807180853g524677cbk578f322b049010cc@mail.gmail.com> On Fri, Jul 18, 2008 at 12:42 PM, Sidnei da Silva <sidnei at enfoldsystems.com> wrote: > That's a great trick! I seem to remember that there is a way to turn > that off globally though, but not sure where. Maybe running > drwtsn32.exe and un-checking 'Visual Notification' does the trick? FWIW, here's a kb article that describes this. Although it refers to 32-bit apps on 64-bit XP, I'm pretty sure you can do the same for 32-bit XP? http://support.microsoft.com/kb/283150/EN-US/ -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 From status at bugs.python.org Fri Jul 18 18:07:19 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 18 Jul 2008 18:07:19 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080718160719.ECE9778282@psf.upfronthosting.co.za> ACTIVITY SUMMARY (07/11/08 - 07/18/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1967 open (+38) / 13262 closed (+25) / 15229 total (+63) Open issues with patches: 619 Average duration of open issues: 705 days. Median duration of open issues: 1617 days. Open Issues Breakdown open 1939 (+38) pending 28 ( +0) Issues Created Or Reopened (63) _______________________________ IDLE - use enumerate instead of zip(count(), ...) 07/11/08 http://bugs.python.org/issue3344 created taleinat patch Patch for CGIHTTPServer.is_cgi function documentation 07/11/08 CLOSED http://bugs.python.org/issue3345 created tebeka patch test_ossaudiodev fails 07/12/08 http://bugs.python.org/issue3346 created cartman urllib.robotparser doesn't work in Py3k 07/12/08 http://bugs.python.org/issue3347 created mgiuca patch Cannot start wsgiref simple server in Py3k 07/12/08 http://bugs.python.org/issue3348 created mgiuca patch search for 'patch' produces roundup error 07/12/08 CLOSED http://bugs.python.org/issue3349 created techtonik multiprocessing adds built-in types to the global copyreg.dispat 07/13/08 http://bugs.python.org/issue3350 created alexandre.vassalotti Python crashed 07/14/08 CLOSED http://bugs.python.org/issue3351 created yiyuan Deficiencies in multiprocessing/threading API 07/14/08 http://bugs.python.org/issue3352 created ncoghlan make built-in tokenizer available via Python C API 07/14/08 http://bugs.python.org/issue3353 created effbot Improve error reporting for the argument parsing API 07/14/08 http://bugs.python.org/issue3354 reopened benjamin.peterson Display bug in :show-inheritance: for class with standard docstr 07/14/08 http://bugs.python.org/issue3355 created kumar303 some tests fail in debug mode (test_distutils, test_set) 07/14/08 http://bugs.python.org/issue3356 created pitrou A bug in the __doc__ string of the sys module 07/14/08 CLOSED http://bugs.python.org/issue3357 created cheDu 2to3 Iterative Wildcard Matching 07/15/08 http://bugs.python.org/issue3358 created nedds patch add 'rbU' mode to open() 07/15/08 CLOSED http://bugs.python.org/issue3359 created techtonik Inconsistent type-deduction of decimal floating-point 07/15/08 CLOSED http://bugs.python.org/issue3360 created richyk pypi issue tracker link (python.org) 07/15/08 CLOSED http://bugs.python.org/issue3361 created tds333 locale.getpreferredencoding() gives bus error on Mac OS X 10.4.1 07/15/08 http://bugs.python.org/issue3362 created cfr python version incorrectly reported in crash reports on Mac OS X 07/15/08 http://bugs.python.org/issue3363 created cfr An ortographical typo in Zen of Python text 07/15/08 CLOSED http://bugs.python.org/issue3364 created cheDu in module math, PI value seems to be wrong after digit 17th 07/15/08 CLOSED http://bugs.python.org/issue3365 created TanaT Add gamma and error functions to math module 07/15/08 http://bugs.python.org/issue3366 created tjreedy Uninitialized value read in parsetok.c 07/15/08 http://bugs.python.org/issue3367 created krisvale patch, patch, easy Memory leak in import.c 07/15/08 http://bugs.python.org/issue3368 created krisvale patch, patch, easy memory leak in floatobject.c 07/15/08 http://bugs.python.org/issue3369 created krisvale patch, patch, easy importing with_statement causes exec to raise syntax error on bl 07/15/08 http://bugs.python.org/issue3370 created mccredie 2to3 fails in Python 2.6 07/16/08 CLOSED http://bugs.python.org/issue3371 created benjamin.peterson socket.setsockopt() is broken for multicast TTL and Loop options 07/16/08 CLOSED http://bugs.python.org/issue3372 created niallo sys recursion limit a lot shorter on trunk? 07/16/08 http://bugs.python.org/issue3373 created esrever_otua Bisect upgrades: key/cmp/reverse, parameterized handedness 07/16/08 CLOSED http://bugs.python.org/issue3374 created dan.uznanski patch _multiprocessing.so build problems 07/16/08 CLOSED http://bugs.python.org/issue3375 created barry Use Python 3 lexer for 3.0 docs 07/16/08 http://bugs.python.org/issue3376 created georg.brandl Invalid child node access in ast.c 07/16/08 CLOSED http://bugs.python.org/issue3377 created krisvale Memory leak in pythonrun.c 07/16/08 http://bugs.python.org/issue3378 created krisvale patch, patch Option to not-exit on test 07/16/08 http://bugs.python.org/issue3379 created pupeno patch documentation for ElementTree is unusable 07/16/08 CLOSED http://bugs.python.org/issue3380 created jrw at pobox.com `./configure --enable-framework --enable-universalsdk` fails bec 07/16/08 CLOSED http://bugs.python.org/issue3381 created trentm Make '%F' and float.__format__('F') convert results to upper cas 07/17/08 http://bugs.python.org/issue3382 reopened eric.smith easy ctypes.util fails to find libc in some environments 07/16/08 http://bugs.python.org/issue3383 created exarkun patch Documentation for re.findall and re.finditer lacks "ordering" in 07/16/08 http://bugs.python.org/issue3384 created jkugler cPickle to pickle conversion in py3k missing methods 07/16/08 http://bugs.python.org/issue3385 created jnoller [PATCH] distutils.sysconfig.get_python_lib prefix argument broke 07/16/08 http://bugs.python.org/issue3386 created pjenvey patch undefined array passed to CryptGenRandomBytes 07/16/08 http://bugs.python.org/issue3387 created krisvale patch, patch, easy With keyword not mentioned in Input Output tutorial 07/16/08 CLOSED http://bugs.python.org/issue3388 created segfaulthunter patch [PATCH] Allow custom logging Handlers in logging config files 07/16/08 CLOSED http://bugs.python.org/issue3389 created pjenvey patch [PATCH] replace last has_key in unittest by in operator 07/17/08 http://bugs.python.org/issue3390 created grubert patch Idle uses old showwarning signature 07/17/08 http://bugs.python.org/issue3391 created schuppenies patch subprocess fails in select when descriptors are large 07/17/08 http://bugs.python.org/issue3392 created yorick `cd Mac && make installmacsubtree` fails on Mac OS X < 10.5 beca 07/17/08 CLOSED http://bugs.python.org/issue3393 created trentm zipfile.writestr doesn't set external attributes, so files are e 07/17/08 http://bugs.python.org/issue3394 created swarren patch typo in test_multiprocessing.py: should _debugInfo be _debug_in 07/17/08 CLOSED http://bugs.python.org/issue3395 created marketdickinson easy rlcompleter can't autocomplete members of callable objects 07/17/08 http://bugs.python.org/issue3396 created pitrou Mac 3.0 build cannot find cachersrc.py 07/17/08 CLOSED http://bugs.python.org/issue3397 created barry-scott mac build 3.0 no MacOS module 07/17/08 CLOSED http://bugs.python.org/issue3398 created barry-scott Memory corruption in multiprocessing module, OS X 10.5.4 07/17/08 http://bugs.python.org/issue3399 created marketdickinson dis module: undocumented new bytecodes 07/17/08 http://bugs.python.org/issue3400 created tjreedy wsgiref can't handle unicode environments 07/17/08 http://bugs.python.org/issue3401 created benjamin.peterson test_nis is hanging on Solaris 07/18/08 http://bugs.python.org/issue3402 created benjamin.peterson Unexpected default arguments behaviour 07/18/08 CLOSED http://bugs.python.org/issue3403 created SukkoPera wrong precision in float formatting or doc error 07/18/08 CLOSED http://bugs.python.org/issue3404 created hagen Add support for the new data option supported by event generate 07/18/08 http://bugs.python.org/issue3405 created gpolo patch LocaleTextCalendar and LocaleHTMLCalendar break without a locale 07/18/08 CLOSED http://bugs.python.org/issue3406 created WoLpH Issues Now Closed (67) ______________________ Add a factorial function 121 days http://bugs.python.org/issue2138 georg.brandl parser module chokes on unusual characters 122 days http://bugs.python.org/issue2280 benjamin.peterson patch isinstance is 4x as slow as in 2.5 because the common case raise 119 days http://bugs.python.org/issue2303 gregory.p.smith patch decide what to do with gettext API 107 days http://bugs.python.org/issue2512 benjamin.peterson patch pickling of large recursive structures crashes cPickle 21 days http://bugs.python.org/issue2702 facundobatista patch Clean up undoc.rst 62 days http://bugs.python.org/issue2828 brett.cannon easy Remove plat-mac from 3.0 55 days http://bugs.python.org/issue2910 benjamin.peterson Test_imports fails in 2.6 52 days http://bugs.python.org/issue2969 benjamin.peterson Let bin/oct/hex show floats 21 days http://bugs.python.org/issue3008 marketdickinson patch Windows online help broken when spaces in TEMP environ 41 days http://bugs.python.org/issue3045 georg.brandl patch Add alternate (#) formatting for bin, oct, hex output for str.fo 34 days http://bugs.python.org/issue3083 eric.smith test_multiprocessing hangs intermittently on POSIX platforms 16 days http://bugs.python.org/issue3088 tebeka patch ARCHFLAGS parsing/concatenation in unixccompiler.py breaks when 34 days http://bugs.python.org/issue3090 jnoller patch implement PEP 3134 exception reporting 31 days http://bugs.python.org/issue3112 benjamin.peterson patch os.listdir randomly fails on occasions when it shouldn't 32 days http://bugs.python.org/issue3115 georg.brandl patch sys.getsizeof() gives an AttributeError for _sre objects. 30 days http://bugs.python.org/issue3122 schuppenies patch, patch sqlite leaks on error 23 days http://bugs.python.org/issue3153 alexandre.vassalotti bytes type has inconsistent methods (append, insert) 26 days http://bugs.python.org/issue3156 georg.brandl patch function annotation for builtin and C function 19 days http://bugs.python.org/issue3208 bhy SystemError: Parent module 'foo' not loaded on import statement 16 days http://bugs.python.org/issue3221 schmir can't install on OSX 10.4 18 days http://bugs.python.org/issue3226 benjamin.peterson ctypes assertion failure in trunk 13 days http://bugs.python.org/issue3258 theller Py_CLEAR(tmp) seg faults 10 days http://bugs.python.org/issue3274 alexandre.vassalotti Use Py_XDECREF() instead of Py_DECREF() in MultibyteCodec and Mu 10 days http://bugs.python.org/issue3305 georg.brandl patch Out-of-date example 3.0b1 Tutorial Classes page, 'issubclass' 10 days http://bugs.python.org/issue3310 georg.brandl patch bugs in _sqlite module 9 days http://bugs.python.org/issue3312 georg.brandl patch dlopen() error with no error message from dlerror() 8 days http://bugs.python.org/issue3313 theller patch Proposal for fix_urllib 8 days http://bugs.python.org/issue3316 brett.cannon patch duplicate lines in zipfile.py 5 days http://bugs.python.org/issue3317 alanmcintyre patch Documentation: timeit: "lower bound" should read "upper bound" 9 days http://bugs.python.org/issue3318 georg.brandl various doc typos 4 days http://bugs.python.org/issue3320 benjamin.peterson patch Broken link in online doc 8 days http://bugs.python.org/issue3324 georg.brandl subprocess lib - opening same command fails 4 days http://bugs.python.org/issue3335 benjamin.peterson datetime weekday() function 5 days http://bugs.python.org/issue3336 georg.brandl dummy_thread LockType.acquire() always returns None, should be T 2 days http://bugs.python.org/issue3339 brett.cannon patch Tracebacks are not properly indented 0 days http://bugs.python.org/issue3342 brett.cannon patch Py_DisplaySourceLine is not documented 0 days http://bugs.python.org/issue3343 amaury.forgeotdarc Patch for CGIHTTPServer.is_cgi function documentation 5 days http://bugs.python.org/issue3345 georg.brandl patch search for 'patch' produces roundup error 0 days http://bugs.python.org/issue3349 techtonik Python crashed 0 days http://bugs.python.org/issue3351 loewis A bug in the __doc__ string of the sys module 0 days http://bugs.python.org/issue3357 benjamin.peterson add 'rbU' mode to open() 2 days http://bugs.python.org/issue3359 techtonik Inconsistent type-deduction of decimal floating-point 1 days http://bugs.python.org/issue3360 marketdickinson pypi issue tracker link (python.org) 0 days http://bugs.python.org/issue3361 loewis An ortographical typo in Zen of Python text 2 days http://bugs.python.org/issue3364 goodger in module math, PI value seems to be wrong after digit 17th 0 days http://bugs.python.org/issue3365 marketdickinson 2to3 fails in Python 2.6 0 days http://bugs.python.org/issue3371 benjamin.peterson socket.setsockopt() is broken for multicast TTL and Loop options 2 days http://bugs.python.org/issue3372 gpolo Bisect upgrades: key/cmp/reverse, parameterized handedness 0 days http://bugs.python.org/issue3374 rhettinger patch _multiprocessing.so build problems 1 days http://bugs.python.org/issue3375 gvanrossum Invalid child node access in ast.c 1 days http://bugs.python.org/issue3377 jhylton documentation for ElementTree is unusable 0 days http://bugs.python.org/issue3380 facundobatista `./configure --enable-framework --enable-universalsdk` fails bec 2 days http://bugs.python.org/issue3381 trentm With keyword not mentioned in Input Output tutorial 0 days http://bugs.python.org/issue3388 georg.brandl patch [PATCH] Allow custom logging Handlers in logging config files 1 days http://bugs.python.org/issue3389 vsajip patch `cd Mac && make installmacsubtree` fails on Mac OS X < 10.5 beca 1 days http://bugs.python.org/issue3393 trentm typo in test_multiprocessing.py: should _debugInfo be _debug_in 0 days http://bugs.python.org/issue3395 jnoller easy Mac 3.0 build cannot find cachersrc.py 0 days http://bugs.python.org/issue3397 benjamin.peterson mac build 3.0 no MacOS module 0 days http://bugs.python.org/issue3398 benjamin.peterson Unexpected default arguments behaviour 0 days http://bugs.python.org/issue3403 georg.brandl wrong precision in float formatting or doc error 0 days http://bugs.python.org/issue3404 georg.brandl LocaleTextCalendar and LocaleHTMLCalendar break without a locale 0 days http://bugs.python.org/issue3406 gpolo rlcompleter add "(" to callables feature 2535 days http://bugs.python.org/issue449227 pitrou patch, easy re.split emptyok flag (fix for #852532) 1467 days http://bugs.python.org/issue988761 georg.brandl patch Move firewall warning to "about" menu 716 days http://bugs.python.org/issue1529018 taleinat patch Sloppy error checking in listdir() for Posix 590 days http://bugs.python.org/issue1608818 georg.brandl patch, easy robotparser.py fixes 327 days http://bugs.python.org/issue1778443 benjamin.peterson patch Top Issues Most Discussed (10) ______________________________ 18 threading module can deadlock after fork 1650 days open http://bugs.python.org/issue874900 13 _multiprocessing.so build problems 1 days closed http://bugs.python.org/issue3375 11 `./configure --enable-framework --enable-universalsdk` fails be 2 days closed http://bugs.python.org/issue3381 10 sys recursion limit a lot shorter on trunk? 3 days open http://bugs.python.org/issue3373 9 locale.getpreferredencoding() gives bus error on Mac OS X 10.4. 3 days open http://bugs.python.org/issue3362 9 Deficiencies in multiprocessing/threading API 4 days open http://bugs.python.org/issue3352 9 Proposal for fix_urllib 8 days closed http://bugs.python.org/issue3316 9 Let bin/oct/hex show floats 21 days closed http://bugs.python.org/issue3008 8 Memory corruption in multiprocessing module, OS X 10.5.4 1 days open http://bugs.python.org/issue3399 8 2to3 Fix_imports optimization 21 days open http://bugs.python.org/issue3218 From db3l.net at gmail.com Fri Jul 18 18:18:34 2008 From: db3l.net at gmail.com (David Bolen) Date: Fri, 18 Jul 2008 12:18:34 -0400 Subject: [Python-Dev] Windows buildbot trick References: <g5pfrp$d9e$1@ger.gmane.org> Message-ID: <87iqv3tbc5.fsf@valheru.db3l.homeip.net> Thomas Heller <theller at ctypes.org> writes: > The most annoying thing on the windows buildbots is when Python crashes hard > (with a general protection fault or stack overflow, for example). > Usually the system pops up a dialog in this case which allows to > attach a debugger to the process. This dialog will stay open until > the maintainer manually closes it, and will prevent the next builds. > > On my boxes I have inserted these two lines into the PythonXY/Scripts/buildbot > script, right at the top: > > import win32api; win32api.SetErrorMode(7) > print "SetErrorMode(7) called." > > These lines prevent the dialog box to be displayed for critical errors. For my buildbot, I've been running similarly modified buildbot code since late 2007 (in commands.py, surrounding the spawning of a process to execute a step) so it covers any use of the buildbot (mine was also building MSIs). I had to modify buildbot to fix a file upload bug anyway, so didn't mind running a patched version. Technically using 0x8007 rather than just 7 would also cover a possible missing file dialog, but not crucial. It hasn't been 100% foolproof, as I seem to recall once or twice still having to clear a dialog, but I think that was from the CRT within Python itself, which there was an old discussion about changing the Python code to initialize differently. It definitely catches the Windows hard failures though. -- David From eric+python-dev at trueblade.com Fri Jul 18 19:10:28 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 18 Jul 2008 13:10:28 -0400 Subject: [Python-Dev] [Python-checkins] r65099 - python/trunk/Doc/library/string.rst In-Reply-To: <20080718111506.867BF1E4007@bag.python.org> References: <20080718111506.867BF1E4007@bag.python.org> Message-ID: <4880CE84.7050003@trueblade.com> georg.brandl wrote: > Author: georg.brandl > Date: Fri Jul 18 13:15:06 2008 > New Revision: 65099 > > Log: > Document the different meaning of precision for {:f} and {:g}. > Also document how inf and nan are formatted. #3404. Thanks for doing this. But see this output: http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/builds/255/step-test/0 which shows that on Solaris with gcc it's 'NaN', not 'nan'. This is one of the reasons I didn't get into documenting it. And on Windows, it's even worse (no Windows box handy, sorry). Do we want to document the actual behavior, or do we want to normalize all platforms so that they all return 'nan' and 'inf'? I'm still hoping to fix "F" formatting (issue 3382) on Windows, assuming that it's acceptable to do that before the last beta. It mostly comes down to deleting the tests, since I can't match on 'nan' and 'inf'. There is some additional work mapping 'F' to 'f', and then uppercasing the result, but that part is easy. Eric. From josiah.carlson at gmail.com Fri Jul 18 19:45:06 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 18 Jul 2008 10:45:06 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> Message-ID: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> On Fri, Jul 18, 2008 at 8:11 AM, Guido van Rossum <guido at python.org> wrote: > On Fri, Jul 18, 2008 at 7:57 AM, Josiah Carlson > <josiah.carlson at gmail.com> wrote: >> Invariably, when someone goes and removes a module, someone else is >> going to complain, "but I used feature X, not having feature X will >> break my code." We, as maintainers can then say, "if you cared, >> maintain it." But I'm not sure that is the greatest thing to tell >> people. I suspect that we may have to include some sort of >> "work-alike" for 2.7 and if not 3.0, 3.1 . If I were to vote for a >> work-alike, it would be based on sqlite. For one of the most common >> use-cases (bsddb.btree), simple sqlite code can be written to do the >> right thing. Recno is a little more tricky, but can also be done. >> The bsddb hash may not be possible, because sqlite doesn't support >> hashed indices :/. > > In my mind, BSDDB is pretty much the most heavy-weight extension we're > maintaining. I think it's an illusion that a sqlite-based look-alike > is going to fool anyone. The correct solution is to take support for > bsddb to a separate project where those who care about it can maintain > it together. That also makes it a lot easier to track the versions of > Berkeley DB as they come out. > > Of course, you're free to try writing the work-alike you're proposing. :-) It's entirely possible that I know very little about what was being made available via the bsddb module, but to match the API of what is included in the documentation (plus the dictionary interface that it supports) shouldn't be terribly difficult. Now, if there were *other* things that were undocumented, well, there's not much I can do about those. ;) - Josiah From fdrake at acm.org Fri Jul 18 20:03:27 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 18 Jul 2008 14:03:27 -0400 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> Message-ID: <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote: > It's entirely possible that I know very little about what was being > made available via the bsddb module, but to match the API of what is > included in the documentation (plus the dictionary interface that it > supports) shouldn't be terribly difficult. It's also entirely possible that the API isn't interesting if you don't support existing databases, for many applications. -Fred -- Fred Drake <fdrake at acm.org> From brett at python.org Fri Jul 18 20:04:17 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 11:04:17 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> Message-ID: <bbaeab100807181104ra7e7cdcl98183746e0016daa@mail.gmail.com> On Fri, Jul 18, 2008 at 7:21 AM, Guido van Rossum <guido at python.org> wrote: > On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote: >> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote: >>> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: >>>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: >>>>> >>>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into >>>>> 3k. I somewhat doubt that this gets resolved before the release, so >>>>> bsddb users might need to skip 3.0. >>>> >>>> >>>> In fact, bsddb as packages in core Python has rarely been in good shape. >>>> >>>> Has anyone actually considered that bsddb might do better if maintained >>>> strictly as an external package? That would make it easier for the any >>>> maintainers to release updates, and removes a source of frustration for >>>> users who either don't need it or would rather wait for a happier version. >>>> >>>> I think this is worth considering. I vaguely recall that the bsddb module >>>> was maintained before it was incorporated into the core Python release. >>> >>> +1. In my recollection maintaining bsddb has been nothing but trouble >>> right from the start when we were all sitting together at "Zope Corp >>> North" in a rented office in McLean... We can remove it from 3.0. We >>> can't really remove it from 2.6, but we can certainly start >>> end-of-lifing it in 2.6. > >> Unless I hear otherwise, I will add it to PEP 3108. > > Please do! > Done. -Brett From brett at python.org Fri Jul 18 20:12:41 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 11:12:41 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <20080718065430.GA6991@arctrix.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> Message-ID: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote: > [back on the list] > > On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote: >> Turned out to be a rebuild:: >> >> .... >> r65077 = 82d954e8c20c91562c4c660859d17756cba10992 >> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a >> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536 >> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771 >> >> How do I know what is going to be sent? ``git log`` seems to suggest >> something by not listing a git-svn-id for my last commit, but is that >> really the best I got? > > The command > > git log git-svn.. > And those two periods are significant for people who think they are line noise. Damn is Git quirky. > will show you changes in your HEAD (by default "master") tree that > are not in the remote tree (git-svn). > >> Is there some other way to see what will be pushed? > > I like running "gitk" before I push something. > Sure, but I don't know what the heck I am looking at. There is some green stuff for both master and my branch, but then there is some more green stuff just for my branch. Now if it was just the one commit I did on my branch I would assume that is just what I have not pushed, but considering there is also a marker on the master branch which is pristine I am not sure what I am looking at. >> And how do I diff easily between commits? > > It depends on what you want, exactly. Maybe you can describe some > use cases. A DVCS can't use identify revisions like SVN does. > Generally I find myself using heads or tags to identify versions in > combination with the ^ operator. For example, > I assume the ^ operator means "just before this commit". > git diff HEAD^ > > would show the difference between the current working tree and the > commit before the head of the stored tree. If you want the patch > for a single commit, use "git show <object>". For example, "git > show" will display the last commit. To see amk's typo fix: > > git show 6cadb9c1b7e30a8b66cdba01cd79aa6397a07080 > > You can also abbreviate the commit id, eg. > > git show 6cadb9 > Does the abbreviation have to be exactly six characters? > As I say in my guide, "git format-patch" and "git am" are very handy > when slinging patches around (e.g. to and from a bug tracker or > mailing list). I tried that, but but format-patch didn't show me anything since I had just committed. And when I run ``git format-patch HEAD^`` it spits out what looks like a file name, but I don't see it anywhere. -Brett From ncoghlan at gmail.com Fri Jul 18 20:16:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 19 Jul 2008 04:16:57 +1000 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> Message-ID: <4880DE19.7050704@gmail.com> Fred Drake wrote: > On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote: >> It's entirely possible that I know very little about what was being >> made available via the bsddb module, but to match the API of what is >> included in the documentation (plus the dictionary interface that it >> supports) shouldn't be terribly difficult. > > > It's also entirely possible that the API isn't interesting if you don't > support existing databases, for many applications. And downloading pybsddb and installing really shouldn't be all that difficult :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From amk at amk.ca Fri Jul 18 20:32:15 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 18 Jul 2008 14:32:15 -0400 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> Message-ID: <20080718183215.GA7748@amk-desktop.matrixgroup.net> yOn Fri, Jul 18, 2008 at 10:45:06AM -0700, Josiah Carlson wrote: >> It's entirely possible that I know very little about what was being > made available via the bsddb module, but to match the API of what is > included in the documentation (plus the dictionary interface that it > supports) shouldn't be terribly difficult. It would pose a problem for the external maintainer if Python came with a bsddb module or package; the external package would also want to provide a 'bsddb' package containing the 'bsddb.db' subpackage. This would be a replay of the 'xml'/PyXML situation, and lots of people think that was a big mistake in the first place. If we do want to let bsddb go, we should just let the new external package own the package name. We can obviously drop the module for 3.0. For 2.x, should we just shrug and disable most of the BerkeleyDB tests (maybe just on Windows) by adding a new resource to enable them? If we're stuck with it for the remaining 2.x, at least we can stop the buildbots from going red for bugs that we can't debug and probably won't fix. --amk From brett at python.org Fri Jul 18 20:37:09 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 11:37:09 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> Message-ID: <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> On Fri, Jul 18, 2008 at 11:12 AM, Brett Cannon <brett at python.org> wrote: > On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote: >> [back on the list] >> >> On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote: >>> Turned out to be a rebuild:: >>> >>> .... >>> r65077 = 82d954e8c20c91562c4c660859d17756cba10992 >>> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a >>> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536 >>> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771 >>> >>> How do I know what is going to be sent? ``git log`` seems to suggest >>> something by not listing a git-svn-id for my last commit, but is that >>> really the best I got? >> >> The command >> >> git log git-svn.. >> > > And those two periods are significant for people who think they are > line noise. Damn is Git quirky. > OK, so I decided to trying committing through Git by doing ``git svn dcommit``. But it told me that Misc/NEWS was out of date. So I then did ``git fetch git-svn`` with a ``git merge git-svn``, which I realize now is a mistake since I am used to other DVCSs using "merge" for when there are changes and "update" when there are none. So Git tells me there is a conflict; fine. I go in, fix the file, and then try to commit Misc/NEWS. It says I can't do a partial commit, I have to commit everything; fine. So I do a full commit, but now I get: Misc/NEWS: needs merge Misc/NEWS: unmerged (3dc56ad4e5918676358c4e20be738b37ce8d50d0) Misc/NEWS: unmerged (ad4c19cecfb584be37d8b9c138791daa83ad9285) Misc/NEWS: unmerged (853df55fc6ac71bcea0eb340a8ab52b348db9e8c) error: Error building trees This is not winning me over on the usability side of things. =) So what do I do now to get this tree back in a usable state so I can commit (although, thanks to ``git show``, I might patch a svn checkout so I don't have to wait on this)? -Brett From amk at amk.ca Fri Jul 18 20:42:06 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 18 Jul 2008 14:42:06 -0400 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> Message-ID: <20080718184206.GB7748@amk-desktop.matrixgroup.net> On Fri, Jul 18, 2008 at 11:12:41AM -0700, Brett Cannon wrote: > And those two periods are significant for people who think they are > line noise. Damn is Git quirky. Oh my, yes. We use git at work; there's a reason I now use Bazaar for personal projects. > I assume the ^ operator means "just before this commit". Correct; HEAD^^ is two commits ago, and you can do HEAD~35 for 35 commits ago. There's a git-rev-parse man page describing these. > Does the abbreviation have to be exactly six characters? No, it has to be long enough to be unambiguous. 6cadb9c1b7 would also work, and 6cadb might, depending on the other commits in your repository. > I tried that, but but format-patch didn't show me anything since I had > just committed. And when I run ``git format-patch HEAD^`` it spits out > what looks like a file name, but I don't see it anywhere. I think it writes the file to the current working directory, so I don't know why you're not seeing it. (The file is something like 0001-<commit-message>.patch.) --amk From brett at python.org Fri Jul 18 20:57:21 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 11:57:21 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> Message-ID: <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> On Fri, Jul 18, 2008 at 11:37 AM, Brett Cannon <brett at python.org> wrote: > On Fri, Jul 18, 2008 at 11:12 AM, Brett Cannon <brett at python.org> wrote: >> On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote: >>> [back on the list] >>> >>> On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote: >>>> Turned out to be a rebuild:: >>>> >>>> .... >>>> r65077 = 82d954e8c20c91562c4c660859d17756cba10992 >>>> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a >>>> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536 >>>> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771 >>>> >>>> How do I know what is going to be sent? ``git log`` seems to suggest >>>> something by not listing a git-svn-id for my last commit, but is that >>>> really the best I got? >>> >>> The command >>> >>> git log git-svn.. >>> >> >> And those two periods are significant for people who think they are >> line noise. Damn is Git quirky. >> > > OK, so I decided to trying committing through Git by doing ``git svn > dcommit``. But it told me that Misc/NEWS was out of date. So I then > did ``git fetch git-svn`` with a ``git merge git-svn``, which I > realize now is a mistake since I am used to other DVCSs using "merge" > for when there are changes and "update" when there are none. > > So Git tells me there is a conflict; fine. I go in, fix the file, and > then try to commit Misc/NEWS. It says I can't do a partial commit, I > have to commit everything; fine. So I do a full commit, but now I get: > > Misc/NEWS: needs merge > Misc/NEWS: unmerged (3dc56ad4e5918676358c4e20be738b37ce8d50d0) > Misc/NEWS: unmerged (ad4c19cecfb584be37d8b9c138791daa83ad9285) > Misc/NEWS: unmerged (853df55fc6ac71bcea0eb340a8ab52b348db9e8c) > error: Error building trees > > This is not winning me over on the usability side of things. =) So > what do I do now to get this tree back in a usable state so I can > commit (although, thanks to ``git show``, I might patch a svn checkout > so I don't have to wait on this)? > I figured this out. I just did ``git reset --hard``, did the proper "fetch;rebase" dance, resolved the conflict, did ``git add`` and then continued with the rebase. It all looks fine now. -Brett From nas at arctrix.com Fri Jul 18 20:57:42 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Fri, 18 Jul 2008 12:57:42 -0600 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> Message-ID: <20080718185741.GA9433@arctrix.com> On Fri, Jul 18, 2008 at 11:12:41AM -0700, Brett Cannon wrote: > > git log git-svn.. > > And those two periods are significant for people who think they are > line noise. Damn is Git quirky. I guess it would have been clearer if I had used "git-svn..HEAD". The ".." is similar to SVN's ":" so I don't see that as much of a quirk. However, if you look at the git-rev-parse man page you will see that git supports a very rich set of revision specifications and that can be overwhelming to a new user. You can probably mostly get by with "<rev>" and "<rev1>..<rev2>" combined with ^. > Sure, but I don't know what the heck I am looking at. The top left window shows a DAG of the commits. Each commit is a little ball. Parents are connected with a colored line. Tags and heads are shown as labels on specific commits. You can click on any commit to see the details (shown in the lower panels). > I assume the ^ operator means "just before this commit". Yes and you can use it more than once (e.g. ^^^). This is all documented in the git-rev-parse man page. > Does the abbreviation have to be exactly six characters? No. > I tried that, but but format-patch didn't show me anything since I had > just committed. And when I run ``git format-patch HEAD^`` it spits out > what looks like a file name, but I don't see it anywhere. By default it creates a file for each commit, prefixed by 0001, 0002, etc. Use "git format-patch --stdout" to have it spit the patches out as a mbox to stdout. From brett at python.org Fri Jul 18 21:07:19 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 12:07:19 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> Message-ID: <bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com> On Fri, Jul 18, 2008 at 11:57 AM, Brett Cannon <brett at python.org> wrote: > On Fri, Jul 18, 2008 at 11:37 AM, Brett Cannon <brett at python.org> wrote: >> On Fri, Jul 18, 2008 at 11:12 AM, Brett Cannon <brett at python.org> wrote: >>> On Thu, Jul 17, 2008 at 11:54 PM, Neil Schemenauer <nas at arctrix.com> wrote: >>>> [back on the list] >>>> >>>> On Thu, Jul 17, 2008 at 11:24:16PM -0700, Brett Cannon wrote: >>>>> Turned out to be a rebuild:: >>>>> >>>>> .... >>>>> r65077 = 82d954e8c20c91562c4c660859d17756cba10992 >>>>> r65082 = 1c75cce93c2ef2ec87e801888638cfdf5d2ff29a >>>>> r65085 = 3143c2fbe7315afd29496dc0cdac3122bed30536 >>>>> Done rebuilding .git/svn/git-svn/.rev_map.6015fed2-1504-0410-9fe1-9d1591cc4771 >>>>> >>>>> How do I know what is going to be sent? ``git log`` seems to suggest >>>>> something by not listing a git-svn-id for my last commit, but is that >>>>> really the best I got? >>>> >>>> The command >>>> >>>> git log git-svn.. >>>> >>> >>> And those two periods are significant for people who think they are >>> line noise. Damn is Git quirky. >>> >> >> OK, so I decided to trying committing through Git by doing ``git svn >> dcommit``. But it told me that Misc/NEWS was out of date. So I then >> did ``git fetch git-svn`` with a ``git merge git-svn``, which I >> realize now is a mistake since I am used to other DVCSs using "merge" >> for when there are changes and "update" when there are none. >> >> So Git tells me there is a conflict; fine. I go in, fix the file, and >> then try to commit Misc/NEWS. It says I can't do a partial commit, I >> have to commit everything; fine. So I do a full commit, but now I get: >> >> Misc/NEWS: needs merge >> Misc/NEWS: unmerged (3dc56ad4e5918676358c4e20be738b37ce8d50d0) >> Misc/NEWS: unmerged (ad4c19cecfb584be37d8b9c138791daa83ad9285) >> Misc/NEWS: unmerged (853df55fc6ac71bcea0eb340a8ab52b348db9e8c) >> error: Error building trees >> >> This is not winning me over on the usability side of things. =) So >> what do I do now to get this tree back in a usable state so I can >> commit (although, thanks to ``git show``, I might patch a svn checkout >> so I don't have to wait on this)? >> > > I figured this out. I just did ``git reset --hard``, did the proper > "fetch;rebase" dance, resolved the conflict, did ``git add`` and then > continued with the rebase. It all looks fine now. > I lied. Trying again complained about Mac/IDLE/Makefile.in which I have not touched, nor is it listed as changed. OK, so I deleted it in hopes that "fetch; rebase" would fix it; nope. So I did another ``git reset --hard``, but I am still stuck with being told that Mac/IDLE/Makefile.in is out of date:: Committing to svn+ssh://svn.python.org/python/trunk ... M Mac/IDLE/Makefile.in Transaction is out of date: Out of date: '/python/trunk/Mac/IDLE/Makefile.in' in transaction '65108-1' at /unix/bin/git-svn line 461 So I don't know how to deal with this since Git does not even list the file as being modified. -Brett From josiah.carlson at gmail.com Fri Jul 18 21:15:42 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 18 Jul 2008 12:15:42 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> Message-ID: <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> On Fri, Jul 18, 2008 at 11:03 AM, Fred Drake <fdrake at acm.org> wrote: > On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote: >> >> It's entirely possible that I know very little about what was being >> made available via the bsddb module, but to match the API of what is >> included in the documentation (plus the dictionary interface that it >> supports) shouldn't be terribly difficult. > > > It's also entirely possible that the API isn't interesting if you don't > support existing databases, for many applications. I see where the confusion was. I'm not suggesting that someone write a new bsddb module, I'm suggesting that we can provide something called, perhaps, on_disk_dictionary, which offers the behavior of bsddb, without using bsddb anywhere, or supporting bsddb files. - Josiah From barry at python.org Fri Jul 18 21:19:32 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 18 Jul 2008 15:19:32 -0400 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <20080718183215.GA7748@amk-desktop.matrixgroup.net> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <20080718183215.GA7748@amk-desktop.matrixgroup.net> Message-ID: <2750F8D4-898A-4176-AE36-9E37CA17ABDB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 18, 2008, at 2:32 PM, A.M. Kuchling wrote: > We can obviously drop the module for 3.0. For 2.x, should we just > shrug and disable most of the BerkeleyDB tests (maybe just on Windows) > by adding a new resource to enable them? If we're stuck with it for > the remaining 2.x, at least we can stop the buildbots from going red > for bugs that we can't debug and probably won't fix. +1 - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSIDsxXEjvBPtnXfVAQJ6egP+O0HQPj9X7te1X3CbPBRN8RhadJucaDmF 13a5bf13D/5D+gQ4iKlYMovd9l66J6lpFpSX+cHLgU3g/BZvFUXJFl4gZegWyD2p idQvGVFUrjGDL5TFIL4Dvg5D/b8pYSPfr5xWgJNSPHMeM5sEM6hDt/WhuvYSzuv+ lMROoc5c0NI= =isdz -----END PGP SIGNATURE----- From stephen at xemacs.org Fri Jul 18 21:06:01 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 19 Jul 2008 04:06:01 +0900 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <4880DE19.7050704@gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <4880DE19.7050704@gmail.com> Message-ID: <87ljzzovvq.fsf@uwakimon.sk.tsukuba.ac.jp> Nick Coghlan writes: > And downloading pybsddb and installing really shouldn't be all that > difficult :) It shouldn't be, but lots of "enterprise"[1] environments will require qualifying the "new" package according to corporate standards. I won't argue that this is a sufficient reason to keep a package with bsddb's history in stdlib, but let's not joke about the difficulties. Footnotes: [1] Which apparently means that the company can be enterprising, but the people who do the work mustn't be! From nas at arctrix.com Fri Jul 18 21:31:00 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Fri, 18 Jul 2008 13:31:00 -0600 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> References: <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> Message-ID: <20080718193100.GB9433@arctrix.com> On Fri, Jul 18, 2008 at 11:57:21AM -0700, Brett Cannon wrote: > I figured this out. I just did ``git reset --hard``, did the proper > "fetch;rebase" dance, resolved the conflict, did ``git add`` and then > continued with the rebase. It all looks fine now. Doing a fetch followed by a rebase is similar to what "svn up" does and is what you want in this case. Rebase rewrites patches (commits) so that they apply to a different version of a tree (they will get new SHA ids). If you use merge then your patches (commits) stay unchanged and a new commit object is created that contains the info required to integrate them with the new tree. Using merge is also useful in certain cases (e.g. in a distributed environment, if you have published your changes already and people have them in their repositories) but the downside is the extra merge commit object. However, since in this specific case you are always pushing back to a central repo you should avoid merge. BTW, the rebase command is very handy if you are maintaing some change over a longer term. Here's an example workflow: <start with git checkout> git checkout -b my_feature # create a private branch <edit code, commit changes> git checkout master # back to main branch <time passes> git fetch git-svn && git rebase git-svn # update master to latest code git checkout my_feature # back to private branch git rebase master # make my changes apply to latest code To generate a series of patches to send somewhere: git format-patch --stdout master..my_feature > my_feature.txt Cheers, Neil From brett at python.org Fri Jul 18 21:34:24 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 12:34:24 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <20080718193100.GB9433@arctrix.com> References: <20080715200411.GA26998@arctrix.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> <20080718193100.GB9433@arctrix.com> Message-ID: <bbaeab100807181234j6ea71da9i550eb29cfef5b46b@mail.gmail.com> On Fri, Jul 18, 2008 at 12:31 PM, Neil Schemenauer <nas at arctrix.com> wrote: > On Fri, Jul 18, 2008 at 11:57:21AM -0700, Brett Cannon wrote: >> I figured this out. I just did ``git reset --hard``, did the proper >> "fetch;rebase" dance, resolved the conflict, did ``git add`` and then >> continued with the rebase. It all looks fine now. > > Doing a fetch followed by a rebase is similar to what "svn up" does > and is what you want in this case. Rebase rewrites patches > (commits) so that they apply to a different version of a tree (they > will get new SHA ids). If you use merge then your patches (commits) > stay unchanged and a new commit object is created that contains the > info required to integrate them with the new tree. > > Using merge is also useful in certain cases (e.g. in a distributed > environment, if you have published your changes already and people > have them in their repositories) but the downside is the extra merge > commit object. However, since in this specific case you are always > pushing back to a central repo you should avoid merge. > > BTW, the rebase command is very handy if you are maintaing some > change over a longer term. Here's an example workflow: > > <start with git checkout> > git checkout -b my_feature # create a private branch > <edit code, commit changes> > git checkout master # back to main branch > <time passes> > git fetch git-svn && git rebase git-svn # update master to latest code > git checkout my_feature # back to private branch > git rebase master # make my changes apply to latest code > That's what I have been doing, except using "merge" in the master branch. > To generate a series of patches to send somewhere: > > git format-patch --stdout master..my_feature > my_feature.txt OK, so that's how you use format-patch. -Brett From nas at arctrix.com Fri Jul 18 21:35:31 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Fri, 18 Jul 2008 13:35:31 -0600 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com> References: <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> <bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com> Message-ID: <20080718193531.GC9433@arctrix.com> On Fri, Jul 18, 2008 at 12:07:19PM -0700, Brett Cannon wrote: > I lied. Trying again complained about Mac/IDLE/Makefile.in which I > have not touched, nor is it listed as changed. I think you are running into the fact that the git tree on code.python.org is only updated every 30 minutes. Using the git:// protocol is fast but can't get changes that are in SVN and not yet in the git tree. If your repo is fairly up-to-date, you can just use "git svn rebase". That will grab the latest changes from the SVN server and rebase your local changes, very similar to what "svn up" does. BTW, I'm on #python-dev at the moment if you run into more trouble. Neil From charleshixsn at earthlink.net Fri Jul 18 21:35:46 2008 From: charleshixsn at earthlink.net (Charles Hixson) Date: Fri, 18 Jul 2008 12:35:46 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> Message-ID: <200807181235.46259.charleshixsn@earthlink.net> On Friday 18 July 2008 07:57:01 am Josiah Carlson wrote: > On Fri, Jul 18, 2008 at 7:21 AM, Guido van Rossum <guido at python.org> wrote: > > On Thu, Jul 17, 2008 at 10:43 PM, Brett Cannon <brett at python.org> wrote: > >> On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote: > >>> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: > >>>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: > >>>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into > >>>>> 3k. I somewhat doubt that this gets resolved before the release, so > >>>>> bsddb users might need to skip 3.0. > >>>> > >>>> In fact, bsddb as packages in core Python has rarely been in good > >>>> shape. > >>>> > >>>> Has anyone actually considered that bsddb might do better if > >>>> maintained strictly as an external package? That would make it easier > >>>> for the any maintainers to release updates, and removes a source of > >>>> frustration for users who either don't need it or would rather wait > >>>> for a happier version. > >>>> > >>>> I think this is worth considering. I vaguely recall that the bsddb > >>>> module was maintained before it was incorporated into the core Python > >>>> release. > >>> > >>> +1. In my recollection maintaining bsddb has been nothing but trouble > >>> right from the start when we were all sitting together at "Zope Corp > >>> North" in a rented office in McLean... We can remove it from 3.0. We > >>> can't really remove it from 2.6, but we can certainly start > >>> end-of-lifing it in 2.6. > >> > >> Unless I hear otherwise, I will add it to PEP 3108. > > > > Please do! > > Invariably, when someone goes and removes a module, someone else is > going to complain, "but I used feature X, not having feature X will > break my code." We, as maintainers can then say, "if you cared, > maintain it." But I'm not sure that is the greatest thing to tell > people. I suspect that we may have to include some sort of > "work-alike" for 2.7 and if not 3.0, 3.1 . If I were to vote for a > work-alike, it would be based on sqlite. For one of the most common > use-cases (bsddb.btree), simple sqlite code can be written to do the > right thing. Recno is a little more tricky, but can also be done. > The bsddb hash may not be possible, because sqlite doesn't support > hashed indices :/. > > Just an idea. > > - Josiah > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/charleshixsn%40earthlink. >net Were I to vote for "something" it would be a B+Tree in collections. One that didn't impose a requirement that the key be a string (and not, e.g., an integer or a float). OTOH, I don't care enough to build it. (I've proven this to myself repeatedly, as I've started to create such a thing, and then kludged a different solution.) From brett at python.org Fri Jul 18 21:38:13 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 12:38:13 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <20080718193531.GC9433@arctrix.com> References: <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> <bbaeab100807181157uf05bce1hf5772f07e8b6a695@mail.gmail.com> <bbaeab100807181207ta65c275hff3c64c594916900@mail.gmail.com> <20080718193531.GC9433@arctrix.com> Message-ID: <bbaeab100807181238r75f9ef43pb75d07a3ca4a8c0b@mail.gmail.com> On Fri, Jul 18, 2008 at 12:35 PM, Neil Schemenauer <nas at arctrix.com> wrote: > On Fri, Jul 18, 2008 at 12:07:19PM -0700, Brett Cannon wrote: >> I lied. Trying again complained about Mac/IDLE/Makefile.in which I >> have not touched, nor is it listed as changed. > > I think you are running into the fact that the git tree on > code.python.org is only updated every 30 minutes. Using the git:// > protocol is fast but can't get changes that are in SVN and not yet > in the git tree. If your repo is fairly up-to-date, you can just > use "git svn rebase". That will grab the latest changes from the > SVN server and rebase your local changes, very similar to what "svn > up" does. > Did that, still complained. > BTW, I'm on #python-dev at the moment if you run into more trouble. If I get daring again I will try you in the IRC channel, but I just gave up and used a patch against svn so the change could get committed. -Brett From g.brandl at gmx.net Fri Jul 18 21:38:46 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 18 Jul 2008 21:38:46 +0200 Subject: [Python-Dev] [Python-checkins] r65099 - python/trunk/Doc/library/string.rst In-Reply-To: <4880CE84.7050003@trueblade.com> References: <20080718111506.867BF1E4007@bag.python.org> <4880CE84.7050003@trueblade.com> Message-ID: <g5qrgh$2iu$1@ger.gmane.org> Eric Smith schrieb: > georg.brandl wrote: >> Author: georg.brandl >> Date: Fri Jul 18 13:15:06 2008 >> New Revision: 65099 >> >> Log: >> Document the different meaning of precision for {:f} and {:g}. >> Also document how inf and nan are formatted. #3404. > > Thanks for doing this. But see this output: > http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/builds/255/step-test/0 > which shows that on Solaris with gcc it's 'NaN', not 'nan'. This is one > of the reasons I didn't get into documenting it. And on Windows, it's > even worse (no Windows box handy, sorry). I see. > Do we want to document the actual behavior, or do we want to normalize > all platforms so that they all return 'nan' and 'inf'? I'd find a consistent behavior nice, if it isn't too much work to get one. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From janssen at parc.com Fri Jul 18 21:54:13 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 18 Jul 2008 12:54:13 PDT Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> References: <20080714224343.GA23048@arctrix.com> <1B9E1F51-F57A-44A9-900A-FBD19614F68C@python.org> <20080715200411.GA26998@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> Message-ID: <08Jul18.125419pdt."58698"@synergy1.parc.xerox.com> Why is it you're trying to use "git"? Bill From theller at ctypes.org Fri Jul 18 22:27:53 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 18 Jul 2008 22:27:53 +0200 Subject: [Python-Dev] Windows buildbot trick In-Reply-To: <a7a2b76b0807180853g524677cbk578f322b049010cc@mail.gmail.com> References: <g5pfrp$d9e$1@ger.gmane.org> <a7a2b76b0807180842k7d64205dqa77864749f3b6851@mail.gmail.com> <a7a2b76b0807180853g524677cbk578f322b049010cc@mail.gmail.com> Message-ID: <g5qub5$be9$1@ger.gmane.org> Sidnei da Silva schrieb: > On Fri, Jul 18, 2008 at 12:42 PM, Sidnei da Silva > <sidnei at enfoldsystems.com> wrote: >> That's a great trick! I seem to remember that there is a way to turn >> that off globally though, but not sure where. Maybe running >> drwtsn32.exe and un-checking 'Visual Notification' does the trick? > > FWIW, here's a kb article that describes this. Although it refers to > 32-bit apps on 64-bit XP, I'm pretty sure you can do the same for > 32-bit XP? > > http://support.microsoft.com/kb/283150/EN-US/ > Well, generally it is a good idea on a developer machine to automatically start the debugger when a problem in the tested application happens, so I prefer to not change this setting globally. However, some time ago I have tried to turn the notifications off in every way I could find but did not succeed. Although I'm not sure I found the article you mentioned. -- Thanks, Thomas From dickinsm at gmail.com Fri Jul 18 23:08:47 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 18 Jul 2008 22:08:47 +0100 Subject: [Python-Dev] [Python-checkins] r65099 - python/trunk/Doc/library/string.rst In-Reply-To: <4880CE84.7050003@trueblade.com> References: <20080718111506.867BF1E4007@bag.python.org> <4880CE84.7050003@trueblade.com> Message-ID: <5c6f2a5d0807181408g595cde7fv8a9888914291c4bd@mail.gmail.com> On Fri, Jul 18, 2008 at 6:10 PM, Eric Smith <eric+python-dev at trueblade.com> wrote: > Do we want to document the actual behavior, or do we want to normalize all > platforms so that they all return 'nan' and 'inf'? I seem to recall that Christian Heimes recently put some work into making sure that repr() or str() of an infinity or nan is 'inf' or 'nan' (or '-inf'), regardless of platform. +1 for normalizing '%f' and '%F' behaviour across platforms. Mark From mal at egenix.com Fri Jul 18 23:42:34 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 18 Jul 2008 23:42:34 +0200 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <200807181235.46259.charleshixsn@earthlink.net> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <200807181235.46259.charleshixsn@earthlink.net> Message-ID: <48810E4A.6070304@egenix.com> On 2008-07-18 21:35, Charles Hixson wrote: >> Invariably, when someone goes and removes a module, someone else is >> going to complain, "but I used feature X, not having feature X will >> break my code." We, as maintainers can then say, "if you cared, >> maintain it." But I'm not sure that is the greatest thing to tell >> people. I suspect that we may have to include some sort of >> "work-alike" for 2.7 and if not 3.0, 3.1 . If I were to vote for a >> work-alike, it would be based on sqlite. For one of the most common >> use-cases (bsddb.btree), simple sqlite code can be written to do the >> right thing. Recno is a little more tricky, but can also be done. >> The bsddb hash may not be possible, because sqlite doesn't support >> hashed indices :/. >> >> Just an idea. >> >> - Josiah > > Were I to vote for "something" it would be a B+Tree in collections. One that > didn't impose a requirement that the key be a string (and not, e.g., an > integer or a float). > > OTOH, I don't care enough to build it. (I've proven this to myself > repeatedly, as I've started to create such a thing, and then kludged a > different solution.) If pybsddb is sufficient as work-around for the stdlib bsddb module (or perhaps even better), then I don't see much of a problem removing the module from the stdlib using a PEP 4 process. Can't really say, since I've never used any of these myself... for on-disk dictionaries, we use mxBeeBase: http://www.egenix.com/products/python/mxBase/mxBeeBase/ For anything more complex, a SQL database. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 18 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From brett at python.org Fri Jul 18 23:57:41 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 14:57:41 -0700 Subject: [Python-Dev] git repositories for trunk and py3k In-Reply-To: <7815067257654475794@unknownmsgid> References: <20080714224343.GA23048@arctrix.com> <1afaf6160807151326o3cb2d5a0j6a7dd589fd304026@mail.gmail.com> <g5j4v3$6hi$1@ger.gmane.org> <bbaeab100807172252x74796b4eh555dcbbae2eed746@mail.gmail.com> <20080718061113.GA6848@arctrix.com> <bbaeab100807172324j500928d8m50e3e1001f8c1c51@mail.gmail.com> <20080718065430.GA6991@arctrix.com> <bbaeab100807181112v52401e65p271120883f92fb53@mail.gmail.com> <bbaeab100807181137g4e561f15n424b01a3fee88f6b@mail.gmail.com> <7815067257654475794@unknownmsgid> Message-ID: <bbaeab100807181457m6d8c380bt9a7a11fb9ab184e0@mail.gmail.com> On Fri, Jul 18, 2008 at 12:54 PM, Bill Janssen <janssen at parc.com> wrote: > Why is it you're trying to use "git"? Neil set up a mirror and I was curious. Same with the bzr mirror. -Brett From shane at hathawaymix.org Thu Jul 17 20:42:21 2008 From: shane at hathawaymix.org (Shane Hathaway) Date: Thu, 17 Jul 2008 12:42:21 -0600 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> Message-ID: <487F928D.50501@hathawaymix.org> ranjith kannikara wrote: > As a student I am not familiar with Restricted Python and python AST > implementation.And in need of help to start the Restricted Python > implementation. Here is some context for Python-Dev. RestrictedPython is a custom Python compiler that, when combined with a restricted environment, provides a sandbox safe enough to allow partly-trusted people to write and execute scripts on a Zope server. It has been used in Zope 2 for a long time and will have a future in Zope 3. The sandbox is more extensive than what the rexec module provides. The safety of RestrictedPython has been validated in a somewhat formal process with Python 2.4. Ranjith is working to validate it with Python 2.5. He is first working to discover all changes between Python 2.4 and 2.5 that might have affected the safety of a RestrictedPython sandbox. Any changes to the AST, builtin functions, methods of builtin types, etc., need to be evaluated for safety. So, in general, he is looking for detailed lists of changes between Python 2.4 and 2.5--more than the "What's New" doc. Shane From shane at hathawaymix.org Fri Jul 18 16:36:45 2008 From: shane at hathawaymix.org (Shane Hathaway) Date: Fri, 18 Jul 2008 08:36:45 -0600 Subject: [Python-Dev] Test, please ignore Message-ID: <4880AA7D.6090302@hathawaymix.org> ... From brett at python.org Sat Jul 19 01:36:33 2008 From: brett at python.org (Brett Cannon) Date: Fri, 18 Jul 2008 16:36:33 -0700 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <487F928D.50501@hathawaymix.org> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <487F928D.50501@hathawaymix.org> Message-ID: <bbaeab100807181636s286d6ab2u618c2b823ae33197@mail.gmail.com> On Thu, Jul 17, 2008 at 11:42 AM, Shane Hathaway <shane at hathawaymix.org> wrote: > ranjith kannikara wrote: >> >> As a student I am not familiar with Restricted Python and python AST >> implementation.And in need of help to start the Restricted Python >> implementation. > > Here is some context for Python-Dev. > > RestrictedPython is a custom Python compiler that, when combined with a > restricted environment, provides a sandbox safe enough to allow > partly-trusted people to write and execute scripts on a Zope server. It has > been used in Zope 2 for a long time and will have a future in Zope 3. The > sandbox is more extensive than what the rexec module provides. > > The safety of RestrictedPython has been validated in a somewhat formal > process with Python 2.4. Ranjith is working to validate it with Python 2.5. > He is first working to discover all changes between Python 2.4 and 2.5 that > might have affected the safety of a RestrictedPython sandbox. Any changes to > the AST, builtin functions, methods of builtin types, etc., need to be > evaluated for safety. > > So, in general, he is looking for detailed lists of changes between Python > 2.4 and 2.5--more than the "What's New" doc. > As it has been said, his best chance is to either diff between versions or look at the log output from svn on the files he cares about. -Brett From jcea at jcea.es Sat Jul 19 03:41:32 2008 From: jcea at jcea.es (Jesus Cea) Date: Sat, 19 Jul 2008 03:41:32 +0200 Subject: [Python-Dev] No beta2 tonight In-Reply-To: <487FD54C.2040308@v.loewis.de> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> Message-ID: <4881464C.3000908@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: | bsddb is in a very bad shape, as the 2.6 code hasn't been merged into | 3k. I somewhat doubt that this gets resolved before the release, so | bsddb users might need to skip 3.0. Working on the 3.0 port just now. 03:40 in the morning, in Spain. bsddb will be ready for python 3.0. (writting this while compiling new 2.6 and 3.0 beta2 :-) ) - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIFGTJlgi5GaxT1NAQKTBAP/f2AiHwEmMCSE6xhYVKj/0Rkh3A9/7XWG f8CrJ8Dn//QoG7+FZ63mF1BoT9aJ3RGUIrkIePe7+XJUpa8Fq264sQMoz94Estkc KyzSN69j+O9fTBGdI3PByFuCkfrrcl3+i0sEg0WQylOzzTpEfwKcRT5hGG2qb8c1 RZKV8BxQVXA= =4fO2 -----END PGP SIGNATURE----- From jcea at jcea.es Sat Jul 19 03:36:27 2008 From: jcea at jcea.es (Jesus Cea) Date: Sat, 19 Jul 2008 03:36:27 +0200 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> Message-ID: <4881451B.9010304@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brett Cannon wrote: | On Thu, Jul 17, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote: |> On Thu, Jul 17, 2008 at 7:30 PM, Fred Drake <fdrake at acm.org> wrote: |>> On Jul 17, 2008, at 7:27 PM, Martin v. L?wis wrote: |>>> bsddb is in a very bad shape, as the 2.6 code hasn't been merged into |>>> 3k. I somewhat doubt that this gets resolved before the release, so |>>> bsddb users might need to skip 3.0. |>> |>> In fact, bsddb as packages in core Python has rarely been in good shape. |>> |>> Has anyone actually considered that bsddb might do better if maintained |>> strictly as an external package? That would make it easier for the any |>> maintainers to release updates, and removes a source of frustration for |>> users who either don't need it or would rather wait for a happier version. |>> |>> I think this is worth considering. I vaguely recall that the bsddb module |>> was maintained before it was incorporated into the core Python release. |> +1. In my recollection maintaining bsddb has been nothing but trouble |> right from the start when we were all sitting together at "Zope Corp |> North" in a rented office in McLean... We can remove it from 3.0. We |> can't really remove it from 2.6, but we can certainly start |> end-of-lifing it in 2.6. |> | | Unless I hear otherwise, I will add it to PEP 3108. 03:31 AM in the morning in Spain. Just working in porting bsddb to python3000. I have a lot of issues compiling the C code and with the 2to3 automated conversion, but I'm pretty sure can get it ready for october release. I'm still loooking for a *GOOD* python2->python3 conversion guide for C language modules. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIFFFJlgi5GaxT1NAQLHlQP/cXmobkxlEs5gL9MRVNOAhcCtmATJqkQd lpAkzExa+75cQSoP28iIa+GhxRuJJCAz9NxAw22VIkwab/93R1eFYVjdC9jqYprK gwZsZDHKs12RlRRHFk0562ZDMgtdFB4Ne9iJXsFOKNz6ZvOhMFHfoj/blZyyocsq zdS6SoxTtXM= =aIrU -----END PGP SIGNATURE----- From jcea at jcea.es Sat Jul 19 04:25:41 2008 From: jcea at jcea.es (Jesus Cea) Date: Sat, 19 Jul 2008 04:25:41 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> Message-ID: <488150A5.8050804@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lennart Regebro wrote: | On Sun, May 25, 2008 at 6:25 AM, Jesus Cea <jcea at jcea.es> wrote: |> Since I need to port bsddb3 to py3k, what I need to know?. Is any |> *updated* document out there?. | | No, but there is a not yet complete, but quite updated set of examples. | | http://code.google.com/p/python-incompatibility/ | | This is a collection of code snippets that will show code that does | work under 2.x but not under 3.x, and the 3.x way of doing it (as well | as a way that works under both 2.6 and 3.0, in applicable cases). | | There is no tests for changes in the C-API, if you (or somebody else) | would like to add that (or any other tests) that would be very | appreciated! I can't checkout the code. What username/password must I use?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIFQnZlgi5GaxT1NAQL3LQP/ZxblxpJwZAQ9qIUXHAZpFlmK86y7UPfT DX707wR1YMOujiNazE2NuJSVCVkbbOlc1G2dxTAqDm7FAThkjuIeO74ijueUeFdN p6LP2dS5pHH8h8G9bUfDynGCjRQ4t1TUblksQgPDtrYiXlzIBqYL1qkHRf72w2c3 fyuZWBpaH0w= =nnN4 -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Sat Jul 19 04:43:42 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 19 Jul 2008 14:43:42 +1200 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <1afaf6160807170841x7cf14947x1276878b4fd50cd4@mail.gmail.com> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> Message-ID: <488154DE.8050405@canterbury.ac.nz> Josiah Carlson wrote: > It's entirely possible that I know very little about what was being > made available via the bsddb module, but to match the API of what is > included in the documentation (plus the dictionary interface that it > supports) shouldn't be terribly difficult. Maybe for new databases, but what about people with existing bsddb files that they need to read? -- Greg From kmtracey at gmail.com Sat Jul 19 05:05:11 2008 From: kmtracey at gmail.com (Karen Tracey) Date: Fri, 18 Jul 2008 23:05:11 -0400 Subject: [Python-Dev] Change in repr of Decimal in 2.6 Message-ID: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com> [Originally posted to python-list but on further reflection and some feedback I think it might be more appropriate here.] I noticed when trying out Python's 2.6b2 release that the repr of Decimal has changed since 2.5. On 2.5: Python 2.5.1 (r251:54863, Mar 7 2008, 04:10:12) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import decimal >>> decimal.Decimal(7) Decimal("7") >>> double quotes were used whereas on 2.6b2: Python 2.6b2 (r26b2:65082, Jul 18 2008, 13:36:54) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import decimal >>> decimal.Decimal(7) Decimal('7') >>> single quotes are used. Searching around I see this was done in r60773 with the log message: Fix decimal repr which should have used single quotes like other reprs. but I can't find any discussion other than that. My problem is this breaks a bunch of doctests that were written assuming the prior repr. I can't just update the tests to assume the new single quotes because they are for code that is supposed to run on everything back to Python 2.3. So my questions: Is this backwards-incompatible change really necessary and could it be reconsidered? If it's here to stay, is there some straightforward way that I am unaware of to construct tests that use Decimal repr but will work correctly on Python 2.3-2.6? Thanks for any feedback, Karen -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080718/efa07deb/attachment.htm> From python at rcn.com Sat Jul 19 05:19:05 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 18 Jul 2008 20:19:05 -0700 Subject: [Python-Dev] Change in repr of Decimal in 2.6 References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com> Message-ID: <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1> From: Karen Tracey > I noticed when trying out Python's 2.6b2 release that the repr of Decimal has changed since 2.5. On 2.5: ... > quotes were used whereas on 2.6b2: ... > single quotes are used. Searching around I see this was done in r60773 with the log message: > Fix decimal repr which should have used single quotes like other reprs. > > but I can't find any discussion other than that. Guido requested this change so that the Decimal repr would match the style used elsewhere (i.e. str(7) --> '7'). > If it's here to stay, is there some straightforward way that I am unaware > of to construct tests that use Decimal repr but will work correctly on > Python 2.3-2.6? A quick hack is to monkey patch the decimal module: >>> import decimal >>> decimal.Decimal.__repr__ = lambda s: 'Decimal("%s")' % str(s) A better fix is to amend to doctests to not rely on the __repr__ format. Instead of: >>> Decimal('7.1') Decimal("7.1") use: >>> print Decimal('7.1') 7.1 Raymond _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com From guido at python.org Sat Jul 19 05:55:51 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Jul 2008 20:55:51 -0700 Subject: [Python-Dev] Change in repr of Decimal in 2.6 In-Reply-To: <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1> References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com> <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1> Message-ID: <ca471dc20807182055k2e3b52c6h25b6478320a9a4b4@mail.gmail.com> This is an example of the problem with doctest -- it's easy to overspecify the tests. I don't think that whether the repr() of a Decimal uses single or double quotes should be considered a spec cast in stone by doctests. On Fri, Jul 18, 2008 at 8:19 PM, Raymond Hettinger <python at rcn.com> wrote: > From: Karen Tracey >> >> I noticed when trying out Python's 2.6b2 release that the repr of Decimal >> has changed since 2.5. On 2.5: > > ... >> >> quotes were used whereas on 2.6b2: > > ... >> >> single quotes are used. Searching around I see this was done in r60773 >> with the log message: >> Fix decimal repr which should have used single quotes like other reprs. >> >> but I can't find any discussion other than that. > > Guido requested this change so that the Decimal repr would match the style > used elsewhere (i.e. str(7) --> '7'). > > >> If it's here to stay, is there some straightforward way that I am unaware >> of to construct tests that use Decimal repr but will work correctly on >> Python 2.3-2.6? > > A quick hack is to monkey patch the decimal module: > > >>> import decimal > >>> decimal.Decimal.__repr__ = lambda s: 'Decimal("%s")' % str(s) > > A better fix is to amend to doctests to not rely on the __repr__ format. > Instead of: > > >>> Decimal('7.1') > Decimal("7.1") > > use: > > >>> print Decimal('7.1') > 7.1 > > > Raymond > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python%40rcn.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From kmtracey at gmail.com Sat Jul 19 05:59:49 2008 From: kmtracey at gmail.com (Karen Tracey) Date: Fri, 18 Jul 2008 23:59:49 -0400 Subject: [Python-Dev] Fwd: Change in repr of Decimal in 2.6 In-Reply-To: <af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com> References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com> <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1> <af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com> Message-ID: <af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com> Meant to copy list on reply, sorry. ---------- Forwarded message ---------- From: Karen Tracey <kmtracey at gmail.com> Date: Fri, Jul 18, 2008 at 11:48 PM Subject: Re: [Python-Dev] Change in repr of Decimal in 2.6 To: Raymond Hettinger <python at rcn.com> On Fri, Jul 18, 2008 at 11:19 PM, Raymond Hettinger <python at rcn.com> wrote: > From: Karen Tracey > >> I noticed when trying out Python's 2.6b2 release that the repr of Decimal >> has changed since 2.5. On 2.5: >> > ... > >> quotes were used whereas on 2.6b2: >> > ... > >> single quotes are used. Searching around I see this was done in r60773 >> with the log message: >> Fix decimal repr which should have used single quotes like other reprs. >> >> but I can't find any discussion other than that. >> > > Guido requested this change so that the Decimal repr would match the style > used elsewhere (i.e. str(7) --> '7'). > Thanks for the background. Can't say I ever noticed the inconsistency myself. > > If it's here to stay, is there some straightforward way that I am unaware >> of to construct tests that use Decimal repr but will work correctly on >> Python 2.3-2.6? >> > > A quick hack is to monkey patch the decimal module: > > >>> import decimal > >>> decimal.Decimal.__repr__ = lambda s: 'Decimal("%s")' % str(s) > That is quick, but does seem hackish. > A better fix is to amend to doctests to not rely on the __repr__ format. > Instead of: > > >>> Decimal('7.1') > Decimal("7.1") > > use: > > >>> print Decimal('7.1') > 7.1 > Yeah, but the testcases are not quite that simple. They're often testing return values from functions and as much verifying that the type is correct as the value, so I think I'd have to change stuff like: >>> f.clean('1') Decimal("1") to: >>> x = f.clean('1') >>> print type(x), x <class 'decimal.Decimal'> 1 right? Thanks for the answer, Karen -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080718/0fbbfe7e/attachment-0001.htm> From python at rcn.com Sat Jul 19 06:06:54 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 18 Jul 2008 21:06:54 -0700 Subject: [Python-Dev] Issue 3008: Binary repr of floats Message-ID: <D2EA66CFD5F84761AB5E2D634E99426D@RaymondLaptop1> The new float.hex() is really nice. Would like to augment it with a matching float.bin() method using the same notation and normalization and leaving all the rightmost bits as Guido suggested. I think this would help demystify floats and make it straightforward to show exactly what is happening during a floating point calculation that is losing precision. def float_as_bin(x): '3.125 --> -0b1.1101000000000000000000000000000000000000000000000000p+1' hex2bin = {'0' : '0000', '1' : '0001', '2' : '0010', '3' : '0011', '4' : '0100', '5' : '0101', '6' : '0110', '7' : '0111', '8' : '1000', '9' : '1001', 'a' : '1010', 'b' : '1011', 'c' : '1100', 'd' : '1101', 'e' : '1110', 'f' : '1111'} hex_pattern = '(\-)?0x([0-9a-f]+)\.([0-9a-f]*)(.*)' sign, intpart, fracpart, exp = re.search(hex_pattern, x.hex().lower()).groups() return ((sign or '') + '0b' + intpart + '.' + ''.join(hex2bin[d] for d in fracpart)[:53] + exp) The implementation would re-use Mark's code, substituting binary output for hex in the fractional part. Raymond From kmtracey at gmail.com Sat Jul 19 06:18:38 2008 From: kmtracey at gmail.com (Karen Tracey) Date: Sat, 19 Jul 2008 00:18:38 -0400 Subject: [Python-Dev] Change in repr of Decimal in 2.6 In-Reply-To: <ca471dc20807182055k2e3b52c6h25b6478320a9a4b4@mail.gmail.com> References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com> <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1> <ca471dc20807182055k2e3b52c6h25b6478320a9a4b4@mail.gmail.com> Message-ID: <af3536270807182118o4ea1a59eh1647830bcc5d1e98@mail.gmail.com> On Fri, Jul 18, 2008 at 11:55 PM, Guido van Rossum <guido at python.org> wrote: > This is an example of the problem with doctest -- it's easy to > overspecify the tests. I don't think that whether the repr() of a > Decimal uses single or double quotes should be considered a spec cast > in stone by doctests. > OK, but I'm just thinking of all the places in tests we have stuff like: >>> var.method(params) expected_repr_of_return_value We should really be doing something else to guard against a potential future change in repr of whatever it is? Thanks, Karen -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080719/1deba7d9/attachment.htm> From jcea at jcea.es Sat Jul 19 10:54:38 2008 From: jcea at jcea.es (Jesus Cea) Date: Sat, 19 Jul 2008 10:54:38 +0200 Subject: [Python-Dev] Progress on switching Windows build to Berkeley DB 4.7.25... In-Reply-To: <6167796BFEB5D0438720AC212E89A6B00787275B@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B00787275B@exchange.onresolve.com> Message-ID: <4881ABCE.3000303@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: | Hi all, | | Jesus, apologies that this has taken so long for me to get back | to, I've been completely and utterly swamped with client work the | past few weeks. However, thanks to a couple of hours spare at | Detroit airport yesterday, I was finally able to make some | progress on updating the Windows Berkeley DB build to 4.7.25. | I've checked in the work I've done so far to | branches/tnelson-trunk-bsddb-47-upgrade. One thing I wanted | to double check with you is the following change: | | Modified: python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py | ============================================================================== | --- python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py (original) | +++ python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py Wed Jun 18 06:13:44 2008 | @@ -94,7 +94,7 @@ | # The timeout is necessary in BDB 4.5, since DB_EVENT_REP_STARTUPDONE | # is not generated if the master has no new transactions. | # This is solved in BDB 4.6 (#15542). | - timeout = time.time()+2 | + timeout = time.time()+10 | while (time.time()<timeout) and not (self.confirmed_master and self.client_startupdone) : | time.sleep(0.02) | if db.version() >= (4,6) : | | Basically, when using +2, the test failed every so often when running the entire test_bsddb3 suite. I picked 10 arbitrarily; it improves things, but it's still not 100%, I still encounter the following failure every so often: [...] | Can you comment on this? Yes. The problem is a race condition between the client and the server in the replication. If you have bad luck, the starting order will be reversed and the sync will fail... temporally. BDB Replication Manager has some configurable timeouts for reconnection, master election, etc. The right thing to do is to configure them in the right way. This work is already done in my SVN version. It will be in the 4.7.2 bsddb release. In the meanwhile, your workaround would be enough, although the test checking will be slower. My intent is to publish 4.7.2 as soon as I have a Python 3.0 compatible version. Hope 2-4 weeks from now. Less if I have help solving some ugly issues. | Apart from this small issue, the other 311 tests pass on x86 and | x64 with flying colours, so nice work, whatever you've been doing ;-) Nice to see the tests passing on MS Windows :-p. I'm a bit surprised, actually O:-). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIGrx5lgi5GaxT1NAQI6wQP/f3oDcbPfjHQa6YcSy8KH5DYk971eNMrl Phg0mDmy5XGgzMZArVEexVZTq1ykTBaqJ3S9hjM1RMcnNkTlB+pYS8zJxQ09psYA YDJDnSHzPDDhLsElTzfHs/nIKaAGyEnSpsm3RCdONCJgu0LGnsjgfTmyh2bBZNol IcAd/+dCMVU= =Mh4Y -----END PGP SIGNATURE----- From ncoghlan at gmail.com Sat Jul 19 12:22:06 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 19 Jul 2008 20:22:06 +1000 Subject: [Python-Dev] Removing bsddb module from py3k (was Re: [Python-3000] No beta2 tonight) In-Reply-To: <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> Message-ID: <4881C04E.5060208@gmail.com> Josiah Carlson wrote: > On Fri, Jul 18, 2008 at 11:03 AM, Fred Drake <fdrake at acm.org> wrote: >> On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote: >>> It's entirely possible that I know very little about what was being >>> made available via the bsddb module, but to match the API of what is >>> included in the documentation (plus the dictionary interface that it >>> supports) shouldn't be terribly difficult. >> >> It's also entirely possible that the API isn't interesting if you don't >> support existing databases, for many applications. > > I see where the confusion was. I'm not suggesting that someone write > a new bsddb module, I'm suggesting that we can provide something > called, perhaps, on_disk_dictionary, which offers the behavior of > bsddb, without using bsddb anywhere, or supporting bsddb files. No, I knew what you were suggesting, I just don't see the point in doing it. If an app depends on bsddb specifically, they can either stick with the 2.x series, or they can move to 3.0 and download the externally maintained pybsddb (modulo any additional licensing checks required by a company's contracts department) or they can switch to a simpler file-based database format like sqlite3. I'm not clear on what problem you are attempting to solve with the idea of a module with the bsddb API but without an actual bsddb backend. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Jul 19 12:43:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 19 Jul 2008 20:43:05 +1000 Subject: [Python-Dev] Implementing restricted Python in Zope2 In-Reply-To: <487F928D.50501@hathawaymix.org> References: <20aa8c370807171054n1918161ch9ffa83d49915aa49@mail.gmail.com> <487F928D.50501@hathawaymix.org> Message-ID: <4881C539.5030500@gmail.com> Shane Hathaway wrote: > ranjith kannikara wrote: >> As a student I am not familiar with Restricted Python and python AST >> implementation.And in need of help to start the Restricted Python >> implementation. > > Here is some context for Python-Dev. > > RestrictedPython is a custom Python compiler that, when combined with a > restricted environment, provides a sandbox safe enough to allow > partly-trusted people to write and execute scripts on a Zope server. It > has been used in Zope 2 for a long time and will have a future in Zope > 3. The sandbox is more extensive than what the rexec module provides. > > The safety of RestrictedPython has been validated in a somewhat formal > process with Python 2.4. Ranjith is working to validate it with Python > 2.5. He is first working to discover all changes between Python 2.4 and > 2.5 that might have affected the safety of a RestrictedPython sandbox. > Any changes to the AST, builtin functions, methods of builtin types, > etc., need to be evaluated for safety. As others have noted, Python 2.4 didn't really have an AST - it had a concrete syntax tree that it called an AST. Python 2.5 introduced an actual AST written in ASDL and the parsing and compilation process was rewritten on that basis. The most relevant areas of the source tree to compare are the respective Parser subdirectories in 2.4 and 2.5: http://svn.python.org/projects/python/branches/release24-maint/Parser/ http://svn.python.org/projects/python/branches/release25-maint/Parser/ The changes to symtable.c and compile.c in the Python subdirectory between the two versions are also highly relevant. There may be other changes of relevance, but even going over just the changes I mentioned should keep you busy for quite a while (I don't think there was too much of the old compiler left once the AST compiler went into the tree). It's easy to get a diff between files in the two versions using the read-only access to the SVN server: svn diff --old <Python 2.4 URL> --new <Python 2.5 URL> (e.g. using the two parser directory URLs given above). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From victor.stinner at haypocalc.com Sat Jul 19 13:23:12 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 19 Jul 2008 13:23:12 +0200 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed Message-ID: <200807191323.13124.victor.stinner@haypocalc.com> Hi, I filled 14 issues about bugs found by fuzzing (see my other email "Play with fuzzing" for more informations). Most bugs are now closed, cool :-) Last bugs: == Trivial open bugs == segfault on locale.gettext(None) - http://bugs.python.org/issue3302 - attached patch is trivial: fix the PyArg_ParseTuple() to block None value, and reject empty domain string for bindtextdomain() (to avoid strange error "OSError(0): success") invalid ref count on locale.strcoll() error - http://bugs.python.org/issue3303 - attached patch is trivial: add "if (rel1)" _multiprocessing.Connection() doesn't check handle - http://bugs.python.org/issue3321 - _multiprocessing.Connection(fd) doesn't check that fd is a valid file handle and so may crash on poll (the "evil" FD_SET() call) - my patch add "|| fstat(handle, &statbuf)" to make sure that the file descriptor is valid == Complex open bugs == block operation on closed socket/pipe for multiprocessing - http://bugs.python.org/issue3311 - close() method sets the file handle to -1 but most methods don't check the handle and so may fail or crash. Especially poll() calls FD_SET((SOCKET)conn->handle, &rfds); with handle=-1 => crash. - my patch creates a new MP error: "return MP_CLOSED_FILE;", used if handle is INVALID_HANDLE_VALUE to block operations (send, receive, poll) on closed files for socket and pipe. bugs in scanstring_str() and scanstring_unicode() of _json module - http://bugs.python.org/issue3322 - scanstring() function crashs if second argument is a big negative integer. There is no attached patch because I don't understand this function enough to fix it correctly, but I suggest to raise a ValueError if end is too small/big invalid object destruction in re.finditer() - or "PyObject_DEL inconsistency if pydebug option is used" - http://bugs.python.org/issue3299 - It's the most complex bug, I prefer to write a new email :-) == Need backport / port to python 3.0 == invalid call to PyMem_Free() in fileio_init() - http://bugs.python.org/issue3304 - patch applied in Python 2.6 (trunk) but not in Python 3000: "i'm assuming that'll be merged into py3k automagically." wrote Gregory P. Smith missing lock release in BZ2File_iternext() - http://bugs.python.org/issue3309 - patch applied in Python 2.6 but "Needs backporting to release25-maint." wrote Gregory P. Smith When all bugs will be closed, I will restart a fuzzing Python ;-) But I also tried with my patches and I was unable to find new bugs, great! Victor From victor.stinner at haypocalc.com Sat Jul 19 13:40:34 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 19 Jul 2008 13:40:34 +0200 Subject: [Python-Dev] Problem with PyObject_DEL in pydebug mode Message-ID: <200807191340.34939.victor.stinner@haypocalc.com> Hi, I filled an issue about the crash: "import re; re.finditer("a", {})" http://bugs.python.org/issue3299 It appears quickly that the bug is specific to Python compiled in pydebug mode, or to be exact: when Py_TRACE_REFS is defined. == The Py_TRACE_REFS option == The problem is that PyObject_Del(obj) and PyObject_DEL(obj) don't remove obj from the "object list" (_ob_prev and _ob_next attributes of all objects when Py_TRACE_REFS is defined). And so obj will be reused later (maybe removed by garbage collector? or something like that) whereas it's invalid (memory freed). PyObject_NEW(obj) and PyObject_NEW_VAR(obj) call PyObject_Init() which registers the obj to the object list. So a developer can expect that PyObject_DEL(obj) will remove the object from this list. PyObject_Del(obj) and PyObject_DEL(obj) are used on object initialization failure, but also in "dealloc" callbacks. Classic object destruction is done by Py_DECREF(): when reference count is zero, dealloc() is called. PyObject_Del/DEL are different because they just free memory but don't call dealloc() whereas some attributes may be set (and some other may be uninitialized or NULL). == Solutions == (a) Replace PyObject_Del/PyObject_DEL by Py_DECREF to call dealloc callback which will call PyObject_Del/PyObject_DEL. I prefer this solution because dealloc is called and so we make that all attributes are deinitialized. New problem: dealloc expects that the object is fully initialized (all attributes are set and are not NULL), which is wrong is initialization fails. Eg. with re module,scanner_dealloc() calls Py_DECREF(self->pattern); whereas pattern attribute is NULL. Fix: replace Py_DECREF by Py_XDECREF. But all dealloc have to be reviewed. (b) Fix PyObject_Del/PyObject_DEL to remove object from the object list (c) Create new macro which is PyObject_Del/PyObject_DEL + remove the object from the list (d) Never use Py_TRACE_REFS :-) I wrote many informations in http://bugs.python.org/issue3299. -- Victor Stinner aka haypo http://fusil.hachoir.org/ From jnoller at gmail.com Sat Jul 19 15:14:44 2008 From: jnoller at gmail.com (Jesse Noller) Date: Sat, 19 Jul 2008 09:14:44 -0400 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <200807191323.13124.victor.stinner@haypocalc.com> References: <200807191323.13124.victor.stinner@haypocalc.com> Message-ID: <4222a8490807190614m370447d9s30d4af12de3e3152@mail.gmail.com> On Sat, Jul 19, 2008 at 7:23 AM, Victor Stinner <victor.stinner at haypocalc.com> wrote: > Hi, > > I filled 14 issues about bugs found by fuzzing (see my other email "Play with > fuzzing" for more informations). Most bugs are now closed, cool :-) Last > bugs: > > > == Trivial open bugs == > > segfault on locale.gettext(None) > - http://bugs.python.org/issue3302 > - attached patch is trivial: fix the PyArg_ParseTuple() to block None value, > and reject empty domain string for bindtextdomain() (to avoid strange > error "OSError(0): success") > > invalid ref count on locale.strcoll() error > - http://bugs.python.org/issue3303 > - attached patch is trivial: add "if (rel1)" > > _multiprocessing.Connection() doesn't check handle > - http://bugs.python.org/issue3321 > - _multiprocessing.Connection(fd) doesn't check that fd is a valid file handle > and so may crash on poll (the "evil" FD_SET() call) > - my patch add "|| fstat(handle, &statbuf)" to make sure that the > file descriptor is valid > > > == Complex open bugs == > > block operation on closed socket/pipe for multiprocessing > - http://bugs.python.org/issue3311 > - close() method sets the file handle to -1 but most methods don't check > the handle and so may fail or crash. Especially poll() calls > FD_SET((SOCKET)conn->handle, &rfds); with handle=-1 => crash. > - my patch creates a new MP error: "return MP_CLOSED_FILE;", used if handle > is INVALID_HANDLE_VALUE to block operations (send, receive, poll) on > closed files for socket and pipe. > > bugs in scanstring_str() and scanstring_unicode() of _json module > - http://bugs.python.org/issue3322 > - scanstring() function crashs if second argument is a big negative > integer. There is no attached patch because I don't understand this > function enough to fix it correctly, but I suggest to raise a ValueError > if end is too small/big > > invalid object destruction in re.finditer() > - or "PyObject_DEL inconsistency if pydebug option is used" > - http://bugs.python.org/issue3299 > - It's the most complex bug, I prefer to write a new email :-) > > > == Need backport / port to python 3.0 == > > invalid call to PyMem_Free() in fileio_init() > - http://bugs.python.org/issue3304 > - patch applied in Python 2.6 (trunk) but not in Python 3000: > "i'm assuming that'll be merged into py3k automagically." > wrote Gregory P. Smith > > missing lock release in BZ2File_iternext() > - http://bugs.python.org/issue3309 > - patch applied in Python 2.6 but "Needs backporting to release25-maint." > wrote Gregory P. Smith > > > When all bugs will be closed, I will restart a fuzzing Python ;-) But I also > tried with my patches and I was unable to find new bugs, great! > > Victor Thank you Victor - I didn't want to change any underlying multiprocessing code until we had the test suite in a better state (which we do now). Now that beta 2 is out, I will address the multiprocessing ones asap. One suggestion would be to include tests to prove the bugs is fixed if possible (to the patch), so we can add them to the suite. I know that that is not always possible, but it does help. I worry about making code changes without appropriate tests. If anything, a snippet of code "exploiting" the flaw may help generate a test case on my end. Thanks again for doing this. -jesse From josiah.carlson at gmail.com Sat Jul 19 16:54:39 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sat, 19 Jul 2008 07:54:39 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <488154DE.8050405@canterbury.ac.nz> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <487FD54C.2040308@v.loewis.de> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <488154DE.8050405@canterbury.ac.nz> Message-ID: <e6511dbf0807190754p6c311945l3dabcc8168754f46@mail.gmail.com> On Fri, Jul 18, 2008 at 7:43 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Josiah Carlson wrote: >> >> It's entirely possible that I know very little about what was being >> made available via the bsddb module, but to match the API of what is >> included in the documentation (plus the dictionary interface that it >> supports) shouldn't be terribly difficult. > > Maybe for new databases, but what about people with > existing bsddb files that they need to read? Classic data transition issue. Dump the data with the old software, reload the data with the new software. Both bsddb and sqlite are pretty fast, the latter of which has nice bulk inserts, after-the-fact index creation, ..., which can reduce the time it takes to do the transition. - Josiah From josiah.carlson at gmail.com Sat Jul 19 17:43:28 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sat, 19 Jul 2008 08:43:28 -0700 Subject: [Python-Dev] [Python-3000] No beta2 tonight In-Reply-To: <e6511dbf0807190754p6c311945l3dabcc8168754f46@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <0A94A999-9CF4-4864-BF26-829BB2798ED5@acm.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <488154DE.8050405@canterbury.ac.nz> <e6511dbf0807190754p6c311945l3dabcc8168754f46@mail.gmail.com> Message-ID: <e6511dbf0807190843y16a7a26dwb54d29130918592c@mail.gmail.com> On Sat, Jul 19, 2008 at 7:54 AM, Josiah Carlson <josiah.carlson at gmail.com> wrote: > On Fri, Jul 18, 2008 at 7:43 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >> Josiah Carlson wrote: >>> >>> It's entirely possible that I know very little about what was being >>> made available via the bsddb module, but to match the API of what is >>> included in the documentation (plus the dictionary interface that it >>> supports) shouldn't be terribly difficult. >> >> Maybe for new databases, but what about people with >> existing bsddb files that they need to read? > > Classic data transition issue. Dump the data with the old software, > reload the data with the new software. Both bsddb and sqlite are > pretty fast, the latter of which has nice bulk inserts, after-the-fact > index creation, ..., which can reduce the time it takes to do the > transition. After discussing with Guido off-list about how some companies have come to rely on deep features in bsddb, concerns that these companies wouldn't be able to transition away from bsddb, plus Jesus' picking up the code for 3.0 maintenance, I'm taking the offer to write a wrapper for sqlite off the table. Not because I don't think it could be done, but because no one seems to want what I'm offering (a 95% solution that doesn't require weeks/months of maintenance every release). - Josiah From josiah.carlson at gmail.com Sat Jul 19 18:27:15 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sat, 19 Jul 2008 09:27:15 -0700 Subject: [Python-Dev] Removing bsddb module from py3k (was Re: [Python-3000] No beta2 tonight) In-Reply-To: <4881C04E.5060208@gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> <4881C04E.5060208@gmail.com> Message-ID: <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com> On Sat, Jul 19, 2008 at 3:22 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: > Josiah Carlson wrote: >> >> On Fri, Jul 18, 2008 at 11:03 AM, Fred Drake <fdrake at acm.org> wrote: >>> >>> On Jul 18, 2008, at 1:45 PM, Josiah Carlson wrote: >>>> >>>> It's entirely possible that I know very little about what was being >>>> made available via the bsddb module, but to match the API of what is >>>> included in the documentation (plus the dictionary interface that it >>>> supports) shouldn't be terribly difficult. >>> >>> It's also entirely possible that the API isn't interesting if you don't >>> support existing databases, for many applications. >> >> I see where the confusion was. I'm not suggesting that someone write >> a new bsddb module, I'm suggesting that we can provide something >> called, perhaps, on_disk_dictionary, which offers the behavior of >> bsddb, without using bsddb anywhere, or supporting bsddb files. > > No, I knew what you were suggesting, I just don't see the point in doing it. > If an app depends on bsddb specifically, they can either stick with the 2.x > series, or they can move to 3.0 and download the externally maintained > pybsddb (modulo any additional licensing checks required by a company's > contracts department) or they can switch to a simpler file-based database > format like sqlite3. > > I'm not clear on what problem you are attempting to solve with the idea of a > module with the bsddb API but without an actual bsddb backend. On-disk key -> value dictionary. In every use of bsddb that I've seen (or done myself), that's been the extent of it's use. That's what I *was* offering. But it seems that everyone has had experience with bsddb on a far deeper level (beyond dictionary + cursor access), and would find (like you) absolutely no use in an on-disk dictionary with a sqlite backend (which hasn't had the same maintenance issues as bsddb), which is why I withdrew the offer. - Josiah From steve at holdenweb.com Thu Jul 17 07:50:48 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 17 Jul 2008 01:50:48 -0400 Subject: [Python-Dev] Unittest PEP do's and don'ts (BDFL pronouncement) In-Reply-To: <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com> References: <ca471dc20807161257s2059f707rbbf9343ce1777e4f@mail.gmail.com> <B8F029CB9D584A5D9055A64ED1D3EB0D@RaymondLaptop1> <487E5EB6.8070306@voidspace.org.uk> <20080717002252.GG26808@steerpike.home.puzzling.org> <87ej5tz6fh.fsf@benfinney.id.au> <20080717010730.GA29299@steerpike.home.puzzling.org> <87abghz4cu.fsf@benfinney.id.au> <20080717014556.GB29299@steerpike.home.puzzling.org> <871w1tz219.fsf@benfinney.id.au> <87sku9xmcr.fsf@benfinney.id.au> <ca471dc20807162023p65661957u5cc9e156de4a78ab@mail.gmail.com> Message-ID: <487EDDB8.1030803@holdenweb.com> Guido van Rossum wrote: > On Wed, Jul 16, 2008 at 7:42 PM, Ben Finney <ben+python at benfinney.id.au> wrote: >> The result I'm trying to avoid by this is that of having the >> externally visible behaviour of functions drift from the promise made >> by their names. Either rename the function, or create a new one for >> the new functionality. > > Oh get over it. This is getting ridiculous. Where did you *get* these > design "principles" you are so fond of? Read the zen of Python and > come back after you've memorized it. > Thank you. I was about to suggest that len() should be renamed length_or_raise_exception_if_not_a_container(). But then I did remark (before I stopped adding to this thread's interminable length, some aeons ago) that this was getting silly. Someone would surely have replied that the proposal had negative connotations and the correct name was really length_as_long_as_a_container. Or something equally trivial. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From jcea at jcea.es Sun Jul 20 03:44:10 2008 From: jcea at jcea.es (Jesus Cea) Date: Sun, 20 Jul 2008 03:44:10 +0200 Subject: [Python-Dev] Removing bsddb module from py3k (was Re: [Python-3000] No beta2 tonight) In-Reply-To: <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807171937x21a4ba4bv6063b5ff91baed01@mail.gmail.com> <bbaeab100807172243s2b5bd743pb6a1817489599443@mail.gmail.com> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> <4881C04E.5060208@gmail.com> <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com> Message-ID: <4882986A.3060509@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Josiah Carlson wrote: | On-disk key -> value dictionary. In every use of bsddb that I've seen | (or done myself), that's been the extent of it's use. That's what I | *was* offering. But it seems that everyone has had experience with | bsddb on a far deeper level (beyond dictionary + cursor access), and | would find (like you) absolutely no use in an on-disk dictionary with | a sqlite backend (which hasn't had the same maintenance issues as | bsddb), which is why I withdrew the offer. For such a simple application, you can use the already in stdlib "gdbm" module. Just remember it is not transactional, so beware diskfulls, application crashes, etc. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIKXKJlgi5GaxT1NAQLmSQP+PEDrOy5AM6IRhIDFriyks0nTpA5a3vYi bgrVRNJYQXTkV43AFD0bUkDFT1VBsROSlSO0jEsSfHXG2FG51TJa7Hp1LCX62ryV H6NtztvqlA5OpTxEcWGPdha0XRueRwjYe/a+cVFXjAo+fXirlIac2W6ZhNJiAehg VP9lxrffQSM= =KV8P -----END PGP SIGNATURE----- From tjreedy at udel.edu Sun Jul 20 04:52:21 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 19 Jul 2008 22:52:21 -0400 Subject: [Python-Dev] Fwd: Change in repr of Decimal in 2.6 In-Reply-To: <af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com> References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com> <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1> <af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com> <af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com> Message-ID: <g5u993$3p2$1@ger.gmane.org> Karen Tracey wrote: > Yeah, but the testcases are not quite that simple. They're often > testing return values from functions and as much verifying that the type > is correct as the value, so I think I'd have to change stuff like: > > >>> f.clean('1') > Decimal("1") > > to: > > >>> x = f.clean('1') > >>> print type(x), x > <class 'decimal.Decimal'> 1 > > right? >>> f.clean('1') == Decimal('1') True Since 'True' is a keyword, and Guido is *very* reluctant to even add keywords, let alone change their spelling, I think you can depend on that working indefinitely. Similarly, if you now have >>> type(a) <type 'int'> you test will fail in 3.0 which instead prints '<class 'int'>. But >>>type(a) is int True will continue working. Doctest has two quite different uses. One is to check text with interactive examples to make sure there are no mistakes in the examples. For that, printing representations is usually the natural style. Then one must accept that representations change faster that object behavior. The other use is to check code. For that, the more stilted style is more future-proof. For text doing double duty as a formal reference and code test, I would go for cross-version dependability. Terry Jan Reedy From charleshixsn at earthlink.net Sun Jul 20 05:01:15 2008 From: charleshixsn at earthlink.net (Charles Hixson) Date: Sat, 19 Jul 2008 20:01:15 -0700 Subject: [Python-Dev] Python-2.6b2.tar.bz missing sig file on web site Message-ID: <200807192001.16011.charleshixsn@earthlink.net> Python-2.6b2.tar.bz missing sig file on web site. That's about all the info I have, except that the tgz is also missing the sig file, and that 3.0b2 has it's sig file. Hope this is the correct place to report this. From barry at python.org Sun Jul 20 06:20:55 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 20 Jul 2008 00:20:55 -0400 Subject: [Python-Dev] Python-2.6b2.tar.bz missing sig file on web site In-Reply-To: <200807192001.16011.charleshixsn@earthlink.net> References: <200807192001.16011.charleshixsn@earthlink.net> Message-ID: <64701199-2C4F-4BAD-9E7D-57D8F74855F6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 19, 2008, at 11:01 PM, Charles Hixson wrote: > Python-2.6b2.tar.bz missing sig file on web site. That's about all > the info I > have, except that the tgz is also missing the sig file, and that > 3.0b2 has > it's sig file. > > Hope this is the correct place to report this. It is, thanks. Looks like I forgot to svn add those files. Please try again now. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSIK9KHEjvBPtnXfVAQLBdwQAok5D6yqAkfOEHjvq5YEVuOwnLLWz1UbV Y/bH6Q7lxnh4aIKJn2Mty82fLM7MpJQhF7zM+wLwLp2ilqgvRhZ1B9H8ykSErT23 Xpa7NAUzlxlL3Ca5r1jF7G10L+O46IpTz9+F1j5uvY1wXCjK9EP+HctpLzBRf6Hz ng6EyO/cfto= =nPmM -----END PGP SIGNATURE----- From kmtracey at gmail.com Sun Jul 20 06:55:25 2008 From: kmtracey at gmail.com (Karen Tracey) Date: Sun, 20 Jul 2008 00:55:25 -0400 Subject: [Python-Dev] Fwd: Change in repr of Decimal in 2.6 In-Reply-To: <g5u993$3p2$1@ger.gmane.org> References: <af3536270807182005r9418665jbde7a4b614bedef4@mail.gmail.com> <7476654F2F8D4542941C18DB53D4E81E@RaymondLaptop1> <af3536270807182048y7f84d293l7fd938aa3a17effe@mail.gmail.com> <af3536270807182059p2e371695y92ed1f096f910d6a@mail.gmail.com> <g5u993$3p2$1@ger.gmane.org> Message-ID: <af3536270807192155q3381a21cl3f94ca72794acf88@mail.gmail.com> On Sat, Jul 19, 2008 at 10:52 PM, Terry Reedy <tjreedy at udel.edu> wrote: > >>> f.clean('1') == Decimal('1') > True > Ah, yes, why didn't I think of that? > > Since 'True' is a keyword, and Guido is *very* reluctant to even add > keywords, let alone change their spelling, I think you can depend on that > working indefinitely. Similarly, if you now have > > >>> type(a) > <type 'int'> > > you test will fail in 3.0 which instead prints '<class 'int'>. But > > >>>type(a) is int > True > > will continue working. > > Doctest has two quite different uses. One is to check text with > interactive examples to make sure there are no mistakes in the examples. > For that, printing representations is usually the natural style. Then one > must accept that representations change faster that object behavior. > > The other use is to check code. For that, the more stilted style is more > future-proof. For text doing double duty as a formal reference and code > test, I would go for cross-version dependability. > > Thanks, I see your point about the different styles. I think we have a fair number of tests that use the print-style (probably because that's easiest, and then people see that style everywhere and do the same when adding new tests) when to be future proof they really should be using the alternative. Karen -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080720/0853ceb9/attachment.htm> From barry at barrys-emacs.org Sun Jul 20 09:44:05 2008 From: barry at barrys-emacs.org (Barry Scott) Date: Sun, 20 Jul 2008 08:44:05 +0100 Subject: [Python-Dev] Web site type: Python 2.6b2 Released: 18-Jun-2008 Message-ID: <AA73256D-736B-4923-9102-6A8D6CDB05A0@barrys-emacs.org> I think you mean july. Barry From barry at barrys-emacs.org Sun Jul 20 10:57:21 2008 From: barry at barrys-emacs.org (Barry Scott) Date: Sun, 20 Jul 2008 09:57:21 +0100 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <488150A5.8050804@jcea.es> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> Message-ID: <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> See http://code.google.com/p/python-incompatibility/source/checkout Barry On Jul 19, 2008, at 03:25, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Lennart Regebro wrote: > | On Sun, May 25, 2008 at 6:25 AM, Jesus Cea <jcea at jcea.es> wrote: > |> Since I need to port bsddb3 to py3k, what I need to know?. Is any > |> *updated* document out there?. > | > | No, but there is a not yet complete, but quite updated set of > examples. > | > | http://code.google.com/p/python-incompatibility/ > | > | This is a collection of code snippets that will show code that does > | work under 2.x but not under 3.x, and the 3.x way of doing it (as > well > | as a way that works under both 2.6 and 3.0, in applicable cases). > | > | There is no tests for changes in the C-API, if you (or somebody > else) > | would like to add that (or any other tests) that would be very > | appreciated! > > I can't checkout the code. What username/password must I use?. > > - -- > Jesus Cea Avion _/_/ _/_/_/ _/_/_/ > jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ > _/_/ > jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ > . _/_/ _/_/ _/_/ _/_/ _/_/ > "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ > "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ > "El amor es poner tu felicidad en la felicidad de otro" - Leibniz > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.8 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iQCVAwUBSIFQnZlgi5GaxT1NAQL3LQP/ZxblxpJwZAQ9qIUXHAZpFlmK86y7UPfT > DX707wR1YMOujiNazE2NuJSVCVkbbOlc1G2dxTAqDm7FAThkjuIeO74ijueUeFdN > p6LP2dS5pHH8h8G9bUfDynGCjRQ4t1TUblksQgPDtrYiXlzIBqYL1qkHRf72w2c3 > fyuZWBpaH0w= > =nnN4 > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/barry > %40barrys-emacs.org > From barry at python.org Sun Jul 20 14:23:28 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 20 Jul 2008 08:23:28 -0400 Subject: [Python-Dev] Web site type: Python 2.6b2 Released: 18-Jun-2008 In-Reply-To: <AA73256D-736B-4923-9102-6A8D6CDB05A0@barrys-emacs.org> References: <AA73256D-736B-4923-9102-6A8D6CDB05A0@barrys-emacs.org> Message-ID: <11B5E8BA-1BEB-4D9F-A2FE-6AA4D49B020C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 20, 2008, at 3:44 AM, Barry Scott wrote: > I think you mean july. Thanks, I'll fix that. - -B -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSIMuQXEjvBPtnXfVAQLFVwP/SYQkNNHnReOOuPxnnJQkNqKTnDYLpZqT 9J0y/fExHPDNxrawPnxQfxKO9ARFv+DrJSO0aBaQrQP3F164xVfudSM/jN1eq9Th w5im9VxdQTDp/QueK2JZA14LkGK+tj0owY7W599GFavX/OE1nUAWA16YwQAXI7EL mJO4KyFciOk= =v/Pl -----END PGP SIGNATURE----- From josiah.carlson at gmail.com Sun Jul 20 19:24:56 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 20 Jul 2008 10:24:56 -0700 Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was Re: No beta2 tonight) In-Reply-To: <4882986A.3060509@jcea.es> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> <4881C04E.5060208@gmail.com> <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com> <4882986A.3060509@jcea.es> Message-ID: <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com> On Sat, Jul 19, 2008 at 6:44 PM, Jesus Cea <jcea at jcea.es> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Josiah Carlson wrote: > | On-disk key -> value dictionary. In every use of bsddb that I've seen > | (or done myself), that's been the extent of it's use. That's what I > | *was* offering. But it seems that everyone has had experience with > | bsddb on a far deeper level (beyond dictionary + cursor access), and > | would find (like you) absolutely no use in an on-disk dictionary with > | a sqlite backend (which hasn't had the same maintenance issues as > | bsddb), which is why I withdrew the offer. > > For such a simple application, you can use the already in stdlib "gdbm" > module. Just remember it is not transactional, so beware diskfulls, > application crashes, etc. But sqlite is transactional, can offer cursors, getrange, etc., etc. I'm still curious as to what deep features people are using in bsddb. Anyone have any pointers to open source software? - Josiah From greg at krypto.org Sun Jul 20 20:19:13 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 20 Jul 2008 11:19:13 -0700 Subject: [Python-Dev] Search broken on the python dev docs? Message-ID: <52dc1c820807201119k5b3589cfwb0b378c8da2c008e@mail.gmail.com> http://docs.python.org/dev/ the search box worked for earlier releases but has been broken and returns nothing useful of late. If I enter simple terms like 'time' or 'os' or 'os.walk' what is returned is pathetic. how does this work? is an index corrupt or not being regenerated? -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080720/e473c036/attachment.htm> From victor.stinner at haypocalc.com Sun Jul 20 22:42:05 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sun, 20 Jul 2008 22:42:05 +0200 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <4222a8490807190614m370447d9s30d4af12de3e3152@mail.gmail.com> References: <200807191323.13124.victor.stinner@haypocalc.com> <4222a8490807190614m370447d9s30d4af12de3e3152@mail.gmail.com> Message-ID: <200807202242.05347.victor.stinner@haypocalc.com> Hi, Le Saturday 19 July 2008 15:14:44, vous avez ?crit : > Thank you Victor - I didn't want to change any underlying > multiprocessing code until we had the test suite in a better state > (which we do now) (...) > > One suggestion would be to include tests to prove the bugs is fixed if > possible (to the patch), so we can add them to the suite. Ok, here is a patch for test_multiprocessing.py. - TestClosedFile.test_open() verify that Connection() rejects closed file descriptor - TestClosedFile.test_operations() verify that Connection() raises IOError for operations on closed file I don't know if Connection() is a socket or a pipe. Both should be tested. Victor -------------- next part -------------- A non-text attachment was scrubbed... Name: test_multiprocessing.patch Type: text/x-diff Size: 1883 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-dev/attachments/20080720/87610750/attachment.patch> From victor.stinner at haypocalc.com Sun Jul 20 22:45:39 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sun, 20 Jul 2008 22:45:39 +0200 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <20080719195209.GA18271@amk.local> References: <200807191323.13124.victor.stinner@haypocalc.com> <20080719195209.GA18271@amk.local> Message-ID: <200807202245.39874.victor.stinner@haypocalc.com> Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit?: > Excellent work! Another fruitful area for fuzzing might be the > miniature virtual machine used by the re module. It's possible to > import _sre and call the compile() function directly (see the end of > Lib/sre_compile.py for how it's invoked); I wonder how the regex VM > copes with random strings of bytecode. Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted _sre.compile() in my fuzzer. For information, it's also very easy to crash CPython with fuzzed .pyc file. It's hard to check bytecode without execute it. It's maybe better to add checks directly in the VM. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From g.brandl at gmx.net Sun Jul 20 23:39:54 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 20 Jul 2008 23:39:54 +0200 Subject: [Python-Dev] Search broken on the python dev docs? In-Reply-To: <52dc1c820807201119k5b3589cfwb0b378c8da2c008e@mail.gmail.com> References: <52dc1c820807201119k5b3589cfwb0b378c8da2c008e@mail.gmail.com> Message-ID: <g60bbq$nv3$1@ger.gmane.org> Gregory P. Smith schrieb: > http://docs.python.org/dev/ > > the search box worked for earlier releases but has been broken and > returns nothing useful of late. > > If I enter simple terms like 'time' or 'os' or 'os.walk' what is > returned is pathetic. > > how does this work? is an index corrupt or not being regenerated? Something is wrong; I'll check this and try to fix it. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From andrewm at object-craft.com.au Mon Jul 21 02:13:02 2008 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Mon, 21 Jul 2008 10:13:02 +1000 Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was Re: No beta2 tonight) In-Reply-To: <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> <4881C04E.5060208@gmail.com> <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com> <4882986A.3060509@jcea.es> <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com> Message-ID: <20080721001302.EAED5600808@longblack.object-craft.com.au> >But sqlite is transactional, can offer cursors, getrange, etc., etc. > >I'm still curious as to what deep features people are using in bsddb. It's not using "deep features", unless you define their on-disk layout as deep, but it does get used for things such as interactions with other systems - for example, using it to maintain Radius user databases for a (proprietary/commercial) Radius auth daemon. But dropping it from the core won't stop this. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From steve at holdenweb.com Mon Jul 21 03:37:47 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 20 Jul 2008 21:37:47 -0400 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com> References: <200807191323.13124.victor.stinner@haypocalc.com> <20080719195209.GA18271@amk.local> <200807202245.39874.victor.stinner@haypocalc.com> Message-ID: <4883E86B.1000901@holdenweb.com> Victor Stinner wrote: > Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit : >> Excellent work! Another fruitful area for fuzzing might be the >> miniature virtual machine used by the re module. It's possible to >> import _sre and call the compile() function directly (see the end of >> Lib/sre_compile.py for how it's invoked); I wonder how the regex VM >> copes with random strings of bytecode. > > Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted > _sre.compile() in my fuzzer. > > For information, it's also very easy to crash CPython with fuzzed .pyc file. > > It's hard to check bytecode without execute it. It's maybe better to add > checks directly in the VM. > I think you'll find most developers (and many users too, come to that) reluctant to add any checking that would slow down eval.c, the heart of the virtual machine. So unless you can find a way to add the checks without slowing it down, an external checker might be better. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Mon Jul 21 03:37:47 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 20 Jul 2008 21:37:47 -0400 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com> References: <200807191323.13124.victor.stinner@haypocalc.com> <20080719195209.GA18271@amk.local> <200807202245.39874.victor.stinner@haypocalc.com> Message-ID: <4883E86B.1000901@holdenweb.com> Victor Stinner wrote: > Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit : >> Excellent work! Another fruitful area for fuzzing might be the >> miniature virtual machine used by the re module. It's possible to >> import _sre and call the compile() function directly (see the end of >> Lib/sre_compile.py for how it's invoked); I wonder how the regex VM >> copes with random strings of bytecode. > > Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted > _sre.compile() in my fuzzer. > > For information, it's also very easy to crash CPython with fuzzed .pyc file. > > It's hard to check bytecode without execute it. It's maybe better to add > checks directly in the VM. > I think you'll find most developers (and many users too, come to that) reluctant to add any checking that would slow down eval.c, the heart of the virtual machine. So unless you can find a way to add the checks without slowing it down, an external checker might be better. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From jcea at jcea.es Mon Jul 21 11:06:51 2008 From: jcea at jcea.es (Jesus Cea) Date: Mon, 21 Jul 2008 11:06:51 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> Message-ID: <488451AB.50406@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Barry Scott wrote: | See http://code.google.com/p/python-incompatibility/source/checkout Thanks. I'm *VERY* interested in 2.6->3.0 migration guide for C module extensions. 3.0 is around the corner and the API is changing almost daily :-p. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIRRpZlgi5GaxT1NAQLUlwP9GDwp2zKFfHQWmRnhOyGfTwa0GqWGW1rZ IrHrPXgFweh1mnbhGIjVkvlMJxHnhP2yGKStkN4641zYJ5KgdRJBaIdfrqoOkPTb xzzHJ7DqaAGmRSKRoCZ8jzjCuC4BKeuLalii3V7ofYBimDgsHL42lb0MWxZBVNK5 SnafyX+rqGU= =re/E -----END PGP SIGNATURE----- From g.brandl at gmx.net Mon Jul 21 11:11:50 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 21 Jul 2008 11:11:50 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <488451AB.50406@jcea.es> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> Message-ID: <g61jt8$hag$2@ger.gmane.org> Jesus Cea schrieb: > Barry Scott wrote: > | See http://code.google.com/p/python-incompatibility/source/checkout > > Thanks. > > I'm *VERY* interested in 2.6->3.0 migration guide for C module > extensions. 3.0 is around the corner and the API is changing almost > daily :-p. So it's good that nobody has written a migration guide yet; he'd have to rewrite it daily. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From regebro at gmail.com Mon Jul 21 11:26:07 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 21 Jul 2008 11:26:07 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <488451AB.50406@jcea.es> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> Message-ID: <319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com> On Mon, Jul 21, 2008 at 11:06, Jesus Cea <jcea at jcea.es> wrote: > I'm *VERY* interested in 2.6->3.0 migration guide for C module > extensions. 3.0 is around the corner and the API is changing almost > daily :-p. It would be great if python-incompatibility would have examples of the C-api changes as well. That would really help in migrating and writing a migration guide. It would be great if you could help with this! -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From jcea at jcea.es Mon Jul 21 11:26:43 2008 From: jcea at jcea.es (Jesus Cea) Date: Mon, 21 Jul 2008 11:26:43 +0200 Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was Re: No beta2 tonight) In-Reply-To: <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807180721u5bd34bflac00cd319bdf90c0@mail.gmail.com> <e6511dbf0807180757p2a36ff38xae7445378f13b0ef@mail.gmail.com> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> <4881C04E.5060208@gmail.com> <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com> <4882986A.3060509@jcea.es> <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com> Message-ID: <48845653.5000100@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Josiah Carlson wrote: | I'm still curious as to what deep features people are using in bsddb. | Anyone have any pointers to open source software? I'm using replication and distributed transactions. Database encryption and page integrity checks. Abusing the master election BDB infrastructure for some evil but fun purposes. Managing databases in the 200 terabyte range. Locking & logging infrastructure to implement application logic integrated with database operation. MVCC everywhere. Nothing I can show you without killing you after, though. Independently of current bsddb usage profile, current 2.6 code support for distributed transactions and replication enables new application uses that sqlite can't match. Since I'm taking full responsability over bsddb both in 2.6 and 3.0, I don't really see what the issue is. Past mistakes and maintenance nightmares WILL NOT be repeated. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIRWSplgi5GaxT1NAQL2CgQAnbv7hrxb4I5KzdZ1T05A/pGzzuX4NATm SN5TBeKsesl1LlEwI1+5xaBv9uxuJFfnPYMFLlW7NZdGumIdNCo7QXMhPmEdvuNx 2BxcmlS2Zuag8piAbmgq31Ov2RnnskOCf5ZxmuK4smk5Y0H2L+hUGG96Rk8dz9BU s+AVkiaorKY= =szyU -----END PGP SIGNATURE----- From jcea at jcea.es Mon Jul 21 11:34:23 2008 From: jcea at jcea.es (Jesus Cea) Date: Mon, 21 Jul 2008 11:34:23 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <g61jt8$hag$2@ger.gmane.org> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org> Message-ID: <4884581F.8010201@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Georg Brandl wrote: | So it's good that nobody has written a migration guide yet; he'd have | to rewrite it daily. Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid redoing the same work 15 times XDDDDDDDDDD I'm not lazy, but it is so sunny outside... - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIRYHplgi5GaxT1NAQL4hwP+LUSeAZxP/6q2K8GBs9ptqMnZUPh6ct0j mhJqnAGDrI3XgcyHa3DeWwAltIfbLj9KC+UcgBNSvVQvqqMaPxKhc+tsbXqmsude +o2e9IhZCgWg91FyJv64LPJr/tgMG4g7QBxHR1G9GF76YYrjCQCzi1f06lNkplQh o/KIbRBmwRc= =Q6o6 -----END PGP SIGNATURE----- From jcea at jcea.es Mon Jul 21 11:48:05 2008 From: jcea at jcea.es (Jesus Cea) Date: Mon, 21 Jul 2008 11:48:05 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com> Message-ID: <48845B55.3090306@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lennart Regebro wrote: | On Mon, Jul 21, 2008 at 11:06, Jesus Cea <jcea at jcea.es> wrote: |> I'm *VERY* interested in 2.6->3.0 migration guide for C module |> extensions. 3.0 is around the corner and the API is changing almost |> daily :-p. | | It would be great if python-incompatibility would have examples of the | C-api changes as well. That would really help in migrating and writing | a migration guide. It would be great if you could help with this! I can comment about some issues I had this weekend. Remember that my intention is to keep a single C codebase for 2.6 and 3.0. * Int/Long integration. In Python 3.0 "PyInt_*" has vanished. But using "PyLong_*" in Python 2.x surfaces so many issues that I have decided to define "NUMBER_*" macros to be conditionally expanded to "PyInt"/"PyLong" when compiling to 2.x/3.0. Not nice, but I can't see a better way. * The module initialization has changed completely. In fact I am fighting this, still. Some help needed. * "Py_FindMember" was eliminated ?last week?. I workaround this, but I don't know really how to write a "smart" getattr function now. Any pointer? At least my current code compiles under 3.0, but it still bombs when importing it. Life sucks. PS: I'm learning the hard way, doing "diff" between 2.6 and 3.0 module sourcecode. It must be a better way!. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIRbTplgi5GaxT1NAQJTBAP/cHndbYfnpNF+UoJP0vswoJrMtrNRu6F4 TAAMGbv4sfE5a308sfjoXxFuvUA8kXAyqdJCMXlNWDfX6CEsozQwoS06d6HNkkZn k9iGEbjwiWVdcw0y6gXvt9rL+klK004Rg4pUHMYtO3yUbaFSqvCI1kox6Rf1Q4dB 3CywJ2/fAu4= =V42Q -----END PGP SIGNATURE----- From regebro at gmail.com Mon Jul 21 11:59:42 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 21 Jul 2008 11:59:42 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <48845B55.3090306@jcea.es> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <319e029f0807210226p35a6860fl6789d702925f8a11@mail.gmail.com> <48845B55.3090306@jcea.es> Message-ID: <319e029f0807210259q36250024vc1147c47ff3953ee@mail.gmail.com> On Mon, Jul 21, 2008 at 11:48, Jesus Cea <jcea at jcea.es> wrote: > I can comment about some issues I had this weekend. I don't do C development myself, so comments aren't that useful for me, but code examples are, so we can stick them into python-incompatibility. > Remember that my intention is to keep a single C codebase for 2.6 and 3.0. Cool. > * Int/Long integration. In Python 3.0 "PyInt_*" has vanished. But using > "PyLong_*" in Python 2.x surfaces so many issues that I have decided to > define "NUMBER_*" macros to be conditionally expanded to > "PyInt"/"PyLong" when compiling to 2.x/3.0. Not nice, but I can't see a > better way. Seems resonable. > PS: I'm learning the hard way, doing "diff" between 2.6 and 3.0 module > sourcecode. It must be a better way!. Yeah, these changes should be properly documented in the CHANGES.txt. I've seen some C-API chnges mentiones at least. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From jcea at jcea.es Mon Jul 21 12:07:04 2008 From: jcea at jcea.es (Jesus Cea) Date: Mon, 21 Jul 2008 12:07:04 +0200 Subject: [Python-Dev] Subversion 1.5 and better merge support Message-ID: <48845FC8.1030506@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Subversion 1.5 solves the "repeated merge" problem. At last!!. Somebody is considering upgrading python svn to 1.5, and allowing only commits coming from 1.5 clients?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIRfxZlgi5GaxT1NAQLvggQAj8+RlRxL6Vtntr9GhMIcQ4WnyY7FezKn XyoMwN7mMzOPOxcyiKhakk/bQFLwezBW2tYa5Z40tI5oD8t/xJx6+1gJ2e3aMQAm NttgAke0ix8Q79kOI8iak4Kp60AeO/eD/80VIY6yOkaEAygOPT2WqzPofMO5bkCK E/MwQaqwwK4= =hyn0 -----END PGP SIGNATURE----- From bristow.thankachan at gmail.com Mon Jul 21 13:13:21 2008 From: bristow.thankachan at gmail.com (Bristow Thankachan) Date: Mon, 21 Jul 2008 16:43:21 +0530 Subject: [Python-Dev] Syntax error in python2.6 Message-ID: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> Hi everybody, During the porting of Zope2 to Python2.6, I am stuck with a syntax error in the module AccessControl, which is given below. def reorder(s, with=None, without=()): ^ SyntaxError: invalid syntax in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py. The same code when run in python2.4 and python2.5 didn't give any syntax errors. Can anybody suggest the reason for this syntax error in python2.6. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/88126a0e/attachment.htm> From jnoller at gmail.com Mon Jul 21 13:25:55 2008 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 21 Jul 2008 07:25:55 -0400 Subject: [Python-Dev] Syntax error in python2.6 In-Reply-To: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> Message-ID: <5F6472C2-10A8-4227-8F2D-D78B6FABD3EE@gmail.com> Because as of 2.6 the 'with' word is now part of the with statement, see pep 343. On Jul 21, 2008, at 7:13 AM, "Bristow Thankachan" <bristow.thankachan at gmail.com > wrote: > Hi everybody, > > During the porting of Zope2 to Python2.6, I am stuck with a syntax > error in the module AccessControl, which is given below. > > def reorder(s, with=None, without=()): > ^ > SyntaxError: invalid syntax > > in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/ > Utilities.py. > The same code when run in python2.4 and python2.5 didn't give any > syntax errors. Can anybody suggest the reason for this syntax error > in python2.6. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com From bristow.thankachan at gmail.com Mon Jul 21 13:45:09 2008 From: bristow.thankachan at gmail.com (Bristow Thankachan) Date: Mon, 21 Jul 2008 17:15:09 +0530 Subject: [Python-Dev] [Zope-dev] Syntax error in python2.6 In-Reply-To: <488472BB.7070608@wildenhain.de> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> <488472BB.7070608@wildenhain.de> Message-ID: <d886b2060807210445t1e4fa661k4a49f5a2b18aa023@mail.gmail.com> Thank you for the quick response. I made the change and got the syntax error corrected. with regards Bristow. On Mon, Jul 21, 2008 at 4:57 PM, Tino Wildenhain <tino at wildenhain.de> wrote: > Bristow Thankachan wrote: > >> Hi everybody, >> >> During the porting of Zope2 to Python2.6, I am stuck with a syntax error >> in the module AccessControl, which is given below. >> >> def reorder(s, with=None, without=()): >> ^ >> SyntaxError: invalid syntax >> >> in line 56 of >> /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py. >> The same code when run in python2.4 and python2.5 didn't give any syntax >> errors. Can anybody suggest the reason for this syntax error in python2.6. >> > > I'd say, "with" is now a keyword. > > http://docs.python.org/ref/keywords.html > > btw, shouldn't this already give a warning in 2.5? > > Cheers > Tino > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/b98cf97c/attachment.htm> From mal at egenix.com Mon Jul 21 14:03:08 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 21 Jul 2008 14:03:08 +0200 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com> References: <200807191323.13124.victor.stinner@haypocalc.com> <20080719195209.GA18271@amk.local> <200807202245.39874.victor.stinner@haypocalc.com> Message-ID: <48847AFC.8000206@egenix.com> On 2008-07-20 22:45, Victor Stinner wrote: > Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez ?crit : >> Excellent work! Another fruitful area for fuzzing might be the >> miniature virtual machine used by the re module. It's possible to >> import _sre and call the compile() function directly (see the end of >> Lib/sre_compile.py for how it's invoked); I wonder how the regex VM >> copes with random strings of bytecode. > > Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted > _sre.compile() in my fuzzer. > > For information, it's also very easy to crash CPython with fuzzed .pyc file. > > It's hard to check bytecode without execute it. It's maybe better to add > checks directly in the VM. I don't see that as a big problem: if you execute untrusted byte code, you are on your own anyway... whether that's byte code for the re engine or ceval. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 21 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From tino at wildenhain.de Mon Jul 21 13:27:55 2008 From: tino at wildenhain.de (Tino Wildenhain) Date: Mon, 21 Jul 2008 13:27:55 +0200 Subject: [Python-Dev] [Zope-dev] Syntax error in python2.6 In-Reply-To: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> Message-ID: <488472BB.7070608@wildenhain.de> Bristow Thankachan wrote: > Hi everybody, > > During the porting of Zope2 to Python2.6, I am stuck with a syntax error > in the module AccessControl, which is given below. > > def reorder(s, with=None, without=()): > ^ > SyntaxError: invalid syntax > > in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py. > The same code when run in python2.4 and python2.5 didn't give any syntax > errors. Can anybody suggest the reason for this syntax error in python2.6. I'd say, "with" is now a keyword. http://docs.python.org/ref/keywords.html btw, shouldn't this already give a warning in 2.5? Cheers Tino -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3241 bytes Desc: S/MIME Cryptographic Signature URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/bff9aa2c/attachment.bin> From bristow.thankachan at gmail.com Mon Jul 21 14:09:08 2008 From: bristow.thankachan at gmail.com (Bristow Thankachan) Date: Mon, 21 Jul 2008 17:39:08 +0530 Subject: [Python-Dev] Syntax error in python2.6 In-Reply-To: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> Message-ID: <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com> hi all, I changed the attribute 'with' in /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py to 'wit' and I ran the tests in python2.4 and I got the following result. Running tests at level 1 Running unit tests: Running: Ran 285 tests with 0 failures and 0 errors in 6.587 seconds. Running Testing.ZopeTestCase.layer.ZopeLite tests: Set up Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds. Running: Ran 1 tests with 0 failures and 0 errors in 0.064 seconds. Tearing down left over layers: Tear down Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds. Total: 286 tests, 0 failures, 0 errors in 1.044 seconds. Can anybody tell me whether this implies it is backward compatible in python2.4? with regards Bristow On Mon, Jul 21, 2008 at 4:43 PM, Bristow Thankachan < bristow.thankachan at gmail.com> wrote: > Hi everybody, > > > During the porting of Zope2 to Python2.6, I am stuck with a syntax error in > the module AccessControl, which is given below. > > def reorder(s, with=None, without=()): > ^ > SyntaxError: invalid syntax > > in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py. > The same code when run in python2.4 and python2.5 didn't give any syntax > errors. Can anybody suggest the reason for this syntax error in python2.6. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/dce77047/attachment-0001.htm> From bristow.thankachan at gmail.com Mon Jul 21 14:12:38 2008 From: bristow.thankachan at gmail.com (Bristow Thankachan) Date: Mon, 21 Jul 2008 17:42:38 +0530 Subject: [Python-Dev] import error in python2.6 Message-ID: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com> Hi all, During the porting of Zope2, I am stuck with import errors in many modules. The same code works well in python2.5 and 2.4. Can anybody give the details of this import error in python2.6 and how to get the error corrected? with regards Bristow -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/c996c210/attachment.htm> From qgallet at gmail.com Mon Jul 21 14:20:36 2008 From: qgallet at gmail.com (Quentin Gallet-Gilles) Date: Mon, 21 Jul 2008 14:20:36 +0200 Subject: [Python-Dev] import error in python2.6 In-Reply-To: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com> References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com> Message-ID: <8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com> Hi Bristow, You didn't provide any broken code that could help us give an explanation. Also this kind of question is best answered on the python-users mailing list. Python-dev is reserved for discussion about the evolution of Python, not its use. Cheers, Quentin On Mon, Jul 21, 2008 at 2:12 PM, Bristow Thankachan < bristow.thankachan at gmail.com> wrote: > Hi all, > > During the porting of Zope2, I am stuck with import errors in many modules. > The same code works well in python2.5 and 2.4. Can anybody give the details > of this import error in python2.6 and how to get the error corrected? > > with regards > > Bristow > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/2f47ed17/attachment.htm> From regebro at gmail.com Mon Jul 21 14:25:57 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 21 Jul 2008 14:25:57 +0200 Subject: [Python-Dev] Syntax error in python2.6 In-Reply-To: <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com> Message-ID: <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com> On Mon, Jul 21, 2008 at 14:09, Bristow Thankachan <bristow.thankachan at gmail.com> wrote: > Can anybody tell me whether this implies it is backward compatible in > python2.4? If you change the API it isn't backwards compatible. The question is if this is a problem or not, if anything outside Zope itself is using this call, which is hard to say. Options here are 1. Deprecating the "with" parameter for "with_" and supporting both in 2.12 but not supporting Python2.6. 2. Using **kw in the argument and looking for noth "with" and "with_", that way, which will be backwards compatible. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From mborch at gmail.com Mon Jul 21 14:30:23 2008 From: mborch at gmail.com (Malthe Borch) Date: Mon, 21 Jul 2008 14:30:23 +0200 Subject: [Python-Dev] Syntax error in python2.6 In-Reply-To: <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com> <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com> Message-ID: <4884815F.5080201@gmail.com> Lennart Regebro wrote: > 2. Using **kw in the argument and looking for noth "with" and "with_", > that way, which will be backwards compatible. +1 \malthe From mborch at gmail.com Mon Jul 21 14:30:23 2008 From: mborch at gmail.com (Malthe Borch) Date: Mon, 21 Jul 2008 14:30:23 +0200 Subject: [Python-Dev] Syntax error in python2.6 In-Reply-To: <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com> <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com> Message-ID: <4884815F.5080201@gmail.com> Lennart Regebro wrote: > 2. Using **kw in the argument and looking for noth "with" and "with_", > that way, which will be backwards compatible. +1 \malthe From musiccomposition at gmail.com Mon Jul 21 15:16:31 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 21 Jul 2008 08:16:31 -0500 Subject: [Python-Dev] Subversion 1.5 and better merge support In-Reply-To: <48845FC8.1030506@jcea.es> References: <48845FC8.1030506@jcea.es> Message-ID: <1afaf6160807210616j57f637a2s4fedb1a9396f1e02@mail.gmail.com> On Mon, Jul 21, 2008 at 5:07 AM, Jesus Cea <jcea at jcea.es> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Subversion 1.5 solves the "repeated merge" problem. At last!!. > > Somebody is considering upgrading python svn to 1.5, and allowing only > commits coming from 1.5 clients?. While Subversion 1.5 has merge tracking, it doesn't support blocking revisions to be merged as well as svnmerge.py. (This is important for merging trunk -> py3k). Basically, I think we can get more from svnmerge.py than builtin merge tracking. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Mon Jul 21 15:20:20 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 21 Jul 2008 08:20:20 -0500 Subject: [Python-Dev] import error in python2.6 In-Reply-To: <8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com> References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com> <8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com> Message-ID: <1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com> On Mon, Jul 21, 2008 at 7:20 AM, Quentin Gallet-Gilles <qgallet at gmail.com> wrote: > Hi Bristow, > > You didn't provide any broken code that could help us give an explanation. > Also this kind of question is best answered on the python-users mailing > list. Python-dev is reserved for discussion about the evolution of Python, > not its use. Since this is prelease software, it's probably ok to talk about issues with it. You could file a bug next time. However, AFAIK, Zope hasn't even been ported to 2.5. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From amk at amk.ca Mon Jul 21 15:33:19 2008 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 21 Jul 2008 09:33:19 -0400 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <200807202245.39874.victor.stinner@haypocalc.com> References: <200807191323.13124.victor.stinner@haypocalc.com> <20080719195209.GA18271@amk.local> <200807202245.39874.victor.stinner@haypocalc.com> Message-ID: <20080721133319.GA15912@amk-desktop.matrixgroup.net> On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote: > Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted > _sre.compile() in my fuzzer. We should certainly try to fix those issues, then; people usually assume the re module is safe for use inside a sandbox and probably aren't careful enough to block importing of the _sre module. --amk From fdrake at acm.org Mon Jul 21 15:33:36 2008 From: fdrake at acm.org (Fred Drake) Date: Mon, 21 Jul 2008 09:33:36 -0400 Subject: [Python-Dev] import error in python2.6 In-Reply-To: <1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com> References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com> <8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com> <1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com> Message-ID: <A31A031A-B209-45A6-9AC9-6A6986AC173E@acm.org> On Jul 21, 2008, at 9:20 AM, Benjamin Peterson wrote: > Since this is prelease software, it's probably ok to talk about issues > with it. You could file a bug next time. However, AFAIK, Zope hasn't > even been ported to 2.5. Many people are using Zope 3 with Python 2.5 without problems, though Python 2.5 isn't "officially" supported (whatever that means). I think current versions of Zope 2 work with Python 2.5 as well. -Fred -- Fred Drake <fdrake at acm.org> From victor.stinner at haypocalc.com Mon Jul 21 15:54:11 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 21 Jul 2008 15:54:11 +0200 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <20080721133319.GA15912@amk-desktop.matrixgroup.net> References: <200807191323.13124.victor.stinner@haypocalc.com> <200807202245.39874.victor.stinner@haypocalc.com> <20080721133319.GA15912@amk-desktop.matrixgroup.net> Message-ID: <200807211554.11468.victor.stinner@haypocalc.com> Le Monday 21 July 2008 15:33:19 A.M. Kuchling, vous avez ?crit?: > On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote: > > Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted > > _sre.compile() in my fuzzer. > > We should certainly try to fix those issues, then; people usually > assume the re module is safe for use inside a sandbox and probably > aren't careful enough to block importing of the _sre module. Why is this function public? Is it used by re module? Only _sre module should be allowed to generated "regex bytecode". -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From bristow.thankachan at gmail.com Mon Jul 21 15:56:05 2008 From: bristow.thankachan at gmail.com (Bristow Thankachan) Date: Mon, 21 Jul 2008 19:26:05 +0530 Subject: [Python-Dev] import error in python2.6 Message-ID: <d886b2060807210656i57c99a3fgc844c1de50b06deb@mail.gmail.com> Hi all, I asked about the import error in python2.6 while trying to port zope2. The error message is given below. File "/home/zope/ztrunk26/lib/python/AccessControl/ImplC.py", line 30, in <module> from ImplPython import RestrictedDTML, SecurityManager, ZopeSecurityPolicy ImportError: No module named ImplPython It works well in python2.4 and 2.5. Please give your suggestions. with regards Bristow -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/a97dd32d/attachment.htm> From regebro at gmail.com Mon Jul 21 16:15:57 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 21 Jul 2008 16:15:57 +0200 Subject: [Python-Dev] [Zope-dev] import error in python2.6 In-Reply-To: <d886b2060807210656i57c99a3fgc844c1de50b06deb@mail.gmail.com> References: <d886b2060807210656i57c99a3fgc844c1de50b06deb@mail.gmail.com> Message-ID: <319e029f0807210715i4776bd67p57dcdcde2f4bfce0@mail.gmail.com> On Mon, Jul 21, 2008 at 15:56, Bristow Thankachan <bristow.thankachan at gmail.com> wrote: > Hi all, > > I asked about the import error in python2.6 while trying to port zope2. The > error message is given below. > > File "/home/zope/ztrunk26/lib/python/AccessControl/ImplC.py", line 30, in > <module> > from ImplPython import RestrictedDTML, SecurityManager, ZopeSecurityPolicy > ImportError: No module named ImplPython > > It works well in python2.4 and 2.5. Please give your suggestions. ImplPython is a python module in Zope. It likely has some sort of syntax error under 2.6 that is not a syntaxerror under 2.5, so the import fails, or the C-modules that it uses fail it's compile during setup, so import of them failed. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From tseaver at palladion.com Mon Jul 21 16:35:44 2008 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 21 Jul 2008 10:35:44 -0400 Subject: [Python-Dev] Syntax error in python2.6 In-Reply-To: <4884815F.5080201@gmail.com> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> <d886b2060807210509l2a3efa32j3cd47a705bcc5bdc@mail.gmail.com> <319e029f0807210525s7943ad81q59ffbdceb641e4b0@mail.gmail.com> <4884815F.5080201@gmail.com> Message-ID: <48849EC0.5020502@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Malthe Borch wrote: > Lennart Regebro wrote: >> 2. Using **kw in the argument and looking for noth "with" and "with_", >> that way, which will be backwards compatible. > > +1 The implementation of this function is already so obscure that using keywords should not be a problem. OTOH, the reorder function in RestrictedPython was ported from a version originally designed for doing ordered select lists in DTML: $ cvs log lib/python/DocumentTemplate/DT_Util.py ... revision 1.52 date: 1999/03/22 17:28:37; author: jim; state: Exp; lines: +31 -2 Added a new "builtin" function reorder: reorder(s, [with, without]]) -- Reorder the items in s according to the order given in with and with items mentioned in without removed. Items from s not mentioned in with are removed. s, with, and without are all either sequences if strings or sequences of key-value tuples, with ordering done on the keys. This function is useful for constructing ordered select lists. ... The only use of the method in Zope's own DTML is in the addZClass.dtml template. I'm willing to bet that *nobody* who is still using it would be surprised if they had to recode it when upgrading to a recent Zope, so maybe we can just change the trunk without worrying about it. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIhJ7A+gerLs4ltQ4RAk6lAJ0T6JJC/4nr6kA7jzSQqKSNZUGP7ACeIO2R H7k2D46BKQ+DL0girE8EGOk= =HLxo -----END PGP SIGNATURE----- From musiccomposition at gmail.com Mon Jul 21 16:47:08 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 21 Jul 2008 09:47:08 -0500 Subject: [Python-Dev] Deprecate parser's "ast" functions? In-Reply-To: <g2ej5h$tnl$1@ger.gmane.org> References: <g2ej5h$tnl$1@ger.gmane.org> Message-ID: <1afaf6160807210747v7756589bnac5d07c6d3bbb6f8@mail.gmail.com> On Sat, Jun 7, 2008 at 3:13 PM, Georg Brandl <g.brandl at gmx.net> wrote: > The parser module exports each function and type twice, once with "AST" in > the name, once with "ST". Since "AST" now has a different meaning for > Python code compilation, I propose to deprecate the "ast" variants in 2.6 > and remove them in Python 3000. +1 This personally has confused me! > > (Also, all keyword arguments are called "ast", that cannot change in 2.6 > but should in 3.0.) > > I'm at the moment changing the documentation of parser to only refer to > the "st" variants anymore. > > Georg -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From solipsis at pitrou.net Mon Jul 21 17:53:18 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 21 Jul 2008 15:53:18 +0000 (UTC) Subject: [Python-Dev] Fuzzing bugs: most bugs are closed References: <200807191323.13124.victor.stinner@haypocalc.com> <200807202245.39874.victor.stinner@haypocalc.com> <20080721133319.GA15912@amk-desktop.matrixgroup.net> <200807211554.11468.victor.stinner@haypocalc.com> Message-ID: <loom.20080721T154744-225@post.gmane.org> Victor Stinner <victor.stinner <at> haypocalc.com> writes: > > Le Monday 21 July 2008 15:33:19 A.M. Kuchling, vous avez ?crit?: > > On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote: > > > Hum... how can I say it? It's trivial to crash _sre So I blacklisted > > > _sre.compile() in my fuzzer. > > > > We should certainly try to fix those issues, then; people usually > > assume the re module is safe for use inside a sandbox and probably > > aren't careful enough to block importing of the _sre module. > > Why is this function public? Is it used by re module? Only _sre module should > be allowed to generated "regex bytecode". The underscore at the beginning of _sre clearly indicates that the module is not recommended for direct consumption, IMO. Even the functions that don't themselves start with an underscore... From cadr4u at gmail.com Mon Jul 21 17:58:20 2008 From: cadr4u at gmail.com (Jakob Sievers) Date: Mon, 21 Jul 2008 17:58:20 +0200 Subject: [Python-Dev] Python VM Message-ID: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com> Hi, I've been reading the Python VM sources over the last few afternoons and I took some notes, which I thought I'd share (and if anyone familiar with the VM internals could have a quick look at them, I'd really appreciate it). Cheers, -jakob Unless otherwise noted, the source file in question is Python/ceval.c. Control Flow ============ The calling sequence is: main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) -> run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx() (ceval.c) -> PyEval_EvalFrameEx() (ceval.c). EvalCodeEx() does some initialization (creating a new execution frame, argument processing, and some generator-specific stuff I haven't looked at yet) before calling EvalFrameEx() which contains the main interpreter loop. Threads ======= PyEval_InitThreads() initializes the GIL (interpreter_lock) and sets main_thread to the (threading package dependent) ID of the current thread. Thread switching is done using PyThreadState_Swap(), which sets _PyThreadState_Current (both defined in pystate.c) and PyThreadState_GET() (an alias for _PyThreadState_Current) (pystate.h). Async Callbacks =============== Asynchronous callbacks can be registered by adding the function to be called to pendingcalls[] (see Py_AddPendingCall()). The state of this queue is communicated to the main loop via things_to_do. State ===== The global state is recorded in a (per-process?) PyInterpreterState struct and a per-thread PyThreadState struct. Each execution frame's state is contained in that frame's PyFrameObject (which includes the instruction stream, the environment (globals, locals, builtins, etc.), the value stack and so forth). EvalFrameEx()'s local variables are initialized from this frame object. Instruction Stream ================== The instruction stream looks as follows (c.f. assemble_emit() in compile.c): A byte stream where each instruction consists of either 1) a single byte opcode: OP 2) a single byte opcode plus a two-byte immediate argument: OP LO HI 3) a special opcode followed by the real opcode followed by a four byte argument: EXTENDED_ARG OP BYTE2 BYTE1 BYTE4 BYTE3 Opcode Prediction ================= One nice trick used to speed up opcode dispatch is the following: Using the macros PREDICT() and PREDICTED() it is sometimes possible to jump directly to the code implementing the next instruction rather than having to go through the whole loop preamble, e.g. case FOO: // ... PREDICT(BAR); continue; PREDICTED(BAR); case BAR: // ... expands to case FOO: // ... if (*next_instr == BAR) goto PRED_BAR; continue; PRED_BAR: next_instr++; case BAR: // ... Main Loop ========= Variables and macros used in EvalFrameEx() ------------------------------------------ The value stack: PyObject **stack_pointer; The instruction stream: unsigned char *next_instr; NEXTOP(), NEXTARG(), PEEKARG(), JUMPTO(), and JUMPBY() simply fiddle with next_instr. Likewise for TOP(), SET_SECOND(), PUSH(), POP(), etc. and stack_pointer. Current opcode plus argument: int opcode; int oparg; Error status: enum why_code why; // no, exn, exn re-raised, return, break, continue, yield int err; // non-zero is error The environment: PyObject *names; PyObject *consts; and PyObject **fastlocals; which is accessed via GETLOCAL() and SETLOCAL(). Finally, there are some more PyObject *'s (v, w, u, and so forth, used as temporary variables) as well as PyObject *retval; Basic structure --------------- EvalFrameEx() { why = WHY_NOT; err = 0; for (;;) { <------------------+---+ // do periodic tasks | | | | fast_next_opcode: | | opcode = NEXTOP(); | | if (HAS_ARG(opcode)) | | oparg = NEXTARG(); | | | | dispatch_opcode: | | switch(opcode) { | | | | continue; -------------------+ | | break; ----------------------+ | | | // Also, opcode prediction | | // jumps around inside the | | // switch statement | | | | } <-----------------------+ | | on_error: | // no error: continue -----------+ // otherwise why == WHY_EXCEPTION after this fast_block_end: // unwind stacks if there was an error } // more unwinding fast_yield: // reset current thread's exception info exit_eval_frame: // set thread's execution frame to previous execution frame return retval; } Periodic Tasks -------------- By checking and decrementing _Py_Ticker, the main loop executes certain tasks once every _Py_CheckInterval iterations (in fact Py_AddPendingCall() sets _Py_Ticker to zero, ensuring that pending calls are executed right after the next instruction which doesn't jump to fast_next_opcode): - If there are things_to_do, Py_MakePendingCalls() is called. - The GIL is releases and re-acquired, giving other threads a chance to run. Instruction implementation -------------------------- Some general notes: - Successful instructions either goto fast_next_opcode or continue. - Unsuccessful instructions break out of the switch. - The value stack holds only PyObject *'s. - Instructions must take care to Py_INCREF() and Py_DECREF() the reference counts of PyObject's whose addresses have been pushed onto/popped off the value stack. - Objects are transferred onto the value stack by GETITEM()'ing them from consts or names, or by GETLOCAL()'ing them using oparg as an offset into fastlocals (c.f. LOAD_* instructions). - err is used as a general `error occurred' flag, both inside the code implementing an opcode and `globally' for the entire loop. Nested blocks ------------- Nested loop and try blocks are handled as follows: Each frame maintains a block stack; when entering a nested block, a SETUP_* instruction adds a PyTryBlock to the PyFrameObject's f_blockstack and registers that block's type (instruction which created the block), handler (offset into the instruction stream) and level (value stack level before the nested block was entered). When such blocks are exited normally (e.g. last iteration of a loop), the final POP_BLOCK instruction restores the value stack to the state it was in before the block. If a block is exited abnormally (e.g. a break instruction), the code following fast_block_end unwinds the value stack and jumps to the block's handler. Certain instructions, e.g. RETURN_VALUE, cause the entire block stack to be unwound (leading to multiple unwinds of the value stack). See also the comments in compile.c (compiler_try_finally()) which include nice ASCII art diagrams (incidentally, there's a typo on line 2382: s/en/an/). Error handling -------------- Internal errors (bad oparg, say) generally result in why being set to WHY_EXCEPTION and breaking out of the switch (if the code implementing the instruction doesn't set why, the code after on_error will). The code following fast_block_end will jump to the correct exception handler and set the global exception related variables (exception information is stored in the current execution frame and the thread state. See set_exc_info() and reset_exc_info()). // eof From amk at amk.ca Mon Jul 21 19:41:41 2008 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 21 Jul 2008 13:41:41 -0400 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <loom.20080721T154744-225@post.gmane.org> References: <200807191323.13124.victor.stinner@haypocalc.com> <200807202245.39874.victor.stinner@haypocalc.com> <20080721133319.GA15912@amk-desktop.matrixgroup.net> <200807211554.11468.victor.stinner@haypocalc.com> <loom.20080721T154744-225@post.gmane.org> Message-ID: <20080721174141.GA16598@amk-desktop.matrixgroup.net> yOn Mon, Jul 21, 2008 at 03:53:18PM +0000, Antoine Pitrou wrote: > The underscore at the beginning of _sre clearly indicates that the module is > not recommended for direct consumption, IMO. Even the functions that don't > themselves start with an underscore... Sure, but if someone is trying to break in or DoS your application server, they don't care if the module starts with an underscore or not. To answer Victor's original question: the parser & compiler that turn a regex into bytecode is written in Python. I can't think of a way to prevent other Python modules from importing _sre or accessing the compile() function; if nothing else, code could always do 'import re ; re.sre_compile._sre.compile(...)'. --amk From brett at python.org Mon Jul 21 20:16:22 2008 From: brett at python.org (Brett Cannon) Date: Mon, 21 Jul 2008 11:16:22 -0700 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <4884581F.8010201@jcea.es> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org> <4884581F.8010201@jcea.es> Message-ID: <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> On Mon, Jul 21, 2008 at 2:34 AM, Jesus Cea <jcea at jcea.es> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Georg Brandl wrote: > | So it's good that nobody has written a migration guide yet; he'd have > | to rewrite it daily. > > Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid > redoing the same work 15 times XDDDDDDDDDD > But waiting until all the betas have gone out totally defeats the purpose of the betas! It has already been stated that new code changes that are even remotely shaky or anything not small needs a code review at this point since we only have one beta left. Barry can correct me if I am wrong, but dropping a major rewrite of bsddb into 3.0 between b3 and rc1 is not acceptable. I wouldn't be surprised if all code after b3 requires a code review since at that point we should just be squashing bugs, not rewriting code. And if you are migrating all of bsddb3 at once then that is going to be a massive code review that should be happening in prep for rc1. I really hate feeling like I am coming off as a jerk harping on this, Jesus, as I don't want to discourage your contributions to pybsddb, but it just doesn't feel reasonable to me to be doing this so late in the release cycle. -Brett From brett at python.org Mon Jul 21 20:18:38 2008 From: brett at python.org (Brett Cannon) Date: Mon, 21 Jul 2008 11:18:38 -0700 Subject: [Python-Dev] Subversion 1.5 and better merge support In-Reply-To: <1afaf6160807210616j57f637a2s4fedb1a9396f1e02@mail.gmail.com> References: <48845FC8.1030506@jcea.es> <1afaf6160807210616j57f637a2s4fedb1a9396f1e02@mail.gmail.com> Message-ID: <bbaeab100807211118i6d6d1c35if2d6699b061da423@mail.gmail.com> On Mon, Jul 21, 2008 at 6:16 AM, Benjamin Peterson <musiccomposition at gmail.com> wrote: > On Mon, Jul 21, 2008 at 5:07 AM, Jesus Cea <jcea at jcea.es> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Subversion 1.5 solves the "repeated merge" problem. At last!!. >> >> Somebody is considering upgrading python svn to 1.5, and allowing only >> commits coming from 1.5 clients?. > > While Subversion 1.5 has merge tracking, it doesn't support blocking > revisions to be merged as well as svnmerge.py. (This is important for > merging trunk -> py3k). Basically, I think we can get more from > svnmerge.py than builtin merge tracking. > This has also been discussed by the infrastructure committee and the person in charge of maintaining the svn install does not want to upgrade until 1.5 lands in debian-stable. So as of right now there are no immediate plans to upgrade. -Brett From musiccomposition at gmail.com Mon Jul 21 20:50:44 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 21 Jul 2008 13:50:44 -0500 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org> <4884581F.8010201@jcea.es> <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> Message-ID: <1afaf6160807211150m64ec7b46ha5d71c5492520f03@mail.gmail.com> On Mon, Jul 21, 2008 at 1:16 PM, Brett Cannon <brett at python.org> wrote: > On Mon, Jul 21, 2008 at 2:34 AM, Jesus Cea <jcea at jcea.es> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Georg Brandl wrote: > > | So it's good that nobody has written a migration guide yet; he'd have > > | to rewrite it daily. > > > > Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid > > redoing the same work 15 times XDDDDDDDDDD > > > > But waiting until all the betas have gone out totally defeats the > purpose of the betas! It has already been stated that new code changes > that are even remotely shaky or anything not small needs a code review > at this point since we only have one beta left. Barry can correct me > if I am wrong, but dropping a major rewrite of bsddb into 3.0 between > b3 and rc1 is not acceptable. I wouldn't be surprised if all code > after b3 requires a code review since at that point we should just be > squashing bugs, not rewriting code. And if you are migrating all of > bsddb3 at once then that is going to be a massive code review that > should be happening in prep for rc1. I have a sickening feeling that this isn't the only major change that may hit the betas. PEP 3118 is only about 2/3's implemented (with hardly any tests, I might add), and Travis has said the earliest he can get to it is August... -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/a4326ae9/attachment.htm> From victor.stinner at haypocalc.com Mon Jul 21 21:17:14 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 21 Jul 2008 21:17:14 +0200 Subject: [Python-Dev] fileobj.read(float): warning or error? Message-ID: <200807212117.14485.victor.stinner@haypocalc.com> Hi, Since Python 2.4 (maybe 2.2 or older), fileobj.read(4.2) displays an error and works as fileobj.read(4). >>> i=open('/etc/issue') >>> i.read(4.2) __main__:1: DeprecationWarning: integer argument expected, got float It should raises an error instead of a warning, it has no sense to read a partial byte :-) But that should breaks some applications? Well, the real problem is os.urandom(4.2) which goes to an unlimited loop: while len(bytes) < n: bytes += read(_urandomfd, n - len(bytes)) because read(0.2) works as read(0) :-/ Victor From musiccomposition at gmail.com Mon Jul 21 21:23:21 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 21 Jul 2008 14:23:21 -0500 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <200807212117.14485.victor.stinner@haypocalc.com> References: <200807212117.14485.victor.stinner@haypocalc.com> Message-ID: <1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com> On Mon, Jul 21, 2008 at 2:17 PM, Victor Stinner < victor.stinner at haypocalc.com> wrote: > Hi, > > Since Python 2.4 (maybe 2.2 or older), fileobj.read(4.2) displays an error > and > works as fileobj.read(4). > > >>> i=open('/etc/issue') > >>> i.read(4.2) > __main__:1: DeprecationWarning: integer argument expected, got float This warning is actually given by the argument parser when "i" gets a Python non-integer. > > > It should raises an error instead of a warning, it has no sense to read a > partial byte :-) But that should breaks some applications? This doesn't come into effect until 3.0. > > > Well, the real problem is os.urandom(4.2) which goes to an unlimited loop: > > while len(bytes) < n: > bytes += read(_urandomfd, n - len(bytes)) > > because read(0.2) works as read(0) :-/ > > Victor > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080721/de60ff54/attachment.htm> From srichter at cosmos.phy.tufts.edu Mon Jul 21 16:42:16 2008 From: srichter at cosmos.phy.tufts.edu (Stephan Richter) Date: Mon, 21 Jul 2008 07:42:16 -0700 Subject: [Python-Dev] [Zope-dev] Syntax error in python2.6 In-Reply-To: <488472BB.7070608@wildenhain.de> References: <d886b2060807210413k18464a03p1a735b253d91dd06@mail.gmail.com> <488472BB.7070608@wildenhain.de> Message-ID: <200807210742.16411.srichter@cosmos.phy.tufts.edu> On Monday 21 July 2008, Tino Wildenhain wrote: > btw, shouldn't this already give a warning in 2.5? It does. I see it. Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. "Zope Stephan Richter" From optilude at gmx.net Mon Jul 21 22:59:28 2008 From: optilude at gmx.net (Martin Aspeli) Date: Mon, 21 Jul 2008 21:59:28 +0100 Subject: [Python-Dev] import error in python2.6 In-Reply-To: <A31A031A-B209-45A6-9AC9-6A6986AC173E@acm.org> References: <d886b2060807210512w2a385cd4g79a04f807ed6ca0f@mail.gmail.com> <8b943f2b0807210520v2001c601u5d8354a8a362f0f4@mail.gmail.com> <1afaf6160807210620p591b0177u40904823f7740794@mail.gmail.com> <A31A031A-B209-45A6-9AC9-6A6986AC173E@acm.org> Message-ID: <g62tbi$3p9$2@ger.gmane.org> Fred Drake wrote: > On Jul 21, 2008, at 9:20 AM, Benjamin Peterson wrote: >> Since this is prelease software, it's probably ok to talk about issues >> with it. You could file a bug next time. However, AFAIK, Zope hasn't >> even been ported to 2.5. > > > Many people are using Zope 3 with Python 2.5 without problems, though > Python 2.5 isn't "officially" supported (whatever that means). > > I think current versions of Zope 2 work with Python 2.5 as well. No, they don't. Work is ongoing to port it, though. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book From tjreedy at udel.edu Mon Jul 21 23:15:40 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 21 Jul 2008 17:15:40 -0400 Subject: [Python-Dev] Python VM In-Reply-To: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com> References: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com> Message-ID: <g62u9p$7r4$1@ger.gmane.org> Jakob Sievers wrote: > Hi, > I've been reading the Python VM sources over the last few afternoons and > I took some notes, which I thought I'd share (and if anyone familiar > with the VM internals could have a quick look at them, I'd really > appreciate it). This is the CPython VM. Other implementations do what they do. Perhaps similar, but certainly different in details. There have been various requests over the years on Python-list/comp.lang.python for documents describing the C-VM. The usual answer has been 'read the source' as you did. So I encourage you to announce it there and perhaps consider adding a PyWiki page (I assume there is not one now). tjr From regebro at gmail.com Mon Jul 21 23:37:04 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 21 Jul 2008 23:37:04 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org> <4884581F.8010201@jcea.es> <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> Message-ID: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com> On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote: > But waiting until all the betas have gone out totally defeats the > purpose of the betas! I agree. Writing an actual *guide* can wait, but documenting the differences with code examples is a work that can start now, and I agree that it would be great if this would start now. But writing a guide might not be a good idea until we know what the changes are, and if the API is changing quickly now we don't. :-) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From martin at v.loewis.de Tue Jul 22 00:17:56 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 Jul 2008 00:17:56 +0200 Subject: [Python-Dev] Python VM In-Reply-To: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com> References: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com> Message-ID: <48850B14.5020806@v.loewis.de> Jakob, This looks fairly correct. A few comments below. > Control Flow > ============ > The calling sequence is: > main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) -> > run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx() > (ceval.c) -> PyEval_EvalFrameEx() (ceval.c). What this misses is the compiler stuff, i.e. PyParser_ASTFromFoo and PyAST_Compile, which precedes the call to PyEval_ (atleast, no byte code file is available). > Threads > ======= > PyEval_InitThreads() initializes the GIL (interpreter_lock) and sets > main_thread to the (threading package dependent) ID of the current thread. > Thread switching is done using PyThreadState_Swap(), which sets > _PyThreadState_Current (both defined in pystate.c) and PyThreadState_GET() > (an alias for _PyThreadState_Current) (pystate.h). True, however, in most cases, this is triggered through Py_BEGIN_ALLOW_THREADS, which passes NULL for the new thread. The actual *switching* occurs by releasing the GIL, not by ThreadState_Swap. Actually, Python doesn't dispatch threads at all. It just releases the GIL, giving the operating system permission to wake up a different thread - which the operating system may or may not chose to do. After some time, the original thread will try to reacquire the GIL. Assuming the OS applies fairness, it will not get it back if a different thread was also waiting for it, so our thread will block - and *then* the OS will dispatch (at latest). > State > ===== > The global state is recorded in a (per-process?) PyInterpreterState struct and > a per-thread PyThreadState struct. Yes and no. In principle, multiple interpreter states are supported per process (and the current interpreter is identified by thread). However, there are many limitations and quirks in the multiple-interpreter code. > Each execution frame's state is contained in that frame's PyFrameObject > (which includes the instruction stream, the environment (globals, locals, > builtins, etc.), the value stack and so forth). > EvalFrameEx()'s local variables are initialized from this frame object. Not only. A lot of stuff also lives on the regular C stack, which exists in parallel to the frame object stack (the latter being a spaghetti stack). > The instruction stream looks as follows (c.f. assemble_emit() in compile.c): See also dis.py for the inverse operation. > Basic structure > --------------- > EvalFrameEx() { Somewhere you need to merge the thread-switching for threads that have been executing a lot of instructions. > - Objects are transferred onto the value stack by GETITEM()'ing them from > consts or names, or by GETLOCAL()'ing them using oparg as an offset into > fastlocals (c.f. LOAD_* instructions). Or, of course, as the result from some operation or function call, or load from a global variable, or import, or ... Regards, Martin From tom at vector-seven.com Tue Jul 22 02:28:08 2008 From: tom at vector-seven.com (Thomas Lee) Date: Tue, 22 Jul 2008 10:28:08 +1000 Subject: [Python-Dev] Python VM In-Reply-To: <48850B14.5020806@v.loewis.de> References: <e59aede0807210858l761885b9j276b40d694f61794@mail.gmail.com> <48850B14.5020806@v.loewis.de> Message-ID: <48852998.6050405@vector-seven.com> Martin v. L?wis wrote: > Jakob, > > This looks fairly correct. A few comments below. > > >> Control Flow >> ============ >> The calling sequence is: >> main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) -> >> run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx() >> (ceval.c) -> PyEval_EvalFrameEx() (ceval.c). >> > > What this misses is the compiler stuff, i.e. PyParser_ASTFromFoo and > PyAST_Compile, which precedes the call to PyEval_ (atleast, no byte code > file is available). > > Further, if I have my way with the AST optimization code, the symtable construction will be an explicit step in between these >:) In any case, this is awesome work Jakob. It'd be great for this stuff to be documented in such detail -- I sure wish I had something like this to go by when I first started hacking on the source -- but the details seem to change quite often. Still, seeing the detail distilled once in a while is sort of nice, and great for anybody looking to get their teeth into the code. Thanks for doing the hard yards. :) Cheers, T From martin at v.loewis.de Tue Jul 22 07:01:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 Jul 2008 07:01:42 +0200 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <200807212117.14485.victor.stinner@haypocalc.com> References: <200807212117.14485.victor.stinner@haypocalc.com> Message-ID: <488569B6.8050108@v.loewis.de> > Well, the real problem is os.urandom(4.2) which goes to an unlimited loop: > > while len(bytes) < n: > bytes += read(_urandomfd, n - len(bytes)) > > because read(0.2) works as read(0) :-/ I can't quite accept that as a bug in the library. If you give invalid parameters, Python should not crash, but it may start to behave in a nonsensical way. Of course, it would be possible to move the conversion warning one layer up, into os.urandom; if the argument is float, raise a warning, and then truncate. Regards, Martin From aleaxit at gmail.com Tue Jul 22 07:22:38 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 21 Jul 2008 22:22:38 -0700 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <488569B6.8050108@v.loewis.de> References: <200807212117.14485.victor.stinner@haypocalc.com> <488569B6.8050108@v.loewis.de> Message-ID: <e8a0972d0807212222v9e0b289h312bf33bee344170@mail.gmail.com> I thought that's what we had __index__ for -- reject arguments that don't SMOOTHLY turn into integers when an integer is actually required! Alex On Mon, Jul 21, 2008 at 10:01 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote: >> Well, the real problem is os.urandom(4.2) which goes to an unlimited loop: >> >> while len(bytes) < n: >> bytes += read(_urandomfd, n - len(bytes)) >> >> because read(0.2) works as read(0) :-/ > > I can't quite accept that as a bug in the library. If you give invalid > parameters, Python should not crash, but it may start to behave in a > nonsensical way. > > Of course, it would be possible to move the conversion warning one layer > up, into os.urandom; if the argument is float, raise a warning, and then > truncate. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > From martin at v.loewis.de Tue Jul 22 07:44:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 Jul 2008 07:44:00 +0200 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <e8a0972d0807212222v9e0b289h312bf33bee344170@mail.gmail.com> References: <200807212117.14485.victor.stinner@haypocalc.com> <488569B6.8050108@v.loewis.de> <e8a0972d0807212222v9e0b289h312bf33bee344170@mail.gmail.com> Message-ID: <488573A0.5020408@v.loewis.de> Alex Martelli wrote: > I thought that's what we had __index__ for -- reject arguments that > don't SMOOTHLY turn into integers when an integer is actually > required! Sure. However, using that would create an incompatibility, that's why you only get a warning when it falls back to not using __index__. Regards, Martin From cs at zip.com.au Tue Jul 22 07:37:44 2008 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 22 Jul 2008 15:37:44 +1000 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <200807212117.14485.victor.stinner@haypocalc.com> Message-ID: <20080722053744.GA11290@cskk.homeip.net> On 21Jul2008 21:17, Victor Stinner <victor.stinner at haypocalc.com> wrote: | Well, the real problem is os.urandom(4.2) which goes to an unlimited loop: | while len(bytes) < n: | bytes += read(_urandomfd, n - len(bytes)) | because read(0.2) works as read(0) :-/ Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an exception if asked for < 1 bytes? Or is there a legitimate use for read(0) with which I was not previously aware? -- Cameron Simpson <cs at zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ From leif.walsh at gmail.com Tue Jul 22 08:35:54 2008 From: leif.walsh at gmail.com (Leif Walsh) Date: Mon, 21 Jul 2008 23:35:54 -0700 (PDT) Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <20080722053744.GA11290@cskk.homeip.net> References: <20080722053744.GA11290@cskk.homeip.net> Message-ID: <Pine.LNX.4.64.0807212328340.23485@lappy> On Tue, 22 Jul 2008, Cameron Simpson wrote: > Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an > exception if asked for < 1 bytes? Or is there a legitimate use for > read(0) with which I was not previously aware? I think read(0) should be a no-op, just like it is in libc. This lets you write 'read(bytes)' without worrying about checking bytes, and also lets you silently stop reading when you have no more space, like in the following: buf = f.read(max(bytes_left, page_size)) while buf: process(buf) # updates bytes_left buf = f.read(max(bytes_left, page_size)) -- Cheers, Leif From cadr4u at gmail.com Tue Jul 22 15:54:27 2008 From: cadr4u at gmail.com (Jakob Sievers) Date: Tue, 22 Jul 2008 15:54:27 +0200 Subject: [Python-Dev] Python VM Message-ID: <e59aede0807220654x6ef57210g7f2d043099921ee6@mail.gmail.com> I added a page to wiki.python.org: http://wiki.python.org/moin/CPythonVmInternals This incorporates most of Martin v. L?wis's additions (except for a few bits which I need to look into more). In any case, thanks for the feedback! Cheers, -jakob From jjl at pobox.com Tue Jul 22 21:56:31 2008 From: jjl at pobox.com (John J Lee) Date: Tue, 22 Jul 2008 20:56:31 +0100 (BST) Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <20080722053744.GA11290@cskk.homeip.net> References: <20080722053744.GA11290@cskk.homeip.net> Message-ID: <alpine.DEB.1.00.0807222053270.6381@alice> On Tue, 22 Jul 2008, Cameron Simpson wrote: [...] > Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an > exception if asked for < 1 bytes? Or is there a legitimate use for read(0) > with which I was not previously aware? http://docs.python.org/lib/bltin-file-objects.html read([size]) ... If the size argument is negative or omitted, read all data until EOF is reached. ... John From cs at zip.com.au Wed Jul 23 00:28:44 2008 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 23 Jul 2008 08:28:44 +1000 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <alpine.DEB.1.00.0807222053270.6381@alice> Message-ID: <20080722222844.GA23074@cskk.homeip.net> On 22Jul2008 20:56, John J Lee <jjl at pobox.com> wrote: > On Tue, 22 Jul 2008, Cameron Simpson wrote: > [...] >> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an >> exception if asked for < 1 bytes? Or is there a legitimate use for read(0) >> with which I was not previously aware? > > http://docs.python.org/lib/bltin-file-objects.html > > read([size]) > > ... If the size argument is negative or omitted, read all data until EOF > is reached. ... Hmm, yeah, but 0 is not negative and not omitted so this does not apply. Personally I'm not very fond of that spec; I'm good with the omitted size provoking a "read everything" mode but I'd rather a non-numeric value like None rather than a negative one (eg the conventional "def read(size=None)") if an explicit size should do so. That way bad arithmetic in the caller could have a chance of triggering an exception from read instead of a silent (and to my taste, nasty) "slurp the file" mode. -- Cameron Simpson <cs at zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ From jjl at pobox.com Wed Jul 23 00:45:56 2008 From: jjl at pobox.com (John J Lee) Date: Tue, 22 Jul 2008 23:45:56 +0100 (BST) Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <20080722222844.GA23074@cskk.homeip.net> References: <20080722222844.GA23074@cskk.homeip.net> Message-ID: <alpine.DEB.1.00.0807222339120.7713@alice> On Wed, 23 Jul 2008, Cameron Simpson wrote: > On 22Jul2008 20:56, John J Lee <jjl at pobox.com> wrote: >> On Tue, 22 Jul 2008, Cameron Simpson wrote: >> [...] >>> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an >>> exception if asked for < 1 bytes? Or is there a legitimate use for read(0) >>> with which I was not previously aware? >> >> http://docs.python.org/lib/bltin-file-objects.html >> >> read([size]) >> >> ... If the size argument is negative or omitted, read all data until EOF >> is reached. ... > > Hmm, yeah, but 0 is not negative and not omitted so this does not apply. Well, -1 *is* < 1 (and is in the domain of the function), but yes -- sorry, read too quickly, took your "< 1" too literally. John From cs at zip.com.au Wed Jul 23 00:46:29 2008 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 23 Jul 2008 08:46:29 +1000 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <Pine.LNX.4.64.0807212328340.23485@lappy> Message-ID: <20080722224629.GA24798@cskk.homeip.net> On 21Jul2008 23:35, Leif Walsh <leif.walsh at gmail.com> wrote: | On Tue, 22 Jul 2008, Cameron Simpson wrote: | > Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an | > exception if asked for < 1 bytes? Or is there a legitimate use for | > read(0) with which I was not previously aware? | | I think read(0) should be a no-op, just like it is in libc. This lets | you write 'read(bytes)' without worrying about checking bytes, and | also lets you silently stop reading when you have no more space, like | in the following: | | buf = f.read(max(bytes_left, page_size)) | while buf: | process(buf) # updates bytes_left | buf = f.read(max(bytes_left, page_size)) [ Don't you mean "min()"? Unimportant. ] I see the convenience here, but doubt I'd ever do that myself. I'd write the above like this: while bytes_left > 0: buf = f.read(max(bytes_left, page_size)) if buf == 0: break process(buf) # updates bytes_left I'm kind of picky about doing things exactly as often as required and no more. Especially things that call another facility. read(0) itself must internally have a check for size == 0 anyway, so it's not like the overall system is less complex. If we're unlucky it could trickle all the way down to an OS system call to read(2) (UNIX, substitute as suitable elsewhere) and for a no-op that would be overkill by far. The only way the read() implementation would avoid that is by doing the test on size anyway. But since read() is opaque IMO it is better to avoid it at the upper level if we know it will produce nothing. Which leaves me unconvinced of the utility of this mode. Cheers, -- Cameron Simpson <cs at zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ From victor.stinner at haypocalc.com Wed Jul 23 01:43:18 2008 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 23 Jul 2008 01:43:18 +0200 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com> References: <200807212117.14485.victor.stinner@haypocalc.com> <1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com> Message-ID: <200807230143.18189.victor.stinner@haypocalc.com> Le Monday 21 July 2008 21:23:21, vous avez ?crit?: > > It should raises an error instead of a warning, it has no sense to read a > > partial byte :-) But that should breaks some applications? > > This doesn't come into effect until 3.0. Would it possible to create an option "strict mode" which disallow dangerous cast? Especially in PyArg_Parse*() functions. -- I hate "transparent" cast, like C and PHP do everywhere. The worst "transparent" cast in Python (2.x) is between str (bytes) and unicode (characters) types. Victor Stinner aka haypo From musiccomposition at gmail.com Wed Jul 23 01:47:20 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 22 Jul 2008 18:47:20 -0500 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <200807230143.18189.victor.stinner@haypocalc.com> References: <200807212117.14485.victor.stinner@haypocalc.com> <1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com> <200807230143.18189.victor.stinner@haypocalc.com> Message-ID: <1afaf6160807221647s3b6274b9nacc016fddea20b8e@mail.gmail.com> On Tue, Jul 22, 2008 at 6:43 PM, Victor Stinner < victor.stinner at haypocalc.com> wrote: > Le Monday 21 July 2008 21:23:21, vous avez ?crit : > > > It should raises an error instead of a warning, it has no sense to read > a > > > partial byte :-) But that should breaks some applications? > > > > This doesn't come into effect until 3.0. > > Would it possible to create an option "strict mode" which disallow > dangerous > cast? Especially in PyArg_Parse*() functions. You could use -Wall to make the warning an error. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080722/ba03076c/attachment.htm> From musiccomposition at gmail.com Wed Jul 23 01:47:49 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 22 Jul 2008 18:47:49 -0500 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <1afaf6160807221647s3b6274b9nacc016fddea20b8e@mail.gmail.com> References: <200807212117.14485.victor.stinner@haypocalc.com> <1afaf6160807211223l70bac7a0t8c00479eb7fd2e6b@mail.gmail.com> <200807230143.18189.victor.stinner@haypocalc.com> <1afaf6160807221647s3b6274b9nacc016fddea20b8e@mail.gmail.com> Message-ID: <1afaf6160807221647l7d85aa36r606e0ec1aef48ee8@mail.gmail.com> On Tue, Jul 22, 2008 at 6:47 PM, Benjamin Peterson < musiccomposition at gmail.com> wrote: > > > On Tue, Jul 22, 2008 at 6:43 PM, Victor Stinner < > victor.stinner at haypocalc.com> wrote: > >> Le Monday 21 July 2008 21:23:21, vous avez ?crit : >> > > It should raises an error instead of a warning, it has no sense to >> read a >> > > partial byte :-) But that should breaks some applications? >> > >> > This doesn't come into effect until 3.0. >> >> Would it possible to create an option "strict mode" which disallow >> dangerous >> cast? Especially in PyArg_Parse*() functions. > > > You could use -Wall to make the warning an error. > I meant -Werr. > > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080722/70fca1da/attachment.htm> From leif.walsh at gmail.com Wed Jul 23 03:19:49 2008 From: leif.walsh at gmail.com (Leif Walsh) Date: Tue, 22 Jul 2008 18:19:49 -0700 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <20080722224629.GA24798@cskk.homeip.net> References: <Pine.LNX.4.64.0807212328340.23485@lappy> <20080722224629.GA24798@cskk.homeip.net> Message-ID: <cc7430500807221819j6384105dq4c3b0114cc9d62a7@mail.gmail.com> On Tue, Jul 22, 2008 at 3:46 PM, Cameron Simpson <cs at zip.com.au> wrote: > [ Don't you mean "min()"? Unimportant. ] Haha, that's what I get for not actually _running_ the code example. > I see the convenience here, but doubt I'd ever do that myself. > I'd write the above like this: > > while bytes_left > 0: > buf = f.read(max(bytes_left, page_size)) > if buf == 0: > break > process(buf) # updates bytes_left > > I'm kind of picky about doing things exactly as often as required and no > more. Especially things that call another facility. I do the same, but I know lots of people that prefer the example I sent earlier. Also, if we ever adopt the "while ... as ..." syntax (here's not hoping) currently being discussed in another thread, having read(0) return None or an empty buffer will cause that idiom to short circuit as well. > read(0) itself must internally have a check for size == 0 anyway, so > it's not like the overall system is less complex. If we're unlucky it > could trickle all the way down to an OS system call to read(2) (UNIX, > substitute as suitable elsewhere) and for a no-op that would be overkill > by far. The only way the read() implementation would avoid that is by > doing the test on size anyway. But since read() is opaque IMO it is > better to avoid it at the upper level if we know it will produce > nothing. If we are going to make read(0) a no-op, we should definitely do it before it hits the underlying implementation, for portability's sake. On Tue, Jul 22, 2008 at 4:43 PM, Victor Stinner <victor.stinner at haypocalc.com> wrote: > Would it possible to create an option "strict mode" which disallow dangerous > cast? Especially in PyArg_Parse*() functions. Ack! We're not writing Perl here, guys. Can we please not start having multiple subsets of the language that are separately valid? -- Cheers, Leif From jcea at jcea.es Wed Jul 23 17:33:01 2008 From: jcea at jcea.es (Jesus Cea) Date: Wed, 23 Jul 2008 17:33:01 +0200 Subject: [Python-Dev] bsddb: Test failures on windows (HELP!) Message-ID: <48874F2D.5070201@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent, I was wondering if you could look at some test failures in MS Windows builds. I can't debug Windows issues myself :-(. This is a MS free environment... Details: <http://www.python.org/dev/buildbot/all/x86%20XP-4%20trunk/builds/1364/step-test/0> Thanks in advance for your time and attention. Of course, any other help appreciated :-) - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIdPJJlgi5GaxT1NAQLzFwP9E0huY63jOZWx9v/H7NhDGwIqFl5OGfhO EZ6uCdRBuyH4Y4jQBaIKTFN9jXu2sSONNMBDgOZznkBuYtFIA8Xmn9KDrkuZ5k8G 1GlcG+DcoG1aP1PANgrPMkEpptw5/TNBlA3s6p/F4oDSye1kMW0CbGcSHeECYF9R CIhZztqZvWk= =aZoO -----END PGP SIGNATURE----- From loisel at temple.edu Wed Jul 23 23:14:22 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Wed, 23 Jul 2008 17:14:22 -0400 Subject: [Python-Dev] Infix operators Message-ID: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> Dear Pythonistas, I've googled for this but I wasn't able to find anything definitive. I was recently looking at scipy to see if I could use it in stead of MATLAB for my class on numerical PDEs, but I noticed that there's some difficulty related to the notation; mainly, the matrix multiplication, and the linear solver (i.e., MATLAB's \ operator). After giving it some thought, I've decided to hold back for now and use MATLAB. Since scipy/numeric python have been around for a while and some of it is even integrated in Python 2.5, I'm sure this conversation has happened before, so please just educate me. Now I think that there's always going to be people wanting more and more operators in Python (although the operators I'm talking about are privileged -- the Chinese were using matrices some 2000 years ago), so I was thinking that it would be simpler to have a way for defining new infix operators, which would simply be binary functions whose names are punctuation marks; see any language with this feature, e.g., Haskell. Now since this is such a simple idea, I'm guessing it occured to pythonistas at some point in 1992, and got voted down and that decision has now become scripture. Is that the case? The one thing which I found was PEP 211 http://www.python.org/dev/peps/pep-0211/, which has an amazing quote from James Rawlings. His comment about inv is completely absurd, and while he claims not to know \ and /, he is quoted as an authority on these operators. This particular PEP should probably be deleted from history. For a more authoritative explanation, a quick search yields MathWorks' own Loren Shure: http://blogs.mathworks.com/loren/2007/05/16/purpose-of-inv/ Essentially, in almost all applications, inv(A) is entirely wrong. You can ask any numerical analyst who works with large problems, and they will confirm it. One of the main reason is that, even if A is sparse, inv(A) is full. That absurdity aside, if I were a language designer, I would be reticent to add one operator to Python (like in PEP 211), because there will always be more operators that people want (how long until someone asks for an operator for the Kronecker product or the convolution?) But why not put in this infix possibility? Sincerely, -- S?bastien Loisel -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/6d68d438/attachment.htm> From josiah.carlson at gmail.com Thu Jul 24 01:11:19 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 23 Jul 2008 16:11:19 -0700 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> Message-ID: <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> On Wed, Jul 23, 2008 at 2:14 PM, Sebastien Loisel <loisel at temple.edu> wrote: > Dear Pythonistas, > > I've googled for this but I wasn't able to find anything definitive. I was > recently looking at scipy to see if I could use it in stead of MATLAB for my > class on numerical PDEs, but I noticed that there's some difficulty related > to the notation; mainly, the matrix multiplication, and the linear solver > (i.e., MATLAB's \ operator). After giving it some thought, I've decided to > hold back for now and use MATLAB. > > Since scipy/numeric python have been around for a while and some of it is > even integrated in Python 2.5, I'm sure this conversation has happened > before, so please just educate me. Now I think that there's always going to > be people wanting more and more operators in Python (although the operators > I'm talking about are privileged -- the Chinese were using matrices some > 2000 years ago), so I was thinking that it would be simpler to have a way > for defining new infix operators, which would simply be binary functions > whose names are punctuation marks; see any language with this feature, e.g., > Haskell. > > Now since this is such a simple idea, I'm guessing it occured to pythonistas > at some point in 1992, and got voted down and that decision has now become > scripture. Is that the case? > > The one thing which I found was PEP 211 > http://www.python.org/dev/peps/pep-0211/, which has an amazing quote from > James Rawlings. His comment about inv is completely absurd, and while he > claims not to know \ and /, he is quoted as an authority on these operators. > This particular PEP should probably be deleted from history. For a more > authoritative explanation, a quick search yields MathWorks' own Loren Shure: > http://blogs.mathworks.com/loren/2007/05/16/purpose-of-inv/ > > Essentially, in almost all applications, inv(A) is entirely wrong. You can > ask any numerical analyst who works with large problems, and they will > confirm it. One of the main reason is that, even if A is sparse, inv(A) is > full. > > That absurdity aside, if I were a language designer, I would be reticent to > add one operator to Python (like in PEP 211), because there will always be > more operators that people want (how long until someone asks for an operator > for the Kronecker product or the convolution?) But why not put in this infix > possibility? Why not add the possibility for arbitrary infix operators? Others will most likely disagree with me, but I would claim that adding arbitrary infix operators are a great way to destroy readability. What the heck does 'x = 4 $ 6' mean in Python? Oh, that's right, it's a custom infix operator. But where is it defined? In the module? In some other module that is imported? Can I override or do some pre-processing to the arguments and pass it on to the previously defined $ operator? So many questions, none of which tell me what '$' does. Really though, PEP 211 was a perfect example of a k-line function that someone attempted to make syntax that really didn't need to be syntax. And arguably, any infix operator will need to result in a lookup for the new infix operator, which will be as fast (or slow) as a globals lookup, so you may as well define your operator as a two-argument function, which has defined semantics, and can be imported and passed just like all functions defined for years. I would argue current operators are *convenience* for the 99%+ majority of "mathematical" operations done in Python, with well-defined and understood semantics in the 99%+ cases they are used for. Further, the addition of further infix operators are, strictly speaking, a domain-specific language, which Python (as a language) doesn't frown upon, but doesn't encourage either. If you still think that X \ Y would honestly make Python a better language, I would ask you if logix (http://www.livelogix.net/logix/intro.html) would suit you better. It seems to allow you to use arbitrary punctuation for operators. - Josiah From loisel at temple.edu Thu Jul 24 01:48:06 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Wed, 23 Jul 2008 19:48:06 -0400 Subject: [Python-Dev] Infix operators In-Reply-To: <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> Message-ID: <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> Dear Josiah, Thank you for your email. On Wed, Jul 23, 2008 at 7:11 PM, Josiah Carlson <josiah.carlson at gmail.com> wrote: > What the heck does 'x = 4 $ 6' mean in Python? Oh, that's right, it's > a custom infix operator. But where is it defined? In the module? In > some other module that is imported? Can I override or do some > pre-processing to the arguments and pass it on to the previously > defined $ operator? So many questions, none of which tell me what '$' > does. > My question is really what the history of this thing is and what sort of opposition there is to this idea, in the python community. From your email, I suspect that this conversation has already taken place. I'm not going to become a python developer/politician and try to push this, my job involves teaching and research in mathematics and the programming language is just a tool. If I want to "put food on my family" (in the words of a wise man), I can't really spend a whole lot of time on this mailing list debating this. However, just for posterity (and I'm not going to pursue the argument further than this), I'll say this. The problem of determining the meaning (or overridability or whatever) of x=4$6 is the same as the problem of determining the meaning of x=fooz(4,6). Since it's not a good argument against user-defined functions, I don't see it as a good argument against user-defined operators. Of course, obviously the LISP people have decided that fooz(4,6) (or rather, (fooz 4 6)) is okay, and 4$6 is not, so who am I to dispute that. > If you still think that X \ Y would honestly make Python a better Well, I'm not arguing for adding one more special-purpose operator, because probably the need for more operators will never end. If arbitrary operators were defined like ordinary functions, they could be brought in with the import numeric or whatever, and docstrings could be looked up, etc... > > language, I would ask you if logix > (http://www.livelogix.net/logix/intro.html) would suit you better. It > seems to allow you to use arbitrary punctuation for operators. > Thank you for the pointer. I have taken a look and it does look interesting, on first blush I would love to use that language. The main thing is that I worry about being out on the fringe, using a language that nobody uses, and which may get abandoned without warning (like Sun abandoned `self'), or be buggier just because people don't use it... Although I love metaprogramming and I would use it if it were in python, I don't really need to go that far (arbitrary infix operators would suffice to me.) Sincerely, -- S?bastien Loisel -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/4e31179b/attachment.htm> From anthonybaxter at gmail.com Thu Jul 24 01:50:45 2008 From: anthonybaxter at gmail.com (Anthony Baxter) Date: Wed, 23 Jul 2008 16:50:45 -0700 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org> <4884581F.8010201@jcea.es> <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com> Message-ID: <e69d3ed20807231650x1cc1716ew3789b2aacc280d1d@mail.gmail.com> As a data point, I just presented a tutorial here at OSCON on Python 3 Porting, and had a number of people asking about C API changes. I punted for my talk (*) because things are still in flux. Plus I only had 3 hours. I did suggest that if your extension is just glue to a C library, you should consider using ctypes. So yes, collecting this information, even if it's just in a wiki page, would be a good and popular thing. Anthony (*) slides: http://www.interlink.com.au/anthony/tech/talks/OSCON2008/porting3.pdf On Mon, Jul 21, 2008 at 2:37 PM, Lennart Regebro <regebro at gmail.com> wrote: > On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote: >> But waiting until all the betas have gone out totally defeats the >> purpose of the betas! > > I agree. Writing an actual *guide* can wait, but documenting the > differences with code examples is a work that can start now, and I > agree that it would be great if this would start now. > > But writing a guide might not be a good idea until we know what the > changes are, and if the API is changing quickly now we don't. :-) > > -- > Lennart Regebro: Zope and Plone consulting. > http://www.colliberty.com/ > +33 661 58 14 64 > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/anthony%40interlink.com.au > > From curt at hagenlocher.org Thu Jul 24 02:08:39 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Wed, 23 Jul 2008 17:08:39 -0700 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> Message-ID: <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> On Wed, Jul 23, 2008 at 4:48 PM, Sebastien Loisel <loisel at temple.edu> wrote: > > On Wed, Jul 23, 2008 at 7:11 PM, Josiah Carlson <josiah.carlson at gmail.com> > wrote: >> >> language, I would ask you if logix >> (http://www.livelogix.net/logix/intro.html) would suit you better. It >> seems to allow you to use arbitrary punctuation for operators. > > Thank you for the pointer. I have taken a look and it does look interesting, > on first blush I would love to use that language. The main thing is that I > worry about being out on the fringe, using a language that nobody uses, and > which may get abandoned without warning (like Sun abandoned `self'), or be > buggier just because people don't use it... Have you considered OCaml? (http://en.wikipedia.org/wiki/Ocaml) It's in reasonably broad use and is actively maintained, and it allows user-defined infix operators. A programming language can't be all things to all people. That's why there's room in the world for more than one. -- Curt Hagenlocher curt at hagenlocher.org From loisel at temple.edu Thu Jul 24 02:21:18 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Wed, 23 Jul 2008 20:21:18 -0400 Subject: [Python-Dev] Infix operators In-Reply-To: <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> Message-ID: <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> Dear Curt, Thank you for your email. Have you considered OCaml? (http://en.wikipedia.org/wiki/Ocaml) It's > I have. I've considered a lot of languages, but OCaml isn't for me. My current language is MATLAB. Python is pretty close in syntax, and it's widely accepted, so you can teach students Python and they can go out and use it. You can also publish a paper and write a code snippet without having to explain the syntax. I'm not trying to go out on the blistering leading edge of computer language sophistication; that would not be good for my students, and it would not be good for my research. For most of my programs, what I need is a language where you don't need to give types or declare variables, because that just takes up space and obscures the math. So I need a mainstream dynamic language like Python or MATLAB. OCaml is not dynamic, and linear algebra in OCaml is a pain in the neck because of the explosion of the number of operators for various pairs of operand types. Sincerely, -- S?bastien Loisel -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/b4edbfa0/attachment.htm> From kirklin.mcdonald at gmail.com Thu Jul 24 02:24:28 2008 From: kirklin.mcdonald at gmail.com (Kirk McDonald) Date: Wed, 23 Jul 2008 17:24:28 -0700 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org> <4884581F.8010201@jcea.es> <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com> Message-ID: <25bd58d10807231724u1d60f5e7y8cc57905a599e60c@mail.gmail.com> On Mon, Jul 21, 2008 at 2:37 PM, Lennart Regebro <regebro at gmail.com> wrote: > On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote: >> But waiting until all the betas have gone out totally defeats the >> purpose of the betas! > > I agree. Writing an actual *guide* can wait, but documenting the > differences with code examples is a work that can start now, and I > agree that it would be great if this would start now. > > But writing a guide might not be a good idea until we know what the > changes are, and if the API is changing quickly now we don't. :-) > I am the nominal maintainer of the D programming language's bindings to the Python/C API.[1] Keeping on top of the changes in the C API is therefore of some interest to me. At the heart of these bindings is 3000 or so lines of D code containing the translated C headers. Adding the changes made in 3.0 (and 2.6, for that matter) will probably have to be done by hand. Any document detailing every change, from the broadest refactorings to the tiniest name-changes, will therefore be of invaluable assistance to me. Obviously, I don't plan on even starting this upgrade until the C API is nailed down and such a document exists. But any drafts, or even a bulleted list of changes, would at least give me an idea of the scope of what I'm in for. [1] http://pyd.dsource.org -Kirk McDonald From greg.ewing at canterbury.ac.nz Thu Jul 24 04:05:01 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 24 Jul 2008 14:05:01 +1200 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> Message-ID: <4887E34D.2000608@canterbury.ac.nz> Sebastien Loisel wrote: > Essentially, in almost all applications, inv(A) is entirely wrong. You > can ask any numerical analyst who works with large problems, and they > will confirm it. One of the main reason is that, even if A is sparse, > inv(A) is full. This argues for a function such as solve(A, b). It doesn't argue for an infix operator. > But why not put in this infix possibility? Guido is on record as opposing any kind of macros or other "extensible syntax", and this probably comes under the same heading. Besides which there are technical difficulties, such as how the parser is supposed to know the precedence of these user-defined operators. To address your original problem, I believe that numpy comes with a matrix type that defines * as matrix multiplication. That's the usual Python solution to these kinds of things -- rather than invent a new operator, invent a new type that behaves the way you want. -- Greg From loisel at temple.edu Thu Jul 24 04:28:09 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Wed, 23 Jul 2008 22:28:09 -0400 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> Message-ID: <b7975ab80807231928p468a5e7fj636e4b80d2d88b1e@mail.gmail.com> Dear Greg, Thanks for your email. Guido is on record as opposing any kind of macros > or other "extensible syntax", and this probably comes > under the same heading. > Thanks, that's exactly what I was looking for when I asked: Now since this is such a simple idea, I'm guessing it occured to pythonistas > at some point in 1992, and got voted down and that decision has now become > scripture. Is that the case? > Case closed. Sincerely, -- S?bastien Loisel PS: This argues for a function such as solve(A, b). It > doesn't argue for an infix operator. > I know, I was demolishing PEP 211, not trying to support my point. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080723/abd24192/attachment.htm> From tjreedy at udel.edu Thu Jul 24 05:13:35 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 23 Jul 2008 23:13:35 -0400 Subject: [Python-Dev] Close PEP 211? (was Re: Infix operators) In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> Message-ID: <g68s10$646$1@ger.gmane.org> Sebastien Loisel wrote: > > The one thing which I found was PEP 211 > http://www.python.org/dev/peps/pep-0211/ This now has a status of deferred. Given that itertools.product(A,B) == ((x,y) for x in A for y in B) == the proposed 'A @ B' and given Guido's pronounced distaste for new infix, should this be closed? From greg.ewing at canterbury.ac.nz Thu Jul 24 05:21:07 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 24 Jul 2008 15:21:07 +1200 Subject: [Python-Dev] Infix operators In-Reply-To: <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> Message-ID: <4887F523.7060207@canterbury.ac.nz> Josiah Carlson wrote: > What the heck does 'x = 4 $ 6' mean in Python? Oh, that's right, it's > a custom infix operator. But where is it defined? It's not quite as bad as that -- it would be defined by the relevant operator method on one of the operands. But a convention would be needed for mapping arbitary non- alphanumeric sequences to method names, and it's not obvious how to do that. But the main technical problem I see is that of precedence. It would have to be declared somehow, or a declaration imported from another module. If it's imported, then parsing a module can't be done in isolation any more, since it depends on the contents of other modules. Things could get very messy. > Really though, PEP 211 was a perfect example of a k-line function that > someone attempted to make syntax that really didn't need to be syntax. It looks like a case of someone wanting a matrix multiplication operator in numpy, and then hunting around for something to make it mean in Python. I would actually be in favour of adding a matrix multiplication operator, but I would make it mean matrix multiplication in Python as well, operating on anything that can be treated as a 2D sequence. Why matrix multiplication in particular? Because it's the one thing that you do all the time with matrices for which there isn't an available operator. I think this one addition could be justified on the grounds of very wide usage. -- Greg From greg.ewing at canterbury.ac.nz Thu Jul 24 05:26:23 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 24 Jul 2008 15:26:23 +1200 Subject: [Python-Dev] Close PEP 211? (was Re: Infix operators) In-Reply-To: <g68s10$646$1@ger.gmane.org> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <g68s10$646$1@ger.gmane.org> Message-ID: <4887F65F.9000004@canterbury.ac.nz> Terry Reedy wrote: > Given that itertools.product(A,B) == ((x,y) for x in A for y in B) > == the proposed 'A @ B' and given Guido's pronounced distaste for new > infix, should this be closed? Would there likely be any support for an alternative PEP defining @ as matrix multiplication in both Python and numpy? -- Greg From aahz at pythoncraft.com Thu Jul 24 06:03:17 2008 From: aahz at pythoncraft.com (Aahz) Date: Wed, 23 Jul 2008 21:03:17 -0700 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> Message-ID: <20080724040317.GA5358@panix.com> On Wed, Jul 23, 2008, Sebastien Loisel wrote: > > [...] I was thinking that it would be simpler to have a way > for defining new infix operators, [...] python-dev is the wrong place for this discussion. Please use either comp.lang.python or python-ideas. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Adopt A Process -- stop killing all your children! From mhammond at skippinet.com.au Thu Jul 24 07:44:31 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 24 Jul 2008 15:44:31 +1000 Subject: [Python-Dev] bsddb: Test failures on windows (HELP!) In-Reply-To: <48874F2D.5070201@jcea.es> References: <48874F2D.5070201@jcea.es> Message-ID: <095c01c8ed50$60bdfed0$2239fc70$@com.au> > Trent, I was wondering if you could look at some test failures in MS > Windows builds. I can't debug Windows issues myself :-(. This is a MS > free environment... In these errors I see lots of bsdbd errors, many of the form: | DBFileExistsError: (17, 'File exists -- __fop_file_setup: Retry limit (100) exceeded') Maybe an old test file isn't being nuked? Others of the form: | self.assertTrue(time.time()<timeout) | AssertionError which also look more related to the test suite than to Windows. There are also lots of errors due to the environment having a unicode object in it: | test test_site failed -- Traceback (most recent call last): | ... | TypeError: environment can only contain strings | test test_subprocess failed -- errors occurred; run in verbose mode for details | [Possibly the same error as below?] | test test_sys failed -- Traceback (most recent call last): | File "...\subprocess.py", line 817, in _execute_child | startupinfo) | TypeError: environment can only contain strings * A couple of wsgi related errors, including one about the environment - but this has more information: AssertionError: "AssertionError: Environmental variable LIB is not a string: <type 'unicode'> (value: u'C:\\\\Program Files\\\\Microsoft Visual Studio 9.0\\\\VC\\\\LIB;C:\\\\Program Files\\\\Microsoft SDKs\\\\Windows\\\\v6.0A\\\\lib;c:\\\\program files\\\\microsoft visual studio .NET 2003\\\\vc7\\\\atlmfc\\\\lib;c:\\\\program files\\\\microsoft visual studio .NET 2003\\\\vc7\\\\lib;c:\\\\program files\\\\microsoft visual studio .NET 2003\\\\vc7\\\\PlatformSDK\\\\lib;C:\\\\Program Files\\\\Microsoft Visual Studio .NET 2003\\\\SDK\\\\v1.1\\\\Lib\\\\')" != "AssertionError: Headers (('Content-Type', 'text/plain')) must be of type list: <type 'tuple'>" So LIB has a Unicode value - evn though it has no Unicode characters, and was presumably in the environment Python inherited (but presumably was initially a string). I can't reproduce the Unicode errors: 'python test_sys.py' works and I couldn't run a full test suite (see below). Python allows you to set unicode objects in os.environ, so its possible part of the test suite is modifying the environment. I tried to run a full test, but it hangs in various places before test_sys (bz2, multiprocessing), and I ended up giving up. This was on for either 32 or 64bits with the current trunk, but sadly I've no idea what could be happening there :( So it sounds like just tracking down how a unicode object is getting into the environment will solve the vast majority of the errors - except for the bsddb and wsgi ones, which don't look particularly Windows specific... Hope that helps a bit... Mark From stefan_ml at behnel.de Thu Jul 24 09:27:52 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 24 Jul 2008 09:27:52 +0200 Subject: [Python-Dev] crash in 3.0b2 exception code Message-ID: <g69atp$716$1@ger.gmane.org> Hi, I get a crash in one of lxml's doctests in 3.0b2, and it looks like it's coming from plain Python code (as opposed to Cython code). Just a quick check before I start digging into this, has anyone seen this before? Stefan [...] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1209997120 (LWP 15431)] 0x080f3b89 in BaseException_str (self=0x8d747d4) at Objects/exceptions.c:88 88 switch (PyTuple_GET_SIZE(self->args)) { (gdb) bt #0 0x080f3b89 in BaseException_str (self=0x8d747d4) at Objects/exceptions.c:88 #1 0x0805b158 in PyObject_Str (v=0x8d747d4) at Objects/object.c:414 #2 0x0807951a in unicode_new (type=0x81523a0, args=0x8de77ac, kwds=0x0) at Objects/unicodeobject.c:9247 #3 0x0806068d in type_call (type=0x81523a0, args=0x8de77ac, kwds=0x0) at Objects/typeobject.c:636 #4 0x080d83a9 in PyObject_Call (func=0x81523a0, arg=0x8de77ac, kw=0x0) at Objects/abstract.c:2178 #5 0x0808de50 in PyEval_EvalFrameEx (f=0x8e2d07c, throwflag=0) at Python/ceval.c:3606 #6 0x0808fccd in PyEval_EvalFrameEx (f=0x8e2bf14, throwflag=0) at Python/ceval.c:3481 #7 0x0808fccd in PyEval_EvalFrameEx (f=0x8e2cef4, throwflag=0) at Python/ceval.c:3481 #8 0x0808fccd in PyEval_EvalFrameEx (f=0x8e2d4ec, throwflag=0) at Python/ceval.c:3481 #9 0x08090f6b in PyEval_EvalCodeEx (co=0x8268458, globals=0xb7c182d4, locals=0x0, args=0x8e2cea4, argcount=3, kws=0x8e2ceb0, kwcount=1, defs=0x826c678, defcount=3, kwdefs=0x0, closure=0x0) at Python/ceval.c:2830 [...] From tseaver at palladion.com Thu Jul 24 14:39:08 2008 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 24 Jul 2008 08:39:08 -0400 Subject: [Python-Dev] bsddb: Test failures on windows (HELP!) In-Reply-To: <095c01c8ed50$60bdfed0$2239fc70$@com.au> References: <48874F2D.5070201@jcea.es> <095c01c8ed50$60bdfed0$2239fc70$@com.au> Message-ID: <488877EC.600@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark Hammond wrote: >> Trent, I was wondering if you could look at some test failures in MS >> Windows builds. I can't debug Windows issues myself :-(. This is a MS >> free environment... > > In these errors I see lots of bsdbd errors, many of the form: > > | DBFileExistsError: (17, 'File exists -- __fop_file_setup: Retry limit > (100) exceeded') Maybe this one is due to the fact that Windows, unlike POSIX, doesn't allow unlinking an open file? > Maybe an old test file isn't being nuked? Others of the form: > > | self.assertTrue(time.time()<timeout) > | AssertionError > > which also look more related to the test suite than to Windows. Perhaps this is due to differing timer granularity on Windows? > There are also lots of errors due to the environment having a unicode object > in it: > > | test test_site failed -- Traceback (most recent call last): > | ... > | TypeError: environment can only contain strings > > | test test_subprocess failed -- errors occurred; run in verbose mode for > details > | [Possibly the same error as below?] > > | test test_sys failed -- Traceback (most recent call last): > | File "...\subprocess.py", line 817, in _execute_child > | startupinfo) > | TypeError: environment can only contain strings That definitely shouldn't happen on Unix either. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIiHfr+gerLs4ltQ4RAk5KAJ9It0Am1VfFNQNaE+wA8uWkkTZ6wQCgtwlx o16eVKpEXTHED4X1/Vi0Nk0= =zzKd -----END PGP SIGNATURE----- From musiccomposition at gmail.com Thu Jul 24 14:41:11 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 24 Jul 2008 07:41:11 -0500 Subject: [Python-Dev] crash in 3.0b2 exception code In-Reply-To: <g69atp$716$1@ger.gmane.org> References: <g69atp$716$1@ger.gmane.org> Message-ID: <1afaf6160807240541n2d01b0d9yb7bc3fc5ff56407b@mail.gmail.com> On Thu, Jul 24, 2008 at 2:27 AM, Stefan Behnel <stefan_ml at behnel.de> wrote: > Hi, > > I get a crash in one of lxml's doctests in 3.0b2, and it looks like it's > coming from plain Python code (as opposed to Cython code). Just a quick > check > before I start digging into this, has anyone seen this before? Please file a bug report. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080724/9018b196/attachment.htm> From amauryfa at gmail.com Thu Jul 24 16:23:21 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 24 Jul 2008 16:23:21 +0200 Subject: [Python-Dev] bsddb: Test failures on windows (HELP!) In-Reply-To: <488877EC.600@palladion.com> References: <48874F2D.5070201@jcea.es> <095c01c8ed50$60bdfed0$2239fc70$@com.au> <488877EC.600@palladion.com> Message-ID: <e27efe130807240723v155b05a5x1bd2a9b1a64ab057@mail.gmail.com> Tres Seaver wrote: > Mark Hammond wrote: > >>> Trent, I was wondering if you could look at some test failures in MS >>> Windows builds. I can't debug Windows issues myself :-(. This is a MS >>> free environment... >> >> In these errors I see lots of bsdbd errors, many of the form: >> >> | DBFileExistsError: (17, 'File exists -- __fop_file_setup: Retry limit >> (100) exceeded') > > Maybe this one is due to the fact that Windows, unlike POSIX, doesn't > allow unlinking an open file? I see that some tests use os.unlink. They should use test_support.unlink() instead. -- Amaury Forgeot d'Arc From tom at vector-seven.com Thu Jul 24 16:56:35 2008 From: tom at vector-seven.com (Thomas Lee) Date: Fri, 25 Jul 2008 00:56:35 +1000 Subject: [Python-Dev] lnotab and the AST optimizer Message-ID: <48889823.20506@vector-seven.com> I'm making some good progress with the AST optimizer, and now the main thing standing in my way is lnotab. Currently lnotab expects bytecode sequencing to be roughly in-sync with the order of the source file and a few things that the optimizer does (e.g. swapping the bodies of an if/else after removing negation such that "if not X: A; else: B" becomes "if X: B; else A") breaks this assumption. This will result in either an assertion failure or incorrect line numbers being reported. It seems that lnotab is used in relatively few places in the source code at the moment, but if I'm going to make a change to how lnotab works I want to do so in a way that's going to allow me to move forward while keeping everybody happy. I'm away for a few days so I probably won't be able to get back to anybody until either Sunday or Monday, but I'd appreciate it if anybody in the know can weigh in on this. Cheers, Tom From solipsis at pitrou.net Thu Jul 24 17:19:04 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 24 Jul 2008 15:19:04 +0000 (UTC) Subject: [Python-Dev] lnotab and the AST optimizer References: <48889823.20506@vector-seven.com> Message-ID: <loom.20080724T151343-964@post.gmane.org> Hi, > I'm making some good progress with the AST optimizer, and now the main > thing standing in my way is lnotab. Currently lnotab expects bytecode > sequencing to be roughly in-sync with the order of the source file and a > few things that the optimizer does (e.g. swapping the bodies of an > if/else after removing negation such that "if not X: A; else: B" becomes > "if X: B; else A") breaks this assumption. This will result in either an > assertion failure or incorrect line numbers being reported. In http://bugs.python.org/issue2459 ("speedup for / while / if with better bytecode") I had the same problem and decided to change the lnotab format so that line number increments are signed bytes rather than unsigned. The proposed patch contains many other changes, but with a bit of perseverance you may be able to extract the lnotab-related modifications... ;) This modification will allow many more types of control flow transformations than the current scheme does. By the way: > swapping the bodies of an > if/else after removing negation such that "if not X: A; else: B" becomes > "if X: B; else A") Is this really an optimization? "if" and "if not" should use the same number of opcodes (the former produces JUMP_IF_FALSE and the latter JUMP_IF_TRUE). Regards Antoine. From solipsis at pitrou.net Thu Jul 24 17:43:30 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 24 Jul 2008 15:43:30 +0000 (UTC) Subject: [Python-Dev] lnotab and the AST optimizer References: <48889823.20506@vector-seven.com> <loom.20080724T151343-964@post.gmane.org> Message-ID: <loom.20080724T153939-366@post.gmane.org> Antoine Pitrou <solipsis <at> pitrou.net> writes: > > In http://bugs.python.org/issue2459 ("speedup for / while / if with better > bytecode") I had the same problem and decided to change the lnotab format so > that line number increments are signed bytes rather than unsigned. By the way, the same change could be done for relative jump offsets in the bytecode (change them from unsigned shorts to signed shorts). Taken together, both modifications would release a lot of constraints on the ordering of code blocks. From tom at vector-seven.com Thu Jul 24 17:49:37 2008 From: tom at vector-seven.com (Thomas Lee) Date: Fri, 25 Jul 2008 01:49:37 +1000 Subject: [Python-Dev] lnotab and the AST optimizer In-Reply-To: <loom.20080724T151343-964@post.gmane.org> References: <48889823.20506@vector-seven.com> <loom.20080724T151343-964@post.gmane.org> Message-ID: <4888A491.4070108@vector-seven.com> Antoine Pitrou wrote: > Hi, > > Hi. Thanks for getting back to me so quickly. I can even respond before I have to drag myself off to bed. :) >> I'm making some good progress with the AST optimizer, and now the main >> thing standing in my way is lnotab. Currently lnotab expects bytecode >> sequencing to be roughly in-sync with the order of the source file and a >> few things that the optimizer does (e.g. swapping the bodies of an >> if/else after removing negation such that "if not X: A; else: B" becomes >> "if X: B; else A") breaks this assumption. This will result in either an >> assertion failure or incorrect line numbers being reported. >> > > In http://bugs.python.org/issue2459 ("speedup for / while / if with better > bytecode") I had the same problem and decided to change the lnotab format so > that line number increments are signed bytes rather than unsigned. The proposed > patch contains many other changes, but with a bit of perseverance you may be > able to extract the lnotab-related modifications... ;) > > This modification will allow many more types of control flow transformations > than the current scheme does. > > Great, thanks! I'll check it out next week. > By the way: > >> swapping the bodies of an >> if/else after removing negation such that "if not X: A; else: B" becomes >> "if X: B; else A") >> > > Is this really an optimization? "if" and "if not" should use the same number of > opcodes (the former produces JUMP_IF_FALSE and the latter JUMP_IF_TRUE). > > Not quite. :) Even if we were producing a JUMP_IF_FALSE, it'd still be nice to optimize away the UNARY_NOT in the former. In practice, both actually produce a JUMP_IF_TRUE due to an existing optimization in the peephole optimizer which does just that. I'm trying to emulate this at the AST level because I'm part of a secret, evil conspiracy to be rid of the peephole optimizer. Shh. The relevant code in the peepholer, plus comment: /* Replace UNARY_NOT JUMP_IF_FALSE POP_TOP with with JUMP_IF_TRUE POP_TOP */ case UNARY_NOT: if (codestr[i+1] != JUMP_IF_FALSE || codestr[i+4] != POP_TOP || !ISBASICBLOCK(blocks,i,5)) continue; tgt = GETJUMPTGT(codestr, (i+1)); if (codestr[tgt] != POP_TOP) continue; j = GETARG(codestr, i+1) + 1; codestr[i] = JUMP_IF_TRUE; SETARG(codestr, i, j); codestr[i+3] = POP_TOP; codestr[i+4] = NOP; break; A little hackage with the dis module seems to confirm this is the case. Of course, if you know of a situation where this optimization doesn't apply and we actually wind up with a JUMP_IF_FALSE for an if/else post-peephole, I'm all ears. Thanks again! Cheers, T From tom at vector-seven.com Thu Jul 24 17:59:54 2008 From: tom at vector-seven.com (Thomas Lee) Date: Fri, 25 Jul 2008 01:59:54 +1000 Subject: [Python-Dev] lnotab and the AST optimizer In-Reply-To: <loom.20080724T153939-366@post.gmane.org> References: <48889823.20506@vector-seven.com> <loom.20080724T151343-964@post.gmane.org> <loom.20080724T153939-366@post.gmane.org> Message-ID: <4888A6FA.9050100@vector-seven.com> Antoine Pitrou wrote: > Antoine Pitrou <solipsis <at> pitrou.net> writes: > >> In http://bugs.python.org/issue2459 ("speedup for / while / if with better >> bytecode") I had the same problem and decided to change the lnotab format so >> that line number increments are signed bytes rather than unsigned. >> > > By the way, the same change could be done for relative jump offsets in the > bytecode (change them from unsigned shorts to signed shorts). Taken together, > both modifications would release a lot of constraints on the ordering of code > blocks. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com > By the way, you were right about JUMP_IF_TRUE/JUMP_IF_FALSE. It's far too late. Apologies. I'm still pretty sure this is the peepholer's doing, though, and if that's the case then I want to try and deal with it at the AST level. Which is what's being achieved with the AST optimization I originally proposed, right? Cheers, T From jcea at jcea.es Thu Jul 24 18:03:20 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 24 Jul 2008 18:03:20 +0200 Subject: [Python-Dev] bsddb: Test failures on windows (HELP!) In-Reply-To: <48874F2D.5070201@jcea.es> References: <48874F2D.5070201@jcea.es> Message-ID: <4888A7C8.9010308@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jesus Cea wrote: | Trent, I was wondering if you could look at some test failures in MS | Windows builds. I can't debug Windows issues myself :-(. This is a MS | free environment... I will be out of the city, 100% offline, until monday/tuesday. I will read your suggestions and do some tests as soon as possible. Please, keep the brainstorming going :) Feel free to (minimally };-)) touch the testcases trying to improve the situation. If that does the trick, please let me know to integrate the changes in my code. Remember this code must work in python 2.[3-6] (reason, for example, because I have my own "test_support" code, in pybsddb). Thanks a lot for your invaluable suggestions. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSIinuplgi5GaxT1NAQK6qQP+LOv1lU6G6+GaSrxUqrnFM62bTcmXCMay S0ic3rWYUL4YTvWT/Ips/qBgYvRCPl3uHnmIDia9UAOnYh3EYjkFN+/4GDofGwM+ 1UBRu86C7LsYdJl2VlHJyHGWmz6tgbbtAue306CNX01yD+pwYsCUqMSTuzjiiNCx /q1DHdJv8Qo= =OpNx -----END PGP SIGNATURE----- From solipsis at pitrou.net Thu Jul 24 18:15:57 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 24 Jul 2008 16:15:57 +0000 (UTC) Subject: [Python-Dev] lnotab and the AST optimizer References: <48889823.20506@vector-seven.com> <loom.20080724T151343-964@post.gmane.org> <loom.20080724T153939-366@post.gmane.org> <4888A6FA.9050100@vector-seven.com> Message-ID: <loom.20080724T160958-653@post.gmane.org> Thomas Lee <tom <at> vector-seven.com> writes: > By the way, you were right about JUMP_IF_TRUE/JUMP_IF_FALSE. It's far > too late. Apologies. > > I'm still pretty sure this is the peepholer's doing, Yes indeed. > Which is what's being achieved with the AST optimization I originally > proposed, right? Well, not exactly, your optimization eliminates the UNARY_NOT by swapping the if/else blocks, while the peepholer eliminates the UNARY_NOT by fusing it with the subsequent jump opcode. In this case it doesn't make much of a difference, but if there is only an "if" without an "else", the peepholer's optimization is still possible while yours is not. (bottom line: the peepholer is not dead!) cheers, Antoine. From tom at vector-seven.com Thu Jul 24 18:23:00 2008 From: tom at vector-seven.com (Thomas Lee) Date: Fri, 25 Jul 2008 02:23:00 +1000 Subject: [Python-Dev] lnotab and the AST optimizer In-Reply-To: <loom.20080724T160958-653@post.gmane.org> References: <48889823.20506@vector-seven.com> <loom.20080724T151343-964@post.gmane.org> <loom.20080724T153939-366@post.gmane.org> <4888A6FA.9050100@vector-seven.com> <loom.20080724T160958-653@post.gmane.org> Message-ID: <4888AC64.5060303@vector-seven.com> Antoine Pitrou wrote: > Thomas Lee <tom <at> vector-seven.com> writes: > >> By the way, you were right about JUMP_IF_TRUE/JUMP_IF_FALSE. It's far >> too late. Apologies. >> >> I'm still pretty sure this is the peepholer's doing, >> > > Yes indeed. > > >> Which is what's being achieved with the AST optimization I originally >> proposed, right? >> > > Well, not exactly, your optimization eliminates the UNARY_NOT by swapping the > if/else blocks, while the peepholer eliminates the UNARY_NOT by fusing it with > the subsequent jump opcode. In this case it doesn't make much of a difference, > but if there is only an "if" without an "else", the peepholer's optimization is > still possible while yours is not. > > Unless a pass is injected into the if body, which will generate no additional bytecode and still have the same net effect. > (bottom line: the peepholer is not dead!) > > We'll see ;) Thanks for all your help, I'm looking forward to getting my hands on that patch. Cheers, T From pje at telecommunity.com Thu Jul 24 18:34:26 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 24 Jul 2008 12:34:26 -0400 Subject: [Python-Dev] lnotab and the AST optimizer In-Reply-To: <48889823.20506@vector-seven.com> References: <48889823.20506@vector-seven.com> Message-ID: <20080724163339.D053F3A406B@sparrow.telecommunity.com> At 12:56 AM 7/25/2008 +1000, Thomas Lee wrote: >I'm making some good progress with the AST optimizer, and now the >main thing standing in my way is lnotab. Currently lnotab expects >bytecode sequencing to be roughly in-sync with the order of the >source file and a few things that the optimizer does (e.g. swapping >the bodies of an if/else after removing negation such that "if not >X: A; else: B" becomes "if X: B; else A") breaks this assumption. >This will result in either an assertion failure or incorrect line >numbers being reported. > >It seems that lnotab is used in relatively few places in the source >code at the moment, but if I'm going to make a change to how lnotab >works I want to do so in a way that's going to allow me to move >forward while keeping everybody happy. > >I'm away for a few days so I probably won't be able to get back to >anybody until either Sunday or Monday, but I'd appreciate it if >anybody in the know can weigh in on this. I'd personally love it if the lnotab were capable of handling line numbers from different files as well as out-of-order lines. (For function inlining, among other more esoteric things.) From josiah.carlson at gmail.com Thu Jul 24 19:26:39 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Thu, 24 Jul 2008 10:26:39 -0700 Subject: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was Re: No beta2 tonight) In-Reply-To: <48845653.5000100@jcea.es> References: <1DB64EE3-AEC5-4745-B3F3-BB7321B3E5D4@python.org> <ca471dc20807180811v2478f5ds140810ca486c2bb0@mail.gmail.com> <e6511dbf0807181045x5e7f8b63n8bf90e8c47f8c4f8@mail.gmail.com> <F9D592EE-FB50-412A-9553-B071823E1EEF@acm.org> <e6511dbf0807181215r4774dae9i3c10f128be83c93c@mail.gmail.com> <4881C04E.5060208@gmail.com> <e6511dbf0807190927o2415eb02rd0ae21b7f0b88375@mail.gmail.com> <4882986A.3060509@jcea.es> <e6511dbf0807201024x5f50b46fha0a76ad859dc9e82@mail.gmail.com> <48845653.5000100@jcea.es> Message-ID: <e6511dbf0807241026qc2aa105o11a3b6ca8bdd5a5a@mail.gmail.com> On Mon, Jul 21, 2008 at 2:26 AM, Jesus Cea <jcea at jcea.es> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Josiah Carlson wrote: > | I'm still curious as to what deep features people are using in bsddb. > | Anyone have any pointers to open source software? > > I'm using replication and distributed transactions. Database encryption > and page integrity checks. Abusing the master election BDB > infrastructure for some evil but fun purposes. Managing databases in the > 200 terabyte range. Locking & logging infrastructure to implement > application logic integrated with database operation. MVCC everywhere. > > Nothing I can show you without killing you after, though. That's the kind of answer I was looking for :). Though I don't know if you are the "typical user"; I don't have enough data (the few others that sent messages here and privately weren't using it to your extent). Some of those features are possible to add without huge amounts of work (an afternoon or two), but indeed, some of those are not possible without substantial investment and testing. > Independently of current bsddb usage profile, current 2.6 code support > for distributed transactions and replication enables new application > uses that sqlite can't match. > > Since I'm taking full responsability over bsddb both in 2.6 and 3.0, I > don't really see what the issue is. Past mistakes and maintenance > nightmares WILL NOT be repeated. Your taking over of maintenance is part of the reason why I withdrew my offer ;) . - Josiah From loisel at temple.edu Thu Jul 24 23:19:27 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Thu, 24 Jul 2008 17:19:27 -0400 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> Message-ID: <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> Greg Ewing said: > I would actually be in favour of adding a matrix multiplication > operator That would be helpful to me, for my students as well as my papers. Sincerely, -- S?bastien Loisel From scott+python-dev at scottdial.com Fri Jul 25 00:06:10 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Thu, 24 Jul 2008 18:06:10 -0400 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> Message-ID: <4888FCD2.6060908@scottdial.com> Sebastien Loisel wrote: > Greg Ewing said: >> I would actually be in favour of adding a matrix multiplication >> operator > > That would be helpful to me, for my students as well as my papers. > Perhaps I'm nobody, but I think this would be ridiculous. Matrices are not native objects to the language. There is no type(matrix). The notion of what makes a Python object a matrix is a convention and to have built-in operators dedicated to such objects makes no sense. There are multiple ways to stuff matrices into Python. Please submit a PEP for a type(matrix) first. Until a matrix is a first-order object in Python, there is no logic to making operators for them. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From barry at barrys-emacs.org Fri Jul 25 00:33:46 2008 From: barry at barrys-emacs.org (Barry Scott) Date: Thu, 24 Jul 2008 23:33:46 +0100 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> <g61jt8$hag$2@ger.gmane.org> <4884581F.8010201@jcea.es> <bbaeab100807211116g4df70dc1o81b1075a88282c93@mail.gmail.com> <319e029f0807211437w53619985s85157a60d42966e@mail.gmail.com> Message-ID: <BAF14EE0-6706-40F9-90F1-EBB97EEAB28E@barrys-emacs.org> On Jul 21, 2008, at 22:37, Lennart Regebro wrote: > On Mon, Jul 21, 2008 at 20:16, Brett Cannon <brett at python.org> wrote: >> But waiting until all the betas have gone out totally defeats the >> purpose of the betas! > > I agree. Writing an actual *guide* can wait, but documenting the > differences with code examples is a work that can start now, and I > agree that it would be great if this would start now. > > But writing a guide might not be a good idea until we know what the > changes are, and if the API is changing quickly now we don't. :-) I'm working on getting a version of PyCXX working with Python 3.0. The lack of any docs outside of the header files is making this a slow process. I think its a mistake to expect a serious beta test of extensions when you have no guide to the changes in the C API. If you had a guide then diff it between releases would be a guide to what need fixing up going from beta to beta to rc. Oh and I'm not going to try and make a version of PyCXX that works on 2.x and 3.x as the changes are too fundamental. Barry From fredrik.johansson at gmail.com Fri Jul 25 02:02:41 2008 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Fri, 25 Jul 2008 02:02:41 +0200 Subject: [Python-Dev] Infix operators In-Reply-To: <4888FCD2.6060908@scottdial.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> <4888FCD2.6060908@scottdial.com> Message-ID: <3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com> On Fri, Jul 25, 2008 at 12:06 AM, Scott Dial <scott+python-dev at scottdial.com> wrote: > Perhaps I'm nobody, but I think this would be ridiculous. Matrices are > not native objects to the language. There is no type(matrix). The notion > of what makes a Python object a matrix is a convention and to have > built-in operators dedicated to such objects makes no sense. There are > multiple ways to stuff matrices into Python. Please submit a PEP for a > type(matrix) first. Until a matrix is a first-order object in Python, > there is no logic to making operators for them. Though I would personally find a matrix multiplication operator useful, I have to agree with this. Anyway, it is easy to define pseudo-operators in Python; just create an Operator class and implement its __mul__ and __rmul__ methods appropriately (there are recipes for this around somewhere). Then you can define various custom multiplication operators with syntax like this: A *matrixmul* B A *dot* B A *cross* B A *elementwise* B Some other fun possibilities: A +concat+ B A /solve/ B A **left_inverse** (-1) A **right_inverse** (-1) x **tetrate** y n |choose| k Fredrik From greg.ewing at canterbury.ac.nz Fri Jul 25 03:26:00 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 25 Jul 2008 13:26:00 +1200 Subject: [Python-Dev] lnotab and the AST optimizer In-Reply-To: <48889823.20506@vector-seven.com> References: <48889823.20506@vector-seven.com> Message-ID: <48892BA8.2090603@canterbury.ac.nz> Thomas Lee wrote: > I'm making some good progress with the AST optimizer, and now the main > thing standing in my way is lnotab. My suggestion would be to drop the idea of trying to compress the lnotab in clever ways, and just make it a straightforward list of bytecode offset/line number pairs. I can't imagine that the size of an uncompressed lnotab would be a problem in this day and age. If ordering is an issue, generate it internally as a dict and convert it to a sorted list on output. -- Greg From greg.ewing at canterbury.ac.nz Fri Jul 25 04:02:30 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 25 Jul 2008 14:02:30 +1200 Subject: [Python-Dev] Infix operators In-Reply-To: <4888FCD2.6060908@scottdial.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> <4888FCD2.6060908@scottdial.com> Message-ID: <48893436.5030805@canterbury.ac.nz> Scott Dial wrote: > Perhaps I'm nobody, but I think this would be ridiculous. Matrices are > not native objects to the language. Why should that matter? We already have things like sum(), which operates on any sequence of numbers, without needing a special "array of numbers" data type. I don't see why one shouldn't be able to perform matrix multiplication on a plausible representation of a matrix using the existing built-in sequence types. > Until a matrix is a first-order object in Python, > there is no logic to making operators for them. If there were a dedicated matrix type, there would actually be *less* reason to have a new operator, since that type could just implement * as matrix multiplication. However, there are disadvantages to that, which become apparent when numpy is considered. There are good reasons for wanting * to mean elementwise multiplication of numpy arrays. There are also good reasons for wanting to use numpy arrays as matrices, since you get the benefit of all the other powerful things that numpy can do with arrays. You can use a matrix type that's based somehow on a numpy array, but there are problems with that too. E.g. if you add a matrix and a plain numpy array, what type should the result be? If plain numpy arrays can be used directly as matrices, that problem goes away. -- Greg From greg.ewing at canterbury.ac.nz Fri Jul 25 04:08:42 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 25 Jul 2008 14:08:42 +1200 Subject: [Python-Dev] Infix operators In-Reply-To: <3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> <4888FCD2.6060908@scottdial.com> <3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com> Message-ID: <488935AA.6070509@canterbury.ac.nz> Fredrik Johansson wrote: > Anyway, it is easy to define pseudo-operators in Python; > > A *matrixmul* B > A *dot* B > A *cross* B > A *elementwise* B Urg. This is another one of those recipes that I consider is too clever for its own good. Very nice in theory, but I would never use it in real life. What's more, it doesn't address the real problem at hand, which is providing a notation for matrix multiplication that is concise enough to be used *very* frequently -- like multiple times in every line of code that manipulates matrices. -- Greg From scott+python-dev at scottdial.com Fri Jul 25 04:46:24 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Thu, 24 Jul 2008 22:46:24 -0400 Subject: [Python-Dev] Infix operators In-Reply-To: <48893436.5030805@canterbury.ac.nz> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> <4888FCD2.6060908@scottdial.com> <48893436.5030805@canterbury.ac.nz> Message-ID: <48893E80.6030907@scottdial.com> Greg Ewing wrote: > Scott Dial wrote: >> Perhaps I'm nobody, but I think this would be ridiculous. Matrices are >> not native objects to the language. > > Why should that matter? We already have things like > sum(), which operates on any sequence of numbers, > without needing a special "array of numbers" data > type. I would argue that Python contains a "array of some_type" data type. That sum() performs a left-fold of __add__ on the array is completely independent of them being numbers. In all fact, they could be any type that supports __add__/__radd__ or even a mix of types. I do not believe "array of numbers" to be analogically equivalent to a "matrix of numbers". We have an "array of" type in Python, we do not have a "matrix of" type in Python. sum() is not an operator, it is a builtin; the suggestion was for there to be an operator, not a builtin. If you want to suggest there be a mmul() builtin, then perhaps there is a viable answer there, despite the lack of a "matrix of" type still. > I don't see why one shouldn't be able to > perform matrix multiplication on a plausible > representation of a matrix using the existing > built-in sequence types. What is "a plausible representation of a matrix"? Is it a list of lists? Row-major (m[1][2]) or column-major (m[2][1])? Is it a dictionary of tuple'd indices (m[1,2])? Also, You went on to talk about wanting to using numpy.array. How does this not make it clear that there is not a case of TOOWTDI? -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From josiah.carlson at gmail.com Fri Jul 25 04:47:24 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Thu, 24 Jul 2008 19:47:24 -0700 Subject: [Python-Dev] Infix operators In-Reply-To: <488935AA.6070509@canterbury.ac.nz> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> <4888FCD2.6060908@scottdial.com> <3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com> <488935AA.6070509@canterbury.ac.nz> Message-ID: <e6511dbf0807241947q479e8459y7d424be4ffef2f77@mail.gmail.com> On Thu, Jul 24, 2008 at 7:08 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Fredrik Johansson wrote: > >> Anyway, it is easy to define pseudo-operators in Python; >> >> A *matrixmul* B >> A *dot* B >> A *cross* B >> A *elementwise* B > > Urg. This is another one of those recipes that I consider > is too clever for its own good. Very nice in theory, > but I would never use it in real life. That's unfortunate ;), because it's available in Python today, and with the common local caching of globals, can be as short as 3 characters (M for matrixmul, D for dot, X for cross, E for elementwise). > What's more, it doesn't address the real problem at hand, > which is providing a notation for matrix multiplication > that is concise enough to be used *very* frequently -- > like multiple times in every line of code that manipulates > matrices. This is the first time anyone has mentioned "conciseness" in this thread. And what's a 3 character operator between friends? The "and" and "not" operators don't seem to be bothered by three characters. ;) - Josiah From musiccomposition at gmail.com Fri Jul 25 04:53:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 24 Jul 2008 21:53:37 -0500 Subject: [Python-Dev] Warnings for intra-package imports Message-ID: <1afaf6160807241953s4a4b2d5aq5e704ae852a0ad12@mail.gmail.com> I was just reading up on PEP 328. In the "Timeline" section, it mentions that intra-package imports should raise a DeprecationWarning in 2.6. This doesn't seem to be implemented currently. Is this still the plan? I would like to see Py3k warnings for these kinds of imports at least. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Fri Jul 25 06:17:30 2008 From: brett at python.org (Brett Cannon) Date: Fri, 25 Jul 2008 00:17:30 -0400 Subject: [Python-Dev] Warnings for intra-package imports In-Reply-To: <1afaf6160807241953s4a4b2d5aq5e704ae852a0ad12@mail.gmail.com> References: <1afaf6160807241953s4a4b2d5aq5e704ae852a0ad12@mail.gmail.com> Message-ID: <bbaeab100807242117y19adb39g805b9bc20f60d4a5@mail.gmail.com> On Thu, Jul 24, 2008 at 10:53 PM, Benjamin Peterson <musiccomposition at gmail.com> wrote: > I was just reading up on PEP 328. In the "Timeline" section, it > mentions that intra-package imports should raise a DeprecationWarning > in 2.6. This doesn't seem to be implemented currently. > > Is this still the plan? I would like to see Py3k warnings for these > kinds of imports at least. > It should still be the plan. -Brett From greg.ewing at canterbury.ac.nz Fri Jul 25 07:27:06 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 25 Jul 2008 17:27:06 +1200 Subject: [Python-Dev] Infix operators In-Reply-To: <48893E80.6030907@scottdial.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> <4888FCD2.6060908@scottdial.com> <48893436.5030805@canterbury.ac.nz> <48893E80.6030907@scottdial.com> Message-ID: <4889642A.8040204@canterbury.ac.nz> Scott Dial wrote: > I would argue that Python contains a "array of some_type" data type. > That sum() performs a left-fold of __add__ on the array is completely > independent of them being numbers. That's not strictly true -- it explicitly refuses to operate on strings (or at least it did last time I heard). Guido has said that he *intends* it to only be used for numbers, even if it happens to accidentally do something with other types. However, as I envisage it, the @ operator wouldn't be restricted to numbers either -- it would do whatever the * and + operations do on the elements. > sum() is not an operator, it is a builtin; the > suggestion was for there to be an operator, not a builtin. That's true, and it means that the built-in implementation wouldn't have as wide applicability as the sum() function, since it would be restricted to lists and tuples (and perhaps array.array instances). But I don't think that's a fatal flaw -- if you create your own sequence type, you have to take steps if you want to be able e.g. to use + to concatenate them, and nobody complains about that. > If you want > to suggest there be a mmul() builtin, then perhaps there is a viable > answer there No, that's not a viable answer to people who want a matrix multiplication operator. They want an operator because a function is nowhere near concise enough. Telling these people they have to use a function to multiply matrices is like telling them they have to use a function to multiply numbers. > What is "a plausible representation of a matrix"? Is it a list of lists? That's one. A tuple of tuples would be another. > Row-major (m[1][2]) or column-major (m[2][1])? Pick one and stick to it. Probably row-major, since it's what the numpy matrixmultiply function uses. > Is it a dictionary of tuple'd indices (m[1,2])? I think dictionaries would have to be excluded, because there's no easy way of finding out the dimensions, it's not obvious what to do about missing elements, etc. > Also, You went on to talk about wanting to using numpy.array. Yes -- what's wrong with that? > How does this not make it clear that there is not a case of TOOWTDI? I think there *is* one obvious way of representing a matrix, or any 2D array, using built-in Python types, or rather two ways -- a list of lists if you want mutability, or a tuple of tuples if you want immutability. -- Greg From greg.ewing at canterbury.ac.nz Fri Jul 25 07:30:45 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 25 Jul 2008 17:30:45 +1200 Subject: [Python-Dev] Infix operators In-Reply-To: <e6511dbf0807241947q479e8459y7d424be4ffef2f77@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <d2155e360807231708p2e6c00c3h611cefd0c8a48443@mail.gmail.com> <b7975ab80807231721u68c19785vabe472c8486825d6@mail.gmail.com> <b7975ab80807241419m28dfd4bg2f47bd37e66291b6@mail.gmail.com> <4888FCD2.6060908@scottdial.com> <3d0cebfb0807241702m5b00f07yc612847cc2c23252@mail.gmail.com> <488935AA.6070509@canterbury.ac.nz> <e6511dbf0807241947q479e8459y7d424be4ffef2f77@mail.gmail.com> Message-ID: <48896505.5080205@canterbury.ac.nz> Josiah Carlson wrote: > This is the first time anyone has mentioned "conciseness" in this > thread. I thought it more or less went without saying. After all, if conciseness isn't a goal, there's nothing wrong with a plain function call, which can be as short as 3 characters as well. The trouble is that, for the people who badly want a matrix multiplication operator, 3 characters is far too long! -- Greg From status at bugs.python.org Fri Jul 25 18:06:29 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 25 Jul 2008 18:06:29 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080725160629.C01FD78350@psf.upfronthosting.co.za> ACTIVITY SUMMARY (07/18/08 - 07/25/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1951 open (+29) / 13318 closed (+11) / 15269 total (+40) Open issues with patches: 615 Average duration of open issues: 710 days. Median duration of open issues: 1621 days. Open Issues Breakdown open 1937 (+28) pending 14 ( +1) Issues Created Or Reopened (44) _______________________________ function annotation for builtin and C function 07/21/08 http://bugs.python.org/issue3208 reopened amaury.forgeotdarc `./configure --enable-framework --enable-universalsdk` fails bec 07/21/08 CLOSED http://bugs.python.org/issue3381 reopened georg.brandl patch test_urllib2_localnet fails on MacOS X 10.4.11 (Intel) 07/18/08 http://bugs.python.org/issue3407 created MrJean1 urllib incomplete and urllib2 does not exist 07/18/08 CLOSED http://bugs.python.org/issue3408 created vizcayno ElementPath.Path.findall problem with unicode input 07/18/08 http://bugs.python.org/issue3409 created qual patch platform.version() don't work as expected in Vista in portuguese 07/18/08 http://bugs.python.org/issue3410 created portella str.format() on negative floats 07/18/08 CLOSED http://bugs.python.org/issue3411 created hagen Fraction and Decimal in the Tutorial 07/18/08 http://bugs.python.org/issue3412 created segfaulthunter typo in Mac/Makefile.in breaks installing IDLE 07/19/08 CLOSED http://bugs.python.org/issue3413 created erickt frameworkinstall doesn't create Python.app, which breaks python 07/19/08 CLOSED http://bugs.python.org/issue3414 created erickt Interpreter error when running a script under debugger control 07/19/08 CLOSED http://bugs.python.org/issue3415 created nirai Wrong inherit in PickleHTMLBuilder 07/19/08 CLOSED http://bugs.python.org/issue3416 created is make the fix_dict fixer smarter 07/19/08 http://bugs.python.org/issue3417 created benjamin.peterson patch heavy resource usage with string functions 07/19/08 CLOSED http://bugs.python.org/issue3418 created mgogoulos multiprocessing module is racy 07/19/08 http://bugs.python.org/issue3419 created cartman 2to3 fails to run on Mac OS X 10.4 PPC 3.0b2 07/20/08 http://bugs.python.org/issue3420 created barry-scott Test failure in test_math::testSum 07/20/08 http://bugs.python.org/issue3421 created georg.brandl sphinx.doc.autodoc: Hook for changing argspec 07/20/08 http://bugs.python.org/issue3422 created pv patch DeprecationWarning message applies to wrong context with exec() 07/22/08 http://bugs.python.org/issue3423 created ghazel imghdr test order makes it slow 07/22/08 http://bugs.python.org/issue3424 created biny posixmodule.c always using res = utime(path, NULL) 07/22/08 http://bugs.python.org/issue3425 created oskar86 os.path.abspath with unicode argument should call os.getcwdu 07/22/08 http://bugs.python.org/issue3426 created saturn_mimas urllib documentation: urlopen().info() return type 07/22/08 http://bugs.python.org/issue3427 created ThomasH httplib.HTTPMessage undocumented 07/22/08 http://bugs.python.org/issue3428 created ThomasH urllib.urlopen() return type 07/22/08 http://bugs.python.org/issue3429 created ThomasH httplib.HTTPResponse documentations inconsistent 07/22/08 http://bugs.python.org/issue3430 created ThomasH multiprocessing uses Pickler.dispatch which isn't in 3.0 _pickle 07/23/08 CLOSED http://bugs.python.org/issue3431 created georg.brandl Mac, 2.6 framework install error 07/24/08 http://bugs.python.org/issue3432 created robind Mac, 3.0 framework install error with fink cp 07/24/08 http://bugs.python.org/issue3433 created robind Mac, 3.0 framework install, Python.app not created 07/24/08 CLOSED http://bugs.python.org/issue3434 created robind trace.py tries to get coverage data from non Python files 07/24/08 http://bugs.python.org/issue3435 created Gustavo patch csv.DictReader inconsistency 07/24/08 http://bugs.python.org/issue3436 created mishok13 patch robotparser.py missing one line 07/24/08 http://bugs.python.org/issue3437 created mbloore PyCF_DONT_IMPLY_DEDENT can be used to activate the with statemen 07/24/08 http://bugs.python.org/issue3438 created gpolo patch math.frexp and obtaining the bit size of a large integer 07/24/08 http://bugs.python.org/issue3439 created fredrikj patch Starting Python as a subprocess fails when subprocess.Popen has 07/24/08 http://bugs.python.org/issue3440 created kermode Regression in "module as a script" command-line option 07/25/08 CLOSED http://bugs.python.org/issue3441 created richard wsgiref.validate.InputWrapper.readline does not accept optional 07/25/08 CLOSED http://bugs.python.org/issue3442 created xaka crash on badly initialised AttributeError 07/25/08 http://bugs.python.org/issue3443 created scoder add warnings for intra-package imports 07/25/08 http://bugs.python.org/issue3444 created benjamin.peterson functools.update_wrapper bug 07/25/08 http://bugs.python.org/issue3445 created Antoine d'Otreppe center, ljust and rjust are inconsistent with unicode parameters 07/25/08 http://bugs.python.org/issue3446 created ignas list(obj) can swallow KeyboardInterrupt 07/18/08 http://bugs.python.org/issue1242657 reopened georg.brandl long(float('nan'))!=0L 07/21/08 http://bugs.python.org/issue1481296 reopened georg.brandl patch Issues Now Closed (61) ______________________ python 2.4.4 fails on solaris (sun4u sparc SUNW,Sun-Fire-880) 264 days http://bugs.python.org/issue1363 georg.brandl os.system() fails for commands with multiple quoted file names 237 days http://bugs.python.org/issue1524 likes socket functions that should return unsigned int return signed i 203 days http://bugs.python.org/issue1711 georg.brandl PythonLauncher not working correctly on OS X 10.5/Leopad 180 days http://bugs.python.org/issue1905 georg.brandl UnboundLocalError when trying to raise exceptions inside execfil 126 days http://bugs.python.org/issue2378 amaury.forgeotdarc patch [py3k] Integer floor division (//): small int check omitted 128 days http://bugs.python.org/issue2417 facundobatista patch Py3.0a4 wsgiref simple_server failed to start 109 days http://bugs.python.org/issue2566 pitrou setuptools gets site-packages wrong on Mac 96 days http://bugs.python.org/issue2641 georg.brandl 2to3 discards comments before import statements 64 days http://bugs.python.org/issue2894 georg.brandl patch idlelib/EditorWindow.py uses xrange() 61 days http://bugs.python.org/issue2913 georg.brandl patch urlgrabber.grabber calls setdefaulttimeout 66 days http://bugs.python.org/issue2916 georg.brandl Wrong unicode size detection in pybench 40 days http://bugs.python.org/issue3092 pitrou patch overzealous garbage collector (dict) 37 days http://bugs.python.org/issue3104 georg.brandl Document exception chaining 35 days http://bugs.python.org/issue3113 georg.brandl ConfigParsers are classic classes 27 days http://bugs.python.org/issue3181 georg.brandl patch re.compile fails with some bytes patterns 24 days http://bugs.python.org/issue3231 pitrou patch segfault on gettext(None) 13 days http://bugs.python.org/issue3302 georg.brandl patch invalid ref count on locale.strcoll() error 14 days http://bugs.python.org/issue3303 georg.brandl patch invalid check of _bsddb creation failure 12 days http://bugs.python.org/issue3307 jcea patch pystone.main(10) causes ZeroDivisionError 11 days http://bugs.python.org/issue3319 georg.brandl patch bugs in scanstring_str() and scanstring_unicode() of _json modul 11 days http://bugs.python.org/issue3322 georg.brandl patch Clarify __slots__ behaviour when inheriting 10 days http://bugs.python.org/issue3323 georg.brandl webbrowser module doesn't correctly handle '|' character. 10 days http://bugs.python.org/issue3330 georg.brandl Possible inconsistency in behavior of list comprehensions vs. ge 9 days http://bugs.python.org/issue3331 georg.brandl 2to3 looses indentation on import fix 9 days http://bugs.python.org/issue3334 georg.brandl patch test_ossaudiodev fails 6 days http://bugs.python.org/issue3346 benjamin.peterson urllib.robotparser doesn't work in Py3k 6 days http://bugs.python.org/issue3347 jhylton patch Display bug in :show-inheritance: for class with standard docstr 4 days http://bugs.python.org/issue3355 georg.brandl add 'rbU' mode to open() 10 days http://bugs.python.org/issue3359 amaury.forgeotdarc Memory leak in import.c 4 days http://bugs.python.org/issue3368 georg.brandl patch, patch, easy memory leak in floatobject.c 6 days http://bugs.python.org/issue3369 marketdickinson patch, patch, easy Memory leak in pythonrun.c 3 days http://bugs.python.org/issue3378 georg.brandl patch, patch `./configure --enable-framework --enable-universalsdk` fails bec 1 days http://bugs.python.org/issue3381 trentm patch Documentation for re.findall and re.finditer lacks "ordering" in 3 days http://bugs.python.org/issue3384 jkugler undefined array passed to CryptGenRandomBytes 5 days http://bugs.python.org/issue3387 amaury.forgeotdarc patch, patch, easy [PATCH] replace last has_key in unittest by in operator 1 days http://bugs.python.org/issue3390 georg.brandl patch rlcompleter can't autocomplete members of callable objects 4 days http://bugs.python.org/issue3396 facundobatista dis module: undocumented new bytecodes 3 days http://bugs.python.org/issue3400 georg.brandl urllib incomplete and urllib2 does not exist 0 days http://bugs.python.org/issue3408 georg.brandl str.format() on negative floats 0 days http://bugs.python.org/issue3411 eric.smith typo in Mac/Makefile.in breaks installing IDLE 0 days http://bugs.python.org/issue3413 benjamin.peterson frameworkinstall doesn't create Python.app, which breaks python 0 days http://bugs.python.org/issue3414 georg.brandl Interpreter error when running a script under debugger control 2 days http://bugs.python.org/issue3415 amaury.forgeotdarc Wrong inherit in PickleHTMLBuilder 0 days http://bugs.python.org/issue3416 georg.brandl heavy resource usage with string functions 0 days http://bugs.python.org/issue3418 loewis multiprocessing uses Pickler.dispatch which isn't in 3.0 _pickle 0 days http://bugs.python.org/issue3431 alexandre.vassalotti Mac, 3.0 framework install, Python.app not created 0 days http://bugs.python.org/issue3434 georg.brandl Regression in "module as a script" command-line option 0 days http://bugs.python.org/issue3441 benjamin.peterson wsgiref.validate.InputWrapper.readline does not accept optional 0 days http://bugs.python.org/issue3442 pje pydoc.Helper.topics not based on docs 2096 days http://bugs.python.org/issue628258 georg.brandl test_htmlparser -- more robust SCRIPT tag handling 2003 days http://bugs.python.org/issue674449 georg.brandl patch copy_reg globals in cPickle 1804 days http://bugs.python.org/issue787077 georg.brandl SimpleHTTPServer reports wrong content-length for text files 37 days http://bugs.python.org/issue839496 georg.brandl patch, easy (ref-manual) position docstrings in source not documented 1572 days http://bugs.python.org/issue926501 georg.brandl tp_subclasses grow without bounds 1485 days http://bugs.python.org/issue980092 georg.brandl Make Generators Pickle-able 1299 days http://bugs.python.org/issue1092962 georg.brandl Add proxies arg to urllib.urlretrieve 1170 days http://bugs.python.org/issue1197207 georg.brandl patch floating point literals don't work in non-US locale in 2.5 935 days http://bugs.python.org/issue1391872 georg.brandl 64-bit Universal Binary build broken 578 days http://bugs.python.org/issue1619130 georg.brandl terminalcommand doesn't work under Darwin 513 days http://bugs.python.org/issue1666952 georg.brandl sqlite3.dll cannot be relocated 409 days http://bugs.python.org/issue1733134 georg.brandl Top Issues Most Discussed (10) ______________________________ 10 add 'rbU' mode to open() 10 days closed http://bugs.python.org/issue3359 10 function annotation for builtin and C function 4 days open http://bugs.python.org/issue3208 9 Add gamma and error functions to math module 10 days open http://bugs.python.org/issue3366 9 Clean up usage of map() in the stdlib 759 days open http://bugs.python.org/issue1513299 8 make the fix_dict fixer smarter 6 days open http://bugs.python.org/issue3417 7 bugs in scanstring_str() and scanstring_unicode() of _json modu 11 days closed http://bugs.python.org/issue3322 7 bsddb/test/test_replication.py bus error, segfault, assertion e 62 days open http://bugs.python.org/issue2960 7 Crash in PyObject_Malloc 370 days open http://bugs.python.org/issue1758146 6 binary buffered reading is quadratic 116 days open http://bugs.python.org/issue2523 5 math.frexp and obtaining the bit size of a large integer 1 days open http://bugs.python.org/issue3439 From jek-gmane at kleckner.net Sat Jul 26 03:22:24 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Fri, 25 Jul 2008 18:22:24 -0700 Subject: [Python-Dev] Status of mingw and Python 2.6 ? Message-ID: <g6du8g$cue$1@ger.gmane.org> I gave it a try with cygwin-hosted mingw just to see if that would work as an alternative to VS2008/VC9 to figure out some linkage problems. I tried: python setup.py build_ext --compiler mingw32 and got a version string issue noted below which rejects the version string printed from the loader ld as an inappropriate form. I seem to recall that mingw was expected to work with 2.6. Is it? Thanks - Jim running build_ext Traceback (most recent call last): File "finsim/setup.py", line 96, in <module> package_dir = {'finsim': 'finsim'}, File "c:\Python26\lib\distutils\core.py", line 152, in setup dist.run_commands() File "c:\Python26\lib\distutils\dist.py", line 972, in run_commands self.run_command(cmd) File "c:\Python26\lib\distutils\dist.py", line 992, in run_command cmd_obj.run() File "c:\Python26\lib\distutils\command\build_ext.py", line 309, in run force=self.force) File "c:\Python26\lib\distutils\ccompiler.py", line 1175, in new_compiler return klass (None, dry_run, force) File "c:\Python26\lib\distutils\cygwinccompiler.py", line 306, in __init__ CygwinCCompiler.__init__ (self, verbose, dry_run, force) File "c:\Python26\lib\distutils\cygwinccompiler.py", line 107, in __init__ get_versions() File "c:\Python26\lib\distutils\cygwinccompiler.py", line 430, in get_versions ld_version = StrictVersion(result.group(1)) File "c:\Python26\lib\distutils\version.py", line 40, in __init__ self.parse(vstring) File "c:\Python26\lib\distutils\version.py", line 107, in parse raise ValueError, "invalid version number '%s'" % vstring ValueError: invalid version number '2.18.50.20080523' From greg.ewing at canterbury.ac.nz Sat Jul 26 03:50:58 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 26 Jul 2008 13:50:58 +1200 Subject: [Python-Dev] Matrix product In-Reply-To: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> Message-ID: <488A8302.3050408@canterbury.ac.nz> Sebastien Loisel wrote: > What are the odds of this thing going in? I don't know. Guido has said nothing about it so far this time round, and his is the only opinion that matters in the end. I may write a PEP about this. However, since yesterday I've realised that there's a rather serious problem with part of my proposal. The problem is that being able to multiply matrices isn't much use without also being able to add them and multiply them by numbers, which obviously can't work with the built-in sequence types, since they already have other meanings for + and *. However, I still think that adding an @ operator for numpy to use is a good idea. So I'm now suggesting that the operator be added, with the intended meaning of matrix multiplication, but that no implementation of it be provided in core Python. There is a precedent for this -- the ellipsis notation was added purely for use by Numeric and its successors, and nothing in core Python attaches any meaning to it. > How do the PEPs work? Someone writes a PEP. People talk about it. Eventually, Guido either accepts it or rejects it (although in some cases it is an infinitely long time before that happens:-). -- Greg From amauryfa at gmail.com Sat Jul 26 09:19:32 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Sat, 26 Jul 2008 09:19:32 +0200 Subject: [Python-Dev] Status of mingw and Python 2.6 ? In-Reply-To: <g6du8g$cue$1@ger.gmane.org> References: <g6du8g$cue$1@ger.gmane.org> Message-ID: <e27efe130807260019q405b31cdm1ce9c337f7c313e7@mail.gmail.com> Jim Kleckner wrote: > I gave it a try with cygwin-hosted mingw just to see > if that would work as an alternative to VS2008/VC9 to > figure out some linkage problems. > > I tried: > python setup.py build_ext --compiler mingw32 > > and got a version string issue noted below which > rejects the version string printed from the loader > ld as an inappropriate form. > > I seem to recall that mingw was expected to work with 2.6. > Is it? > [...] > > File "c:\Python26\lib\distutils\version.py", line 107, in parse > raise ValueError, "invalid version number '%s'" % vstring > ValueError: invalid version number '2.18.50.20080523' There are actually two problems with MinGW. Your issue is the same as See http://bugs.python.org/issue2234 But MinGW is also not completely compatible with the msvcr90.dll runtime: http://bugs.python.org/issue3308 For example, if the extension module contains standard date functions (using time_t), it won't load. -- Amaury Forgeot d'Arc From ncoghlan at gmail.com Sat Jul 26 10:23:17 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 26 Jul 2008 18:23:17 +1000 Subject: [Python-Dev] Infix operators In-Reply-To: <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <e6511dbf0807231611jef46516h6dfb0f9408383268@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> Message-ID: <488ADEF5.5080002@gmail.com> Sebastien Loisel wrote: > However, just for posterity (and I'm not going to pursue the argument > further than this), I'll say this. The problem of determining the > meaning (or overridability or whatever) of x=4$6 is the same as the > problem of determining the meaning of x=fooz(4,6). Since it's not a good > argument against user-defined functions, I don't see it as a good > argument against user-defined operators. The namespace of usefully mnemonic function names is infinitely larger than that of usefully mnemonic punctuation marks. User-defined functions are good, but once you have those there is no reason to have user-defined operators *as well*. Cheers, Nick. From daniel.stutzbach at gmail.com Sat Jul 26 18:23:11 2008 From: daniel.stutzbach at gmail.com (daniel.stutzbach at gmail.com) Date: Sat, 26 Jul 2008 11:23:11 -0500 Subject: [Python-Dev] Matrix product In-Reply-To: <488A8302.3050408@canterbury.ac.nz> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> Message-ID: <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com> The desire for a new operator for matrix mutltiplication is because binary prefix operators are horrible for expressin this kind of thing, right? Stuff like this is hard to write, read, and debug (especially when checking it against an infix formula): mmul(mmul(mmul(a, b), c), d) How about just making a matrix multiply function that can take many arguments? I think this is pretty readable: mmul(a, b, c, d) Additionally, mmul could then optimize the order of the multiplications (e.g., depending the dimensions of the matrices, it may be much more efficient to perform a*((b*c)*d) rather than ((a*b)*c)*d). (please forgive typos--writing this on a smartphone) -- Daniel Stutzbach From steve at pearwood.info Sat Jul 26 19:11:16 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 27 Jul 2008 03:11:16 +1000 Subject: [Python-Dev] Matrix product In-Reply-To: <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com> Message-ID: <200807270311.17414.steve@pearwood.info> On Sun, 27 Jul 2008 02:23:11 am daniel.stutzbach at gmail.com wrote: > How about just making a matrix multiply function that can take many > arguments? I think this is pretty readable: > > mmul(a, b, c, d) > > Additionally, mmul could then optimize the order of the > multiplications (e.g., depending the dimensions of the matrices, it > may be much more efficient to perform a*((b*c)*d) rather than > ((a*b)*c)*d). But be careful there: matrix multiplication is associative, so that a*b*c = (a*b)*c = a*(b*c), but that doesn't necessarily apply once the elements of the matrices are floats. For instance, a*(b*c) might underflow some elements to zero, while multiplying (a*b)*c does not. As a general rule, compilers should not mess with the order of floating point calculations. Also, some classes that people might want to multiply may not be associative even in principle. E.g. the vector cross product: (a*b)*c != a*(b*c) in general. I think a product() function that multiplies the arguments from left to right would be useful. But it won't solve the problems that people want custom operators to solve. I'm not even sure if it will solve the problem of matrix multiplication. -- Steven. From regebro at gmail.com Sat Jul 26 21:00:09 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 26 Jul 2008 21:00:09 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <488451AB.50406@jcea.es> References: <4838EA2D.7060402@jcea.es> <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> <488150A5.8050804@jcea.es> <B98D6CBA-6D19-482F-B11D-1FF60B77F17A@barrys-emacs.org> <488451AB.50406@jcea.es> Message-ID: <319e029f0807261200p373bc176ibd70d6e6ef5dee79@mail.gmail.com> I've added a setup.py to the python-incompatibilities projects code, so adding c-extention modules should now be much easier. I don't do much c-development myself, so I'm not the right person to do this, but anybody that feels like adding C-extensions to this project is more than welcome to do so. Just mail me for write access. http://code.google.com/p/python-incompatibility/ From skip at pobox.com Sun Jul 27 05:21:07 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 26 Jul 2008 22:21:07 -0500 Subject: [Python-Dev] Possible small csv.DictReader change Message-ID: <18571.59811.311511.28772@montanaro-dyndns-org.local> A new issue in the tracker: http://bugs.python.org/issue3436 points out that the csv module's DictReader class doesn't know the fieldnames defined in the file until you've fetched the first row of data. It's fairly easy to change the behavior so that __init__ automatically grabs the fieldnames if they aren't passed into it (I attached a patch) but Raymond commented that __init__() shouldn't read any line(s) from the file, instead preferring a separate method to set the fieldnames attribute. There's the issue though of whether __init__() should implicitly read the fieldnames from the file or if that task should be delegated to a separate method which must be called either by the user or from the next() method. If you have an opinion either way, I'd appreciate it if you added a comment to that issue. Regardless which way this issue plays out, is it something that can be put into 2.6 & 3.0 or does it need to wait at this point? Thanks, Skip From greg.ewing at canterbury.ac.nz Sun Jul 27 09:21:39 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 27 Jul 2008 19:21:39 +1200 Subject: [Python-Dev] Matrix product In-Reply-To: <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <eae285400807260923g51441d69j29e033184f408156@mail.gmail.com> Message-ID: <488C2203.3080306@canterbury.ac.nz> daniel.stutzbach at gmail.com wrote: > How about just making a matrix multiply function that can take many > arguments? I think this is pretty readable: > > mmul(a, b, c, d) The multiplications aren't necessarily all together, e.g. a*b + c*d + e*f would become mmul(a, b) + mmul(c, d) + mmul(e, f) -- Greg From charleshixsn at earthlink.net Sun Jul 27 19:43:38 2008 From: charleshixsn at earthlink.net (Charles Hixson) Date: Sun, 27 Jul 2008 10:43:38 -0700 Subject: [Python-Dev] Infix operators In-Reply-To: <488ADEF5.5080002@gmail.com> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <488ADEF5.5080002@gmail.com> Message-ID: <200807271043.38692.charleshixsn@earthlink.net> On Saturday 26 July 2008 01:23:17 am Nick Coghlan wrote: > Sebastien Loisel wrote: > > However, just for posterity (and I'm not going to pursue the argument > > further than this), I'll say this. The problem of determining the > > meaning (or overridability or whatever) of x=4$6 is the same as the > > problem of determining the meaning of x=fooz(4,6). Since it's not a good > > argument against user-defined functions, I don't see it as a good > > argument against user-defined operators. > > The namespace of usefully mnemonic function names is infinitely larger > than that of usefully mnemonic punctuation marks. User-defined functions > are good, but once you have those there is no reason to have > user-defined operators *as well*. > > Cheers, > Nick. > Most mathematicians would disagree with you. I'll grant that it tends to make the code extremely obscure to those who don't work in the field, but it tends to make it much clearer to those who do so work. OTOH, it's also true that there aren't sufficient punctuation symbols. E.g. math people drafted capital sigma as sum of a series, etc. Therefore it seems to me that the appropriate thing is to create a convention that bar-somethingprintable-bar should be interpreted as a user defined operation, e.g. |+| or |@#|. The first one is reasonable as matrix multiplication, the second as some user defined operation that hasn't yet been specified (in this context). And since such things don't yet have a "secret name" I would suggest that they be defined either via an ordinary def, or via a def with their name in quotes, i.e., either: def |+| or def "|+|" If an ordinary functional reference is desireable (probably) there could be a decorators that assigned the name, and possibly the precedence, e.g.: @name=madd @precedence="+" def |+| (a, b): etc. The main drawback that I see is that code that relies heavily on this approach would become much less readable to anyone not in the particular field. Think of the first time you encountered the curl or del operators, or even the kronecker delta. OTOH, it seems far too late in the development process to be inserting such a change in Python 2.6 or 3.0. If this is important to you, you should probably propose it for 2.7/3.1. From regebro at gmail.com Sun Jul 27 20:18:17 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 27 Jul 2008 20:18:17 +0200 Subject: [Python-Dev] Infix operators In-Reply-To: <200807271043.38692.charleshixsn@earthlink.net> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <488ADEF5.5080002@gmail.com> <200807271043.38692.charleshixsn@earthlink.net> Message-ID: <319e029f0807271118r429faaaep309b5b77be1c63d9@mail.gmail.com> On Sun, Jul 27, 2008 at 19:43, Charles Hixson <charleshixsn at earthlink.net> wrote: > Therefore it seems to me that the appropriate thing is to create a convention > that bar-somethingprintable-bar And the "something-printable" shows the main flaw of this approach. Mathematics indeed uses a lot of symbols to make things clear and unambigous. Many of these are hard to print in a line, even with unicode, and entering and editing this from a keyboard would be very difficult. So to make this scheme useable you would have to limit yourself to ascii-codes, and then most of the point goes away since you can't use the proper symbols anyway, and ambiguity is reintroduced. It seems to me that mathematicians who need these things would be better served by dedicated maths-software. Just my 2 cents. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From steve at holdenweb.com Sun Jul 27 23:42:20 2008 From: steve at holdenweb.com (Steve Holden) Date: Sun, 27 Jul 2008 17:42:20 -0400 Subject: [Python-Dev] Infix operators In-Reply-To: <200807271043.38692.charleshixsn@earthlink.net> References: <b7975ab80807231414wacc8d72kd25d2608e5c2ad4d@mail.gmail.com> <b7975ab80807231648s61b1adf2k29e34545120215c3@mail.gmail.com> <488ADEF5.5080002@gmail.com> <200807271043.38692.charleshixsn@earthlink.net> Message-ID: <g6iq4p$knn$1@ger.gmane.org> Charles Hixson wrote: [...] > OTOH, it seems far too late in the development process to be inserting such a > change in Python 2.6 or 3.0. If this is important to you, you should > probably propose it for 2.7/3.1. It's been too late for over three months now, and the suggestions I've seen so far are all way impractical anyway. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ondrej at certik.cz Mon Jul 28 10:28:34 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Mon, 28 Jul 2008 10:28:34 +0200 Subject: [Python-Dev] str(container) should call str(item), not repr(item) Message-ID: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> Hi, as discussed before here: http://mail.python.org/pipermail/python-3000/2008-May/013876.html if you do: >>> from decimal import Decimal >>> a = Decimal(40) >>> print a, a**2, a**3 40 1600 64000 >>> print [a, a**2, a**3] [Decimal("40"), Decimal("1600"), Decimal("64000")] >>> print {a: 3} {Decimal("40"): 3} >>> print {a: a**2} {Decimal("40"): Decimal("1600")} i.e. the str on list (and tuple and dict) calls repr() on the elements, instead of str. This really seems to me like a bug. Because if I wanted the repr() representation, I'd call repr() on the list/tuple/dict. If I want a nice readable representation, I call str(). That's the philosophy, no? I understant there can be technical reasons why this cannot be made into python 3.0, so why not 3.1? or 2.8? If this cannot be made into python for some reason, how would you suggest us we solve the following problem in sympy (symbolic manipulation library): >>> from sympy import Symbol, srepr >>> x = Symbol("x") >>> l = [x, x**2/2, x**3/3, x**4/4] >>> str(l) '[x, 1/2*x**2, 1/3*x**3, 1/4*x**4]' >>> repr(l) '[x, 1/2*x**2, 1/3*x**3, 1/4*x**4]' >>> srepr(l) "[Symbol('x'), Mul(Half(1, 2), Pow(Symbol('x'), Integer(2))), Mul(Rational(1, 3), Pow(Symbol('x'), Integer(3))), Mul(Rational(1, 4), Pow(Symbol('x'), Integer(4)))]" As you can see, we currently have to hack our __repr__ to return the same thing as __str__ so that things look nice in lists and dictionaries. We provide the srepr() function that does what the __repr__ should do instead. And as you can see, the result of srepr is really not very readable, but it is easy to convert it back to a sympy expression using eval. That's the purpose of repr(), no? However, you cannot easily convert the str() back to an expression, because the eval() doesn't know what "x" is. One solution that we'll probably use is that we'll change our repr() to the srepr output above, so str([x, x**2,...]) will not be pretty and we'll use sys.displayhook to hook our own function into the interactive session that overcomes this python behavior. This solves the problem for interactive sessions, but it doesn't solve it if people just call "print [x, x**2, ...]" inside their programs, for example when debugging. Another option is to write a C extension module that monkey patches the list class and changes the C pointer to the (currently null) __str__ method to our own method. But I find it very hackish and also it requires a C module. I was discussing this with other people at EuroSciPy2008 and everyone's first reaction that str(list) calls repr() on the elements was that it's a bug. So was my reaction when I discovered that. So, I think people find this highly unintuitive. So I just wanted to discuss this more with core Python developers what you think about it and if you also find this not very intuitive and if so, if there is any chance to get this fixed. Thanks, Ondrej From skip at pobox.com Mon Jul 28 10:59:42 2008 From: skip at pobox.com (skip at pobox.com) Date: Mon, 28 Jul 2008 03:59:42 -0500 Subject: [Python-Dev] str(container) should call str(item), not repr(item) In-Reply-To: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> Message-ID: <18573.35454.142435.398911@montanaro-dyndns-org.local> Ondrej> i.e. the str on list (and tuple and dict) calls repr() on the Ondrej> elements, instead of str. This really seems to me like a bug. Ondrej> Because if I wanted the repr() representation, I'd call repr() Ondrej> on the list/tuple/dict. If I want a nice readable Ondrej> representation, I call str(). That's the philosophy, no? I think this is the case which calls for the distinction: >>> str(["1", "2", "3"]) "['1', '2', '3']" >>> str([1, 2, 3]) '[1, 2, 3]' If the first case did as you suggested you couldn't distinguish it from the second. Skip From phd at phd.pp.ru Mon Jul 28 11:36:56 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 28 Jul 2008 13:36:56 +0400 Subject: [Python-Dev] str(container) should call str(item), not repr(item) In-Reply-To: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> Message-ID: <20080728093656.GC25461@phd.pp.ru> On Mon, Jul 28, 2008 at 10:28:34AM +0200, Ondrej Certik wrote: > http://mail.python.org/pipermail/python-3000/2008-May/013876.html The PEP is pep-3140: http://python.org/peps/pep-3140.html and it has been rejected. To revive it we need better, more compelling arguments. We also need a plan for implementation that would be good enough to be accepted. Current proposals for implementation are listed in the PEP with their disadvantages. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From haase at msg.ucsf.edu Mon Jul 28 12:04:21 2008 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Mon, 28 Jul 2008 12:04:21 +0200 Subject: [Python-Dev] str(container) should call str(item), not repr(item) In-Reply-To: <18573.35454.142435.398911@montanaro-dyndns-org.local> References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> <18573.35454.142435.398911@montanaro-dyndns-org.local> Message-ID: <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com> On Mon, Jul 28, 2008 at 10:59 AM, <skip at pobox.com> wrote: > > Ondrej> i.e. the str on list (and tuple and dict) calls repr() on the > Ondrej> elements, instead of str. This really seems to me like a bug. > Ondrej> Because if I wanted the repr() representation, I'd call repr() > Ondrej> on the list/tuple/dict. If I want a nice readable > Ondrej> representation, I call str(). That's the philosophy, no? > > I think this is the case which calls for the distinction: > > >>> str(["1", "2", "3"]) > "['1', '2', '3']" > >>> str([1, 2, 3]) > '[1, 2, 3]' > > If the first case did as you suggested you couldn't distinguish it from the > second. > Look at this -- it seems to me that it should work fine.... ---- Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26) >>> str("qwer") 'qwer' >>> repr("qwer") "'qwer'" - Sebastian Haase From matt.giuca at gmail.com Mon Jul 28 12:58:05 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Mon, 28 Jul 2008 20:58:05 +1000 Subject: [Python-Dev] str(container) should call str(item), not repr(item) In-Reply-To: <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com> References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> <18573.35454.142435.398911@montanaro-dyndns-org.local> <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com> Message-ID: <e6268af30807280358j21cf127dw29195e1568e1726a@mail.gmail.com> Another disadvantage of calling str recursively rather than repr is that it places an onus on anyone writing a class to write both a repr and a str method (or be inconsistent with the newly-accepted standard for container types). I personally write a repr method for most classes, which aids debugging. This means all my classes behave like containers currently do - their str will call repr on the items. This proposal will make all of my classes behave inconsistently with the standard container types. - Matt Giuca -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080728/188881f0/attachment.htm> From steve at holdenweb.com Mon Jul 28 13:24:18 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 28 Jul 2008 07:24:18 -0400 Subject: [Python-Dev] str(container) should call str(item), not repr(item) In-Reply-To: <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com> References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> <18573.35454.142435.398911@montanaro-dyndns-org.local> <bc657ead0807280304w3ff25334w6afe7b0a44b5339e@mail.gmail.com> Message-ID: <g6kaa9$cf5$1@ger.gmane.org> Sebastian Haase wrote: > On Mon, Jul 28, 2008 at 10:59 AM, <skip at pobox.com> wrote: >> Ondrej> i.e. the str on list (and tuple and dict) calls repr() on the >> Ondrej> elements, instead of str. This really seems to me like a bug. >> Ondrej> Because if I wanted the repr() representation, I'd call repr() >> Ondrej> on the list/tuple/dict. If I want a nice readable >> Ondrej> representation, I call str(). That's the philosophy, no? >> >> I think this is the case which calls for the distinction: >> >> >>> str(["1", "2", "3"]) >> "['1', '2', '3']" >> >>> str([1, 2, 3]) >> '[1, 2, 3]' >> >> If the first case did as you suggested you couldn't distinguish it from the >> second. >> > Look at this -- it seems to me that it should work fine.... > ---- Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26) >>>> str("qwer") > 'qwer' >>>> repr("qwer") > "'qwer'" > No it doesn't. What's happening in these examples is that the interpreter is calling repr() on the expression result - otherwise you wouldn't see the quotes: >>> str("qwer") 'qwer' >>> print str("qwer") qwer >>> regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From aahz at pythoncraft.com Mon Jul 28 16:58:53 2008 From: aahz at pythoncraft.com (Aahz) Date: Mon, 28 Jul 2008 07:58:53 -0700 Subject: [Python-Dev] str(container) should call str(item), not repr(item) In-Reply-To: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> References: <85b5c3130807280128v2deb411i6cd477b2346cb2fb@mail.gmail.com> Message-ID: <20080728145853.GA4266@panix.com> On Mon, Jul 28, 2008, Ondrej Certik wrote: > > as discussed before here: > > http://mail.python.org/pipermail/python-3000/2008-May/013876.html Precisely because this has been discussed extensively with little to show, I recommend that any further discussion be held on either python-ideas or comp.lang.python -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Adopt A Process -- stop killing all your children! From jcea at jcea.es Tue Jul 29 15:51:06 2008 From: jcea at jcea.es (Jesus Cea) Date: Tue, 29 Jul 2008 15:51:06 +0200 Subject: [Python-Dev] bsddb: Test failures on windows (HELP!) In-Reply-To: <e27efe130807240723v155b05a5x1bd2a9b1a64ab057@mail.gmail.com> References: <48874F2D.5070201@jcea.es> <095c01c8ed50$60bdfed0$2239fc70$@com.au> <488877EC.600@palladion.com> <e27efe130807240723v155b05a5x1bd2a9b1a64ab057@mail.gmail.com> Message-ID: <488F204A.40401@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Amaury Forgeot d'Arc wrote: | I see that some tests use os.unlink. They should use | test_support.unlink() instead. Old stuff. Fix just committed. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSI8gQ5lgi5GaxT1NAQJyHwQAiU6izAcI5eI3tizxWkvsw4MBRmQzlGNi Ib+U/ZxuO9bqYOXZLqIQWkH2Ry1/Li7KeepVRehdkVlSnFEkWVXPhNofxvlXoQpl Rt0T8aGJ1GhWkdojkWE7Ab2L8mdTCunHuVyiAQBagTET1E9iRnjrf5//XsRvd09Z SUfPgEnvqv4= =96Vt -----END PGP SIGNATURE----- From razvan.a.lupusoru at intel.com Tue Jul 29 20:56:43 2008 From: razvan.a.lupusoru at intel.com (Lupusoru, Razvan A) Date: Tue, 29 Jul 2008 11:56:43 -0700 Subject: [Python-Dev] Relocating Python Message-ID: <BAD036B67C739F45BE062D0A03D19286035ACFD7@orsmsx414.amr.corp.intel.com> Hello, I am trying to get Python 2.5.2 working for an IA32 system. The compilation is done on an Ubuntu 8.04.1 dev system. I am using a custom gcc and ld specific to the IA32 system. This is my makefile: ############################## BUILD_DEST = /i686-custom-kernel CC = $(BUILD_DEST)/bin/i686-linux-gcc CPP = $(CC) -E CXX = $(BUILD_DEST)/bin/i686-linux-g++ LD = $(BUILD_DEST)/bin/i686-linux-ld PYTHONINSTALLPATH = $(BUILD_DEST)/usr Export all: tar xzfv Python-2.5.2.tgz ./Python-2.5.2/configure -prefix=${PYTHONINSTALLPATH} -host=i686-linux -enable-shared cd Python-2.5.2 make make install ############################## Everything compiles correctly. I then copy the contents of the $BUILD_DEST and put them on the hard drive for my IA32 system. I basically use the contents of $BUILD_DEST as the root directory on my IA32 system. Python seems to run correctly when I run it, but when I do things like "import pysqlite", it cannot find it. Is there anything special I have to do to relocate my python (since on my IA32 system it runs from /usr/bin/python but it originally gets created in ${BUILD_DEST}/usr/bin/python)? Thank you, Razvan A. Lupusoru -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080729/ad910a7e/attachment.htm> From casey at pandora.com Tue Jul 29 21:54:31 2008 From: casey at pandora.com (Casey Duncan) Date: Tue, 29 Jul 2008 13:54:31 -0600 Subject: [Python-Dev] Relocating Python In-Reply-To: <BAD036B67C739F45BE062D0A03D19286035ACFD7@orsmsx414.amr.corp.intel.com> References: <BAD036B67C739F45BE062D0A03D19286035ACFD7@orsmsx414.amr.corp.intel.com> Message-ID: <A8F28B58-2168-4662-B261-495CB1F29284@pandora.com> On Jul 29, 2008, at 12:56 PM, Lupusoru, Razvan A wrote: > Hello, > > I am trying to get Python 2.5.2 working for an IA32 system. The > compilation is done on an Ubuntu 8.04.1 dev system. I am using a > custom gcc and ld specific to the IA32 system. > > This is my makefile: > ############################## > BUILD_DEST = /i686-custom-kernel > CC = $(BUILD_DEST)/bin/i686-linux-gcc > CPP = $(CC) ?E > CXX = $(BUILD_DEST)/bin/i686-linux-g++ > LD = $(BUILD_DEST)/bin/i686-linux-ld > PYTHONINSTALLPATH = $(BUILD_DEST)/usr > Export > > all: > tar xzfv Python-2.5.2.tgz > ./Python-2.5.2/configure ?prefix=$ > {PYTHONINSTALLPATH} ?host=i686-linux ?enable-shared > cd Python-2.5.2 > make > make install > ############################## > > Everything compiles correctly. I then copy the contents of the > $BUILD_DEST and put them on the hard drive for my IA32 system. I > basically use the contents of $BUILD_DEST as the root directory on > my IA32 system. Python seems to run correctly when I run it, but > when I do things like ?import pysqlite?, it cannot find it. Is there > anything special I have to do to relocate my python (since on my > IA32 system it runs from /usr/bin/python but it originally gets > created in ${BUILD_DEST}/usr/bin/python)? You'll want to pass configure the prefix where python will ultimately be installed, otherwise the paths used during make won't make sense on the destination system. That said, pysqlite is not part of the stdlib, so your actual problem may have more to do with how you've installed it then anything. When you run python once relocated, what does os.path contain? What we do for packaging, is run 'configure' normally and then 'make', then override some make variables for 'make install' to temporarily install it in a different place for package staging. It looks something like this: ./configure && \ make && \ make BINDIR=$(shell pwd)/path/to/tmp/bin \ CONFINCLUDEDIR=$(shell pwd)/path/to/tmp/include \ INCLUDEDIR=$(shell pwd)/path/to/tmp/include \ LIBDIR=$(shell pwd)/path/to/tmp/lib \ MANDIR=$(shell pwd)/path/to/tmp/man \ SCRIPTDIR=$(shell pwd)/path/to/tmp/lib \ install Then when the package is deployed, the files are actually installed under the standard 'configure' prefix (/usr/local I think). hth, -Casey From guido at python.org Wed Jul 30 00:37:11 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Jul 2008 15:37:11 -0700 Subject: [Python-Dev] Deprecate parser's "ast" functions? In-Reply-To: <1afaf6160807210747v7756589bnac5d07c6d3bbb6f8@mail.gmail.com> References: <g2ej5h$tnl$1@ger.gmane.org> <1afaf6160807210747v7756589bnac5d07c6d3bbb6f8@mail.gmail.com> Message-ID: <ca471dc20807291537xc3e03d1r6287b27142335beb@mail.gmail.com> +1 as well. On Mon, Jul 21, 2008 at 7:47 AM, Benjamin Peterson <musiccomposition at gmail.com> wrote: > On Sat, Jun 7, 2008 at 3:13 PM, Georg Brandl <g.brandl at gmx.net> wrote: >> The parser module exports each function and type twice, once with "AST" in >> the name, once with "ST". Since "AST" now has a different meaning for >> Python code compilation, I propose to deprecate the "ast" variants in 2.6 >> and remove them in Python 3000. > > +1 This personally has confused me! > >> >> (Also, all keyword arguments are called "ast", that cannot change in 2.6 >> but should in 3.0.) >> >> I'm at the moment changing the documentation of parser to only refer to >> the "st" variants anymore. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 30 02:11:56 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Jul 2008 17:11:56 -0700 Subject: [Python-Dev] fileobj.read(float): warning or error? In-Reply-To: <20080722053744.GA11290@cskk.homeip.net> References: <200807212117.14485.victor.stinner@haypocalc.com> <20080722053744.GA11290@cskk.homeip.net> Message-ID: <ca471dc20807291711m48d5837ap33c9b58f31eb82b5@mail.gmail.com> On Mon, Jul 21, 2008 at 10:37 PM, Cameron Simpson <cs at zip.com.au> wrote: > Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an > exception if asked for < 1 bytes? Or is there a legitimate use for read(0) > with which I was not previously aware? Indeed. read(0) is quite often generated as an edge case when one is computing buffer sizes, and returning an empty string is most definitely the right thing to do here (otherwise some application code becomes more complex by having to avoid calling read(0) at all). Of course, read(), read(None), read(-1) and read(<any negative int>) should all read all data until EOF. On the main topic here, read(<float>) and read(<anything that supports __int__ but not __index__>) should definitely raise an exception in 3.0. In 2.6 it should show a warning as it does in 2.5. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 30 02:26:59 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Jul 2008 17:26:59 -0700 Subject: [Python-Dev] Matrix product In-Reply-To: <488A8302.3050408@canterbury.ac.nz> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> Message-ID: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> On Fri, Jul 25, 2008 at 6:50 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Sebastien Loisel wrote: > >> What are the odds of this thing going in? > > I don't know. Guido has said nothing about it so far this > time round, and his is the only opinion that matters in the > end. I'd rather stay silent until a PEP exists, but I should point out that last time '@' was considered as a new operator, that character had no uses in the language at all. Now it is the decorator marker. Therefore it may not be so attractive any more. I understand that you can't use A*B as matrix multiplication because it should mean elementwise multiplication instead, just like A+B is elementwise addition (for matrixes, as opposed to Python sequences). But would it be totally outlandish to propose A**B for matrix multiplication? I can't think of what "matrix exponentiation" would mean... --Guido > I may write a PEP about this. However, since yesterday I've > realised that there's a rather serious problem with part > of my proposal. > > The problem is that being able to multiply matrices isn't > much use without also being able to add them and multiply > them by numbers, which obviously can't work with the > built-in sequence types, since they already have other > meanings for + and *. > > However, I still think that adding an @ operator for numpy > to use is a good idea. So I'm now suggesting that the > operator be added, with the intended meaning of matrix > multiplication, but that no implementation of it be > provided in core Python. > > There is a precedent for this -- the ellipsis notation was > added purely for use by Numeric and its successors, and > nothing in core Python attaches any meaning to it. > >> How do the PEPs work? > > Someone writes a PEP. People talk about it. Eventually, Guido > either accepts it or rejects it (although in some cases it > is an infinitely long time before that happens:-). > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 30 02:30:13 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Jul 2008 17:30:13 -0700 Subject: [Python-Dev] bsddb: Test failures on windows (HELP!) In-Reply-To: <095c01c8ed50$60bdfed0$2239fc70$@com.au> References: <48874F2D.5070201@jcea.es> <095c01c8ed50$60bdfed0$2239fc70$@com.au> Message-ID: <ca471dc20807291730tdb9caaewc7e87aa0c25bd88@mail.gmail.com> On Wed, Jul 23, 2008 at 10:44 PM, Mark Hammond <mhammond at skippinet.com.au> wrote: >> Trent, I was wondering if you could look at some test failures in MS >> Windows builds. I can't debug Windows issues myself :-(. This is a MS >> free environment... > > In these errors I see lots of bsdbd errors, many of the form: > > | DBFileExistsError: (17, 'File exists -- __fop_file_setup: Retry limit > (100) exceeded') I've had this a few times in the past (Retry limit (100) exceeded)and it has always been caused by cruft left behind by a previous run of the test that didn't end well. Somehow these tests do not do a good job of cleaning up old cruft before they run. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From sidnei.da.silva at gmail.com Wed Jul 30 02:33:51 2008 From: sidnei.da.silva at gmail.com (Sidnei da Silva) Date: Tue, 29 Jul 2008 21:33:51 -0300 Subject: [Python-Dev] Matrix product In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> Message-ID: <a7a2b76b0807291733j29f30588n6e103457203e49d8@mail.gmail.com> On Tue, Jul 29, 2008 at 9:26 PM, Guido van Rossum <guido at python.org> wrote: > But would it be totally outlandish to propose A**B for matrix > multiplication? I can't think of what "matrix exponentiation" would > mean... Before even reading this paragraph, A**B came to my mind, so I suspect it would be the most intuitive option. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 From loisel at temple.edu Wed Jul 30 02:54:15 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Tue, 29 Jul 2008 20:54:15 -0400 Subject: [Python-Dev] Matrix product In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> Message-ID: <b7975ab80807291754x4053036fv5402951f01c7e082@mail.gmail.com> Dear Guido, Thank you for your email. On Tue, Jul 29, 2008 at 8:26 PM, Guido van Rossum <guido at python.org> wrote: > But would it be totally outlandish to propose A**B for matrix > multiplication? I can't think of what "matrix exponentiation" would > mean... Right now, ** is the pointwise power: >>> from numpy import * >>> A=array([[1,2],[3,4]]) >>> print(A**A) [[ 1 4] [ 27 256]] Since all the scalar operators have meaning as pointwise operator, like you say it's hard to bump one off to give it to the matrix product instead. I don't know if it's a good idea with **, it will destroy the orthogonality of the system. They used to say, ignore LISP at your own peril. In that spirit, let me describe MATLAB's approach to this. It features a complete suite of matrix operators (+-*/\^), and their pointwise variants (.+ .- ./ .* .^), although + and .+ are synonyms, as are - and.-. Right now, numpy's *,**,/ correspond to MATLAB .*,.^,./. MATLAB implements scalar^matrix, matrix^scalar, but not matrix^matrix (although since log and exp are defined, I guess you could clobber together a meaning for matrix^matrix). Since ^ is the matrix-product version of "power", 2^A may not be what you expect: >> 2^A 10.4827 14.1519 21.2278 31.7106 Sincerely, -- S?bastien Loisel From fredrik.johansson at gmail.com Wed Jul 30 02:59:36 2008 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Wed, 30 Jul 2008 02:59:36 +0200 Subject: [Python-Dev] Matrix product In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> Message-ID: <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> On Wed, Jul 30, 2008 at 2:26 AM, Guido van Rossum <guido at python.org> wrote: > On Fri, Jul 25, 2008 at 6:50 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >> Sebastien Loisel wrote: >> >>> What are the odds of this thing going in? >> >> I don't know. Guido has said nothing about it so far this >> time round, and his is the only opinion that matters in the >> end. > > I'd rather stay silent until a PEP exists, but I should point out that > last time '@' was considered as a new operator, that character had no > uses in the language at all. Now it is the decorator marker. Therefore > it may not be so attractive any more. I don't like @. > I understand that you can't use A*B as matrix multiplication because > it should mean elementwise multiplication instead, just like A+B is > elementwise addition (for matrixes, as opposed to Python sequences). > > But would it be totally outlandish to propose A**B for matrix > multiplication? I can't think of what "matrix exponentiation" would > mean... http://mathworld.wolfram.com/MatrixExponential.html :-) In fact Mathematica uses ** to denote general noncommutative multiplication (though . for matrix multiplication in particular). However, this wouldn't solve the problem, because an important reason to introduce a matrix multiplication operator is to distinguish between matrix and elementwise operations for arrays. The ** operator already denotes the obvious elementwise operation in numpy. Further, while A**B is not so common, A**n is quite common (for integral n, in the sense of repeated matrix multiplication). So a matrix multiplication operator really should come with a power operator cousin. Matlab uses * for matrix and .* for elementwise multiplication. Introducing .* for elementwise multiplication in Python would not be compatible with existing numpy code, and introducing .* with the reversed meaning of Matlab would be *very* confusing :-) Maple uses &* for matrix multiplication. However, Maple's syntax is not a good style reference for anything. Besides those alternatives and the regular *, I don't know any other ASCII operators used by existing mathematical software for matrix multiplication. Well, Fortress probably has some unicode symbol for it (I suppose that would be one desperate possibility). Fredrik From greg.ewing at canterbury.ac.nz Wed Jul 30 03:30:20 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 30 Jul 2008 13:30:20 +1200 Subject: [Python-Dev] Matrix product In-Reply-To: <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> Message-ID: <488FC42C.80807@canterbury.ac.nz> Guido van Rossum wrote: > last time '@' was considered as a new operator, that character had no > uses in the language at all. Now it is the decorator marker. The only alternatives left would seem to be ?, ! or $, none of which look particularly multiplicationish. > But would it be totally outlandish to propose A**B for matrix > multiplication? I can't think of what "matrix exponentiation" would > mean... But ** has the same problem -- it already represents an elementwise operation on numpy arrays. -- Greg From greg.ewing at canterbury.ac.nz Wed Jul 30 03:38:34 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 30 Jul 2008 13:38:34 +1200 Subject: [Python-Dev] Matrix product In-Reply-To: <b7975ab80807291754x4053036fv5402951f01c7e082@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> <b7975ab80807291754x4053036fv5402951f01c7e082@mail.gmail.com> Message-ID: <488FC61A.7040303@canterbury.ac.nz> Sebastien Loisel wrote: > let > me describe MATLAB's approach to this. It features a complete suite of > matrix operators (+-*/\^), and their pointwise variants (.+ .- ./ .* > .^) That was considered before as well, but rejected on the grounds that the dot-prefixed operators were too cumbersome to use heavily. In MATLAB, the elementwise operations are probably used fairly infrequently. But numpy arrays are often used to vectorise what are otherwise scalar operations, in which case elementwise operations are used almost exclusively. -- Greg From greg.ewing at canterbury.ac.nz Wed Jul 30 03:43:16 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 30 Jul 2008 13:43:16 +1200 Subject: [Python-Dev] Matrix product In-Reply-To: <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> Message-ID: <488FC734.9030503@canterbury.ac.nz> Fredrik Johansson wrote: > Further, while A**B is not so common, A**n is quite common (for > integral n, in the sense of repeated matrix multiplication). So a > matrix multiplication operator really should come with a power > operator cousin. Which obviously should be @@ :-) > Well, Fortress probably has some unicode symbol for it > (I suppose that would be one desperate possibility). I've been carefully refraining from suggesting that. Although now that unicode is allowed in identifiers, it's not *quite* as heretical as it used to be. -- Greg From loisel at temple.edu Wed Jul 30 03:53:49 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Tue, 29 Jul 2008 21:53:49 -0400 Subject: [Python-Dev] Matrix product In-Reply-To: <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> Message-ID: <b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com> Dear Greg, Thank you for your email. > In MATLAB, the elementwise operations are probably > used fairly infrequently. But numpy arrays are often > used to vectorise what are otherwise scalar operations, > in which case elementwise operations are used almost > exclusively. Your assessment of pointwise operators in MATLAB is incorrect. The pointwise operators in MATLAB are used heavily to vectorise the scalar operations, exactly the same as what you describe in numpy. Recently, the MATLAB JIT has become good enough that looping is fast, however, the pointwise operators remain "the MATLAB way" of programming. -- S?bastien Loisel From python at rcn.com Wed Jul 30 10:29:36 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 30 Jul 2008 01:29:36 -0700 Subject: [Python-Dev] Matrix product References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com><488A8302.3050408@canterbury.ac.nz><ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com><3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> <488FC734.9030503@canterbury.ac.nz> Message-ID: <89992E9D6DEB460988282F7F1080E73A@RaymondLaptop1> >> Further, while A**B is not so common, A**n is quite common (for >> integral n, in the sense of repeated matrix multiplication). So a >> matrix multiplication operator really should come with a power >> operator cousin. > > Which obviously should be @@ :-) I think much of this thread is a repeat of conversations that were held for PEP 225: http://www.python.org/dev/peps/pep-0225/ That PEP is marked as deferred. Maybe it's time to bring it back to life. Raymond From loisel at temple.edu Wed Jul 30 11:13:03 2008 From: loisel at temple.edu (Sebastien Loisel) Date: Wed, 30 Jul 2008 05:13:03 -0400 Subject: [Python-Dev] Matrix product In-Reply-To: <b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> <b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com> Message-ID: <b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com> Dear Raymond, Thank you for your email. > I think much of this thread is a repeat of conversations > that were held for PEP 225: > http://www.python.org/dev/peps/pep-0225/ > > That PEP is marked as deferred. Maybe it's time to > bring it back to life. This is a much better PEP than the one I had found, and would solve all of the numpy problems. The PEP is very well thought-out. Sincerely, -- S?bastien Loisel From matt.giuca at gmail.com Wed Jul 30 16:11:40 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Thu, 31 Jul 2008 00:11:40 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <-6533647106000374959@unknownmsgid> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> <-6533647106000374959@unknownmsgid> Message-ID: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> Hi folks, This issue got some attention a few weeks back but it seems to have fallen quiet, and I haven't had a good chance to sit down and reply again till now. As I've said before this is a serious issue which will affect a great deal of code. However it's obviously not as clear-cut as I originally believed, since there are lots of conflicting opinions. Let us see if we can come to a consensus. (For those who haven't seen the discussion, the thread starts here: http://mail.python.org/pipermail/python-dev/2008-July/081013.html continues here for some reason: http://mail.python.org/pipermail/python-dev/2008-July/081066.html and I've got a bug report with a fully tested and documented patch here: http://bugs.python.org/issue3300) Firstly, it looks like most of the people agree we should add an optional "encoding" argument which lets the caller customize which encoding to use. What we tend to disagree about is what the default encoding should be. Here I present the various options as I see it (and I'm trying to be impartial), and the people who've indicated support for that option (apologies if I've misrepresented anybody's opinion, feel free to correct): 1. Leave it as it is. quote is Latin-1 if range(0,256), fallback to UTF-8. unquote is Latin-1. In favour: Anybody who doesn't reply to this thread Pros: Already implemented; some existing code depends upon ord values of string being the same as they were for byte strings; possible to hack around it. Cons: unquote is not inverse of quote; quote behaviour internally-inconsistent; garbage when unquoting UTF-8-encoded URIs. 2. Default to UTF-8. In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven Pros: Fully working and tested solution is implemented; recommended by RFC 3986 for all future schemes; recommended by W3C for use with HTML; UTF-8 used by all major browsers; supports all characters; most existing code compatible by default; unquote is inverse of quote. Cons: By default, URIs may have invalid octet sequences (not possible to reverse). 3. quote default to UTF-8, unquote default to Latin-1. In favour: Andr? Malo Pros: quote able to handle all characters; unquote able to handle all sequences. Cons: unquote is not inverse of quote; totally inconsistent. 4. quote accepts either bytes or str, unquote default to outputting bytes unless given an encoding argument. In favour: Bill Janssen Pros: Technically does what the spec says, which is treat it as an octet encoding. Cons: unquote will break most existing code; almost 100% of the time people will want it as a string. </impartiality> I'll just comment on #4 since I haven't already. Let's talk about quote and unquote separately. For quote, I'm all for letting it accept a bytes as well as a str. That doesn't break anything or surprise anyone. For unquote, I think it will break a lot and surprise everyone. I think that while this may be "purely" the best option, it's pretty silly. I reckon the vast majority of users will be surprised when they see it spitting out a bytes object, and all that most people will do is decode it as UTF-8. Besides, while you're reading the RFCs as "URLs specify a method for encoding octet sequences", I'm reading them as "URLs specify a method for encoding strings, and leave the character encoding unspecified." The second reading supports the idea that unquote outputs a str. I'm also recommending we add unquote_to_bytes to do what you suggest unquote should do. (So either way we'll get both versions of unquote; I'm just suggesting the one called "unquote" do the thing everybody expects). But that's less of a priority so I want to commit these urgent fixes first. I'm basically saying just two things: 1. The standards are undefined; 2. Therefore we should pick the most useful and/or intuitive default. IMHO choosing UTF-8 *is* the most useful AND intuitive, and will be more so in the future when more technologies are hard-coded as UTF-8 (which this RFC recommends they do in the future). I am also quite adamant that unquote be the inverse of quote. Are there any more opinions on this matter? It would be good to reach a consensus. If anyone seriously wants to push a different alternative to mine, please write a working implementation and attach it to issue 3300. On the technical side of things, does anybody have time to review my patch for this issue? http://bugs.python.org/issue3300 Patch 5. It's just a patch for unquote, quote, and small related functions, as well as numerous changes to test cases and documentation. Cheers Matt From matt.giuca at gmail.com Wed Jul 30 16:34:33 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Thu, 31 Jul 2008 00:34:33 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> Message-ID: <e6268af30807300734h68a848a1pdcff123512b6ec20@mail.gmail.com> Arg! Damnit, why do my replies get split off from the main thread? Sorry about any confusion this may be causing. From phd at phd.pp.ru Wed Jul 30 16:53:06 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 30 Jul 2008 18:53:06 +0400 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> Message-ID: <20080730145306.GA26756@phd.pp.ru> On Thu, Jul 31, 2008 at 12:11:40AM +1000, Matt Giuca wrote: > 2. Default to UTF-8. > In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven Count me too: +1. Most sites I use theese days use UTF-8 for URL encoding. Examples: Wikipedia: http://ru.wikipedia.org/wiki/%D0%93%D0%B2%D0%B8%D0%B4%D0%BE_%D0%B2%D0%B0%D0%BD_%D0%A0%D0%BE%D1%81%D1%81%D1%83%D0%BC LingVo (Russian-English dictionary): http://lingvo.yandex.ru/en?text=%D0%BF%D0%B8%D1%82%D0%BE%D0%BD >>> print urllib.quote(unicode('?????', 'koi8-r').encode('utf-8')) %D0%BF%D0%B8%D1%82%D0%BE%D0%BD Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From facundobatista at gmail.com Wed Jul 30 17:00:02 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 30 Jul 2008 12:00:02 -0300 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> Message-ID: <e04bdf310807300800x3fe77520se6707ff794253d85@mail.gmail.com> 2008/7/30 Matt Giuca <matt.giuca at gmail.com>: > 2. Default to UTF-8. > In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven > Pros: Fully working and tested solution is implemented; recommended by > RFC 3986 for all future schemes; recommended by W3C for use with HTML; > UTF-8 used by all major browsers; supports all characters; most > existing code compatible by default; unquote is inverse of quote. > Cons: By default, URIs may have invalid octet sequences (not possible > to reverse). +1, assuming that if you have a different encoding in the URI you can pass it as a parameter. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From solipsis at pitrou.net Wed Jul 30 17:08:29 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 30 Jul 2008 15:08:29 +0000 (UTC) Subject: [Python-Dev] urllib.quote and unquote - Unicode issues References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <e04bdf310807300800x3fe77520se6707ff794253d85@mail.gmail.com> Message-ID: <loom.20080730T150601-899@post.gmane.org> Facundo Batista <facundobatista <at> gmail.com> writes: > > 2008/7/30 Matt Giuca <matt.giuca <at> gmail.com>: > > > 2. Default to UTF-8. > > In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven > > Pros: Fully working and tested solution is implemented; recommended by > > RFC 3986 for all future schemes; recommended by W3C for use with HTML; > > UTF-8 used by all major browsers; supports all characters; most > > existing code compatible by default; unquote is inverse of quote. > > Cons: By default, URIs may have invalid octet sequences (not possible > > to reverse). > > +1, assuming that if you have a different encoding in the URI you can > pass it as a parameter. +1 for me as well, with an optional encoding parameter to override the default. Also, your "con" is a "pro" to me, since it means errors are reported instead of silently producing garbage (as would be the case with latin1). Regards Antoine. From nd at perlig.de Wed Jul 30 17:09:51 2008 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Wed, 30 Jul 2008 17:09:51 +0200 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> Message-ID: <200807301709.51528.nd@perlig.de> [I was pretty busy these days, so sorry for jumping in late again] * Matt Giuca wrote: > 1. Leave it as it is. quote is Latin-1 if range(0,256), fallback to > UTF-8. unquote is Latin-1. > In favour: Anybody who doesn't reply to this thread > Pros: Already implemented; some existing code depends upon ord values > of string being the same as they were for byte strings; possible to > hack around it. > Cons: unquote is not inverse of quote; quote behaviour > internally-inconsistent; garbage when unquoting UTF-8-encoded URIs. > 2. Default to UTF-8. > In favour: Matt Giuca, Brett Cannon, Jeroen Ruigrok van der Werven > Pros: Fully working and tested solution is implemented; recommended by > RFC 3986 for all future schemes; recommended by W3C for use with HTML; > UTF-8 used by all major browsers; supports all characters; most > existing code compatible by default; unquote is inverse of quote. > Cons: By default, URIs may have invalid octet sequences (not possible > to reverse). Con: URI encoding does not encode characters. > > 3. quote default to UTF-8, unquote default to Latin-1. > In favour: Andr? Malo > Pros: quote able to handle all characters; unquote able to handle all > sequences. Cons: unquote is not inverse of quote; totally inconsistent. I'm not in favour of that. I merely answered a question there ;) I'm actually in favour of encoding bytes only back and forth. A useful extension would be *another* function which wraps quote/unquote and encodes and decodes characters. > 4. quote accepts either bytes or str, unquote default to outputting > bytes unless given an encoding argument. > In favour: Bill Janssen > Pros: Technically does what the spec says, which is treat it as an > octet encoding. > Cons: unquote will break most existing code; almost 100% of the time > people will want it as a string. > > </impartiality> > > I'll just comment on #4 since I haven't already. Let's talk about > quote and unquote separately. For quote, I'm all for letting it accept > a bytes as well as a str. That doesn't break anything or surprise > anyone. > > For unquote, I think it will break a lot and surprise everyone. I > think that while this may be "purely" the best option, it's pretty > silly. I reckon the vast majority of users will be surprised when they > see it spitting out a bytes object, and all that most people will do > is decode it as UTF-8. Besides, while you're reading the RFCs as "URLs > specify a method for encoding octet sequences", I'm reading them as > "URLs specify a method for encoding strings, and leave the character > encoding unspecified." The second reading supports the idea that > unquote outputs a str. > > I'm also recommending we add unquote_to_bytes to do what you suggest > unquote should do. (So either way we'll get both versions of unquote; > I'm just suggesting the one called "unquote" do the thing everybody > expects). But that's less of a priority so I want to commit these > urgent fixes first. > > I'm basically saying just two things: 1. The standards are undefined; That's still disputed... > 2. Therefore we should pick the most useful and/or intuitive default. > IMHO choosing UTF-8 *is* the most useful AND intuitive, and will be > more so in the future when more technologies are hard-coded as UTF-8 > (which this RFC recommends they do in the future). See my suggestion above. nd From guido at python.org Wed Jul 30 18:27:02 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Jul 2008 09:27:02 -0700 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <200807301709.51528.nd@perlig.de> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> Message-ID: <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> On Wed, Jul 30, 2008 at 8:09 AM, Andr? Malo <nd at perlig.de> wrote: > I'm actually in favour of encoding bytes only back and forth. A useful > extension would be *another* function which wraps quote/unquote and encodes > and decodes characters. I'd reverse this. By all means, add a new pair of functions that is bytes in / bytes out. But keep the existing functions purely string in / string out, hardcoded to UTF-8. People wanting another encoding can use the bytes functions and explicit encode / decode calls. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Wed Jul 30 18:46:05 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 30 Jul 2008 09:46:05 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <20080712222801.GC27106@nexus.in-nomine.org> <e6268af30807121615s12a7a15ehb0dddf1de86fabbe@mail.gmail.com> <8249917582531821653@unknownmsgid> <e6268af30807132022p747da583v624b8c95b6052b55@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> Message-ID: <08Jul30.094609pdt."58698"@synergy1.parc.xerox.com> > For unquote, I think it will break a lot and surprise everyone. I > think that while this may be "purely" the best option, it's pretty > silly. I don't mind being silly to do the right thing. Happens to me a lot :-). Bill From janssen at parc.com Wed Jul 30 18:52:26 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 30 Jul 2008 09:52:26 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> Message-ID: <08Jul30.095234pdt."58698"@synergy1.parc.xerox.com> > On Wed, Jul 30, 2008 at 8:09 AM, Andr? Malo <nd at perlig.de> wrote: > > I'm actually in favour of encoding bytes only back and forth. A useful > > extension would be *another* function which wraps quote/unquote and encod= > es > > and decodes characters. > > I'd reverse this. By all means, add a new pair of functions that is > bytes in / bytes out. But keep the existing functions purely string in > / string out, hardcoded to UTF-8. People wanting another encoding can > use the bytes functions and explicit encode / decode calls. Actually (as I pointed out before) the existing functions are not string-in/string-out. They are something-in and bytes-out. just look like string-in/string-out because of the confusion between byte strings and Unicode strings in Python 1 and 2. Look, Matt's suggestion is a degradation of the integrity of the stdlib, because it enthrones a broken understanding, a misreading of the RFC, in a very prominent place. I'd prefer not to have Python contribute to that breakage. Keep the functions the way they are now: bytes-in and bytes-out. Bill From guido at python.org Wed Jul 30 19:12:46 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Jul 2008 10:12:46 -0700 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <6046832403127758775@unknownmsgid> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> Message-ID: <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> On Wed, Jul 30, 2008 at 9:52 AM, Bill Janssen <janssen at parc.com> wrote: >> On Wed, Jul 30, 2008 at 8:09 AM, Andr? Malo <nd at perlig.de> wrote: >> > I'm actually in favour of encoding bytes only back and forth. A useful >> > extension would be *another* function which wraps quote/unquote and encod= >> es >> > and decodes characters. >> >> I'd reverse this. By all means, add a new pair of functions that is >> bytes in / bytes out. But keep the existing functions purely string in >> / string out, hardcoded to UTF-8. People wanting another encoding can >> use the bytes functions and explicit encode / decode calls. > > Actually (as I pointed out before) the existing functions are not > string-in/string-out. They are something-in and bytes-out. just look > like string-in/string-out because of the confusion between byte > strings and Unicode strings in Python 1 and 2. Actually, we'd need to look at the various other APIs in Py3k before we can decide whether these should be considered taking or returning bytes or text. It looks like all other APIs in the Py3k version of urllib treat URLs as text. I don't think switching these to bytes would be a good idea; you might as well claim that filenames should be bytes because that's how the filesystem stores them. > Look, Matt's suggestion is a degradation of the integrity of the > stdlib, because it enthrones a broken understanding, a misreading of > the RFC, in a very prominent place. I'd prefer not to have Python > contribute to that breakage. Keep the functions the way they are now: > bytes-in and bytes-out. I think that would break too much code, without a good way to automatically fix it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Wed Jul 30 19:23:47 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 30 Jul 2008 10:23:47 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <08Jul30.095234pdt."58698"@synergy1.parc.xerox.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <08Jul30.095234pdt."58698"@synergy1.parc.xerox.com> Message-ID: <08Jul30.102355pdt."58698"@synergy1.parc.xerox.com> > Actually (as I pointed out before) the existing functions are not > string-in/string-out. They are something-in and bytes-out. Sorry, this is wrong. "quote" is clearly bytes-in and string-out. "unquote" is clearly string-in and bytes-out. The whole point of "quote" is to take an arbitrary sequence of bytes and represent them as an ASCII string, while unquote reverses this process. Again, I urge everyone participating in this discussion to read RFC 3986. We're not creating in a vacuum here; we're talking about implementation of a standard. Bill From janssen at parc.com Wed Jul 30 19:33:51 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 30 Jul 2008 10:33:51 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> Message-ID: <08Jul30.103400pdt."58698"@synergy1.parc.xerox.com> > It looks like all other APIs in the Py3k version of > urllib treat URLs as text. The URL is text, a string of ASCII characters. We're just talking about urllib.quote() and urllib.unquote(), which are there to support the text-ization of binary values, and the de-text-ization. > I think that would break too much code, without a good way to > automatically fix it. You'd rather break Python? Somehow I don't think so. Here's the signature I'm proposing: quote() -- takes string or bytes, and produces string. If input is a string, looks to optional "encoding" parameter to determine character set encoding to use to transform it to byte before quoting it. If "encoding" is not specified, defaults to UTF-8. unquote() -- takes string, produces bytes or string If optional "encoding" parameter is specified, decodes bytes with that encoding and returns string. Otherwise, returns bytes. Bill From guido at python.org Wed Jul 30 20:00:09 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Jul 2008 11:00:09 -0700 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <3968642431361249330@unknownmsgid> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> Message-ID: <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> On Wed, Jul 30, 2008 at 10:33 AM, Bill Janssen <janssen at parc.com> wrote: >> It looks like all other APIs in the Py3k version of >> urllib treat URLs as text. > > The URL is text, a string of ASCII characters. We're just talking > about urllib.quote() and urllib.unquote(), which are there to support > the text-ization of binary values, and the de-text-ization. > >> I think that would break too much code, without a good way to >> automatically fix it. > > You'd rather break Python? Somehow I don't think so. Let's stop the rhetoric, or I'll have to beat you over the head with the Zen of Python. :-) urllib is not meant as a reference implementation of any RFC; it is meant as a practical tool for Python users writing web apps (servers and clients). > Here's the signature I'm proposing: > > quote() -- takes string or bytes, and produces string. > > If input is a string, looks to optional "encoding" parameter to > determine character set encoding to use to transform it to byte before > quoting it. If "encoding" is not specified, defaults to UTF-8. No contest here, since it supports the common string->string use case. E.g. quote('a%b') returns 'a%25b'. > unquote() -- takes string, produces bytes or string > > If optional "encoding" parameter is specified, decodes bytes with > that encoding and returns string. Otherwise, returns bytes. The default of returning bytes will break almost all uses. Most code will uses the unquoted result as a text string, not as bytes -- e.g. a server has to unquote the values it receives from a form (whether POST or GET), but almost always the unquoted values are text, e.g. someone's name or address, or a draft email message. (Aside: I dislike functions that have a different return type based on the value of a parameter.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 30 20:17:51 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Jul 2008 11:17:51 -0700 Subject: [Python-Dev] Fuzzing bugs: most bugs are closed In-Reply-To: <20080721174141.GA16598@amk-desktop.matrixgroup.net> References: <200807191323.13124.victor.stinner@haypocalc.com> <200807202245.39874.victor.stinner@haypocalc.com> <20080721133319.GA15912@amk-desktop.matrixgroup.net> <200807211554.11468.victor.stinner@haypocalc.com> <loom.20080721T154744-225@post.gmane.org> <20080721174141.GA16598@amk-desktop.matrixgroup.net> Message-ID: <ca471dc20807301117y661306f4r365c08c7483b0bfc@mail.gmail.com> On Mon, Jul 21, 2008 at 10:41 AM, A.M. Kuchling <amk at amk.ca> wrote: > On Mon, Jul 21, 2008 at 03:53:18PM +0000, Antoine Pitrou wrote: >> The underscore at the beginning of _sre clearly indicates that the module is >> not recommended for direct consumption, IMO. Even the functions that don't >> themselves start with an underscore... > > Sure, but if someone is trying to break in or DoS your application > server, they don't care if the module starts with an underscore or > not. > > To answer Victor's original question: the parser & compiler that turn > a regex into bytecode is written in Python. I can't think of a way to > prevent other Python modules from importing _sre or accessing the > compile() function; if nothing else, code could always do 'import re ; > re.sre_compile._sre.compile(...)'. I've written a re-code verifier for the Google App Engine. I have permission to open source this, hopefully I will get to this before 2.6 beta 3. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From hall.jeff at gmail.com Wed Jul 30 20:38:40 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Wed, 30 Jul 2008 14:38:40 -0400 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> Message-ID: <1bc395c10807301138m5e39febbpf2c058720d9b38a0@mail.gmail.com> > > > (Aside: I dislike functions that have a different return type based on > the value of a parameter.) > > I wanted to stay out of the whole discussion as it's largely over my head... But I did want to express support for this idea which I think almost rises to the level of a standard... I see more bugs created in our software because of the above issues then anything else... I have no problem with functions that accept various input but producing various outputs just seems to wreak havoc... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080730/f17696aa/attachment-0001.htm> From janssen at parc.com Wed Jul 30 21:49:06 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 30 Jul 2008 12:49:06 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> Message-ID: <08Jul30.124910pdt."58698"@synergy1.parc.xerox.com> > > unquote() -- takes string, produces bytes or string > > > > If optional "encoding" parameter is specified, decodes bytes with > > that encoding and returns string. Otherwise, returns bytes. > > The default of returning bytes will break almost all uses. Most code > will uses the unquoted result as a text string, not as bytes -- e.g. a > server has to unquote the values it receives from a form (whether POST > or GET), but almost always the unquoted values are text, e.g. > someone's name or address, or a draft email message. I actually do know a lot about the uses of this function... But: OK, OK, I yield. Though I still think this is a bad idea, I'll shut up if we can also add "unquote_as_bytes" which returns a byte sequence instead of a string. I'll just change my code to use that. > (Aside: I dislike functions that have a different return type based on > the value of a parameter.) Fair enough. Bill From guido at python.org Wed Jul 30 22:32:24 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Jul 2008 13:32:24 -0700 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <7769003638770459002@unknownmsgid> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> Message-ID: <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> On Wed, Jul 30, 2008 at 12:49 PM, Bill Janssen <janssen at parc.com> wrote: >> > unquote() -- takes string, produces bytes or string >> > >> > If optional "encoding" parameter is specified, decodes bytes with >> > that encoding and returns string. Otherwise, returns bytes. >> >> The default of returning bytes will break almost all uses. Most code >> will uses the unquoted result as a text string, not as bytes -- e.g. a >> server has to unquote the values it receives from a form (whether POST >> or GET), but almost always the unquoted values are text, e.g. >> someone's name or address, or a draft email message. > > I actually do know a lot about the uses of this function... > > But: OK, OK, I yield. Though I still think this is a bad idea, I'll > shut up if we can also add "unquote_as_bytes" which returns a byte > sequence instead of a string. I'll just change my code to use that. > >> (Aside: I dislike functions that have a different return type based on >> the value of a parameter.) > > Fair enough. I think this is as close as consensus as we can get on this issue. Can whoever wrote the patch adjust the patch to this outcome? (I think the only change is to remove the encoding arguments and make separate functions for bytes.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Thu Jul 31 02:25:25 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 30 Jul 2008 17:25:25 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <-6533647106000374959@unknownmsgid> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> Message-ID: <08Jul30.172526pdt."58698"@synergy1.parc.xerox.com> > I think this is as close as consensus as we can get on this issue. Can > whoever wrote the patch adjust the patch to this outcome? (I think the > only change is to remove the encoding arguments and make separate > functions for bytes.) This is 2.7/3.1 only, right? I'm looking at the bales of code I've got that says something like, v = urlib.quote_plus(x.encode("UTF-8", "strict")) then later on x = unicode(urllib.unquote_plus(v), "UTF-8", "strict") Bill From musiccomposition at gmail.com Thu Jul 31 04:31:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 30 Jul 2008 21:31:37 -0500 Subject: [Python-Dev] critical issues for 2.6 and 3.0 Message-ID: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com> I just went through the disturbingly long list of 67 open issues with a "critical" priority pinging and trying to get things moving. There are ~55 now; I was able to close some, but others I promoted to release blocker for beta 3. Shouldn't all criticals be resolved by the final? I've never been through a Python release before, but I find these statistics rather worrying if we want to make the October release date. It doesn't help that we are low on active core developers, presumably because they are taking full advantage of their summer vacations. :) (Speaking of which, I'm leaving this Saturday.) Please focus getting fixes reviewed, checked in, and their issue's closed so we can bring beta 3 out on time! -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Thu Jul 31 04:51:38 2008 From: brett at python.org (Brett Cannon) Date: Wed, 30 Jul 2008 19:51:38 -0700 Subject: [Python-Dev] critical issues for 2.6 and 3.0 In-Reply-To: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com> References: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com> Message-ID: <bbaeab100807301951k511afae0g86d1e53c0943e199@mail.gmail.com> On Wed, Jul 30, 2008 at 7:31 PM, Benjamin Peterson <musiccomposition at gmail.com> wrote: > I just went through the disturbingly long list of 67 open issues with > a "critical" priority pinging and trying to get things moving. There > are ~55 now; I was able to close some, but others I promoted to > release blocker for beta 3. Shouldn't all criticals be resolved by the > final? > Probably, but at that point they will be promoated to release blocker as necessary. > I've never been through a Python release before, but I find these > statistics rather worrying if we want to make the October release > date. If we don't make the release, then we don't make it. Plus this is one of the more complicated releases that I have been through thanks to the release of two simultaneous major revisions, so having a lot to do is not a shock. But people tend to step up work when a beta release is coming so when we get closer to b3 more work will probably land. Another thing to keep in mind beyond the open issues is the code in 2.6 that is not 3.0 compatible when Python is run with -3. I just finished running regrtest with -3 and have a text file listing all of the code that has some warning thanks to -3. I will try to open an issue with those files listed as some point soon, but I will hopefully be able to plow through them rather quickly since most of them are minor like dict.has_key(), etc. -Brett From matt.giuca at gmail.com Thu Jul 31 05:49:17 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Thu, 31 Jul 2008 13:49:17 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> Message-ID: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> > Con: URI encoding does not encode characters. OK, for all the people who say URI encoding does not encode characters: yes it does. This is not an encoding for binary data, it's an encoding for character data, but it's unspecified how the strings map to octets before being percent-encoded. From RFC 3986, section 1.2.1<http://tools.ietf.org/html/rfc3986#section-1.2.1> : Percent-encoded octets (Section 2.1) may be used within a URI to represent > characters outside the range of the US-ASCII coded character set if this > representation is allowed by the scheme or by the protocol element in which > the URI is referenced. Such a definition should specify the character > encoding used to map those characters to octets prior to being > percent-encoded for the URI. So the string->string proposal is actually correct behaviour. I'm all in favour of a bytes->string version as well, just not with the names "quote" and "unquote". I'll prepare a new patch shortly which has bytes->string and string->bytes versions of the functions as well. (quote will accept either type, while unquote will output a str, there will be a new function unquote_to_bytes which outputs a bytes - is everyone happy with that?) Guido says: > Actually, we'd need to look at the various other APIs in Py3k before we can > decide whether these should be considered taking or returning bytes or text. > It looks like all other APIs in the Py3k version of urllib treat URLs as > text. Yes, as I said in the bug tracker, I've groveled over the entire stdlib to see how my patch affects the behaviour of dependent code. Aside from a few minor bits which assumed octets (and did their own encoding/decoding) (which I fixed), all the code assumes strings and is very happy to go on assuming this, as long as the URIs are encoded with UTF-8, which they almost certainly are. Guido says: > I think the only change is to remove the encoding arguments and ... You really want me to remove the encoding= named argument? And hard-code UTF-8 into these functions? It seems like we may as well have the optional encoding argument, as it does no harm and could be of significant benefit. I'll post a patch with the unquote_to_bytes function, but leave the encoding arguments in until this point is clarified. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/209fd01b/attachment.htm> From guido at python.org Thu Jul 31 06:01:56 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Jul 2008 21:01:56 -0700 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> Message-ID: <ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com> On Wed, Jul 30, 2008 at 8:49 PM, Matt Giuca <matt.giuca at gmail.com> wrote: > >> Con: URI encoding does not encode characters. > > OK, for all the people who say URI encoding does not encode characters: yes > it does. This is not an encoding for binary data, it's an encoding for > character data, but it's unspecified how the strings map to octets before > being percent-encoded. From RFC 3986, section 1.2.1: > >> Percent-encoded octets (Section 2.1) may be used within a URI to represent >> characters outside the range of the US-ASCII coded character set if this >> representation is allowed by the scheme or by the protocol element in which >> the URI is referenced. Such a definition should specify the character >> encoding used to map those characters to octets prior to being >> percent-encoded for the URI. > > So the string->string proposal is actually correct behaviour. I'm all in > favour of a bytes->string version as well, just not with the names "quote" > and "unquote". > > I'll prepare a new patch shortly which has bytes->string and string->bytes > versions of the functions as well. (quote will accept either type, while > unquote will output a str, there will be a new function unquote_to_bytes > which outputs a bytes - is everyone happy with that?) I'd rather have two pairs of functions, so that those who want to give the readers of their code a clue can do so. I'm not opposed to having redundant functions that accept either string or bytes though, unless others prefer not to. > Guido says: >> >> Actually, we'd need to look at the various other APIs in Py3k before we >> can decide whether these should be considered taking or returning bytes or >> text. It looks like all other APIs in the Py3k version of urllib treat URLs >> as text. > > Yes, as I said in the bug tracker, I've groveled over the entire stdlib to > see how my patch affects the behaviour of dependent code. Aside from a few > minor bits which assumed octets (and did their own encoding/decoding) (which > I fixed), all the code assumes strings and is very happy to go on assuming > this, as long as the URIs are encoded with UTF-8, which they almost > certainly are. Sorry, I have yet to look at the tracker (only so many minutes in a day...). > Guido says: >> >> I think the only change is to remove the encoding arguments and ... > > You really want me to remove the encoding= named argument? And hard-code > UTF-8 into these functions? It seems like we may as well have the optional > encoding argument, as it does no harm and could be of significant benefit. > I'll post a patch with the unquote_to_bytes function, but leave the encoding > arguments in until this point is clarified. I don't mind an encoding argument, as long as it isn't used to change the return type (as Bill was proposing). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From sumant.gupta at aricent.com Thu Jul 31 07:01:42 2008 From: sumant.gupta at aricent.com (Sumant Gupta) Date: Thu, 31 Jul 2008 10:31:42 +0530 Subject: [Python-Dev] Memory Error while reading large file Message-ID: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM> Hi I have a problem reading very large text file. When I call the len function to get the total lines in python file.i get memory error . I am reading the list of files in a loop ,2 files are read properly but when the third file is read , It gives an memory error . Sumant Gupta Software Engineer Ext:5105 ________________________________ "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error,please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility forloss or damage arising from the use of the information transmitted by this email including damage from virus." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/7ad461ea/attachment.htm> From turnbull at sk.tsukuba.ac.jp Thu Jul 31 08:36:30 2008 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 31 Jul 2008 15:36:30 +0900 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> Message-ID: <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> Matt Giuca writes: > OK, for all the people who say URI encoding does not encode characters: yes > it does. This is not an encoding for binary data, it's an encoding for > character data, but it's unspecified how the strings map to octets before > being percent-encoded. In other words, it's an encoding for binary data, since the octet sequences that might be encountered are completely unrestricted. I have to side with Bill on this. URIs are sequences of characters, but the character set used must contain the ASCII repertoire as a subset, of which the URI delimiters must be mapped to the corresponding ASCII codes, the rest of the set must be represented as sequences of octets (which need not even be constant; you could gzip them first for all URI-encoding cares). URI-encoding itself is a purely mechanical process which transforms reserved octets (not used as delimiters) to percent codes. > From RFC 3986, section > 1.2.1<http://tools.ietf.org/html/rfc3986#section-1.2.1>: > > Percent-encoded octets (Section 2.1) may be used within a URI to represent > > characters outside the range of the US-ASCII coded character set if this > > representation is allowed by the scheme or by the protocol element in which > > the URI is referenced. Such a definition should specify the character > > encoding used to map those characters to octets prior to being > > percent-encoded for the URI. This is kinda perverted, but suppose you have bytes which are actually a Japanese string represented in packed EUC-JP. AFAICS the paragraph above does *not* say you can't transcode to UTF-8 before percent-encoding, and in fact you might be required to by the definition of the scheme. > So the string->string proposal is actually correct behaviour. Ye-e-es, but. What the RFC clearly envisions is not that the percent-encoder will be handed an unencoded string that looks like a URI, but rather a sequence of octets representing one component (scheme, authority, path, query, etc) of a URI. In other words, a string->string URI encoder should only be called by an URI builder, and never with a precomposed URI-like string. Something like def URIBuilder (strings): """Return an URI built from a list of strings. The first string *must* be the scheme. If the URI follows the generic URI syntax of RFC 3986, the remaining components should be given in the order authority, path, fragment, query part [, query part ...].""" def uriencode (s): """URI encode a string per RFC 3986 Section 3.""" # We all know what this does. if strings[0] == "http": # HTTP scheme, delimiters and authority uri = "http://" + uriencode(strings[1]) + "/" # path, if present if strings[2]: uri = uri + uriencode(strings[2]) # query, if present if strings[4]: uri = uri + "?" + uriencode(strings[4]) # further query parameters, if present for s in strings[4:] uri = uri + ";" + uriencode(s) # fragment, if present if strings[3]: uri = uri + "#" + uriencode(strings[3]) else if strings[0] == "mailto": uri = "mailto:" + uriencode(strings[1]) # etc etc return uri I think you'd have a much easier time enforcing this pedantically correct usage with a bytes->bytes encoder. Of course, it's un-Pythonic to enforce pedantry, and we pedants can use a string->string encoder correctly. > You really want me to remove the encoding= named argument? And hard-code > UTF-8 into these functions? A quoting function that accepts bytes *must* have an encoding argument. There's no point to passing the quoter bytes unless the text is represented in a non-Unicode encoding. From steve at pearwood.info Thu Jul 31 09:02:29 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 31 Jul 2008 17:02:29 +1000 Subject: [Python-Dev] Memory Error while reading large file In-Reply-To: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM> References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM> Message-ID: <200807311702.30163.steve@pearwood.info> On Thu, 31 Jul 2008 03:01:42 pm Sumant Gupta wrote: > Hi > > I have a problem reading very large text file. > When I call the len function to get the total lines in python file.i > get memory error . I am reading the list of files in a loop ,2 files > are read properly but when the third file is read , It gives an > memory error . I'm not completely sure, but I think that means you're out of memory. If you have an actual question, I think you would be better off posting to the comp.lang.python newsgroup. This mailing list is for development of the Python compiler, not for writing Python programs. I'll save you some time: when you post to comp.lang.python, you should post the actual error message you get, and the code that fails. -- Steven From janssen at parc.com Thu Jul 31 09:39:29 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 31 Jul 2008 00:39:29 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> Message-ID: <08Jul31.003936pdt."58698"@synergy1.parc.xerox.com> > Guido says: > > > Actually, we'd need to look at the various other APIs in Py3k before we can > > decide whether these should be considered taking or returning bytes or text. > > It looks like all other APIs in the Py3k version of urllib treat URLs as > > text. > > > Yes, as I said in the bug tracker, I've groveled over the entire stdlib to > see how my patch affects the behaviour of dependent code. Aside from a few > minor bits which assumed octets (and did their own encoding/decoding) (which > I fixed), all the code assumes strings and is very happy to go on assuming > this, as long as the URIs are encoded with UTF-8, which they almost > certainly are. I'm not sure that's sufficient review, though I agree it's necessary. The major consumers of quote/unquote are not in the Python standard library. > (quote will accept either type, while > unquote will output a str, there will be a new function unquote_to_bytes > which outputs a bytes - is everyone happy with that?) No, so don't ask. Bill From janssen at parc.com Thu Jul 31 09:47:12 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 31 Jul 2008 00:47:12 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <08Jul31.004722pdt."58698"@synergy1.parc.xerox.com> > Of course, it's un-Pythonic to enforce pedantry, and we pedants can > use a string->string encoder correctly. Sure. All I was asking was that we not break the existing usage of the standard library "unquote" by producing a string by *assuming* a UTF-8 encoded string is what's in those percent-encoded bytes (instead of, say, ISO 2022-JP). Let the "new" function produce a string: "unquote_as_string". > > You really want me to remove the encoding= named argument? And hard-code > > UTF-8 into these functions? > > A quoting function that accepts bytes *must* have an encoding > argument. Huh? What would it use it for? The string, if string it is, is already encoded as octets. All it needs to do is percent-encode the reserved octets. So far as I can see, the "unquote_as_string" is the function that needs the encoding. Ah, it's too late, I'll pick this up tomorrow. Bill From janssen at parc.com Thu Jul 31 09:52:32 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 31 Jul 2008 00:52:32 PDT Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <08Jul31.005237pdt."58698"@synergy1.parc.xerox.com> Also see <http://en.wikipedia.org/wiki/Percent-encoding>. Bill From stephen at xemacs.org Thu Jul 31 12:13:35 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 31 Jul 2008 19:13:35 +0900 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <08Jul31.004722pdt."58698"@synergy1.parc.xerox.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <e6268af30807300711i30003c8h9884c518eea7e296@mail.gmail.com> <200807301709.51528.nd@perlig.de> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> <87r69apnkh.fsf@uwakimon.sk.tsukuba.ac.jp> <08Jul31.004722pdt."58698"@synergy1.parc.xerox.com> Message-ID: <87od4epdio.fsf@uwakimon.sk.tsukuba.ac.jp> Bill Janssen writes: > > A quoting function that accepts bytes *must* have an encoding > > argument. > > Huh? What would it use it for? Ah, you're right. I was thinking in terms of an URI builder, where the quoter would do any required conversion (eg, if the bytes represented a string in Japanese) to another (possibly scheme-mandated) encoding (typically UTF-8). But that doesn't really make sense; the URI builder should know what to do, and that's a better place to do such conversions. From matt.giuca at gmail.com Thu Jul 31 14:07:36 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Thu, 31 Jul 2008 22:07:36 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> <ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com> Message-ID: <e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com> Alright, I've uploaded the new patch which adds the two requested bytes-oriented functions, as well as accompanying docs and tests. http://bugs.python.org/issue3300 http://bugs.python.org/file11009/parse.py.patch6 I'd rather have two pairs of functions, so that those who want to give > the readers of their code a clue can do so. I'm not opposed to having > redundant functions that accept either string or bytes though, unless > others prefer not to. > Yes, I was in a similar mindset. So the way I've implemented it, quote accepts either a bytes or a str. Then there's a new function quote_from_bytes, which is defined precisely like this: quote_from_bytes = quote > So either name can be used on either input type, with the idea being that you should use quote on a str, and quote_from_bytes on a bytes. Is this a good idea or should it be rewritten so each function permits only one input type? Sorry, I have yet to look at the tracker (only so many minutes in a day...). Ah, I didn't mean offense. Just that one could read the sordid details of my investigation on the tracker if one so desired ;) I don't mind an encoding argument, as long as it isn't used to change > the return type (as Bill was proposing). Yeah, my unquote always outputs a str, and unquote_to_bytes always outputs a bytes. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/85e6eb53/attachment.htm> From matt.giuca at gmail.com Thu Jul 31 14:25:09 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Thu, 31 Jul 2008 22:25:09 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <3166922734065744385@unknownmsgid> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <ca471dc20807300927m6c04de93sf5d77aa4417ca998@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> <3166922734065744385@unknownmsgid> Message-ID: <e6268af30807310525l255052deo5e9ad3dbde776bc6@mail.gmail.com> Bill wrote: I'm not sure that's sufficient review, though I agree it's necessary. > The major consumers of quote/unquote are not in the Python standard > library. I figured that Python 3.0 is designed to fix things, with the breaking third-party code being an acceptable side-effect of that. So the most important thing when 3.0 is released is that the stdlib is internally consistent. All other code is "allowed" to be broken. So I've investigated all the code necessary. Having said this, my patch breaks almost no code. Your suggestion breaks a hell of a lot. Sure. All I was asking was that we not break the existing usage of > the standard library "unquote" by producing a string by *assuming* a > UTF-8 encoded string is what's in those percent-encoded bytes (instead > of, say, ISO 2022-JP). Let the "new" function produce a string: > "unquote_as_string". You're assuming that a Python 2.x "str" is the same thing as a Python 3.0 "bytes". It isn't. (If it was, this transition would be trivial). A Python 2 "str" is a non-Unicode string. It can be printed, concatenated with Unicode strings, etc etc. It has the semantics of a string. The Python 3.0 "bytes" is not a string at all. What you're saying is "the old behaviour was to output a bytes, so the new behaviour should be consistent". But that isn't true - the old behaviour was to output a string (a non-Unicode one). People, and code, expect it to output something with string semantics. So making unquote output a bytes is just as big a change as making it output a (unicode) str. Python 3.0 doesn't have a type which is like Python 2's "str" type (which is good - that type was very messy). So the argument that "Python 2 unquote outputs a bytes, so we should too" is not legitimate. If you want to keep pushing this, please install my new patch (patch 6). Then rename "unquote" to "unquote_to_string" and rename "unquote_to_bytes" to "unquote", and witness the havoc that ensues. Firstly, you break most Internet-related modules in the standard library. 10 tests failed: > test_SimpleHTTPServer test_cgi test_email test_http_cookiejar > test_httpservers test_robotparser test_urllib test_urllib2 > test_urllib2_localnet test_wsgiref > Fixing these isn't a matter of changing test cases (which all but one of my fixes were). It would require changes to all the modules, to get them to deal with bytes instead of strings (which would generally mean spraying .decode("utf-8") all over the place). My code, on the other hand, "tends to be" compatible with 2.x code. Here I'm seeing: BytesWarning: Comparison between bytes and string. TypeError: expected an object with the buffer interface http.client.BadStatusLine For another example, try this: >>> import http.server >>> s = http.server.HTTPServer(('',8000), http.server.SimpleHTTPRequestHandler) >>> s.serve_forever() The current (unpatched) build works, but links to files with non-ASCII filenames (eg. '??') break, because of the URL. This is one example of my patch directly fixing a bug in real code. With my patch applied, the links work fine *because URL quoting and unquoting are consistent, and work on all Unicode characters*. If you change unquote to output a bytes, it breaks completely. You get a "TypeError: expected an object with the buffer interface" as soon as the user visits the page. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/7bf35eb4/attachment.htm> From ncoghlan at gmail.com Thu Jul 31 15:51:42 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 31 Jul 2008 23:51:42 +1000 Subject: [Python-Dev] Matrix product In-Reply-To: <b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> <b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com> <b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com> Message-ID: <4891C36E.4070200@gmail.com> Sebastien Loisel wrote: > Dear Raymond, > > Thank you for your email. > >> I think much of this thread is a repeat of conversations >> that were held for PEP 225: >> http://www.python.org/dev/peps/pep-0225/ >> >> That PEP is marked as deferred. Maybe it's time to >> bring it back to life. > > This is a much better PEP than the one I had found, and would solve > all of the numpy problems. The PEP is very well thought-out. A very interesting read! I wouldn't support some of the more exotic elements tacked on to the end (particularly the replacement of the now thoroughly entrenched bitwise operators), but the basic idea of providing ~op variants of several operators seems fairly sound. I'd be somewhat inclined to add ~not, ~and and ~or to the list even though that would pretty much force the semantics to be elementwise for the ~ variants (since the standard not, and and or are always objectwise and without PEP 335 there's no way for an object to change that). Cheers, Nick. From hall.jeff at gmail.com Thu Jul 31 18:33:47 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Thu, 31 Jul 2008 12:33:47 -0400 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <6046832403127758775@unknownmsgid> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> <ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com> <e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com> Message-ID: <1bc395c10807310933n519ac306v7656c83093e7cb6c@mail.gmail.com> > > > quote_from_bytes = quote >> > > So either name can be used on either input type, with the idea being that > you should use quote on a str, and quote_from_bytes on a bytes. Is this a > good idea or should it be rewritten so each function permits only one input > type? > > so you can use quote_from_bytes on strings? I assumed Guido meant it was okay to have quote accept string/byte input and have a function that was redundant but limited in what it accepted (i.e. quote_from_bytes accepts only bytes) I suppose your implementation doesn't break anything... it just strikes me as "odd" -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080731/e1c9be28/attachment.htm> From matt.giuca at gmail.com Thu Jul 31 19:19:57 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Fri, 1 Aug 2008 03:19:57 +1000 Subject: [Python-Dev] urllib.quote and unquote - Unicode issues In-Reply-To: <1bc395c10807310933n519ac306v7656c83093e7cb6c@mail.gmail.com> References: <e6268af30807121027o48cc010fu369bce99bde0233b@mail.gmail.com> <ca471dc20807301012t3ea09539k85002f8e3879bc21@mail.gmail.com> <3968642431361249330@unknownmsgid> <ca471dc20807301100v48c41a2alf49ebb91a0b95949@mail.gmail.com> <7769003638770459002@unknownmsgid> <ca471dc20807301332r23724cffi7de38373df234938@mail.gmail.com> <e6268af30807302049u2e953d41m4e84f9f9d8c3aaba@mail.gmail.com> <ca471dc20807302101m5ad7f08bv10b22af90262d7ac@mail.gmail.com> <e6268af30807310507m4fd678c9q3926e097cbac811a@mail.gmail.com> <1bc395c10807310933n519ac306v7656c83093e7cb6c@mail.gmail.com> Message-ID: <e6268af30807311019j14519f5eh233e827313b358e3@mail.gmail.com> > so you can use quote_from_bytes on strings? Yes, currently. > I assumed Guido meant it was okay to have quote accept string/byte input and have a function that was redundant but limited in what it accepted (i.e. quote_from_bytes accepts only bytes) > > I suppose your implementation doesn't break anything... it just strikes me as "odd" Yeah. I get exactly what you mean. Worse is it takes an encoding/replace argument. I'm in two minds about whether it should allow this or not. On one hand, it kind of goes with the Python philosophy of not artificially restricting the allowed types. And it avoids redundancy in the code. But I'd be quite happy to let quote_from_bytes restrict its input to just bytes, to avoid confusion. It would basically be a slightly-modified version of quote: def quote_from_bytes(s, safe = '/'): if isinstance(safe, str): safe = safe.encode('ascii', 'ignore') cachekey = (safe, always_safe) if not isinstance(s, bytes) or isinstance(s, bytearray): raise TypeError("quote_from_bytes() expected a bytes") try: quoter = _safe_quoters[cachekey] except KeyError: quoter = Quoter(safe) _safe_quoters[cachekey] = quoter res = map(quoter, s) return ''.join(res) (Passes test suite). I think I'm happier with this option. But the "if not isinstance(s, bytes) or isinstance(s, bytearray)" is not very nice. (The only difference to quote besides the missing arguments is the two lines beginning "if not isinstance". Maybe we can generalise the rest of the function). From cesare at pronto.it Thu Jul 31 21:25:57 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Thu, 31 Jul 2008 21:25:57 +0200 Subject: [Python-Dev] Matrix product In-Reply-To: <4891C36E.4070200@gmail.com> References: <b7975ab80807251504q209d9c39v2f3bc01114e1ec77@mail.gmail.com> <488A8302.3050408@canterbury.ac.nz> <ca471dc20807291726y2e4133f3p15884b4fec135fa0@mail.gmail.com> <3d0cebfb0807291759u3716e68at7046354260c09ebc@mail.gmail.com> <b7975ab80807291853v515b6252g9da2831ee2e02f6b@mail.gmail.com> <b7975ab80807300213p4dba4428n79edfdf4deebd908@mail.gmail.com> <4891C36E.4070200@gmail.com> Message-ID: <op.ue579jzuhlrjc9@conan> Nick Coghlan write: > Sebastien Loisel wrote: >> Dear Raymond, >> >> Thank you for your email. >> >>> I think much of this thread is a repeat of conversations >>> that were held for PEP 225: >>> http://www.python.org/dev/peps/pep-0225/ >>> >>> That PEP is marked as deferred. Maybe it's time to >>> bring it back to life. >> >> This is a much better PEP than the one I had found, and would solve >> all of the numpy problems. The PEP is very well thought-out. > > A very interesting read! I wouldn't support some of the more exotic > elements tacked on to the end (particularly the replacement of the now > thoroughly entrenched bitwise operators), but the basic idea of > providing ~op variants of several operators seems fairly sound. I'd be > somewhat inclined to add ~not, ~and and ~or to the list even though > that would pretty much force the semantics to be elementwise for the ~ > variants (since the standard not, and and or are always objectwise and > without PEP 335 there's no way for an object to change that). > > Cheers, > Nick. I agree: adding ~op will be very interesting. For example, we can easily provide case insensitive comparisons for string: if foo ~== 'Spam': print "It's spam!' equivalent to: if foo.upper() == 'SPAM: print "It's spam!' we can save both CPU time and memory to build a brand new string that will be discarded after the comparison... It will be also useful to redefine /, // and ** operators to do some common operations: 'spam, egg' / ', ' could be equivalent to iter('spam, egg'.split(', ')) # Generates an iterator 'spam, egg' // ', ' could be equivalent to 'spam, egg'.split(', ') # Generates a list and ', ' ** ('spam', 'egg') could be equivalent to ', '.join(('spam', 'egg')) but unfortunately we know that at the moment buil-in types cannot be "extended" through "monkey patching"... Cesare From martin at v.loewis.de Thu Jul 31 22:35:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 31 Jul 2008 22:35:50 +0200 Subject: [Python-Dev] critical issues for 2.6 and 3.0 In-Reply-To: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com> References: <1afaf6160807301931mdce696cvb359fa7a00a0c32@mail.gmail.com> Message-ID: <48922226.90003@v.loewis.de> > I've never been through a Python release before, but I find these > statistics rather worrying if we want to make the October release > date. I don't worry. Every Python release had bugs, and there will be 2.6.1 and 3.0.1 releases. The only sure way to resolve bugs is to revert features. If a certain module is cause of too many serious bugs, it should be dropped from the release (perhaps not from the source repository - just removed from all build processes). Regards, Martin From martin at v.loewis.de Thu Jul 31 22:40:48 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 31 Jul 2008 22:40:48 +0200 Subject: [Python-Dev] Memory Error while reading large file In-Reply-To: <200807311702.30163.steve@pearwood.info> References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM> <200807311702.30163.steve@pearwood.info> Message-ID: <48922350.9030102@v.loewis.de> > If you have an actual question I'd like to stress this point as well. Any good posting one wants an answer to must include a question, and that question must be explicitly phrased, and terminated with a question mark. (maybe the use of the question mark is more typical in German than in English; my stomach turns around when I read a question that ends with a full stop) Regards, Martin From scott+python-dev at scottdial.com Thu Jul 31 23:38:42 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Thu, 31 Jul 2008 17:38:42 -0400 Subject: [Python-Dev] Memory Error while reading large file In-Reply-To: <48922350.9030102@v.loewis.de> References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM> <200807311702.30163.steve@pearwood.info> <48922350.9030102@v.loewis.de> Message-ID: <489230E2.501@scottdial.com> Martin v. L?wis wrote: > (maybe the use of the question mark is more typical in German > than in English; my stomach turns around when I read a question > that ends with a full stop) There is no loss in translation here. Proper English requires the use of a question mark just the same as German, but you can't assume proper English will be used on a forum of communication like this one. The OP stated his problem, and maybe he doesn't know enough English to actually ask his question (I'm guessing by the name "Sumant Gupta"). I don't believe you are native speaker yourself, and I would've expected more sympathy from you. Lord knows I hope the recipients of any German I write will have some. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From guido at python.org Thu Jul 31 23:43:55 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 31 Jul 2008 14:43:55 -0700 Subject: [Python-Dev] Memory Error while reading large file In-Reply-To: <489230E2.501@scottdial.com> References: <FA19E97873301B48B29F217ABF64EA8797F021@GUREXMB01.ASIAN.AD.ARICENT.COM> <200807311702.30163.steve@pearwood.info> <48922350.9030102@v.loewis.de> <489230E2.501@scottdial.com> Message-ID: <ca471dc20807311443u514495eo1d59c913906a20ee@mail.gmail.com> On Thu, Jul 31, 2008 at 2:38 PM, Scott Dial <scott+python-dev at scottdial.com> wrote: > Martin v. L?wis wrote: >> >> (maybe the use of the question mark is more typical in German >> than in English; my stomach turns around when I read a question >> that ends with a full stop) > > There is no loss in translation here. Proper English requires the use of a > question mark just the same as German, but you can't assume proper English > will be used on a forum of communication like this one. The OP stated his > problem, and maybe he doesn't know enough English to actually ask his > question (I'm guessing by the name "Sumant Gupta"). I don't believe you are > native speaker yourself, and I would've expected more sympathy from you. > Lord knows I hope the recipients of any German I write will have some. On the level of mercy: (a) this is python-dev, which is explicitly *not* for user questions; (b) the OP didn't show any actual code nor error messages, which makes it impossible to help him unless you're clairvoyant. Unfortunately we get quite a few of such ill-defined problems in this list, despite it not being the wrong list, and my own patience wears thin at times too. (I also get quite a bit of personal mail of the same nature.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)