From ncoghlan at gmail.com  Fri Jul  1 03:50:53 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 1 Jul 2011 11:50:53 +1000
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <iui5dd$bik$1@dough.gmane.org>
References: <nad-47439B.09372828062011@news.gmane.org>
	<4E0B9B68.6090207@v.loewis.de> <4E0C857C.6040906@netwok.org>
	<20110630153251.751dddfd@snowdog> <iui5dd$bik$1@dough.gmane.org>
Message-ID: <BANLkTikRbj5s9+xiEt52HDz_6HNPJR-uVA@mail.gmail.com>

On Fri, Jul 1, 2011 at 1:40 AM, Tres Seaver <tseaver at palladion.com> wrote:
> On 06/30/2011 10:32 AM, Barry Warsaw wrote:
>> I'm not against adding this to svn, but please be sure these files don't leak
>> into the tarballs should we need to cut another 2.5 or 2.6 release. ?I think
>> that just means tweaking sandbox/release/release.py.
>
> The fact that releases might / will still be made from those branches
> argues against including the file on them at all: ?they are in fact the
> "canonical" repository locations, even if most of the work is done in hg
> and patched into them.

Indeed, 2.5 and 2.6 should be left alone. +1 on adding such a file to
the more recent branches that are handled in hg, though.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From skippy.hammond at gmail.com  Fri Jul  1 04:21:54 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Fri, 01 Jul 2011 12:21:54 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <loom.20110630T140918-870@post.gmane.org>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
Message-ID: <4E0D2F42.4040802@gmail.com>

On 30/06/2011 10:09 PM, Vinay Sajip wrote:
> There's a lot to like in the PEP, and I have some questions relating to the
> latest version:
>
> 1. In the section on shebang line parsing, it says "If the command starts with
> the definition of a customized command followed by a space character, the
> customized command will be used." Sorry if I'm being dense, but what's the
> significance of the trailing space character? In fact, your vpython example in
> the customeised comments section doesn't show a trailing space - you've used
> '#! vpython' as the shebang line.

This is just clumsy wording on my part - what I'm trying to say is 
simply that 'vpython3' will not match a customized command 'vpython'.

> 2. The section on .ini files says that "Commands specified in the [.ini file
> in the] "application directory" [APPDATA] will have precedence over the one
> next to the [launcher] executable." What's the benefit of this?

The idea is that a user without admin permissions can customize the 
behaviour - so the admin can add a .ini file, but the user can override 
any commands in that file with their own definitions.

> If you have
> only one launcher executable of one type (say console, 32-bit) on a system,
> then there's no point in having two locations for .ini files.

The idea is that some users will not have permission to edit the one 
next to the launcher.  I'd be open to considering this yagni if others 
agree.

 > If you have
> multiple copies of the launcher and multiple .ini files next to them, then
> with this precedence order, you can't specialise the behaviour of anything in
> a specific launcher .ini that's also specified in the APPDATA .ini.

The intention is that there only be a single launcher, as only one app 
can be associated with .py files.  OTOH though, file associations can be 
configured per-user IIRC, and assuming that is the case, we could avoid 
my multiple-ini-file usecase above by just allowing a different launcher 
to be registered for a specific user.  This is sounding difficult from 
the UI perspective though (ie, does the installer then need to ask that 
question, will there be a post-install technique for per-user 
registration, etc?)

> Doesn't it
> make more sense to look for a setting in any file next to the launcher, and if
> not found there, look in the APPDATA? That way common things can be defined in
> the APPDATA .ini and overridden for special cases in the launcher-adjacent
> .ini. Or have I got completely the wrong end of the stick?

I came at this from the other angle under the assumption there will only 
be one launcher installed - so common things can be stored in the 
launcher-adjacent file and per-user special cases can be in the per-user 
APPDATA dir.

If the expectation was multiple launchers be installed, then I'd 
probably then prefer to KISS and only have the launcher-adjacent file 
supported at all.

>
> By the way, I've already converted the existing Python reference implementation
> to C (I did it before I saw your response to my post). It basically works in
> that it does what the Python version does, but doesn't include any handling of
> -32 suffixes or .ini files. I can post this on BitBucket if anyone's interested,
> but it's a work in progress. I'm working on some simple tests.

Sure, that would be awesome!  I think that will mean your impl is fairly 
close to the first draft of the PEP I checked into HG, which is nice and 
still quite useful to use :)

Thanks!

Mark

From bernie_bill at rocketmail.com  Fri Jul  1 06:56:05 2011
From: bernie_bill at rocketmail.com (Bernie Bill)
Date: Thu, 30 Jun 2011 21:56:05 -0700 (PDT)
Subject: [Python-Dev] commant
Message-ID: <1309496165.69522.YahooMailNeo@web113320.mail.gq1.yahoo.com>

and thanks!!

lederbill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110630/c1772281/attachment.html>

From ulrich.eckhardt at dominolaser.com  Fri Jul  1 08:51:22 2011
From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt)
Date: Fri, 1 Jul 2011 08:51:22 +0200
Subject: [Python-Dev] time.sleep(-1) behaviour
In-Reply-To: <20110630192145.7C83D2506C1@webabinitio.net>
References: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
	<20110630192145.7C83D2506C1@webabinitio.net>
Message-ID: <201107010851.23193.ulrich.eckhardt@dominolaser.com>

On Thursday 30 June 2011, R. David Murray wrote:
> Please file a bug report at bugs.python.org.

Filed as bug #12459.

Uli

**************************************************************************************
Domino Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at http://www.dominolaser.com
**************************************************************************************
Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden.
E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Domino Laser GmbH ist f?r diese Folgen nicht verantwortlich.
**************************************************************************************


From victor.stinner at haypocalc.com  Fri Jul  1 11:18:11 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 01 Jul 2011 11:18:11 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add
 TemporaryFileTests to the testcase list
In-Reply-To: <E1QcSBk-0005xe-LB@dinsdale.python.org>
References: <E1QcSBk-0005xe-LB@dinsdale.python.org>
Message-ID: <1309511891.3655.2.camel@marge>

I tried to find which commit removes TemporaryFileTests from the
testcase list (to see if there is a good reason to do that, or if it's
just a mistake): it's somewhere between Python 2.x and Python 3.0, but I
didn't find the commit.

Is there a tool to detect that a testcase is never executed to ensure
that there is no other forgiven testcase?

Victor

Le vendredi 01 juillet 2011 ? 03:06 +0200, victor.stinner a ?crit :
> http://hg.python.org/cpython/rev/00a874ad4fc9
> changeset:   71104:00a874ad4fc9
> branch:      3.2
> parent:      71101:7c60c1b41da9
> user:        Victor Stinner <victor.stinner at haypocalc.com>
> date:        Fri Jul 01 02:56:15 2011 +0200
> summary:
>   test_os: add TemporaryFileTests to the testcase list
> 
> The testcase was never executed, it's now fixed.
> 
> files:
>   Lib/test/test_os.py |  1 +
>   1 files changed, 1 insertions(+), 0 deletions(-)
> 
> 
> diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
> --- a/Lib/test/test_os.py
> +++ b/Lib/test/test_os.py
> @@ -1344,6 +1344,7 @@
>          PidTests,
>          LoginTests,
>          LinkTests,
> +        TemporaryFileTests,
>      )



From vinay_sajip at yahoo.co.uk  Fri Jul  1 11:20:18 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 1 Jul 2011 09:20:18 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
Message-ID: <loom.20110701T101646-667@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> The intention is that there only be a single launcher, as only one app 
> can be associated with .py files.  OTOH though, file associations can be 
> configured per-user IIRC, and assuming that is the case, we could avoid 
> my multiple-ini-file usecase above by just allowing a different launcher 
> to be registered for a specific user.  This is sounding difficult from 
> the UI perspective though (ie, does the installer then need to ask that 
> question, will there be a post-install technique for per-user 
> registration, etc?)

I don't like this, for the reasons you state. I think it would be better if the
PEP was changed to say that there is intended to be just one launcher
installation per machine. As the intention is for the launcher to ship with
Python, and there can be multiple Pythons installed, I presume we'll have to
handle this by installing the launcher in some common-to-all-Pythons location
somewhere outside a Python installation location, such as "c:\Program
Files\Python Launcher". This should be stated in the PEP, and we'll also need to
indicate how the launcher will be cleaned up if for some strange reason someone
uninstalls all Pythons from a machine. Then we can just state that there's a
global .ini file (where the launcher is installed) and a local one (in the
user's APPDATA). From that perspective, it makes sense to have items in the
local (APPDATA) override the global (launcher installation location).
 
> Sure, that would be awesome!  I think that will mean your impl is fairly 
> close to the first draft of the PEP I checked into HG, which is nice and 
> still quite useful to use :)

Okay, I'll do this soon - once I've added a few tests. I've updated the
implementation to include help output.

BTW I thought of another thing that perhaps needs handling: what if a customized
command points to the launcher itself? It'd be turtles all the way down :-)

Regards,

Vinay Sajip



From solipsis at pitrou.net  Fri Jul  1 11:24:53 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 1 Jul 2011 11:24:53 +0200
Subject: [Python-Dev] cpython: Issue #12451: Add
	support.create_empty_file()
References: <E1QcOkJ-0006oN-T3@dinsdale.python.org>
Message-ID: <20110701112453.7ad659fa@pitrou.net>

On Thu, 30 Jun 2011 23:25:59 +0200
victor.stinner <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/0c49260e85a0
> changeset:   71103:0c49260e85a0
> user:        Victor Stinner <victor.stinner at haypocalc.com>
> date:        Thu Jun 30 23:25:47 2011 +0200
> summary:
>   Issue #12451: Add support.create_empty_file()
> 
> We don't need to create a temporary buffered binary or text file object just to
> create an empty file.

Is there a reason for this?
I find it quite explicit and obvious what the following does:

    with open("somefile", "wb"):
        pass

so I wonder what replacing it with support.create_empty_file() brings
(except from having to lookup yet another helper function).

Regards

Antoine.



From victor.stinner at haypocalc.com  Fri Jul  1 13:12:07 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 01 Jul 2011 13:12:07 +0200
Subject: [Python-Dev] cpython: Issue #12451: Add
 support.create_empty_file()
In-Reply-To: <20110701112453.7ad659fa@pitrou.net>
References: <E1QcOkJ-0006oN-T3@dinsdale.python.org>
	<20110701112453.7ad659fa@pitrou.net>
Message-ID: <1309518727.3655.10.camel@marge>

Le vendredi 01 juillet 2011 ? 11:24 +0200, Antoine Pitrou a ?crit :
> On Thu, 30 Jun 2011 23:25:59 +0200
> victor.stinner <python-checkins at python.org> wrote:
> > http://hg.python.org/cpython/rev/0c49260e85a0
> > changeset:   71103:0c49260e85a0
> > user:        Victor Stinner <victor.stinner at haypocalc.com>
> > date:        Thu Jun 30 23:25:47 2011 +0200
> > summary:
> >   Issue #12451: Add support.create_empty_file()
> > 
> > We don't need to create a temporary buffered binary or text file object just to
> > create an empty file.
> 
> Is there a reason for this?

The code was correct. I think that a function with an explicit name is
better than the open().close() pattern (which has various flavours, see
the diff). This pattern came sometimes with a comment explaining what it
does (create an empty file), which let me think that it's not easy to
understand what it is supposed to do (if you don't have the comment).

My initial need was to make quiet a warning (of my patched Python, see
#12451) if a file is opened in text mode without an explicit encoding.

> I find it quite explicit and obvious what the following does:
> 
>     with open("somefile", "wb"):
>         pass

For "wb", the only gain is to avoid the creation of temporary FileIO and
BufferedWriter objects, a micro optimisation. Most of the time, "w" mode
was used, so another temporary TextIOWrapper object was also created. I
also saw "w+" mode (use BufferedRandom+TextIOWrapper objects).

Victor


From tlesher at gmail.com  Fri Jul  1 14:17:48 2011
From: tlesher at gmail.com (Tim Lesher)
Date: Fri, 1 Jul 2011 08:17:48 -0400
Subject: [Python-Dev] time.sleep(-1) behaviour
In-Reply-To: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
References: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
Message-ID: <BANLkTimEgOqYz3TcOGr_iKtBt0nu_KXqMw@mail.gmail.com>

On Thu, Jun 30, 2011 at 15:13, Ulrich Eckhardt
<ulrich.eckhardt at dominolaser.com> wrote:
> Hi!
>
> This is a request for clarification for the thread "how to call a function for
> evry 10 seconds" from the user mailinglist/newsgroup.
>
>
> The gist is this:
> 1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError.
> 2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that
> converting this to a 32-bit integer for Sleep() causes an underflow.

> 3. Is the behaviour under MS Windows acceptable or a bug?

On the Windows side, Sleep(-1) as "infinite" is correct and documented:

http://msdn.microsoft.com/en-us/library/ms686298(v=vs.85).aspx


-- 
Tim Lesher <tlesher at gmail.com>

From ncoghlan at gmail.com  Fri Jul  1 14:39:31 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 1 Jul 2011 22:39:31 +1000
Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add
 TemporaryFileTests to the testcase list
In-Reply-To: <1309511891.3655.2.camel@marge>
References: <E1QcSBk-0005xe-LB@dinsdale.python.org>
	<1309511891.3655.2.camel@marge>
Message-ID: <BANLkTikJ3OVmE1ZAEkfteT68kbZwG5pZZA@mail.gmail.com>

On Fri, Jul 1, 2011 at 7:18 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> I tried to find which commit removes TemporaryFileTests from the
> testcase list (to see if there is a good reason to do that, or if it's
> just a mistake): it's somewhere between Python 2.x and Python 3.0, but I
> didn't find the commit.
>
> Is there a tool to detect that a testcase is never executed to ensure
> that there is no other forgiven testcase?

Coverage data for the test suite itself.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Fri Jul  1 14:46:16 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 1 Jul 2011 22:46:16 +1000
Subject: [Python-Dev] time.sleep(-1) behaviour
In-Reply-To: <BANLkTimEgOqYz3TcOGr_iKtBt0nu_KXqMw@mail.gmail.com>
References: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
	<BANLkTimEgOqYz3TcOGr_iKtBt0nu_KXqMw@mail.gmail.com>
Message-ID: <BANLkTi=NFWqTsB=U3Cd8=gOfN6i3yRrO0g@mail.gmail.com>

On Fri, Jul 1, 2011 at 10:17 PM, Tim Lesher <tlesher at gmail.com> wrote:
> On Thu, Jun 30, 2011 at 15:13, Ulrich Eckhardt
> <ulrich.eckhardt at dominolaser.com> wrote:
>> Hi!
>>
>> This is a request for clarification for the thread "how to call a function for
>> evry 10 seconds" from the user mailinglist/newsgroup.
>>
>>
>> The gist is this:
>> 1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError.
>> 2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that
>> converting this to a 32-bit integer for Sleep() causes an underflow.
>
>> 3. Is the behaviour under MS Windows acceptable or a bug?
>
> On the Windows side, Sleep(-1) as "infinite" is correct and documented:
>
> http://msdn.microsoft.com/en-us/library/ms686298(v=vs.85).aspx

For Sleep, yes, but the time.sleep() docs [1] are completely silent on
the behaviour of negative sleep values at the Python level. Question 3
isn't "Is there something wrong with Sleep()?", it is "Is there
something wrong with the way time.sleep() *uses* Sleep()?"

My personal preference would be to standardise on ValueError (since
the negative sleep value is the real problem), or, failing that, at
least raising an exception of some kind.

[1] http://docs.python.org/dev/library/time#time.sleep

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Fri Jul  1 14:48:19 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 01 Jul 2011 14:48:19 +0200
Subject: [Python-Dev] time.sleep(-1) behaviour
In-Reply-To: <BANLkTimEgOqYz3TcOGr_iKtBt0nu_KXqMw@mail.gmail.com>
References: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
	<BANLkTimEgOqYz3TcOGr_iKtBt0nu_KXqMw@mail.gmail.com>
Message-ID: <1309524499.7942.5.camel@marge>

Le vendredi 01 juillet 2011 ? 08:17 -0400, Tim Lesher a ?crit :
> On Thu, Jun 30, 2011 at 15:13, Ulrich Eckhardt
> <ulrich.eckhardt at dominolaser.com> wrote:
> > Hi!
> >
> > This is a request for clarification for the thread "how to call a function for
> > evry 10 seconds" from the user mailinglist/newsgroup.
> >
> >
> > The gist is this:
> > 1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError.
> > 2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that
> > converting this to a 32-bit integer for Sleep() causes an underflow.
> 
> > 3. Is the behaviour under MS Windows acceptable or a bug?
> 
> On the Windows side, Sleep(-1) as "infinite" is correct and documented:
> 
> http://msdn.microsoft.com/en-us/library/ms686298(v=vs.85).aspx

I answered on the bug tracker:
http://bugs.python.org/issue12459

Victor



From rosslagerwall at gmail.com  Fri Jul  1 14:55:39 2011
From: rosslagerwall at gmail.com (Ross Lagerwall)
Date: Fri, 01 Jul 2011 14:55:39 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add
 TemporaryFileTests to the testcase list
In-Reply-To: <1309511891.3655.2.camel@marge>
References: <E1QcSBk-0005xe-LB@dinsdale.python.org>
	<1309511891.3655.2.camel@marge>
Message-ID: <1309524939.1473.4.camel@hobo>

On Fri, 2011-07-01 at 11:18 +0200, Victor Stinner wrote:
> I tried to find which commit removes TemporaryFileTests from the
> testcase list (to see if there is a good reason to do that, or if it's
> just a mistake): it's somewhere between Python 2.x and Python 3.0, but I
> didn't find the commit.

For what it's worth,

$ hg grep --all TemporaryFileTests Lib/test/test_os.py
Lib/test/test_os.py:45773:+:class TemporaryFileTests(unittest.TestCase):
Lib/test/test_os.py:43680:-:class TemporaryFileTests(unittest.TestCase):
Lib/test/test_os.py:43680:-:        TemporaryFileTests,


will show where TemporaryFileTests was removed and added.


From tlesher at gmail.com  Fri Jul  1 15:23:17 2011
From: tlesher at gmail.com (Tim Lesher)
Date: Fri, 1 Jul 2011 09:23:17 -0400
Subject: [Python-Dev] time.sleep(-1) behaviour
In-Reply-To: <BANLkTi=NFWqTsB=U3Cd8=gOfN6i3yRrO0g@mail.gmail.com>
References: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
	<BANLkTimEgOqYz3TcOGr_iKtBt0nu_KXqMw@mail.gmail.com>
	<BANLkTi=NFWqTsB=U3Cd8=gOfN6i3yRrO0g@mail.gmail.com>
Message-ID: <BANLkTikFqnH4cpcJsY3Y88Lk4CGRvgAkeg@mail.gmail.com>

On Fri, Jul 1, 2011 at 08:46, Nick Coghlan <ncoghlan at gmail.com> wrote:
> For Sleep, yes, but the time.sleep() docs [1] are completely silent on
> the behaviour of negative sleep values at the Python level. Question 3
> isn't "Is there something wrong with Sleep()?", it is "Is there
> something wrong with the way time.sleep() *uses* Sleep()?"

That makes sense. Better to be consistent within the time API--I know
the different semantics of time.clock() have confused people around
here.

-- 
Tim Lesher <tlesher at gmail.com>

From victor.stinner at haypocalc.com  Fri Jul  1 15:34:55 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 01 Jul 2011 15:34:55 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.2): test_os: add
 TemporaryFileTests to the testcase list
In-Reply-To: <1309524939.1473.4.camel@hobo>
References: <E1QcSBk-0005xe-LB@dinsdale.python.org>
	<1309511891.3655.2.camel@marge>  <1309524939.1473.4.camel@hobo>
Message-ID: <1309527295.7942.11.camel@marge>

Le vendredi 01 juillet 2011 ? 14:55 +0200, Ross Lagerwall a ?crit :
> On Fri, 2011-07-01 at 11:18 +0200, Victor Stinner wrote:
> > I tried to find which commit removes TemporaryFileTests from the
> > testcase list (to see if there is a good reason to do that, or if it's
> > just a mistake): it's somewhere between Python 2.x and Python 3.0, but I
> > didn't find the commit.
> 
> For what it's worth,
> 
> $ hg grep --all TemporaryFileTests Lib/test/test_os.py
> Lib/test/test_os.py:45773:+:class TemporaryFileTests(unittest.TestCase):
> Lib/test/test_os.py:43680:-:class TemporaryFileTests(unittest.TestCase):
> Lib/test/test_os.py:43680:-:        TemporaryFileTests,
> 
> 
> will show where TemporaryFileTests was removed and added.

It was added in the middle of an horrible trunk->3.x merge: "Merged
revisions
61239-61249,61252-61257,61260-61264,61269-61275,61278-61279,61285-61286,61288-61290,61298,61303-61305,
svn+ssh://pythondev at svn.python.org/python/trunk"

To find which commit removed a function in a file, hg bisect can be used
(trick given on #mercurial):

PREVREV=$(hg id); hg bisect -r; hg bisect -g 0; hg bisect -c 'grep -q
tmpnam Modules/posixmodule.c'; hg update $PREVREV

Even if it searchs in ~70.000 commits, it takes less than a second to
find the commit which removed tmpnam in Python 3:
------------
changeset:   43680:a8818ffe24d1
parent:      43673:d66018ed3ded
user:        Guido van Rossum <guido at python.org>
date:        Thu Oct 25 23:18:51 2007 +0000
files:       Doc/library/os.rst Lib/test/test_os.py
Lib/test/test_posix.py Misc/NEWS Modules/posixmodule.c
description:
Patch 1318 by Christian Heimes: remove os.tmpnam(), os.tempnam(),
and os.tmpfile().
------------

Note: I did a new commit on test_os in Python 3.3 to remove
TemporaryFileTests (again): most tests were useless because os.tmpnam(),
os.tempnam() and os.tmpfile() don't exist anymore in Python 3.3. I moved
remaining useful tests to another testcase.

Victor


From driedel at cox.net  Fri Jul  1 16:01:36 2011
From: driedel at cox.net (David P. Riedel)
Date: Fri, 01 Jul 2011 10:01:36 -0400
Subject: [Python-Dev] What happened to Python 3.2.1?
Message-ID: <iukk00$rar$2@dough.gmane.org>

Hi

Python 3.2.1 was scheduled to be released on 6/19, I believe but there 
is no mention of it anywhere.  Has it been delayed?

Thanks.


From brian.curtin at gmail.com  Fri Jul  1 16:38:04 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Fri, 1 Jul 2011 09:38:04 -0500
Subject: [Python-Dev] What happened to Python 3.2.1?
In-Reply-To: <iukk00$rar$2@dough.gmane.org>
References: <iukk00$rar$2@dough.gmane.org>
Message-ID: <BANLkTimpsHVwkR9ScBk+FiP1i3QvVHLorA@mail.gmail.com>

On Fri, Jul 1, 2011 at 09:01, David P. Riedel <driedel at cox.net> wrote:

> Hi
>
> Python 3.2.1 was scheduled to be released on 6/19, I believe but there is
> no mention of it anywhere.  Has it been delayed?
>
> Thanks.


There are two remaining blockers for the release:
http://bugs.python.org/issue12346 and http://bugs.python.org/issue12291
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110701/11a829ea/attachment.html>

From solipsis at pitrou.net  Fri Jul  1 16:43:22 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 1 Jul 2011 16:43:22 +0200
Subject: [Python-Dev] What happened to Python 3.2.1?
References: <iukk00$rar$2@dough.gmane.org>
	<BANLkTimpsHVwkR9ScBk+FiP1i3QvVHLorA@mail.gmail.com>
Message-ID: <20110701164322.72c6f105@pitrou.net>

On Fri, 1 Jul 2011 09:38:04 -0500
Brian Curtin <brian.curtin at gmail.com> wrote:
> On Fri, Jul 1, 2011 at 09:01, David P. Riedel <driedel at cox.net> wrote:
> 
> > Hi
> >
> > Python 3.2.1 was scheduled to be released on 6/19, I believe but there is
> > no mention of it anywhere.  Has it been delayed?
> >
> > Thanks.
> 
> 
> There are two remaining blockers for the release:
> http://bugs.python.org/issue12346 and http://bugs.python.org/issue12291

Perhaps http://bugs.python.org/issue12213 should also be a blocker?



From status at bugs.python.org  Fri Jul  1 18:07:25 2011
From: status at bugs.python.org (Python tracker)
Date: Fri,  1 Jul 2011 18:07:25 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110701160725.65B1D1CCB4@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-06-24 - 2011-07-01)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2850 ( +5)
  closed 21399 (+64)
  total  24249 (+69)

Open issues with patches: 1233 


Issues opened (50)
==================

#5572: packaging should respect the LIBS configure env var
http://bugs.python.org/issue5572  reopened by eric.araujo

#11363: Curses - add missing functions to doc
http://bugs.python.org/issue11363  reopened by eric.araujo

#12401: unset PYTHON* environment variables when running tests
http://bugs.python.org/issue12401  opened by henry.precheur

#12403: Mention sys.displayhook in code module docs and the compile bu
http://bugs.python.org/issue12403  opened by tebeka

#12405: packaging does not record/remove directories it creates
http://bugs.python.org/issue12405  opened by vinay.sajip

#12406: msi.py needs updating for Python 3.3
http://bugs.python.org/issue12406  opened by vinay.sajip

#12410: Create a new helper function that enable to test that an opera
http://bugs.python.org/issue12410  opened by mouad

#12411: cgi.parse_multipart is broken on 3.x
http://bugs.python.org/issue12411  opened by jonas.wagner

#12412: non defined representation for pwd.struct_passwd
http://bugs.python.org/issue12412  opened by fgarciar

#12413: make faulthandler dump traceback of child processes
http://bugs.python.org/issue12413  opened by neologix

#12414: getsizeof() on code objects is wrong
http://bugs.python.org/issue12414  opened by benjamin.peterson

#12415: Missing: How to checkout the Doc sources
http://bugs.python.org/issue12415  opened by philip

#12416: packaging does not have hooks callable during distribution rem
http://bugs.python.org/issue12416  opened by vinay.sajip

#12418: python should inherit the library search path from the compile
http://bugs.python.org/issue12418  opened by vorlon

#12420: distutils tests fail if PATH is not defined
http://bugs.python.org/issue12420  opened by henry.precheur

#12423: signal handler doesn't handle SIGABRT from os.abort
http://bugs.python.org/issue12423  opened by kisielk

#12424: distutils2: extension section uses bad environment marker sepa
http://bugs.python.org/issue12424  opened by eli.collins

#12425: gettext breaks on empty plural-forms value
http://bugs.python.org/issue12425  opened by djc

#12426: packaging.tests.test_command_install_dist.InstallTestCase fail
http://bugs.python.org/issue12426  opened by haypo

#12427: packaging register fails because "POST data should be bytes"
http://bugs.python.org/issue12427  opened by vinay.sajip

#12428: functools test coverage
http://bugs.python.org/issue12428  opened by Thorney

#12429: test_io.check_interrupted_write() sporadic failures on FreeBSD
http://bugs.python.org/issue12429  opened by haypo

#12432: remove a bunch of unused imports in Lib
http://bugs.python.org/issue12432  opened by vincele

#12434: Strengthen 2.7 io types warning
http://bugs.python.org/issue12434  opened by terry.reedy

#12436: Provide reference to detailed installation instructions
http://bugs.python.org/issue12436  opened by ncoghlan

#12437: _ctypes.dlopen does not include errno in OSError
http://bugs.python.org/issue12437  opened by anacrolix

#12438: IDLE problem displaying warning message
http://bugs.python.org/issue12438  opened by JBernardo

#12439: BaseHTTPServer's send_reponse adds extra "\r\n" when using HTT
http://bugs.python.org/issue12439  opened by Yoav.Weiss

#12440: test_ssl.test_options() failure on Snow Leopard: can't clear o
http://bugs.python.org/issue12440  opened by haypo

#12441: _GLOBAL_DEFAULT_TIMEOUT remains as an object() in HTTPConnecti
http://bugs.python.org/issue12441  opened by juanjux

#12445: dict view values objects are missing tp_richcmp and tp_as_numb
http://bugs.python.org/issue12445  opened by Julian

#12446: StreamReader Readlines behavior odd
http://bugs.python.org/issue12446  opened by Thomas.Barnet-Lamb

#12448: smtplib's __main__ doesn't flush when prompting
http://bugs.python.org/issue12448  opened by anacrolix

#12449: Add accelerator "F" to button "Finish" in all MSI installers m
http://bugs.python.org/issue12449  opened by cool-RR

#12451: open: avoid the locale encoding when possible
http://bugs.python.org/issue12451  opened by haypo

#12452: reuse plistlib in sysconfig; deprecation process in plistlib
http://bugs.python.org/issue12452  opened by haypo

#12453: test_import.test_failing_import_sticks() sporadic failures
http://bugs.python.org/issue12453  opened by haypo

#12454: mailbox: use ASCII to read/write .mh_sequences files
http://bugs.python.org/issue12454  opened by haypo

#12455: urllib2 forces title() on header names, breaking some requests
http://bugs.python.org/issue12455  opened by Cal.Leeming

#12456: Hangs in concurrent.futures
http://bugs.python.org/issue12456  opened by rosslagerwall

#12457: type() returns incorrect type for nested classes
http://bugs.python.org/issue12457  opened by pwil3058

#12458: Tracebacks should contain the first line of continuation lines
http://bugs.python.org/issue12458  opened by psss

#12459: time.sleep(-1.0) behaviour
http://bugs.python.org/issue12459  opened by eckhardt

#12460: SocketServer.shutdown() does not have "timeout=None" parameter
http://bugs.python.org/issue12460  opened by mmarkk

#12461: it's not clear how the shutil.copystat() should work on symlin
http://bugs.python.org/issue12461  opened by mmarkk

#12463: Calling SocketServer.shutdown() when server_forever() was not 
http://bugs.python.org/issue12463  opened by mmarkk

#12464: tempfile.TemporaryDirectory.cleanup follows symbolic links
http://bugs.python.org/issue12464  opened by abacabadabacaba

#12465: gc.get_referents can be used to crash Python
http://bugs.python.org/issue12465  opened by abacabadabacaba

#12466: test_subprocess.test_close_fds() sporadic failures on Mac OS X
http://bugs.python.org/issue12466  opened by haypo

#12467: test_threading.test_6_daemon_threads(): Assertion failed: PyUn
http://bugs.python.org/issue12467  opened by haypo



Most recent 15 issues with no replies (15)
==========================================

#12467: test_threading.test_6_daemon_threads(): Assertion failed: PyUn
http://bugs.python.org/issue12467

#12466: test_subprocess.test_close_fds() sporadic failures on Mac OS X
http://bugs.python.org/issue12466

#12465: gc.get_referents can be used to crash Python
http://bugs.python.org/issue12465

#12464: tempfile.TemporaryDirectory.cleanup follows symbolic links
http://bugs.python.org/issue12464

#12463: Calling SocketServer.shutdown() when server_forever() was not 
http://bugs.python.org/issue12463

#12461: it's not clear how the shutil.copystat() should work on symlin
http://bugs.python.org/issue12461

#12460: SocketServer.shutdown() does not have "timeout=None" parameter
http://bugs.python.org/issue12460

#12458: Tracebacks should contain the first line of continuation lines
http://bugs.python.org/issue12458

#12454: mailbox: use ASCII to read/write .mh_sequences files
http://bugs.python.org/issue12454

#12453: test_import.test_failing_import_sticks() sporadic failures
http://bugs.python.org/issue12453

#12452: reuse plistlib in sysconfig; deprecation process in plistlib
http://bugs.python.org/issue12452

#12448: smtplib's __main__ doesn't flush when prompting
http://bugs.python.org/issue12448

#12439: BaseHTTPServer's send_reponse adds extra "\r\n" when using HTT
http://bugs.python.org/issue12439

#12436: Provide reference to detailed installation instructions
http://bugs.python.org/issue12436

#12434: Strengthen 2.7 io types warning
http://bugs.python.org/issue12434



Most recent 15 issues waiting for review (15)
=============================================

#12459: time.sleep(-1.0) behaviour
http://bugs.python.org/issue12459

#12454: mailbox: use ASCII to read/write .mh_sequences files
http://bugs.python.org/issue12454

#12452: reuse plistlib in sysconfig; deprecation process in plistlib
http://bugs.python.org/issue12452

#12451: open: avoid the locale encoding when possible
http://bugs.python.org/issue12451

#12445: dict view values objects are missing tp_richcmp and tp_as_numb
http://bugs.python.org/issue12445

#12432: remove a bunch of unused imports in Lib
http://bugs.python.org/issue12432

#12429: test_io.check_interrupted_write() sporadic failures on FreeBSD
http://bugs.python.org/issue12429

#12428: functools test coverage
http://bugs.python.org/issue12428

#12425: gettext breaks on empty plural-forms value
http://bugs.python.org/issue12425

#12424: distutils2: extension section uses bad environment marker sepa
http://bugs.python.org/issue12424

#12420: distutils tests fail if PATH is not defined
http://bugs.python.org/issue12420

#12411: cgi.parse_multipart is broken on 3.x
http://bugs.python.org/issue12411

#12410: Create a new helper function that enable to test that an opera
http://bugs.python.org/issue12410

#12406: msi.py needs updating for Python 3.3
http://bugs.python.org/issue12406

#12401: unset PYTHON* environment variables when running tests
http://bugs.python.org/issue12401



Top 10 most discussed issues (10)
=================================

#12398: Sending binary data with a POST request in httplib can cause U
http://bugs.python.org/issue12398  14 msgs

#6721: Locks in python standard library should be sanitized on fork
http://bugs.python.org/issue6721  13 msgs

#10181: Problems with Py_buffer management in memoryobject.c (and else
http://bugs.python.org/issue10181  13 msgs

#12352: multiprocessing.Value() hangs
http://bugs.python.org/issue12352  12 msgs

#12455: urllib2 forces title() on header names, breaking some requests
http://bugs.python.org/issue12455  11 msgs

#8716: test_tk/test_tkk_guionly fails on OS X if run from buildbot sl
http://bugs.python.org/issue8716  10 msgs

#12291: file written using marshal in 3.2 can be read by 2.7, but not 
http://bugs.python.org/issue12291  10 msgs

#2193: Cookie Colon Name Bug
http://bugs.python.org/issue2193   9 msgs

#11870: test_3_join_in_forked_from_thread() of test_threading hangs 1 
http://bugs.python.org/issue11870   9 msgs

#12451: open: avoid the locale encoding when possible
http://bugs.python.org/issue12451   9 msgs



Issues closed (57)
==================

#3435: 3rd party program calls trace.py on non Python files
http://bugs.python.org/issue3435  closed by terry.reedy

#5114: 2.7:  test_threading hangs on Solaris
http://bugs.python.org/issue5114  closed by neologix

#5375: Unified locals/consts array + register-based instructions
http://bugs.python.org/issue5375  closed by pitrou

#8661: FAQ item 2.25 is unclear
http://bugs.python.org/issue8661  closed by brett.cannon

#8746: os.chflags() and os.lchflags() are not built when they should 
http://bugs.python.org/issue8746  closed by ned.deily

#9955: multiprocessing.Pipe segmentation fault when recv of unpicklab
http://bugs.python.org/issue9955  closed by neologix

#10206: python program starting with unmatched quote spews spaces to s
http://bugs.python.org/issue10206  closed by r.david.murray

#10326: Can't pickle unittest.TestCase instances
http://bugs.python.org/issue10326  closed by rhettinger

#10510: packaging upload/register should use CRLF in HTTP requests
http://bugs.python.org/issue10510  closed by eric.araujo

#10736: test_ttk_guionly fails on OS X using ActiveState Tcl 8.5.9 (Co
http://bugs.python.org/issue10736  closed by ned.deily

#10845: test_multiprocessing failure under Windows
http://bugs.python.org/issue10845  closed by ncoghlan

#11163: iter() documentation code doesn't work
http://bugs.python.org/issue11163  closed by rhettinger

#11197: information leakage with SimpleHTTPServer
http://bugs.python.org/issue11197  closed by orsenthil

#11302: Add more tests to test_ast.py
http://bugs.python.org/issue11302  closed by python-dev

#11469: Fix resource warning in test_trailers
http://bugs.python.org/issue11469  closed by sandro.tosi

#11493: Add python.exe-gdb.py to .hgignore
http://bugs.python.org/issue11493  closed by sandro.tosi

#11568: docstring of select.epoll.register() is wrong
http://bugs.python.org/issue11568  closed by python-dev

#11669: Clarify Lang Ref "Compound statements" footnote
http://bugs.python.org/issue11669  closed by ezio.melotti

#11758: increase xml.dom.minidom test coverage
http://bugs.python.org/issue11758  closed by rhettinger

#11802: filecmp.cmp needs a documented way to clear cache
http://bugs.python.org/issue11802  closed by rhettinger

#11889: 'enumerate' 'start' parameter documentation is confusing
http://bugs.python.org/issue11889  closed by rhettinger

#12086: Tutorial doesn't discourage name mangling
http://bugs.python.org/issue12086  closed by rhettinger

#12134: json.dump much slower than dumps
http://bugs.python.org/issue12134  closed by rhettinger

#12139: Add CCC command support to ftplib
http://bugs.python.org/issue12139  closed by giampaolo.rodola

#12141: sysconfig.get_config_vars('srcdir') fails in specific cases
http://bugs.python.org/issue12141  closed by ned.deily

#12164: str.translate docstring doesn't mention that 'table' can be No
http://bugs.python.org/issue12164  closed by mark.dickinson

#12172: IDLE crashes when I use F5 to run
http://bugs.python.org/issue12172  closed by ned.deily

#12228: Stat doc - fix swap of UF_OPAQUE and UF_NOUNLINK description
http://bugs.python.org/issue12228  closed by mark.dickinson

#12303: expose sigwaitinfo() and sigtimedwait() in the signal module
http://bugs.python.org/issue12303  closed by rosslagerwall

#12341: Some additions to .hgignore
http://bugs.python.org/issue12341  closed by ezio.melotti

#12379: build outside source fail in head
http://bugs.python.org/issue12379  closed by ned.deily

#12385: the help for bytearray.maketrans describes bytes.maketrans
http://bugs.python.org/issue12385  closed by python-dev

#12392: pthread_kill() doesn't work on the main thread on FreeBSD6
http://bugs.python.org/issue12392  closed by haypo

#12399: simplify cell var initialization by storing constant data on t
http://bugs.python.org/issue12399  closed by python-dev

#12400: regrtest: always run tests in verbose mode, but hide the outpu
http://bugs.python.org/issue12400  closed by haypo

#12402: Overriding code.InteractiveConsole.write does not work
http://bugs.python.org/issue12402  closed by r.david.murray

#12404: c99 code in mmapmodule
http://bugs.python.org/issue12404  closed by rosslagerwall

#12407: test_subinterps fails on Windows
http://bugs.python.org/issue12407  closed by pitrou

#12408: Relative import used on test_future5
http://bugs.python.org/issue12408  closed by mark.dickinson

#12409: Moving "Documenting Python" to Devguide
http://bugs.python.org/issue12409  closed by rhettinger

#12417: Inappropriate copyright on profile files
http://bugs.python.org/issue12417  closed by python-dev

#12419: Add ident parameter to SysLogHandler
http://bugs.python.org/issue12419  closed by python-dev

#12421: Use PYTHON when calling Parser/asdl_c.py
http://bugs.python.org/issue12421  closed by r.david.murray

#12422: When deepcopying, don't store immutable objects in the memo di
http://bugs.python.org/issue12422  closed by python-dev

#12430: Pip fails to fetch from mirror if PyPi checksum times out
http://bugs.python.org/issue12430  closed by skrah

#12431: urllib2.Request.get_full_url() broken in newer versions of Pyt
http://bugs.python.org/issue12431  closed by orsenthil

#12433: make clean doesn't clean up static libraries on 2.x
http://bugs.python.org/issue12433  closed by ned.deily

#12435: Input function does not strip trailing '\r' from string input
http://bugs.python.org/issue12435  closed by amaury.forgeotdarc

#12442: shutil.disk_usage()
http://bugs.python.org/issue12442  closed by giampaolo.rodola

#12443: locale.setlocale(locale.LC_ALL, locale.getlocale()) fails for 
http://bugs.python.org/issue12443  closed by haypo

#12444: accept sets or collections for str.strip/lstrip/rstrip
http://bugs.python.org/issue12444  closed by rhettinger

#12447: ~True is not False
http://bugs.python.org/issue12447  closed by amaury.forgeotdarc

#12450: Use the Grisu algorithms to convert floats to strings
http://bugs.python.org/issue12450  closed by rhettinger

#12462: time.sleep(1): call PyErr_CheckSignals() if the sleep was inte
http://bugs.python.org/issue12462  closed by haypo

#1067702: urllib fails with multiple ftp transfers
http://bugs.python.org/issue1067702  closed by python-dev

#5722: settimer / gettimer functionality on FreeBSD 6.3 (not 7.x)
http://bugs.python.org/issue5722  closed by terry.reedy

#1669349: make install fails if no previous Python installation
http://bugs.python.org/issue1669349  closed by terry.reedy

From vinay_sajip at yahoo.co.uk  Fri Jul  1 18:17:03 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 1 Jul 2011 16:17:03 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
Message-ID: <loom.20110701T180521-738@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> Sure, that would be awesome!  I think that will mean your impl is fairly 
> close to the first draft of the PEP I checked into HG, which is nice and 
> still quite useful to use :)

My C implementation of the launcher is now available at

https://bitbucket.org/vinay.sajip/pylauncher

It's been built using VS 2008, and tested on Windows 7 (32-bit) with
ActivePython 2.6.2.2 and ActivePython 3.2.0.0 installed.

I've added support for .ini files [commands] section; I haven't looked at
implementing [defaults] yet.

There's incomplete support for -32 suffix at the moment - that can be looked at
in due course.

A couple of points:

I've also allowed for /usr/local/bin/python as a built-in command, this can
always be removed later if YAGNI.

I've assumed that if a customised command is provided with arguments in the
shebang line, these will be ignored - if people want to run with different
options they can always define more customised commands. If you agree with this,
the PEP should probably explicitly state this.

In a couple of cases I've implemented using fixed size arrays - for the lists of
installed Pythons and customised commands. Of course these can be made dynamic,
but what's there is good enough for the moment for exploration.

Do have a play, and let me know what you think.

Regards,

Vinay Sajip


From victor.stinner at haypocalc.com  Fri Jul  1 18:25:33 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 01 Jul 2011 18:25:33 +0200
Subject: [Python-Dev] What happened to Python 3.2.1?
In-Reply-To: <20110701164322.72c6f105@pitrou.net>
References: <iukk00$rar$2@dough.gmane.org>
	<BANLkTimpsHVwkR9ScBk+FiP1i3QvVHLorA@mail.gmail.com>
	<20110701164322.72c6f105@pitrou.net>
Message-ID: <1309537533.13804.2.camel@marge>

Le vendredi 01 juillet 2011 ? 16:43 +0200, Antoine Pitrou a ?crit :
> On Fri, 1 Jul 2011 09:38:04 -0500
> Brian Curtin <brian.curtin at gmail.com> wrote:
> > On Fri, Jul 1, 2011 at 09:01, David P. Riedel <driedel at cox.net> wrote:
> > 
> > > Hi
> > >
> > > Python 3.2.1 was scheduled to be released on 6/19, I believe but there is
> > > no mention of it anywhere.  Has it been delayed?
> > >
> > > Thanks.
> > 
> > 
> > There are two remaining blockers for the release:
> > http://bugs.python.org/issue12346 and http://bugs.python.org/issue12291
> 
> Perhaps http://bugs.python.org/issue12213 should also be a blocker?

If you care of interlaced read-write, you may also see
http://bugs.python.org/issue12215

I don't think that #12213 and #12215 are blocker. Nobody noticed it
since the introduction of the io module (ok, except me), it's not a
regression.

Python 3.2.1 contains fixes of regressions, users are waiting for them
(e.g. "\r" in the Windows console).

Can't we schedule another release later to fix #12213 and #12215?

Victor


From victor.stinner at haypocalc.com  Fri Jul  1 20:46:10 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 01 Jul 2011 20:46:10 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.2): #11873: fix test
 regex so it covers windows os.sep as well.
In-Reply-To: <E1Qcg2c-0000uL-OA@dinsdale.python.org>
References: <E1Qcg2c-0000uL-OA@dinsdale.python.org>
Message-ID: <1309545970.13804.5.camel@marge>

Le vendredi 01 juillet 2011 ? 17:54 +0200, r.david.murray a ?crit :
> http://hg.python.org/cpython/rev/f8ece8c93918
> changeset:   71119:f8ece8c93918
> branch:      3.2
> parent:      71117:3f30cfe51315
> user:        R David Murray <rdmurray at bitdance.com>
> date:        Fri Jul 01 11:51:50 2011 -0400
> summary:
>   #11873: fix test regex so it covers windows os.sep as well.
> 
> files:
>   Lib/test/test_compileall.py |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Lib/test/test_compileall.py b/Lib/test/test_compileall.py
> --- a/Lib/test/test_compileall.py
> +++ b/Lib/test/test_compileall.py
> @@ -248,7 +248,7 @@
>          self.assertEqual(b'', quiet)
>  
>      def test_regexp(self):
> -        self.assertRunOK('-q', '-x', 'ba[^\/]*$', self.pkgdir)
> +        self.assertRunOK('-q', '-x', r'ba[^\/]*$', self.pkgdir)
>          self.assertNotCompiled(self.barfn)
>          self.assertCompiled(self.initfn)

I'm not sure that your regex is correct. You may have to use a double
slash or it is interpreted by the regex :-/

>>> import re
>>> re.match(r'ba[^\/]*$', 'babar')
<_sre.SRE_Match object at 0x7f420559a7e8>
>>> re.match(r'ba[^\/]*$', 'babar/')
>>> re.match(r'ba[^\/]*$', 'babar\\a')
<_sre.SRE_Match object at 0x7f420559a850>

Correct regex:

>>> re.match(r'ba[^\\/]*$', 'baba\\')
>>> re.match(r'ba[^\\/]*$', 'baba/')

Victor


From skippy.hammond at gmail.com  Sat Jul  2 09:16:07 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sat, 02 Jul 2011 17:16:07 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <loom.20110701T101646-667@post.gmane.org>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
Message-ID: <4E0EC5B7.9010607@gmail.com>

On 1/07/2011 7:20 PM, Vinay Sajip wrote:
> Mark Hammond<skippy.hammond<at>  gmail.com>  writes:
>
>> The intention is that there only be a single launcher, as only one app
>> can be associated with .py files.  OTOH though, file associations can be
>> configured per-user IIRC, and assuming that is the case, we could avoid
>> my multiple-ini-file usecase above by just allowing a different launcher
>> to be registered for a specific user.  This is sounding difficult from
>> the UI perspective though (ie, does the installer then need to ask that
>> question, will there be a post-install technique for per-user
>> registration, etc?)
>
> I don't like this, for the reasons you state. I think it would be better if the
> PEP was changed to say that there is intended to be just one launcher
> installation per machine. As the intention is for the launcher to ship with
> Python, and there can be multiple Pythons installed, I presume we'll have to
> handle this by installing the launcher in some common-to-all-Pythons location
> somewhere outside a Python installation location, such as "c:\Program
> Files\Python Launcher". This should be stated in the PEP, and we'll also need to
> indicate how the launcher will be cleaned up if for some strange reason someone
> uninstalls all Pythons from a machine. Then we can just state that there's a
> global .ini file (where the launcher is installed) and a local one (in the
> user's APPDATA). From that perspective, it makes sense to have items in the
> local (APPDATA) override the global (launcher installation location).

The PEP does say "if possible, should be installed somewhere likely to 
already be on the system PATH (eg., the Windows System32) directory." 
It is silent about what to do when that isn't possible, but I'd think it 
OK if the launcher was installed directly in the Python directory - IOW, 
I'd think it OK if the PEP said "should be installed next to the 
PythonXX.dll being installed" - but an important point in the above 
working is the "already be on the system PATH" - ie, I don't really want 
it put in a newly created directory unless the installer also adds that 
directory to the PATH - and what to do on uninstall then becomes an issue.

One problem with all of this is uninstallation and specifically if the 
user is uninstalling the most recent Python installation while leaving 
earlier ones.  I guess there are 2 general answers to this:

* The installer process could remember the previous association and 
restore that on uninstall.

* We treat this as a "wont-fix" and document a work-around of asking the 
user to reinstall the previous version to restore the file association.

We probably need to be mindful of adding too much extra work for the 
installer process as that may well end up blocking us on getting this 
into the next appropriate release.  In particular, Martin's thoughts 
here would be very useful.


This would force the user to reinstall that older one to re-establish 
the associations correctly

> BTW I thought of another thing that perhaps needs handling: what if a customized
> command points to the launcher itself? It'd be turtles all the way down :-)

Yeah - I wonder if we can leverage the "job" api here and refuse to 
start if there are already 2 processes in the job?  OTOH, that is tricky 
as it would also prevent someone using os.startfile with a .py file....

 From your second mail:

> I've assumed that if a customised command is provided with arguments in the
> shebang line, these will be ignored - if people want to run with different
> options they can always define more customised commands. If you agree with this,
> the PEP should probably explicitly state this.

I'm not too bothered to be honest - the customized commands exist purely 
for alternative implementations, so my initial thoughts are that 
additional args would be as useful for them as they are for cpython 
invocations.  IOW, if they don't need it, then CPython invocations don't 
need it either, so maybe it can be dropped completely?

> In a couple of cases I've implemented using fixed size arrays - for the lists of
> installed Pythons and customised commands. Of course these can be made dynamic,
> but what's there is good enough for the moment for exploration.

Sure - I think there is some policy that a Python version number will 
never be > 10, so that sounds fine to me.  So long as the launcher 
doesn't blindly run off the end of such arrays I think it is fine - 
limitations can be addressed in later versions.

It will be a few days until I can look at the implementation, but I'm 
very happy to see it started.  Given it is now ahead of the Python 
reference impl, I wonder if we should just drop all wording about that 
reference impl and just treat the C impl as canonical?

Cheers,

Mark

From vinay_sajip at yahoo.co.uk  Sat Jul  2 11:08:50 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 2 Jul 2011 09:08:50 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
Message-ID: <loom.20110702T102523-59@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> The PEP does say "if possible, should be installed somewhere likely to 
> already be on the system PATH (eg., the Windows System32) directory." 
> It is silent about what to do when that isn't possible, but I'd think it 
> OK if the launcher was installed directly in the Python directory - IOW, 
> I'd think it OK if the PEP said "should be installed next to the 
> PythonXX.dll being installed" - but an important point in the above 
> working is the "already be on the system PATH" - ie, I don't really want 
> it put in a newly created directory unless the installer also adds that 
> directory to the PATH - and what to do on uninstall then becomes an issue.

But "next to PythonXY.dll" implies multiple copies, which we want to avoid,
right?

> One problem with all of this is uninstallation and specifically if the 
> user is uninstalling the most recent Python installation while leaving 
> earlier ones.  I guess there are 2 general answers to this:
> 
> * The installer process could remember the previous association and 
> restore that on uninstall.

We'd need to do that in the case of the earlier Python not having come with a
launcher, i.e. when its version is < 3.3.

> * We treat this as a "wont-fix" and document a work-around of asking the 
> user to reinstall the previous version to restore the file association.

This is not ideal from a user's perspective.

> We probably need to be mindful of adding too much extra work for the 
> installer process as that may well end up blocking us on getting this 
> into the next appropriate release.  In particular, Martin's thoughts 
> here would be very useful.
> 
> This would force the user to reinstall that older one to re-establish 
> the associations correctly

It sounds onerous for users to have to reinstall an older Python. I'm not that
familiar with Windows Installer features, so I don't know if this is too fancy,
but perhaps we could remember the last non-launcher association when we install
the launcher, and either restore that when the launcher is uninstalled (if it's
still pointing to an installed Python) or remove the association altogether,
otherwise. If this logic is too fancy to include in the installer itself,
perhaps we can provide this logic in the launcher itself or via an ancillary
executable or DLL that the installer can just call into.

Is there some mechanism like the SharedDLLs registry key for executables, which
could be used to reference count launcher installations so that it could be
uninstalled at the appropriate time? Could we perhaps used the SharedDLLs
feature itself?

> Yeah - I wonder if we can leverage the "job" api here and refuse to 
> start if there are already 2 processes in the job?  OTOH, that is tricky 
> as it would also prevent someone using os.startfile with a .py file....

If there's only ever one launcher installed (which we could ensure by
installing it in a Windows rather than a Python location) then perhaps we could
perhaps check for the value of a customised command pointing at the one-and-
only launcher, but this is circumventable. Anyway, perhaps we just need to
handle a user error rather than someone deliberately trying to engineer
recursion.

Another approach might be - rather than limit the number of processes in the
job, look to see if the launcher executable is already associated with an
existing job. I'm not au fait with the job API, and hence unsure of how that
would play with usages such as os.startfile, subprocess etc.

> I'm not too bothered to be honest - the customized commands exist purely 
> for alternative implementations, so my initial thoughts are that 
> additional args would be as useful for them as they are for cpython 
> invocations.  IOW, if they don't need it, then CPython invocations don't 
> need it either, so maybe it can be dropped completely?

I think they would be useful, so let me check the implementation again.

> It will be a few days until I can look at the implementation, but I'm 
> very happy to see it started.  Given it is now ahead of the Python 
> reference impl, I wonder if we should just drop all wording about that 
> reference impl and just treat the C impl as canonical?

Once you've taken a closer look, if you think it looks good enough, then that's
fine. If you have a BitBucket account, I can add your account to the repo so
you can push changes to it.

Regards,

Vinay Sajip


From skippy.hammond at gmail.com  Sun Jul  3 09:00:43 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sun, 03 Jul 2011 17:00:43 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <loom.20110702T102523-59@post.gmane.org>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
Message-ID: <4E10139B.9050700@gmail.com>

On 2/07/2011 7:08 PM, Vinay Sajip wrote:
> Mark Hammond<skippy.hammond<at>  gmail.com>  writes:
>
>> The PEP does say "if possible, should be installed somewhere likely to
>> already be on the system PATH (eg., the Windows System32) directory."
>> It is silent about what to do when that isn't possible, but I'd think it
>> OK if the launcher was installed directly in the Python directory - IOW,
>> I'd think it OK if the PEP said "should be installed next to the
>> PythonXX.dll being installed" - but an important point in the above
>> working is the "already be on the system PATH" - ie, I don't really want
>> it put in a newly created directory unless the installer also adds that
>> directory to the PATH - and what to do on uninstall then becomes an issue.
>
> But "next to PythonXY.dll" implies multiple copies, which we want to avoid,
> right?

I believe this will be most useful when people select "install for all 
users", which will force it into system32.  If people select "just for 
me", then there will be multiple copies on disk, but only one will be 
active (ie, we will not support different ones being registered for 
different users.

It isn't ideal, but I think it good enough, avoids some complexity and 
should meet the needs of the majority of users.

>> One problem with all of this is uninstallation and specifically if the
>> user is uninstalling the most recent Python installation while leaving
>> earlier ones.  I guess there are 2 general answers to this:
>>
>> * The installer process could remember the previous association and
>> restore that on uninstall.
>
> We'd need to do that in the case of the earlier Python not having come with a
> launcher, i.e. when its version is<  3.3.

OTOH, we don't do that now AFAIK.

>> * We treat this as a "wont-fix" and document a work-around of asking the
>> user to reinstall the previous version to restore the file association.
>
> This is not ideal from a user's perspective.

Sure, but neither is the current situation - I don't recall enough users 
complaining about that to make the effort worthwhile.

I'm not fundamentally opposed to doing something better here - I'm just 
trying to avoid this kind of stuff being a requirement for acceptance. 
If you are more familiar with the installer than I and feel it can be 
done simply, then I'm happy for you to go for it!

>> We probably need to be mindful of adding too much extra work for the
>> installer process as that may well end up blocking us on getting this
>> into the next appropriate release.  In particular, Martin's thoughts
>> here would be very useful.
>>
>> This would force the user to reinstall that older one to re-establish
>> the associations correctly
>
> It sounds onerous for users to have to reinstall an older Python.

But this only happens when they install a later version, then uninstall 
the later one and continue to use the old one.  I'd suggest that is (a) 
rare and (b) possibly already broken (ie, the old association may not be 
restored now).  If the old association currently is correctly restored, 
then I'd expect things to just magically work in this new model without 
any effort.

> I'm not that
> familiar with Windows Installer features, so I don't know if this is too fancy,
> but perhaps we could remember the last non-launcher association when we install
> the launcher, and either restore that when the launcher is uninstalled (if it's
> still pointing to an installed Python) or remove the association altogether,
> otherwise. If this logic is too fancy to include in the installer itself,
> perhaps we can provide this logic in the launcher itself or via an ancillary
> executable or DLL that the installer can just call into.

I'm still inclined to call YAGNI, but as above, I don't fundamentally 
oppose such bells-and-whistles.

> Is there some mechanism like the SharedDLLs registry key for executables, which
> could be used to reference count launcher installations so that it could be
> uninstalled at the appropriate time? Could we perhaps used the SharedDLLs
> feature itself?
>
>> Yeah - I wonder if we can leverage the "job" api here and refuse to
>> start if there are already 2 processes in the job?  OTOH, that is tricky
>> as it would also prevent someone using os.startfile with a .py file....
>
> If there's only ever one launcher installed (which we could ensure by
> installing it in a Windows rather than a Python location)

We probably can't ensure this while the installer supports a non-UAC 
installer (ie, the "just for me" option.)  OTOH, I'm not sure the "just 
for me" option currently works without needing UAC elevation anyway. 
But assuming it does (or the intention is to fix things if it does not), 
then we can't guarantee a writable system32.

 > then perhaps we could
> perhaps check for the value of a customised command pointing at the one-and-
> only launcher, but this is circumventable. Anyway, perhaps we just need to
> handle a user error rather than someone deliberately trying to engineer
> recursion.

Yeah, I agree.

> Another approach might be - rather than limit the number of processes in the
> job, look to see if the launcher executable is already associated with an
> existing job. I'm not au fait with the job API, and hence unsure of how that
> would play with usages such as os.startfile, subprocess etc.

Exactly - code doing os.startfile on a .py file will correctly cause the 
same launcher to be re-executed and preventing this would break such 
scripts.  However, I think this would be extremely rare and the more 
usual case would be to use sys.executable.  Indeed, any script using 
os.startfile for a .py file is already broken, even if the author 
doesn't know it yet :)

>> I'm not too bothered to be honest - the customized commands exist purely
>> for alternative implementations, so my initial thoughts are that
>> additional args would be as useful for them as they are for cpython
>> invocations.  IOW, if they don't need it, then CPython invocations don't
>> need it either, so maybe it can be dropped completely?
>
> I think they would be useful, so let me check the implementation again.
>
>> It will be a few days until I can look at the implementation, but I'm
>> very happy to see it started.  Given it is now ahead of the Python
>> reference impl, I wonder if we should just drop all wording about that
>> reference impl and just treat the C impl as canonical?
>
> Once you've taken a closer look, if you think it looks good enough, then that's
> fine. If you have a BitBucket account, I can add your account to the repo so
> you can push changes to it.

Great, thanks!

Mark

From vinay_sajip at yahoo.co.uk  Sun Jul  3 12:13:43 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 3 Jul 2011 10:13:43 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com>
Message-ID: <loom.20110703T114330-901@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> I'm not fundamentally opposed to doing something better here - I'm just 
> trying to avoid this kind of stuff being a requirement for acceptance. 
> If you are more familiar with the installer than I and feel it can be 
> done simply, then I'm happy for you to go for it!

No, you're right - I'm just throwing ideas out. I'm not especially familiar with
MSI in general, though I believe you can do the relevant things with logic in
custom actions. AFAIK there are already custom actions used in the Python MSI,
but I wouldn't want to propose any such changes to the MSI unless Martin were to
make encouraging comments :-)

> But this only happens when they install a later version, then uninstall 
> the later one and continue to use the old one.  I'd suggest that is (a) 
> rare and (b) possibly already broken (ie, the old association may not be 
> restored now).  If the old association currently is correctly restored, 
> then I'd expect things to just magically work in this new model without 
> any effort.

As to (a), not uncommon scenarios would be (i) later version breaks something,
so go back to earlier version, or (ii) using 2.x, installing 3.x to try out,
going back to 2.x. I'm not sure about (b).

> I'm still inclined to call YAGNI, but as above, I don't fundamentally 
> oppose such bells-and-whistles.

You're probably right about the cost(effort)-benefit trade-off.

> We probably can't ensure this while the installer supports a non-UAC 
> installer (ie, the "just for me" option.)  OTOH, I'm not sure the "just 
> for me" option currently works without needing UAC elevation anyway. 
> But assuming it does (or the intention is to fix things if it does not), 
> then we can't guarantee a writable system32.

Okay. From a cursory glance at msi.py, UAC elevation appears to be requested.
 
> Exactly - code doing os.startfile on a .py file will correctly cause the 
> same launcher to be re-executed and preventing this would break such 
> scripts.  However, I think this would be extremely rare and the more 
> usual case would be to use sys.executable.  Indeed, any script using 
> os.startfile for a .py file is already broken, even if the author 
> doesn't know it yet :)

I had a closer look at the Job API, and it does seem to offer the required
functionality, so we could tell whether the current process is in a job -> get
all processes in that job -> get their executables and command lines -> see if
recursion is occurring. However, would we class this as an implementation detail
or does it need a mention in the PEP?

> > I think they would be useful, so let me check the implementation again.

I've made some updates so the following works: with the configuration containing

[commands]
shell=cmd /c

with a shebang in doit.py of '#!shell python -v'

the launcher will run 'cmd /c python -v doit.py'.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Sun Jul  3 15:06:43 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 3 Jul 2011 13:06:43 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com>
Message-ID: <loom.20110703T145754-864@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> But this only happens when they install a later version, then uninstall 
> the later one and continue to use the old one.  I'd suggest that is (a) 
> rare and (b) possibly already broken (ie, the old association may not be 
> restored now).  If the old association currently is correctly restored, 
> then I'd expect things to just magically work in this new model without 
> any effort.

I've now checked, and it appears that we don't do the right thing now anyway.

On a clean Win7-x64, I installed Python 2.7 (32-bit), for all users, with
registration of extensions - and Python.File etc. were added to the registry
pointing at Python 2.7.

I then installed Python 3.2 the same way, and rhe registry entries then pointed
to 3.2.

I then uninstalled 3.2, and the registry entries were gone! No magical
restoration to the earlier values :-(

Okay - if users have to do this now anyway, we at least wouldn't be naking
things worse :-)

Regards,

Vinay Sajip



From p.f.moore at gmail.com  Sun Jul  3 17:39:07 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 3 Jul 2011 16:39:07 +0100
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <BANLkTinjAwbTC0fvC+-+VkVozfbn-_z-5A@mail.gmail.com>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk>
	<4E0C5A55.6000706@voidspace.org.uk>
	<BANLkTinjAwbTC0fvC+-+VkVozfbn-_z-5A@mail.gmail.com>
Message-ID: <CACac1F9H=nZC6+=RsEgNc1atyrWXAmi_Q0ukQm8m1QBwK6diLg@mail.gmail.com>

On 30 June 2011 13:50, Paul Moore <p.f.moore at gmail.com> wrote:
> On 30 June 2011 12:13, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> I have that email (the update one from Mark not the silent nodding from Tim)
>> still sitting in my inbox waiting for me to properly read through and
>> comment on... Sorry for being useless, I'll try and move it up the priority
>> list.
>>
>> I really like the PEP and think it will be a *huge* step forward for Windows
>> users - so it's purely the details that need thrashing out (heh).
>
> That's my situation, too. I'll try to look through it properly in the
> next day or two.

OK, having looked through this, it looks pretty solid to me. I might
try installing Vinay's implementation and seeing how it feels in use,
as well...

On restoring associations, I think it's entirely reasonable not to
bother. It would be nice if py.exe remained installed as long as any
(all-users) Python installation remained on the PC, but this may be
complicated (given that older versions won't uninstall it) so maybe
don't even bother with that.

Actually, I'd question whether this shouldn't be packaged as a
separate application, available as an independent download from
python.org (I do think it's important enough to be hosted on
python.org rather than PyPI, though). Future versions of python could
bundle it as well, but that could be purely for convenience and to
ensure that users don't miss it.

If Python does bundle it, then I'd suggest that it remain a separate
item in add/remove programs. That allows the user to choose whether to
uninstall it or not. As py.exe and python live in separate directories
(except in per-user installs) the installers don't have any nasty
interactions like who deletes the directory to worry about. If you
want to bother with an "active installation count" mechanism (whether
SharedDLLs or a simple count/listing in the py.ini file, or something
else) then when the count hits "No", just offer the user the choice to
uninstall as part of the Python uninstall.

I'd be inclined not to worry too much about per-user installations.
Are per-user file associations possible? I've never used them, myself.
As a simple approach, just install py.exe alongside python.exe in the
user dir, don't worry about it being on PATH or having a separate
add/remove programs item, and let the user do what he wants with it.
Of course, if someone with more experience of per-user installs and
the sorts of environment where they are needed wants to suggest
something different, believe them rather than me!

Basically, my feeling is that the core functionality is fine. Nothing
in this will make life worse for anyone, so we can safely leave corner
cases to be addressed later (with a standalone release, if it's urgent
enough).

Paul.

From vinay_sajip at yahoo.co.uk  Sun Jul  3 20:20:32 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 3 Jul 2011 18:20:32 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_397_=28Python_launcher_for_Windows=29_?=
	=?utf-8?q?reference=09implementation?=
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk>
	<4E0C5A55.6000706@voidspace.org.uk>
	<BANLkTinjAwbTC0fvC+-+VkVozfbn-_z-5A@mail.gmail.com>
	<CACac1F9H=nZC6+=RsEgNc1atyrWXAmi_Q0ukQm8m1QBwK6diLg@mail.gmail.com>
Message-ID: <loom.20110703T201627-197@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> OK, having looked through this, it looks pretty solid to me. I might
> try installing Vinay's implementation and seeing how it feels in use,
> as well...

Do have a play, it would be nice to get feedback. It's only available as source,
though - is that OK?
 
> I'd be inclined not to worry too much about per-user installations.
> Are per-user file associations possible? I've never used them, myself.

Yes, and they can be problematic: see

http://www.deadlybloodyserious.com/2007/05/no-script-arguments/

where the problem described has also bitten me. I've no idea what put that
association in the registry.

Regards,

Vinay Sajip


From p.f.moore at gmail.com  Sun Jul  3 20:33:01 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 3 Jul 2011 19:33:01 +0100
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <loom.20110703T201627-197@post.gmane.org>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk>
	<4E0C5A55.6000706@voidspace.org.uk>
	<BANLkTinjAwbTC0fvC+-+VkVozfbn-_z-5A@mail.gmail.com>
	<CACac1F9H=nZC6+=RsEgNc1atyrWXAmi_Q0ukQm8m1QBwK6diLg@mail.gmail.com>
	<loom.20110703T201627-197@post.gmane.org>
Message-ID: <CACac1F85X9U634=5=GyT--bGi8YTXj6qWLPUqcvSMH9v32Vz-g@mail.gmail.com>

On 3 July 2011 19:20, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>
>> OK, having looked through this, it looks pretty solid to me. I might
>> try installing Vinay's implementation and seeing how it feels in use,
>> as well...
>
> Do have a play, it would be nice to get feedback. It's only available as source,
> though - is that OK?

No problem - I can build it from source.

>> I'd be inclined not to worry too much about per-user installations.
>> Are per-user file associations possible? I've never used them, myself.
>
> Yes, and they can be problematic: see
>
> http://www.deadlybloodyserious.com/2007/05/no-script-arguments/
>
> where the problem described has also bitten me. I've no idea what put that
> association in the registry.

Ugh. That's nasty. It doesn't even look like a standard file type association.

Paul.

From listas at alejolp.com  Mon Jul  4 01:24:57 2011
From: listas at alejolp.com (Alejandro Santos)
Date: Sun, 3 Jul 2011 20:24:57 -0300
Subject: [Python-Dev] Extending documentation innacuracies
Message-ID: <CAKMi1TwK5ixKhEt5ZW9ANC=5zgfUZHuJSDUaWt-jn+02VF6W6Q@mail.gmail.com>

Hi,

While reading through the Extending and Embedding docs for Python 3.2
I've noticed something. While the "Py_InitModule" does not exists on
Py3k, it is still mentioned on the docs:

http://docs.python.org/py3k/extending/extending.html#keyword-parameters-for-extension-functions
http://docs.python.org/py3k/extending/windows.html#a-cookbook-approach
http://docs.python.org/py3k/faq/extending.html#what-does-systemerror-pyimport-fixupextension-module-yourmodule-not-loaded-mean

Should I report a new issue?

-- 
Alejandro Santos
http://alejolp.com.ar

From ncoghlan at gmail.com  Mon Jul  4 02:33:42 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 4 Jul 2011 10:33:42 +1000
Subject: [Python-Dev] Extending documentation innacuracies
In-Reply-To: <CAKMi1TwK5ixKhEt5ZW9ANC=5zgfUZHuJSDUaWt-jn+02VF6W6Q@mail.gmail.com>
References: <CAKMi1TwK5ixKhEt5ZW9ANC=5zgfUZHuJSDUaWt-jn+02VF6W6Q@mail.gmail.com>
Message-ID: <CADiSq7dpx1e9VyqL3BkNqkorzWbcKp808dek4_jZdp=PyU_+JQ@mail.gmail.com>

On Mon, Jul 4, 2011 at 9:24 AM, Alejandro Santos <listas at alejolp.com> wrote:
> Should I report a new issue?

Yes please, tracker items are the best way to get that kind of
oversight cleaned up. Thanks for noticing!

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Jul  4 03:20:03 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 4 Jul 2011 11:20:03 +1000
Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here
	(or should anyway)
In-Reply-To: <E1QdUkL-00046L-To@dinsdale.python.org>
References: <E1QdUkL-00046L-To@dinsdale.python.org>
Message-ID: <CADiSq7cFgkxqMzGx0oa=a=SkeOETTqyVu4u8ZqLtVBp2uFyUnQ@mail.gmail.com>

On Mon, Jul 4, 2011 at 8:02 AM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/bbbeddafeec0
> changeset: ? 71160:bbbeddafeec0
> user: ? ? ? ?Benjamin Peterson <benjamin at python.org>
> date: ? ? ? ?Sun Jul 03 17:06:32 2011 -0500
> summary:
> ?no one passes NULL here (or should anyway)
>
> files:
> ?Python/ceval.c | ?3 ---
> ?1 files changed, 0 insertions(+), 3 deletions(-)
>
>
> diff --git a/Python/ceval.c b/Python/ceval.c
> --- a/Python/ceval.c
> +++ b/Python/ceval.c
> @@ -1115,9 +1115,6 @@
>
> ?/* Start of code */
>
> - ? ?if (f == NULL)
> - ? ? ? ?return NULL;
> -

May need to replace that with an assert(f != NULL) to keep static
analysers happy.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From benjamin at python.org  Mon Jul  4 04:54:23 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 3 Jul 2011 21:54:23 -0500
Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here
	(or should anyway)
In-Reply-To: <CADiSq7cFgkxqMzGx0oa=a=SkeOETTqyVu4u8ZqLtVBp2uFyUnQ@mail.gmail.com>
References: <E1QdUkL-00046L-To@dinsdale.python.org>
	<CADiSq7cFgkxqMzGx0oa=a=SkeOETTqyVu4u8ZqLtVBp2uFyUnQ@mail.gmail.com>
Message-ID: <CAPZV6o-sgHuARDgbzsdi-rvSE0PvHM-P043swa-8ECmURrSt2Q@mail.gmail.com>

2011/7/3 Nick Coghlan <ncoghlan at gmail.com>:
> On Mon, Jul 4, 2011 at 8:02 AM, benjamin.peterson
> <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/bbbeddafeec0
>> changeset: ? 71160:bbbeddafeec0
>> user: ? ? ? ?Benjamin Peterson <benjamin at python.org>
>> date: ? ? ? ?Sun Jul 03 17:06:32 2011 -0500
>> summary:
>> ?no one passes NULL here (or should anyway)
>>
>> files:
>> ?Python/ceval.c | ?3 ---
>> ?1 files changed, 0 insertions(+), 3 deletions(-)
>>
>>
>> diff --git a/Python/ceval.c b/Python/ceval.c
>> --- a/Python/ceval.c
>> +++ b/Python/ceval.c
>> @@ -1115,9 +1115,6 @@
>>
>> ?/* Start of code */
>>
>> - ? ?if (f == NULL)
>> - ? ? ? ?return NULL;
>> -
>
> May need to replace that with an assert(f != NULL) to keep static
> analysers happy.

Surely static analyzers don't assume every argument passed in is NULL?


-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Mon Jul  4 05:44:59 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 4 Jul 2011 13:44:59 +1000
Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here
	(or should anyway)
In-Reply-To: <CAPZV6o-sgHuARDgbzsdi-rvSE0PvHM-P043swa-8ECmURrSt2Q@mail.gmail.com>
References: <E1QdUkL-00046L-To@dinsdale.python.org>
	<CADiSq7cFgkxqMzGx0oa=a=SkeOETTqyVu4u8ZqLtVBp2uFyUnQ@mail.gmail.com>
	<CAPZV6o-sgHuARDgbzsdi-rvSE0PvHM-P043swa-8ECmURrSt2Q@mail.gmail.com>
Message-ID: <CADiSq7cQi7qiCKZq=KuHoscWGiEeqV_hUgPmxXwxcU8+G9bA8Q@mail.gmail.com>

On Mon, Jul 4, 2011 at 12:54 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2011/7/3 Nick Coghlan <ncoghlan at gmail.com>:
>> May need to replace that with an assert(f != NULL) to keep static
>> analysers happy.
>
> Surely static analyzers don't assume every argument passed in is NULL?

I didn't check - was this change in a static function? For those, I
think they can figure it out. For functions exposed to the linker, I
think they demand an explicit check for a non-NULL pointer (which may
be in the form of an assertion).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From benjamin at python.org  Mon Jul  4 06:15:53 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 3 Jul 2011 23:15:53 -0500
Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here
	(or should anyway)
In-Reply-To: <CADiSq7cQi7qiCKZq=KuHoscWGiEeqV_hUgPmxXwxcU8+G9bA8Q@mail.gmail.com>
References: <E1QdUkL-00046L-To@dinsdale.python.org>
	<CADiSq7cFgkxqMzGx0oa=a=SkeOETTqyVu4u8ZqLtVBp2uFyUnQ@mail.gmail.com>
	<CAPZV6o-sgHuARDgbzsdi-rvSE0PvHM-P043swa-8ECmURrSt2Q@mail.gmail.com>
	<CADiSq7cQi7qiCKZq=KuHoscWGiEeqV_hUgPmxXwxcU8+G9bA8Q@mail.gmail.com>
Message-ID: <CAPZV6o9Jsp_eFe-53UgGzaxdJexdYrgbNok1vr8WcxgU5kW_LQ@mail.gmail.com>

2011/7/3 Nick Coghlan <ncoghlan at gmail.com>:
> On Mon, Jul 4, 2011 at 12:54 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2011/7/3 Nick Coghlan <ncoghlan at gmail.com>:
>>> May need to replace that with an assert(f != NULL) to keep static
>>> analysers happy.
>>
>> Surely static analyzers don't assume every argument passed in is NULL?
>
> I didn't check - was this change in a static function? For those, I
> think they can figure it out. For functions exposed to the linker, I
> think they demand an explicit check for a non-NULL pointer (which may
> be in the form of an assertion).

If someone's static analysis tool starts complaining about it, I'd be
happy to consider adding an assert...


-- 
Regards,
Benjamin

From greg.ewing at canterbury.ac.nz  Mon Jul  4 07:59:14 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 04 Jul 2011 17:59:14 +1200
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <4E10139B.9050700@gmail.com>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com> <loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com> <loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com> <loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com>
Message-ID: <4E1156B2.6050404@canterbury.ac.nz>

Mark Hammond wrote:

> On 2/07/2011 7:08 PM, Vinay Sajip wrote:

>> perhaps we could remember the last non-launcher association when 
>> we install the launcher,

It might be better to look in the registry for other Python
installations and ask the user which one to restore if there
is more than one. Trying to restore the "last" one would be
prone to breakage if the user didn't uninstall versions in
precisely the reverse of the order of installation.

-- 
Greg

From skippy.hammond at gmail.com  Mon Jul  4 08:27:17 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Mon, 04 Jul 2011 16:27:17 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <4E1156B2.6050404@canterbury.ac.nz>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
Message-ID: <4E115D45.8000101@gmail.com>

On 4/07/2011 3:59 PM, Greg Ewing wrote:
> Mark Hammond wrote:
>
>> On 2/07/2011 7:08 PM, Vinay Sajip wrote:
>
>>> perhaps we could remember the last non-launcher association when we
>>> install the launcher,
>
> It might be better to look in the registry for other Python
> installations and ask the user which one to restore if there
> is more than one. Trying to restore the "last" one would be
> prone to breakage if the user didn't uninstall versions in
> precisely the reverse of the order of installation.

While that makes alot of sense, the fact we are already "broken" in 
exactly the same way means I hope we can treat the restoration of 
associations as a separate issue - a worthwhile one, but not a 
pre-requisite for this PEP being approved.

Cheers,

Mark

From aigarius at gmail.com  Mon Jul  4 12:10:57 2011
From: aigarius at gmail.com (Aigars Mahinovs)
Date: Mon, 4 Jul 2011 13:10:57 +0300
Subject: [Python-Dev] Threaded asynchronous return from functions
Message-ID: <CABpYwDXVfrxFaH3LnkPyzUrWiF6AA2oOH1VGuFCb8jF+u6D5sw@mail.gmail.com>

Short version: Could we get

def funct( args ):
    if args == 'good':
        return 'good'
    async_return "I'll think about it"
    if args == '123':
        do_x()
    elif args == 'abc':
        do_y()
    else:
        do_z()

as equivalent to

def do_thinking( args ):
    if args == '123':
        do_x()
    elif args == 'abc':
        do_y()
    else:
        do_z()

def funct( args ):
    if args == 'good':
        return 'good'
    t = threading.Thread(target=do_thinking, args=args)
    t.start()
    return "I'll think about it"

Longer version:

I have been doing some multithreaded work lately and have found that
often what I find wanting to do is to call a function, have it check
it's arguments, possibly do some work and then return to the caller,
but still do some extra processing right after that. Currently to
accomplish such feat I need to separate the 'extra processing' bit
into a separate function and call that in a separate thread. A nice
convenience would be a function or statement that would allow to
return a value from the current function, but still keep running its
code (in a separate thread). Such approach could then be used in many
places where async processing is required, such as GUI programming,
XMLRPC, web applications, ... with less boilerplate and more obvious
code flow.

-- 
Best regards,
? ? Aigars Mahinovs? ? ? ? mailto:aigarius at debian.org
? #--------------------------------------------------------------#
?| .''`.? ? Debian GNU/Linux (http://www.debian.org)? ? ? ? ? ? |
?| : :' :?? Latvian Open Source Assoc. (http://www.laka.lv)? ?? |
?| `. `'? ? Linux Administration and Free Software Consulting?? |
?|?? `-? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?? (http://www.aiteki.com) |
?#--------------------------------------------------------------#

From amauryfa at gmail.com  Mon Jul  4 13:11:49 2011
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Jul 2011 13:11:49 +0200
Subject: [Python-Dev] Threaded asynchronous return from functions
In-Reply-To: <CABpYwDXVfrxFaH3LnkPyzUrWiF6AA2oOH1VGuFCb8jF+u6D5sw@mail.gmail.com>
References: <CABpYwDXVfrxFaH3LnkPyzUrWiF6AA2oOH1VGuFCb8jF+u6D5sw@mail.gmail.com>
Message-ID: <CAGmFidaw6BXDpOcYa=wqCbLUqCYTa-gKP+iQpomMEyh6mR_rmg@mail.gmail.com>

Hello,

2011/7/4 Aigars Mahinovs <aigarius at gmail.com>

> I have been doing some multithreaded work lately and have found that

often what I find wanting to do is to call a function, have it check
> it's arguments, possibly do some work and then return to the caller,
> but still do some extra processing right after that. Currently to
> accomplish such feat I need to separate the 'extra processing' bit
> into a separate function and call that in a separate thread. A nice
> convenience would be a function or statement that would allow to
> return a value from the current function, but still keep running its
> code (in a separate thread). Such approach could then be used in many
> places where async processing is required, such as GUI programming,
> XMLRPC, web applications, ... with less boilerplate and more obvious
> code flow.
>

This kind of topic is not suitable to python-dev.
Please ask this on the python-list mailing list, or eventually
on python-ideas.
(where someone will probably suggest you to use a nested function)

Cheers,

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110704/96af1a4d/attachment.html>

From peck at us.ibm.com  Mon Jul  4 12:04:18 2011
From: peck at us.ibm.com (Jon K Peck)
Date: Mon, 4 Jul 2011 04:04:18 -0600
Subject: [Python-Dev] AUTO: Jon K Peck is out of the office (returning
	07/15/2011)
Message-ID: <OF1EF42667.4E9E65E1-ON872578C3.00375397-872578C3.00375398@us.ibm.com>



I am out of the office until 07/15/2011.

I am out of the office traveling  Monday July 4 through Friday, July 15
I will have little or no access to email during this time, so I will be
delayed in responding.


Note: This is an automated response to your message  "Python-Dev Digest,
Vol 96, Issue 6" sent on 7/4/11 0:27:26.

This is the only notification you will receive while this person is away.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110704/3c5062f9/attachment.html>

From jnoller at gmail.com  Mon Jul  4 17:51:37 2011
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 4 Jul 2011 11:51:37 -0400
Subject: [Python-Dev] Speed.Python.org
Message-ID: <CACQrdOmjmLEW12pVMPL2L53y+44jFqMuyXNehFAfHbSMd8a0AA@mail.gmail.com>

Now that we have the machine, we need to start working on
collecting/organizing the resources needed to get a shared codespeed
system in place. After speaking with various people, we felt that
overloading codespeed-dev, pypy-dev or python-dev with the discussions
around this would be sub optimal. I've spun up a new mailing list
here:

http://mail.python.org/mailman/listinfo/speed

Those who are interested in working on or contributing to the
speed.python.org project can subscribe there. I personally can not
lead the project, and so I will be looking to the current
speed.pypy.org team, and python-dev contributors for leadership in
this. I got you the hardware and hosting! :)

jesse

From solipsis at pitrou.net  Mon Jul  4 18:23:44 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 4 Jul 2011 18:23:44 +0200
Subject: [Python-Dev] cpython: Issue #12469: replace assertions by
	explicit if+raise
References: <E1Qdlfh-0004B5-BT@dinsdale.python.org>
Message-ID: <20110704182344.7bad6b62@pitrou.net>

On Mon, 04 Jul 2011 18:06:53 +0200
victor.stinner <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/7eef821ab20d
> changeset:   71197:7eef821ab20d
> user:        Victor Stinner <victor.stinner at haypocalc.com>
> date:        Mon Jul 04 18:06:35 2011 +0200
> summary:
>   Issue #12469: replace assertions by explicit if+raise

Instead of generic Exception, it would be better to use AssertionError.

Regards

Antoine.



From g.brandl at gmx.net  Mon Jul  4 19:37:47 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 04 Jul 2011 19:37:47 +0200
Subject: [Python-Dev] cpython: Issue #12469: replace assertions by
	explicit if+raise
In-Reply-To: <20110704182344.7bad6b62@pitrou.net>
References: <E1Qdlfh-0004B5-BT@dinsdale.python.org>
	<20110704182344.7bad6b62@pitrou.net>
Message-ID: <iusto6$ehr$1@dough.gmane.org>

Am 04.07.2011 18:23, schrieb Antoine Pitrou:
> On Mon, 04 Jul 2011 18:06:53 +0200
> victor.stinner <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/7eef821ab20d
>> changeset:   71197:7eef821ab20d
>> user:        Victor Stinner <victor.stinner at haypocalc.com>
>> date:        Mon Jul 04 18:06:35 2011 +0200
>> summary:
>>   Issue #12469: replace assertions by explicit if+raise
> 
> Instead of generic Exception, it would be better to use AssertionError.

What is the reason for this change anyway -- as far as I can see this code
is never run with -O.

Also I don't see how it relates to #12469.

Georg


From georg at python.org  Mon Jul  4 19:48:50 2011
From: georg at python.org (Georg Brandl)
Date: Mon, 04 Jul 2011 19:48:50 +0200
Subject: [Python-Dev] [RELEASED] Python 3.2.1 rc 2
Message-ID: <4E11FD02.3040205@python.org>

On behalf of the Python development team, I am pleased to announce the
second release candidate of Python 3.2.1.

Python 3.2.1 will the first bugfix release for Python 3.2, fixing over 120
bugs and regressions in Python 3.2.

For an extensive list of changes and features in the 3.2 line, see

    http://docs.python.org/3.2/whatsnew/3.2.html

To download Python 3.2.1 visit:

    http://www.python.org/download/releases/3.2.1/

This is a testing release: Please consider trying Python 3.2.1 with your code
and reporting any bugs you may notice to:

    http://bugs.python.org/


Enjoy!

-- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.2's contributors)

From greg at krypto.org  Mon Jul  4 19:48:45 2011
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 4 Jul 2011 10:48:45 -0700
Subject: [Python-Dev] cpython: Issue #12469: replace assertions by
 explicit if+raise
In-Reply-To: <20110704182344.7bad6b62@pitrou.net>
References: <E1Qdlfh-0004B5-BT@dinsdale.python.org>
	<20110704182344.7bad6b62@pitrou.net>
Message-ID: <CAGE7PNJ6DbQ+eGiMV7vD6n8tk0fioGEo3Gs7Pes5Enth4Sw6yg@mail.gmail.com>

On Mon, Jul 4, 2011 at 9:23 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 04 Jul 2011 18:06:53 +0200
> victor.stinner <python-checkins at python.org> wrote:
> > http://hg.python.org/cpython/rev/7eef821ab20d
> > changeset:   71197:7eef821ab20d
> > user:        Victor Stinner <victor.stinner at haypocalc.com>
> > date:        Mon Jul 04 18:06:35 2011 +0200
> > summary:
> >   Issue #12469: replace assertions by explicit if+raise
>
> Instead of generic Exception, it would be better to use AssertionError.
>

or in many cases given this was in unittests... use the self.assertFoo
methods and avoid assert and if statements all together.


>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110704/8d45bc7d/attachment.html>

From victor.stinner at haypocalc.com  Mon Jul  4 20:55:06 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 04 Jul 2011 20:55:06 +0200
Subject: [Python-Dev] cpython: Issue #12469: replace assertions by
 explicit if+raise
In-Reply-To: <20110704182344.7bad6b62@pitrou.net>
References: <E1Qdlfh-0004B5-BT@dinsdale.python.org>
	<20110704182344.7bad6b62@pitrou.net>
Message-ID: <1309805706.30323.4.camel@marge>

Le lundi 04 juillet 2011 ? 18:23 +0200, Antoine Pitrou a ?crit :
> On Mon, 04 Jul 2011 18:06:53 +0200
> victor.stinner <python-checkins at python.org> wrote:
> > http://hg.python.org/cpython/rev/7eef821ab20d
> > changeset:   71197:7eef821ab20d
> > user:        Victor Stinner <victor.stinner at haypocalc.com>
> > date:        Mon Jul 04 18:06:35 2011 +0200
> > summary:
> >   Issue #12469: replace assertions by explicit if+raise


> Instead of generic Exception, it would be better to use AssertionError.

and

> or in many cases given this was in unittests... use the self.assertFoo
> methods and avoid assert and if statements all together.

The code is running in a subprocess (python -c ...), not in an
unittest.TestCase, so I cannot use self.assertFoo and it doesn't really
matter if the exception is an Exception or an AssertionError.

> What is the reason for this change anyway -- as far as I can
> see this code is never run with -O.

I'm not sure that the code will never be running using -O, so I prefer
to use an explicit if+raise. I don't like the assert statement because
it doesn't provide any information about the failure (content of the
variables) by default.

Victor


From digitalxero at gmail.com  Mon Jul  4 21:04:44 2011
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Mon, 4 Jul 2011 15:04:44 -0400
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <4E115D45.8000101@gmail.com>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org> <4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org> <4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org> <4E10139B.9050700@gmail.com>
	<4E1156B2.6050404@canterbury.ac.nz> <4E115D45.8000101@gmail.com>
Message-ID: <CAMPUAFMzAfNJU8+=LmJeyD4wU_O8As9yy-+MQ-OSNDjFKchtMw@mail.gmail.com>

On Mon, Jul 4, 2011 at 2:27 AM, Mark Hammond <skippy.hammond at gmail.com> wrote:
> While that makes alot of sense, the fact we are already "broken" in exactly
> the same way means I hope we can treat the restoration of associations as a
> separate issue - a worthwhile one, but not a pre-requisite for this PEP
> being approved.

I agree let the restoration or not of file association be an
implementation detail and not a requirement. I also agree with Paul
Moore that py.exe should be a separate install/uninstall It can be
bundled with the python MSI but should be a standalone MSI that the
python MSI executes. So uninstalling the version of python that
installed it alone is not enough to remove it. This will make it seem
like the file associations are just working for naive users but will
still let them remove the associations. We can just put a warning in
the uninstall process of py.exe that lets the user know if they still
have python versions installed they will need to re-install them to
get the file association again.

From senthil at uthcode.com  Mon Jul  4 23:30:51 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Mon, 4 Jul 2011 14:30:51 -0700
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTikJGTxQ9Pv-Y7dkHVGNrSdago1ZyA@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind> <iuaie1$qlm$1@dough.gmane.org>
	<iuavjj$9ga$1@dough.gmane.org> <4E09085D.80202@voidspace.org.uk>
	<BANLkTikJGTxQ9Pv-Y7dkHVGNrSdago1ZyA@mail.gmail.com>
Message-ID: <20110704213051.GA4320@mathmagic>

On Tue, Jun 28, 2011 at 10:30:14AM +1000, Nick Coghlan wrote:

> Rather than fighting that convention, we should probably just confront
> the ambiguity head on and update
> http://docs.python.org/dev/glossary.html#term-attribute to describe
> both uses of the term (and add a separate entry for "data attribute",
> with a definition which basically says "attributes which are not
> methods").

http://bugs.python.org/issue12491 is the issue to track it.  The
glossary term should give us a stance on what is meant by attributes.

For the other issue (10403), I just concentrated on removing the term
members and used attributes and methods appropriately focussing on
clarity rather than presenting the detail on the object model. For our
rescue, sphinx reST provide :attr: for attribute and :meth: for
methods. :)


-- 
Senthil

From vinay_sajip at yahoo.co.uk  Tue Jul  5 00:05:31 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 4 Jul 2011 22:05:31 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
Message-ID: <loom.20110705T000517-753@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> > It might be better to look in the registry for other Python
> > installations and ask the user which one to restore if there
> > is more than one. Trying to restore the "last" one would be
> > prone to breakage if the user didn't uninstall versions in
> > precisely the reverse of the order of installation.
> 
> While that makes alot of sense, the fact we are already "broken" in 
> exactly the same way means I hope we can treat the restoration of 
> associations as a separate issue - a worthwhile one, but not a 
> pre-requisite for this PEP being approved.

I agree, but there's one aspect of associations which is perhaps worth
exploring: the installation of a pre-3.3 version of Python after Python 3.3 is
installed with the launcher will, if the user selects "Register Extensions",
hijack the laumcher's associations to that earlier Python. Then bye bye launcher
- how do we deal with that? Or don't we? There'll be no warning for the user,
and this problem will occur even if the launcher is packaged separately from
Python. so I think we need to think about this a little more. What say?

Regards,

Vinay Sajip


From jimjjewett at gmail.com  Tue Jul  5 00:27:33 2011
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 4 Jul 2011 18:27:33 -0400
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <E1QdQNb-0003Ir-03@dinsdale.python.org>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
Message-ID: <CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>

If you're going to get rid of the pun, you might as well change the
whole sentence...

On Sun, Jul 3, 2011 at 1:22 PM, georg.brandl <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/76452b892838
> changeset: ? 71146:76452b892838
> parent: ? ? ?71144:ce52310f61a0
> user: ? ? ? ?Georg Brandl <georg at python.org>
> date: ? ? ? ?Sun Jul 03 19:22:42 2011 +0200
> summary:
> ?Remove mention of medical condition from the test suite.
>
> files:
> ?Lib/test/test_csv.py | ?8 ++++----
> ?1 files changed, 4 insertions(+), 4 deletions(-)
>
>
> diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
> --- a/Lib/test/test_csv.py
> +++ b/Lib/test/test_csv.py
> @@ -459,20 +459,20 @@
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'5', '6']])
>
> ? ? def test_quoted_quote(self):
> - ? ? ? ?self.readerAssertEqual('1,2,3,"""I see,"" said the blind man","as he picked up his hammer and saw"',
> + ? ? ? ?self.readerAssertEqual('1,2,3,"""I see,"" said the happy man","as he picked up his hammer and saw"',
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[['1', '2', '3',
> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see," said the blind man',
> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see," said the happy man',
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'as he picked up his hammer and saw']])
>
> ? ? def test_quoted_nl(self):
> ? ? ? ? input = '''\
> ?1,2,3,"""I see,""
> -said the blind man","as he picked up his
> +said the happy man","as he picked up his
> ?hammer and saw"
> ?9,8,7,6'''
> ? ? ? ? self.readerAssertEqual(input,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[['1', '2', '3',
> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see,"\nsaid the blind man',
> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? '"I see,"\nsaid the happy man',
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'as he picked up his\nhammer and saw'],
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ['9','8','7','6']])
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>

From vinay_sajip at yahoo.co.uk  Tue Jul  5 01:19:39 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 4 Jul 2011 23:19:39 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
Message-ID: <loom.20110705T011036-184@post.gmane.org>

One more thing about associations - we've got pyw.exe for Python.NoConFile
and py.exe for Python.file, but how do we handle Python.CompiledFile? It
doesn't really make sense to have the association not handled by the launcher.
Unfortunately, of course, both pyw and py compile to pyo, so we don't know
which launcher to use. It is, of course, easy for either py or pyw to
determine which version of python is to be used to invoke a .pyc - just not
which Windows variant.

BTW just as a test I implemented .pyc support in the C implementation - it
works fine apart from the "python.exe or pythonw.exe?" question.

Regards,

Vinay Sajip


From greg.ewing at canterbury.ac.nz  Tue Jul  5 03:23:20 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 05 Jul 2011 13:23:20 +1200
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <loom.20110705T000517-753@post.gmane.org>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com> <loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com> <loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com> <loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com> <loom.20110705T000517-753@post.gmane.org>
Message-ID: <4E126788.7020705@canterbury.ac.nz>

Vinay Sajip wrote:
> the installation of a pre-3.3 version of Python after Python 3.3 is
> installed with the launcher will, if the user selects "Register Extensions",
> hijack the laumcher's associations to that earlier Python. Then bye bye launcher

I don't see how anything can be done about that. It
also doesn't seem like too serious a problem -- it's
no worse than what currently happens if you install an
older version over a newer one, i.e. associations revert
to the old version.

Making the launcher a separate install at least offers
a way of fixing that if it happens and isn't desired,
i.e. reinstall the launcher.

-- 
Greg

From skippy.hammond at gmail.com  Tue Jul  5 04:12:54 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Tue, 05 Jul 2011 12:12:54 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <4E126788.7020705@canterbury.ac.nz>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
	<loom.20110705T000517-753@post.gmane.org>
	<4E126788.7020705@canterbury.ac.nz>
Message-ID: <4E127326.7090202@gmail.com>

On 5/07/2011 11:23 AM, Greg Ewing wrote:
> Vinay Sajip wrote:
>> the installation of a pre-3.3 version of Python after Python 3.3 is
>> installed with the launcher will, if the user selects "Register
>> Extensions",
>> hijack the laumcher's associations to that earlier Python. Then bye
>> bye launcher
>
> I don't see how anything can be done about that. It
> also doesn't seem like too serious a problem -- it's
> no worse than what currently happens if you install an
> older version over a newer one, i.e. associations revert
> to the old version.
>
> Making the launcher a separate install at least offers
> a way of fixing that if it happens and isn't desired,
> i.e. reinstall the launcher.

Or an MSI installer may be able to offer a "repair" feature without too 
much pain.

However, I'm a bit torn on the separate installer issue.  I really think 
it should be installed with later Python versions even if it is made 
available as a separate installer for the benefit of earlier ones as it 
is one less step for people to get confused about - eg, in a few years 
when 3.3+ is the most common Python being downloaded and used, books or 
people helping on python-list could start referring to the launcher 
under the assumption it is installed, rather than avoiding mention of it 
simply to avoid the whole spiel about "this will only work if you have 
installed an optional package."  IOW, it will be impossible to give 
unconditional advice on certain aspects of launching Python.

If the launcher is such that we can unconditionally recommend its use, 
IMO we should just install it with Python.  I'll go with the consensus 
though...

Mark

From ncoghlan at gmail.com  Tue Jul  5 04:26:26 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 5 Jul 2011 12:26:26 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <4E127326.7090202@gmail.com>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
	<loom.20110705T000517-753@post.gmane.org>
	<4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com>
Message-ID: <CADiSq7eLu9=R088wLLnHbckEx1CtnjVQG_L8eweGFyR=Jp0kgQ@mail.gmail.com>

On Tue, Jul 5, 2011 at 12:12 PM, Mark Hammond <skippy.hammond at gmail.com> wrote:
> If the launcher is such that we can unconditionally recommend its use, IMO
> we should just install it with Python. ?I'll go with the consensus though...

I've installed other WIndows apps that create multiple add/remove
programs entries from a single installer. I believe people are
suggesting a similar thing here (i.e. have the launcher installed
automatically when installing python, but create a separate add/remove
entry so uninstallation leaves it behind unless removal is explicitly
requested)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From stephen at xemacs.org  Tue Jul  5 08:27:00 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 05 Jul 2011 15:27:00 +0900
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
Message-ID: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>

Jim Jewett writes:

 > If you're going to get rid of the pun, you might as well change the
 > whole sentence...

In fact, he should, since this is such a well-known pun that many
people will consciously make the reverse substitution, and wonder WTF
python-dev was thinking when they put the amended sentence in the
test.  Maybe somebody was *trying* to offend people who make such
corrections habitually by demonstrating the kind of nonsense they
occasionally produce.  I've seen that target hit several times.

Also, the log should say why this was done.

-?Remove mention of medical condition from the test suite.
+ Change potentially offensive language in the test suite.

One could also use the somewhat euphemistic "unprofessional language".
That points to why such changes are justified despite an author's
right to have her mode of expression respected -- the Python project
aims at professionalism, and offensive language detracts from it.

From skippy.hammond at gmail.com  Tue Jul  5 08:59:42 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Tue, 05 Jul 2011 16:59:42 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <4E0EC5B7.9010607@gmail.com>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
Message-ID: <4E12B65E.5010504@gmail.com>

On 2/07/2011 5:16 PM, I wrote:

> Given [the C implementation] is now ahead of the Python
> reference impl, I wonder if we should just drop all wording about that
> reference impl and just treat the C impl as canonical?

I'm looking to update the PEP based on this discussion - does anyone 
object to the above?  Or to put it another way, does anyone believe 
dropping the Python reference implementation is to the detriment of the PEP?

Thanks,

Mark

From vinay_sajip at yahoo.co.uk  Tue Jul  5 10:01:58 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 5 Jul 2011 08:01:58 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_397_=28Python_launcher_for_Windows=29_?=
	=?utf-8?q?reference=09implementation?=
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
	<loom.20110705T000517-753@post.gmane.org>
	<4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com>
	<CADiSq7eLu9=R088wLLnHbckEx1CtnjVQG_L8eweGFyR=Jp0kgQ@mail.gmail.com>
Message-ID: <loom.20110705T095246-691@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> I've installed other WIndows apps that create multiple add/remove
> programs entries from a single installer. I believe people are
> suggesting a similar thing here (i.e. have the launcher installed
> automatically when installing python, but create a separate add/remove
> entry so uninstallation leaves it behind unless removal is explicitly
> requested)

Were those other Windows apps packaged as .msi, or .exe? AFAICT, although you
can embed an MSI inside another one, the practice of concurrent/nested
installations is strongly discouraged by Microsoft - see http://goo.gl/FJx1S
(Rule 20).

Also, IIUC, each entry in Add/Remove programs would correspond to a specific MSI
(since you can e.g. repair that specific entry, it would imply its own installer
database).

So you could package Python and the launcher as separate MSIs (this would make
sense so that you could restore associations to the launcher just by repairing
its installation), but since nested MSIs are a no-no, that means installing via
a bootstrapping .exe. This is a bigger change to our Windows packaging than some
people might be comfortable with ...

Regards,

Vinay Sajip


From p.f.moore at gmail.com  Tue Jul  5 10:18:56 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 5 Jul 2011 09:18:56 +0100
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <CADiSq7eLu9=R088wLLnHbckEx1CtnjVQG_L8eweGFyR=Jp0kgQ@mail.gmail.com>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
	<loom.20110705T000517-753@post.gmane.org>
	<4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com>
	<CADiSq7eLu9=R088wLLnHbckEx1CtnjVQG_L8eweGFyR=Jp0kgQ@mail.gmail.com>
Message-ID: <CACac1F9G0wQxDnPShoHuaf=-wmarJN=fyUXdSnPv7dtcNZ5dZQ@mail.gmail.com>

On 5 July 2011 03:26, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Jul 5, 2011 at 12:12 PM, Mark Hammond <skippy.hammond at gmail.com> wrote:
>> If the launcher is such that we can unconditionally recommend its use, IMO
>> we should just install it with Python. ?I'll go with the consensus though...
>
> I've installed other WIndows apps that create multiple add/remove
> programs entries from a single installer. I believe people are
> suggesting a similar thing here (i.e. have the launcher installed
> automatically when installing python, but create a separate add/remove
> entry so uninstallation leaves it behind unless removal is explicitly
> requested)

That's certainly what I was meaning.

I'm 100% in favour of Python 3.3 and later containing the installer as
part of the core Python installer. One download, one install. (And two
add/remove entries).

I'd like to see a standalone installer for users of Python 2.7/3.2 and
earlier. It's too useful a feature to not make it available for people
who haven't installed 3.3 yet. And I'd prefer it if that standalone
installer was hosted on python.org for visibility, rather than on
PyPI.

I'm not enough of an MSI expert to know if this can be implemented by
having a standalone MSI, and then "embedding" it in the Python 3.3
MSI. That was what I'd thought of, but Vinay's later email suggests it
might not be advisable:

> AFAICT, although you can embed an MSI inside another one, the practice
> of concurrent/nested installations is strongly discouraged by Microsoft -
> see http://goo.gl/FJx1S (Rule 20).
[...]
> So you could package Python and the launcher as separate MSIs (this would
> make sense so that you could restore associations to the launcher just by
> repairing its installation), but since nested MSIs are a no-no, that means
> installing via a bootstrapping .exe. This is a bigger change to our Windows
> packaging than some people might be comfortable with ...

Paul.

From solipsis at pitrou.net  Tue Jul  5 11:47:48 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 5 Jul 2011 11:47:48 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
	<87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20110705114748.5c9fca6d@pitrou.net>

On Tue, 05 Jul 2011 15:27:00 +0900
"Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> 
> One could also use the somewhat euphemistic "unprofessional language".
> That points to why such changes are justified despite an author's
> right to have her mode of expression respected -- the Python project
> aims at professionalism, and offensive language detracts from it.

I sincerely hope we don't start using the word "professional" to denote
"careful" or "good quality". Most professional work is crap, and that's
why we have open source.

Regards

Antoine.



From solipsis at pitrou.net  Tue Jul  5 11:49:03 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 5 Jul 2011 11:49:03 +0200
Subject: [Python-Dev] cpython (3.2): Issue #12497: Install test/data to
 prevent failures of the various codecmaps
References: <E1Qdv73-0004Lf-OT@dinsdale.python.org>
Message-ID: <20110705114903.4503d138@pitrou.net>

On Tue, 05 Jul 2011 04:11:45 +0200
ned.deily <python-checkins at python.org> wrote:
>  LIBSUBDIRS=	tkinter tkinter/test tkinter/test/test_tkinter \
>  		tkinter/test/test_ttk site-packages test \
> -		test/capath \
> +		test/capath test/data \
>  		test/cjkencodings test/decimaltestdata test/xmltestdata test/subprocessdata \
>  		test/tracedmodules test/encoded_modules \
>  		concurrent concurrent/futures encodings \

Shouldn't we have something less dumb than a hardcoded list of
directories? :)

Regards

Antoine.



From victor.stinner at haypocalc.com  Tue Jul  5 12:09:39 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 5 Jul 2011 12:09:39 +0200
Subject: [Python-Dev] cpython (3.2): Issue #12497: Install test/data to
	prevent failures of the various codecmaps
In-Reply-To: <20110705114903.4503d138@pitrou.net>
References: <E1Qdv73-0004Lf-OT@dinsdale.python.org>
	<20110705114903.4503d138@pitrou.net>
Message-ID: <201107051209.39742.victor.stinner@haypocalc.com>

Le mardi 05 juillet 2011 11:49:03, Antoine Pitrou a ?crit :
> On Tue, 05 Jul 2011 04:11:45 +0200
> 
> ned.deily <python-checkins at python.org> wrote:
> >  LIBSUBDIRS=	tkinter tkinter/test tkinter/test/test_tkinter \
> >  
> >  		tkinter/test/test_ttk site-packages test \
> > 
> > -		test/capath \
> > +		test/capath test/data \
> > 
> >  		test/cjkencodings test/decimaltestdata test/xmltestdata
> >  		test/subprocessdata \ test/tracedmodules test/encoded_modules \
> >  		concurrent concurrent/futures encodings \
> 
> Shouldn't we have something less dumb than a hardcoded list of
> directories? :)

Yes we should, especially because Makefile is not the only file that should be 
fixed: Tools/msi/msi.py

I recently added Lib/test/cjkencodings directory, see issue #12057:

R. David Murray: "Haypo, since you've created a new directory there are 
makefile (and PC build file, I think) updates that will need to be made.  (This 
should be documented in the dev guide if it isn't already.)"

Terry J. Reedy: "I presume and hope David meant the process, as I would have 
no idea how to add a directory. And David did not seem completely sure."

Victor

From vinay_sajip at yahoo.co.uk  Tue Jul  5 13:24:48 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 5 Jul 2011 11:24:48 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
	<loom.20110705T000517-753@post.gmane.org>
	<4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com>
Message-ID: <loom.20110705T131132-838@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> Or an MSI installer may be able to offer a "repair" feature without too 
> much pain.

A few more observations to do with installation:

1. It's been mentioned that a standalone version should be available for use
with earlier Python versions. This could be done with a merge module which can
be used either with a standalone installer or the Python .msi.
2. For the standalone MSI, we will most likely need separate 32-bit and 64-bit
MSIs, because the MSI system will look at the MSI type to determine whether
registry stuff goes in the main registry or the Wow6432Node used for 32-bit
applications.

Regards,

Vinay Sajip


From murman at gmail.com  Tue Jul  5 15:38:54 2011
From: murman at gmail.com (Michael Urman)
Date: Tue, 5 Jul 2011 08:38:54 -0500
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <loom.20110705T095246-691@post.gmane.org>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
	<loom.20110705T000517-753@post.gmane.org>
	<4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com>
	<CADiSq7eLu9=R088wLLnHbckEx1CtnjVQG_L8eweGFyR=Jp0kgQ@mail.gmail.com>
	<loom.20110705T095246-691@post.gmane.org>
Message-ID: <CAOpBPYUVzgiG3tMQtim9wfHbWpksvy-Xwt5jUmvAvQoB2fOahA@mail.gmail.com>

On Tue, Jul 5, 2011 at 03:01, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Were those other Windows apps packaged as .msi, or .exe? AFAICT, although you
> can embed an MSI inside another one, the practice of concurrent/nested
> installations is strongly discouraged by Microsoft - see http://goo.gl/FJx1S
> (Rule 20).

Right, you cannot sanely embed one .msi inside another .msi; the
support for "nested" or "concurrent" installations referred to in that
link is indeed something to avoid.

> So you could package Python and the launcher as separate MSIs (this would make
> sense so that you could restore associations to the launcher just by repairing
> its installation), but since nested MSIs are a no-no, that means installing via
> a bootstrapping .exe. This is a bigger change to our Windows packaging than some
> people might be comfortable with ...

You can certainly jump through all these hoops, but the pieces here
are much more suited towards a component definition that can be shared
among multiple products. If the component always installs to the same
place, has the same GUID, and otherwise only changes by versions the
exe, this is a completely safe correct use of one. Last I knew, msi.py
allocates random GUIDs, so may or may not be suitable for generating
this. If there is only rare need to update this exe, and msi.py has
support for merge modules, that could be one approach.

My recommendation for distributing this: each .msi which wants to
include it should have a component that includes the following. Note
that each .exe (py, pyw) and each architecture (x86, x64) need their
own component with their own static GUID.
 * Defined unchanging GUID
 * Defined target location (perhaps SystemFolder)
 * msidbComponentAttributesSharedDllRefCount (cooperate with non-MSI
installers), msidbComponentAttributesShared (keep highest version),
and possibly msidbComponentAttributesPermanent flags set on the
component
 * Versioned .exe (using a Windows version block)
 * File association information

Then these components can be included in the python 3.3 installer,
future releases, and even a standalone installer, and reference count
correctly. Again, these can optionally be made available as merge
modules for other consumers, but there's likely no need.

-- 
Michael Urman

From ncoghlan at gmail.com  Tue Jul  5 15:57:24 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 5 Jul 2011 23:57:24 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
	<87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7e9KQ8AN3-_O8RDaYO7mGj0KqB4W1h5MwJosgFU1TrtMQ@mail.gmail.com>

On Tue, Jul 5, 2011 at 4:27 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> That points to why such changes are justified despite an author's
> right to have her mode of expression respected -- the Python project
> aims at professionalism, and offensive language detracts from it.

Given that the contents of many test strings are quite arbitrary, I
personally consider a bit of inoffensive humour or cultural references
to be a fine thing to include rather than yet another instance of
"foobar" (which itself has humorous *and* offensive origins). Heck,
stripping just the Monty Python quotes from the test suite would
probably take a while :)

Avoiding offensive text has nothing to do with a desire to appear
"professional" (at least for me) - it's about demonstrating common
courtesy to the potentially wide variety of people that will read the
Python source code in the future.

(In the specific case, I thought quoting the venerable pun was fine,
but I also don't have any real problem with modifying it)

Cheers,
Nick.

P.S. 'twas a sad day when copyright concerns cost us the old test audio file :(

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Tue Jul  5 17:43:53 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 5 Jul 2011 15:43:53 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_397_=28Python_launcher_for_Windows=29_?=
	=?utf-8?q?reference=09implementation?=
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
	<loom.20110630T140918-870@post.gmane.org>
	<4E0D2F42.4040802@gmail.com>
	<loom.20110701T101646-667@post.gmane.org>
	<4E0EC5B7.9010607@gmail.com>
	<loom.20110702T102523-59@post.gmane.org>
	<4E10139B.9050700@gmail.com> <4E1156B2.6050404@canterbury.ac.nz>
	<4E115D45.8000101@gmail.com>
	<loom.20110705T000517-753@post.gmane.org>
	<4E126788.7020705@canterbury.ac.nz> <4E127326.7090202@gmail.com>
	<CADiSq7eLu9=R088wLLnHbckEx1CtnjVQG_L8eweGFyR=Jp0kgQ@mail.gmail.com>
	<loom.20110705T095246-691@post.gmane.org>
	<CAOpBPYUVzgiG3tMQtim9wfHbWpksvy-Xwt5jUmvAvQoB2fOahA@mail.gmail.com>
Message-ID: <loom.20110705T173703-221@post.gmane.org>

Michael Urman <murman <at> gmail.com> writes:

> You can certainly jump through all these hoops, but the pieces here
> are much more suited towards a component definition that can be shared
> among multiple products. If the component always installs to the same
> place, has the same GUID, and otherwise only changes by versions the
> exe, this is a completely safe correct use of one. Last I knew, msi.py
> allocates random GUIDs, so may or may not be suitable for generating
> this. If there is only rare need to update this exe, and msi.py has
> support for merge modules, that could be one approach.

I'm currently experimenting with a merge module approach: launcher_module.msm ->
launcher.msi, and x64 variants in separate .amd64.ms? files.

> My recommendation for distributing this: each .msi which wants to
> include it should have a component that includes the following. Note
> that each .exe (py, pyw) and each architecture (x86, x64) need their
> own component with their own static GUID.
>  * Defined unchanging GUID
>  * Defined target location (perhaps SystemFolder)

I'm doing that.

>  * msidbComponentAttributesSharedDllRefCount (cooperate with non-MSI
> installers), msidbComponentAttributesShared (keep highest version),
> and possibly msidbComponentAttributesPermanent flags set on the
> component

Thanks, that's interesting information. I'll read up about these. I'm a Windows
installer tyro :-)

>  * Versioned .exe (using a Windows version block)

I'm doing that, too.

>  * File association information

Currently I'm putting the file association information in the same component as
the files, since the registry values cross reference those files.

Regards,

Vinay Sajip



From stephen at xemacs.org  Tue Jul  5 18:23:55 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 06 Jul 2011 01:23:55 +0900
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <20110705114748.5c9fca6d@pitrou.net>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
	<87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20110705114748.5c9fca6d@pitrou.net>
Message-ID: <87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > I sincerely hope we don't start using the word "professional" to denote
 > "careful" or "good quality".

No, by "professional" I mean "of a profession," which is a service
that is provided by experts to laymen, and therefore demands adherence
to certain standards since the clients are not able to judge the
product for themselves, but in general must trust the vendor.  The
care and good quality proceed from the commitment of the professional.

 > Most professional work is crap, and that's why we have open source.

Open source is not a substitute for professionalism.



From solipsis at pitrou.net  Tue Jul  5 18:42:29 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 5 Jul 2011 18:42:29 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
	<87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20110705114748.5c9fca6d@pitrou.net>
	<87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20110705184229.037a15ed@pitrou.net>

On Wed, 06 Jul 2011 01:23:55 +0900
"Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Antoine Pitrou writes:
> 
>  > I sincerely hope we don't start using the word "professional" to denote
>  > "careful" or "good quality".
> 
> No, by "professional" I mean "of a profession," which is a service
> that is provided by experts to laymen, and therefore demands adherence
> to certain standards since the clients are not able to judge the
> product for themselves, but in general must trust the vendor.  The
> care and good quality proceed from the commitment of the professional.

I see. But the "experts" are not necessarily vendors (we aren't), and
our users aren't "clients". Besides, we are not merely providing a
service, we are building a community where everyone can take part in
the shared work, thereby blurring the barrier between "experts" and
"laymen".

Regards

Antoine.

From stephen at xemacs.org  Tue Jul  5 19:33:12 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 06 Jul 2011 02:33:12 +0900
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <20110705184229.037a15ed@pitrou.net>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
	<87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20110705114748.5c9fca6d@pitrou.net>
	<87aacselwk.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20110705184229.037a15ed@pitrou.net>
Message-ID: <878vsceip3.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:
 > On Wed, 06 Jul 2011 01:23:55 +0900
 > "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
 > > Antoine Pitrou writes:
 > > 
 > >  > I sincerely hope we don't start using the word "professional" to denote
 > >  > "careful" or "good quality".
 > > 
 > > No, by "professional" I mean "of a profession,"

 > I see. But the "experts" are not necessarily vendors (we aren't), and
 > our users aren't "clients". Besides, we are not merely providing a
 > service, we are building a community where everyone can take part in
 > the shared work, thereby blurring the barrier between "experts" and
 > "laymen".

*sigh*  And another good word bites the dust.  OK, I will reserve the
adjective "professional" for those who appreciate it.  Nick's "common
courtesy" will do for the current purpose.



From drsalists at gmail.com  Tue Jul  5 21:12:24 2011
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 5 Jul 2011 12:12:24 -0700
Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails
In-Reply-To: <CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
References: <ipp0u8$q8f$1@news-cypress.fernuni-hagen.de>
	<0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com>
	<4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com>
	<CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
Message-ID: <CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>

On Tue, Jul 5, 2011 at 7:25 AM, David Robinow <drobinow at gmail.com> wrote:

>
>  Cygwin is not really a supported platform.

...

> [Ultimately somebody with an
> interest in cygwin will need to get active in python development. I've
> been meaning to do this but life gets in the way.]
>

I was bitten by the lack of Cygwin support in 3.2 as well.

IMO,  python-dev needs continuous integration on a build farm that includes
representative platforms.  Most of the machines in the farm could be
virtualboxes.

I don't think the problem is so much that the right people haven't gotten
involved, as that the currently-involved people don't know when they're
breaking something for someone else due to the lack of continuous
integration.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/65f47db6/attachment.html>

From brian.curtin at gmail.com  Tue Jul  5 21:18:22 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jul 2011 14:18:22 -0500
Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails
In-Reply-To: <CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>
References: <ipp0u8$q8f$1@news-cypress.fernuni-hagen.de>
	<0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com>
	<4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com>
	<CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
	<CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>
Message-ID: <CAD+XWwo_dcrjcQjP3Lf_jijE8LhWmcj6GDyTUqkgEhLd4SAnTw@mail.gmail.com>

On Tue, Jul 5, 2011 at 14:12, Dan Stromberg <drsalists at gmail.com> wrote:

>
> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow <drobinow at gmail.com> wrote:
>
>>
>>  Cygwin is not really a supported platform.
>
> ...
>
>> [Ultimately somebody with an
>> interest in cygwin will need to get active in python development. I've
>> been meaning to do this but life gets in the way.]
>>
>
> I was bitten by the lack of Cygwin support in 3.2 as well.
>
> IMO,  python-dev needs continuous integration on a build farm that
> includes representative platforms.  Most of the machines in the farm could
> be virtualboxes.
>
> I don't think the problem is so much that the right people haven't gotten
> involved, as that the currently-involved people don't know when they're
> breaking something for someone else due to the lack of continuous
> integration.
>

We've had that for some time now: http://www.python.org/dev/buildbot/

There are several issues on the bug tracker about cygwin build issues, but
to my knowledge, none of them have included successful patches.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/520c851e/attachment.html>

From drsalists at gmail.com  Tue Jul  5 21:41:33 2011
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 5 Jul 2011 12:41:33 -0700
Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails
In-Reply-To: <CAD+XWwo_dcrjcQjP3Lf_jijE8LhWmcj6GDyTUqkgEhLd4SAnTw@mail.gmail.com>
References: <ipp0u8$q8f$1@news-cypress.fernuni-hagen.de>
	<0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com>
	<4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com>
	<CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
	<CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>
	<CAD+XWwo_dcrjcQjP3Lf_jijE8LhWmcj6GDyTUqkgEhLd4SAnTw@mail.gmail.com>
Message-ID: <CAGGBd_p_Y6hv7zyfUC_kHor5DBebXJr34agdH=dAXc0JisAMVw@mail.gmail.com>

On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Tue, Jul 5, 2011 at 14:12, Dan Stromberg <drsalists at gmail.com> wrote:
>
>>
>> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow <drobinow at gmail.com> wrote:
>>
>>>
>>>  Cygwin is not really a supported platform.
>>
>> ...
>>
>>> [Ultimately somebody with an
>>> interest in cygwin will need to get active in python development. I've
>>> been meaning to do this but life gets in the way.]
>>>
>>
>> I was bitten by the lack of Cygwin support in 3.2 as well.
>>
>> IMO,  python-dev needs continuous integration on a build farm that
>> includes representative platforms.  Most of the machines in the farm could
>> be virtualboxes.
>>
>> I don't think the problem is so much that the right people haven't gotten
>> involved, as that the currently-involved people don't know when they're
>> breaking something for someone else due to the lack of continuous
>> integration.
>>
>
> We've had that for some time now: http://www.python.org/dev/buildbot/
>

Good to know.  Apologies for my incorrect assumption.  Where do the e-mail
notifications of build and/or test failures go?

Shouldn't Cygwin be represented here?  I don't see it in the list of builds.

Some shops have a policy that nothing gets merged into trunk unless it's
passing critical automated tests...  Would that work here?

There are several issues on the bug tracker about cygwin build issues, but
> to my knowledge, none of them have included successful patches.
>

I think you'll find that most people using Cygwin would rather be working on
some other OS, but are forced to use Windows for policy reasons.  It's
remains a rather significant need in many cases.

How does the buildbot initiate builds?  ssh?  Happily Cygwin mostly allows
sshd (as long as you don't need a CIFS share or something).

Native Windows builds do appear to be represented.  Is there any reason not
to set up a buildbot for Cygwin on the same (virtual?) hardware?

I'm inclined to believe that whoever rearranged the shared object stuff in
CPython 3.x's build system would've been more careful if they'd had feedback
about what it was doing to Cygwin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/df5b73b4/attachment-0001.html>

From brian.curtin at gmail.com  Tue Jul  5 22:00:24 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jul 2011 15:00:24 -0500
Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails
In-Reply-To: <CAGGBd_p_Y6hv7zyfUC_kHor5DBebXJr34agdH=dAXc0JisAMVw@mail.gmail.com>
References: <ipp0u8$q8f$1@news-cypress.fernuni-hagen.de>
	<0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com>
	<4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com>
	<CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
	<CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>
	<CAD+XWwo_dcrjcQjP3Lf_jijE8LhWmcj6GDyTUqkgEhLd4SAnTw@mail.gmail.com>
	<CAGGBd_p_Y6hv7zyfUC_kHor5DBebXJr34agdH=dAXc0JisAMVw@mail.gmail.com>
Message-ID: <CAD+XWwpJzmTQSP7UdM+sR=h3wrd_xNjBN3ZVo+NiM_mHVMdj1A@mail.gmail.com>

On Tue, Jul 5, 2011 at 14:41, Dan Stromberg <drsalists at gmail.com> wrote:

>
> On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin <brian.curtin at gmail.com>wrote:
>
>> On Tue, Jul 5, 2011 at 14:12, Dan Stromberg <drsalists at gmail.com> wrote:
>>
>>>
>>> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow <drobinow at gmail.com>wrote:
>>>
>>>>
>>>>  Cygwin is not really a supported platform.
>>>
>>> ...
>>>
>>>> [Ultimately somebody with an
>>>> interest in cygwin will need to get active in python development. I've
>>>> been meaning to do this but life gets in the way.]
>>>>
>>>
>>> I was bitten by the lack of Cygwin support in 3.2 as well.
>>>
>>> IMO,  python-dev needs continuous integration on a build farm that
>>> includes representative platforms.  Most of the machines in the farm could
>>> be virtualboxes.
>>>
>>> I don't think the problem is so much that the right people haven't gotten
>>> involved, as that the currently-involved people don't know when they're
>>> breaking something for someone else due to the lack of continuous
>>> integration.
>>>
>>
>> We've had that for some time now: http://www.python.org/dev/buildbot/
>>
>
> Good to know.  Apologies for my incorrect assumption.  Where do the e-mail
> notifications of build and/or test failures go?
>

There might be an RSS feed or something, but I don't think there's any email
notification. #python-dev on IRC receives live failure info. Other than
that, you'd just have to look at one of the views of the fleet to see which
build slaves are failing.


> Shouldn't Cygwin be represented here?  I don't see it in the list of
> builds.
>

Probably, but it isn't represented because no one contributed a build slave
for it. I know some of the other Windows build slave operators use Cygwin to
some degree, but I'm not sure if anyone has looked into actually setting up
a build slave for it.

Some shops have a policy that nothing gets merged into trunk unless it's
> passing critical automated tests...  Would that work here?
>

We don't make much use of branching, but that would work if we did. If no
one is actively contributing work on the Cygwin build then I don't see us
holding up work in order to figure out any Cygwin-specific issues.

There are several issues on the bug tracker about cygwin build issues, but
>> to my knowledge, none of them have included successful patches.
>>
>
> I think you'll find that most people using Cygwin would rather be working
> on some other OS, but are forced to use Windows for policy reasons.  It's
> remains a rather significant need in many cases.
>

I don't disagree with that, but if there's no one contributing Cygwin
patches then it will probably just die off and we'll have situations like
the current one where it doesn't build. A great majority of the contributing
developers are on UNIX-based systems with no access to Windows. A small
handful, myself included, are Windows users, but I don't think any of us use
Cygwin (I don't).

Native Windows builds do appear to be represented.  Is there any reason not
> to set up a buildbot for Cygwin on the same (virtual?) hardware?
>

Besides the time and effort needed to set it up and occasionally look over
it, no. We'd have to have a successfully compiling Cygwin build before we
think about adding a build slave for it.

I wouldn't be opposed to hosting this myself, but I need to steal some time
and get my Windows 2008 build slave back to some form of a functional system
- it has been up and down for a few months now. If someone else is
interested, go ahead.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/526ad820/attachment.html>

From drsalists at gmail.com  Tue Jul  5 22:10:14 2011
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 5 Jul 2011 13:10:14 -0700
Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails
In-Reply-To: <CAD+XWwpJzmTQSP7UdM+sR=h3wrd_xNjBN3ZVo+NiM_mHVMdj1A@mail.gmail.com>
References: <ipp0u8$q8f$1@news-cypress.fernuni-hagen.de>
	<0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com>
	<4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com>
	<CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
	<CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>
	<CAD+XWwo_dcrjcQjP3Lf_jijE8LhWmcj6GDyTUqkgEhLd4SAnTw@mail.gmail.com>
	<CAGGBd_p_Y6hv7zyfUC_kHor5DBebXJr34agdH=dAXc0JisAMVw@mail.gmail.com>
	<CAD+XWwpJzmTQSP7UdM+sR=h3wrd_xNjBN3ZVo+NiM_mHVMdj1A@mail.gmail.com>
Message-ID: <CAGGBd_pfOESg4r_oovbcmMoRKj=NsV_avDML_pp0-gmvQR68Lg@mail.gmail.com>

On Tue, Jul 5, 2011 at 1:00 PM, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Tue, Jul 5, 2011 at 14:41, Dan Stromberg <drsalists at gmail.com> wrote:
>
>>
>> On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin <brian.curtin at gmail.com>wrote:
>>
>>> On Tue, Jul 5, 2011 at 14:12, Dan Stromberg <drsalists at gmail.com> wrote:
>>>
>>>>
>>>> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow <drobinow at gmail.com>wrote:
>>>>
>>>>>
>>>>>  Cygwin is not really a supported platform.
>>>>
>>>> ...
>>>>
>>>>> [Ultimately somebody with an
>>>>> interest in cygwin will need to get active in python development. I've
>>>>> been meaning to do this but life gets in the way.]
>>>>>
>>>>
>>>> I was bitten by the lack of Cygwin support in 3.2 as well.
>>>>
>>>> IMO,  python-dev needs continuous integration on a build farm that
>>>> includes representative platforms.  Most of the machines in the farm could
>>>> be virtualboxes.
>>>>
>>>> I don't think the problem is so much that the right people haven't
>>>> gotten involved, as that the currently-involved people don't know when
>>>> they're breaking something for someone else due to the lack of continuous
>>>> integration.
>>>>
>>>
>>> We've had that for some time now: http://www.python.org/dev/buildbot/
>>>
>>
>> Good to know.  Apologies for my incorrect assumption.  Where do the e-mail
>> notifications of build and/or test failures go?
>>
>
> There might be an RSS feed or something, but I don't think there's any
> email notification. #python-dev on IRC receives live failure info. Other
> than that, you'd just have to look at one of the views of the fleet to see
> which build slaves are failing.
>

Am I correct in assuming that "stable" buildbots are required to be
reasonably functional before a release is tagged?


> Shouldn't Cygwin be represented here?  I don't see it in the list of
>> builds.
>>
>
> Probably, but it isn't represented because no one contributed a build slave
> for it. I know some of the other Windows build slave operators use Cygwin to
> some degree, but I'm not sure if anyone has looked into actually setting up
> a build slave for it.
>

I see.


> Some shops have a policy that nothing gets merged into trunk unless it's
>> passing critical automated tests...  Would that work here?
>>
>
> We don't make much use of branching, but that would work if we did. If no
> one is actively contributing work on the Cygwin build then I don't see us
> holding up work in order to figure out any Cygwin-specific issues.
>

I might suggest that doing so (using branching, keeping trunk stable) might
be of benefit, especially with a DVCS.


>
> There are several issues on the bug tracker about cygwin build issues, but
>>> to my knowledge, none of them have included successful patches.
>>>
>>
>> I think you'll find that most people using Cygwin would rather be working
>> on some other OS, but are forced to use Windows for policy reasons.  It's
>> remains a rather significant need in many cases.
>>
>
> I don't disagree with that, but if there's no one contributing Cygwin
> patches then it will probably just die off and we'll have situations like
> the current one where it doesn't build. A great majority of the contributing
> developers are on UNIX-based systems with no access to Windows. A small
> handful, myself included, are Windows users, but I don't think any of us use
> Cygwin (I don't).
>

I see.

Is there a python.org resource for setting up mailing lists - say, for a
python-cygwin mailing list?



> Native Windows builds do appear to be represented.  Is there any reason not
>> to set up a buildbot for Cygwin on the same (virtual?) hardware?
>>
>
> Besides the time and effort needed to set it up and occasionally look over
> it, no. We'd have to have a successfully compiling Cygwin build before we
> think about adding a build slave for it.
>

That makes sense.


> I wouldn't be opposed to hosting this myself, but I need to steal some time
> and get my Windows 2008 build slave back to some form of a functional system
> - it has been up and down for a few months now. If someone else is
> interested, go ahead.
>

I might contribute some elbow grease if someone could get me VNC access to a
suitable Windows server.

BTW, is there someone available who is familiar with the meanings of the
various shared object-related make symbols?  I glanced at them and didn't
find their naming astonishingly clear - seems like something to document,
or...  maybe it already is, and I just haven't seen where it is yet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/19af5f58/attachment.html>

From victor.stinner at haypocalc.com  Tue Jul  5 23:41:22 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 05 Jul 2011 23:41:22 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #12452: Plist and
 Dict are now deprecated
In-Reply-To: <4E12FC8E.2070609@trueblade.com>
References: <E1QdiGg-0003FE-O4@dinsdale.python.org>
	<4E12FC8E.2070609@trueblade.com>
Message-ID: <1309902082.24795.2.camel@marge>

Le mardi 05 juillet 2011 ? 07:59 -0400, Eric Smith a ?crit :
> On 7/4/2011 8:28 AM, victor.stinner wrote:
> > http://hg.python.org/cpython/rev/4f14050a963f
> > changeset:   71194:4f14050a963f
> > user:        Victor Stinner <victor.stinner at haypocalc.com>
> > date:        Mon Jul 04 14:28:45 2011 +0200
> > summary:
> >   Issue #12452: Plist and Dict are now deprecated
> > 
> > Replace PendingDeprecationWarning warnings by DeprecationWarning.
> 
> Shouldn't this be in MISC/News, be accompanied by documentation changes,
> and have tests?

Plist and Dict were never documented (in Doc/library/plistlib.rst).
These classes have no test.

You mean that I should add an entry to Misc/NEWS saying that these
classe are now deprecated? Should I also mention the deprecation to the
"What's new in Python 3.3?" document?

Victor


From eric at trueblade.com  Tue Jul  5 23:52:21 2011
From: eric at trueblade.com (Eric Smith)
Date: Tue, 05 Jul 2011 17:52:21 -0400
Subject: [Python-Dev] [Python-checkins] cpython: Issue #12452: Plist and
 Dict are now deprecated
In-Reply-To: <1309902082.24795.2.camel@marge>
References: <E1QdiGg-0003FE-O4@dinsdale.python.org>	<4E12FC8E.2070609@trueblade.com>
	<1309902082.24795.2.camel@marge>
Message-ID: <4E138795.4090707@trueblade.com>

> Plist and Dict were never documented (in Doc/library/plistlib.rst).
> These classes have no test.

Ouch!

> You mean that I should add an entry to Misc/NEWS saying that these
> classe are now deprecated? Should I also mention the deprecation to the
> "What's new in Python 3.3?" document?

Yes. I think this should make it into a "What's new" document, usually
via Misc/NEWS. Thanks.

Eric.

From guido at python.org  Wed Jul  6 00:29:34 2011
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jul 2011 15:29:34 -0700
Subject: [Python-Dev] cpython: Issue #12469: replace assertions by
 explicit if+raise
In-Reply-To: <1309805706.30323.4.camel@marge>
References: <E1Qdlfh-0004B5-BT@dinsdale.python.org>
	<20110704182344.7bad6b62@pitrou.net>
	<1309805706.30323.4.camel@marge>
Message-ID: <CAP7+vJLwSQ=MT5S5OpgwHdO4F65S2FhyYwwHL18NJhenVGxyJg@mail.gmail.com>

Exception is for catching, not raising.
On Jul 4, 2011 11:57 AM, "Victor Stinner" <victor.stinner at haypocalc.com>
wrote:
> Le lundi 04 juillet 2011 ? 18:23 +0200, Antoine Pitrou a ?crit :
>> On Mon, 04 Jul 2011 18:06:53 +0200
>> victor.stinner <python-checkins at python.org> wrote:
>> > http://hg.python.org/cpython/rev/7eef821ab20d
>> > changeset: 71197:7eef821ab20d
>> > user: Victor Stinner <victor.stinner at haypocalc.com>
>> > date: Mon Jul 04 18:06:35 2011 +0200
>> > summary:
>> > Issue #12469: replace assertions by explicit if+raise
>
>
>> Instead of generic Exception, it would be better to use AssertionError.
>
> and
>
>> or in many cases given this was in unittests... use the self.assertFoo
>> methods and avoid assert and if statements all together.
>
> The code is running in a subprocess (python -c ...), not in an
> unittest.TestCase, so I cannot use self.assertFoo and it doesn't really
> matter if the exception is an Exception or an AssertionError.
>
>> What is the reason for this change anyway -- as far as I can
>> see this code is never run with -O.
>
> I'm not sure that the code will never be running using -O, so I prefer
> to use an explicit if+raise. I don't like the assert statement because
> it doesn't provide any information about the failure (content of the
> variables) by default.
>
> Victor
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/6625e35e/attachment.html>

From guido at python.org  Wed Jul  6 00:50:09 2011
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jul 2011 15:50:09 -0700
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <E1QdQNb-0003Ir-03@dinsdale.python.org>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
Message-ID: <CAP7+vJ+QYb13HLkuR=2=vd8f9u5f5AB_opDuSoC9tbJzBK3YAA@mail.gmail.com>

It's not a bug and shouldn't be "fixed". We leave lots of minor infractions
in the code because the code churn of fixing them all would be too
distracting.
On Jul 3, 2011 10:22 AM, "georg.brandl" <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/76452b892838
> changeset: 71146:76452b892838
> parent: 71144:ce52310f61a0
> user: Georg Brandl <georg at python.org>
> date: Sun Jul 03 19:22:42 2011 +0200
> summary:
> Remove mention of medical condition from the test suite.
>
> files:
> Lib/test/test_csv.py | 8 ++++----
> 1 files changed, 4 insertions(+), 4 deletions(-)
>
>
> diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
> --- a/Lib/test/test_csv.py
> +++ b/Lib/test/test_csv.py
> @@ -459,20 +459,20 @@
> '5', '6']])
>
> def test_quoted_quote(self):
> - self.readerAssertEqual('1,2,3,"""I see,"" said the blind man","as he
picked up his hammer and saw"',
> + self.readerAssertEqual('1,2,3,"""I see,"" said the happy man","as he
picked up his hammer and saw"',
> [['1', '2', '3',
> - '"I see," said the blind man',
> + '"I see," said the happy man',
> 'as he picked up his hammer and saw']])
>
> def test_quoted_nl(self):
> input = '''\
> 1,2,3,"""I see,""
> -said the blind man","as he picked up his
> +said the happy man","as he picked up his
> hammer and saw"
> 9,8,7,6'''
> self.readerAssertEqual(input,
> [['1', '2', '3',
> - '"I see,"\nsaid the blind man',
> + '"I see,"\nsaid the happy man',
> 'as he picked up his\nhammer and saw'],
> ['9','8','7','6']])
>
>
> --
> Repository URL: http://hg.python.org/cpython
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/f8fc5b4e/attachment.html>

From guido at python.org  Wed Jul  6 02:35:44 2011
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jul 2011 17:35:44 -0700
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <CAP7+vJ+QYb13HLkuR=2=vd8f9u5f5AB_opDuSoC9tbJzBK3YAA@mail.gmail.com>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CAP7+vJ+QYb13HLkuR=2=vd8f9u5f5AB_opDuSoC9tbJzBK3YAA@mail.gmail.com>
Message-ID: <CAP7+vJL7j6kCo8cCrgUR2U5XwtKS9KkLTrzex25ZVp8On7tiLA@mail.gmail.com>

To clarify, now that I have access to an actual keyboard instead of
just a cellphone: I think it should be rolled back, since the proper
process for controversial changes was not followed. Our process (part
of our culture, if you will) for anything controversial is to discuss
the change first, then, if deemed necessary, fix the code. And from
the size of this thread it clearly is controversial. Georg, can you
please revert this change?

Note that another part of our process/culture is that we try not to
engage in battling commits, i.e. generally we don't unilaterally roll
back a change just to make the point that it is incorrect; we ask the
original committer to roll it back.

As to the controversy itself, if you want to make blind people fell
more at home in the Python community surely there are more useful
things to do than to remove puns involving blindness; e.g. improve
accessibility of python.org or some part of it. Or maybe find some
blind programmers and ask them what would help them.

--Guido

On Tue, Jul 5, 2011 at 3:50 PM, Guido van Rossum <guido at python.org> wrote:
> It's not a bug and shouldn't be "fixed". We leave lots of minor infractions
> in the code because the code churn of fixing them all would be too
> distracting.
>
> On Jul 3, 2011 10:22 AM, "georg.brandl" <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/76452b892838
>> changeset: 71146:76452b892838
>> parent: 71144:ce52310f61a0
>> user: Georg Brandl <georg at python.org>
>> date: Sun Jul 03 19:22:42 2011 +0200
>> summary:
>> Remove mention of medical condition from the test suite.
>>
>> files:
>> Lib/test/test_csv.py | 8 ++++----
>> 1 files changed, 4 insertions(+), 4 deletions(-)
>>
>>
>> diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
>> --- a/Lib/test/test_csv.py
>> +++ b/Lib/test/test_csv.py
>> @@ -459,20 +459,20 @@
>> '5', '6']])
>>
>> def test_quoted_quote(self):
>> - self.readerAssertEqual('1,2,3,"""I see,"" said the blind man","as he
>> picked up his hammer and saw"',
>> + self.readerAssertEqual('1,2,3,"""I see,"" said the happy man","as he
>> picked up his hammer and saw"',
>> [['1', '2', '3',
>> - '"I see," said the blind man',
>> + '"I see," said the happy man',
>> 'as he picked up his hammer and saw']])
>>
>> def test_quoted_nl(self):
>> input = '''\
>> 1,2,3,"""I see,""
>> -said the blind man","as he picked up his
>> +said the happy man","as he picked up his
>> hammer and saw"
>> 9,8,7,6'''
>> self.readerAssertEqual(input,
>> [['1', '2', '3',
>> - '"I see,"\nsaid the blind man',
>> + '"I see,"\nsaid the happy man',
>> 'as he picked up his\nhammer and saw'],
>> ['9','8','7','6']])
>>
>>
>> --
>> Repository URL: http://hg.python.org/cpython
>



-- 
--Guido van Rossum (python.org/~guido)

From brian.curtin at gmail.com  Wed Jul  6 02:38:29 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jul 2011 19:38:29 -0500
Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails
In-Reply-To: <CAGGBd_pfOESg4r_oovbcmMoRKj=NsV_avDML_pp0-gmvQR68Lg@mail.gmail.com>
References: <ipp0u8$q8f$1@news-cypress.fernuni-hagen.de>
	<0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com>
	<4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com>
	<CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
	<CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>
	<CAD+XWwo_dcrjcQjP3Lf_jijE8LhWmcj6GDyTUqkgEhLd4SAnTw@mail.gmail.com>
	<CAGGBd_p_Y6hv7zyfUC_kHor5DBebXJr34agdH=dAXc0JisAMVw@mail.gmail.com>
	<CAD+XWwpJzmTQSP7UdM+sR=h3wrd_xNjBN3ZVo+NiM_mHVMdj1A@mail.gmail.com>
	<CAGGBd_pfOESg4r_oovbcmMoRKj=NsV_avDML_pp0-gmvQR68Lg@mail.gmail.com>
Message-ID: <CAD+XWwpa54pUD40cOO8SOwRhZuM25bc-rbKMQzAWea4A=fSdBg@mail.gmail.com>

On Tue, Jul 5, 2011 at 15:10, Dan Stromberg <drsalists at gmail.com> wrote:

>
> On Tue, Jul 5, 2011 at 1:00 PM, Brian Curtin <brian.curtin at gmail.com>wrote:
>
>> On Tue, Jul 5, 2011 at 14:41, Dan Stromberg <drsalists at gmail.com> wrote:
>>
>>>
>>> On Tue, Jul 5, 2011 at 12:18 PM, Brian Curtin <brian.curtin at gmail.com>wrote:
>>>
>>>> On Tue, Jul 5, 2011 at 14:12, Dan Stromberg <drsalists at gmail.com>wrote:
>>>>
>>>>>
>>>>> On Tue, Jul 5, 2011 at 7:25 AM, David Robinow <drobinow at gmail.com>wrote:
>>>>>
>>>>>>
>>>>>>  Cygwin is not really a supported platform.
>>>>>
>>>>> ...
>>>>>
>>>>>> [Ultimately somebody with an
>>>>>> interest in cygwin will need to get active in python development. I've
>>>>>> been meaning to do this but life gets in the way.]
>>>>>>
>>>>>
>>>>> I was bitten by the lack of Cygwin support in 3.2 as well.
>>>>>
>>>>> IMO,  python-dev needs continuous integration on a build farm that
>>>>> includes representative platforms.  Most of the machines in the farm could
>>>>> be virtualboxes.
>>>>>
>>>>> I don't think the problem is so much that the right people haven't
>>>>> gotten involved, as that the currently-involved people don't know when
>>>>> they're breaking something for someone else due to the lack of continuous
>>>>> integration.
>>>>>
>>>>
>>>> We've had that for some time now: http://www.python.org/dev/buildbot/
>>>>
>>>
>>> Good to know.  Apologies for my incorrect assumption.  Where do the
>>> e-mail notifications of build and/or test failures go?
>>>
>>
>> There might be an RSS feed or something, but I don't think there's any
>> email notification. #python-dev on IRC receives live failure info. Other
>> than that, you'd just have to look at one of the views of the fleet to see
>> which build slaves are failing.
>>
>
> Am I correct in assuming that "stable" buildbots are required to be
> reasonably functional before a release is tagged?
>

Yep - all green is the goal.


>
>
>>  Shouldn't Cygwin be represented here?  I don't see it in the list of
>>> builds.
>>>
>>
>> Probably, but it isn't represented because no one contributed a build
>> slave for it. I know some of the other Windows build slave operators use
>> Cygwin to some degree, but I'm not sure if anyone has looked into actually
>> setting up a build slave for it.
>>
>
> I see.
>
>
>> Some shops have a policy that nothing gets merged into trunk unless it's
>>> passing critical automated tests...  Would that work here?
>>>
>>
>> We don't make much use of branching, but that would work if we did. If no
>> one is actively contributing work on the Cygwin build then I don't see us
>> holding up work in order to figure out any Cygwin-specific issues.
>>
>
> I might suggest that doing so (using branching, keeping trunk stable) might
> be of benefit, especially with a DVCS.
>
>
>>
>> There are several issues on the bug tracker about cygwin build issues, but
>>>> to my knowledge, none of them have included successful patches.
>>>>
>>>
>>> I think you'll find that most people using Cygwin would rather be working
>>> on some other OS, but are forced to use Windows for policy reasons.  It's
>>> remains a rather significant need in many cases.
>>>
>>
>> I don't disagree with that, but if there's no one contributing Cygwin
>> patches then it will probably just die off and we'll have situations like
>> the current one where it doesn't build. A great majority of the contributing
>> developers are on UNIX-based systems with no access to Windows. A small
>> handful, myself included, are Windows users, but I don't think any of us use
>> Cygwin (I don't).
>>
>
> I see.
>
> Is there a python.org resource for setting up mailing lists - say, for a
> python-cygwin mailing list?
>

You might want to suggest something like cygwin-sig as a mailing list. Check
out http://www.python.org/community/sigs/guidelines/ for more info.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110705/51a9fe8f/attachment.html>

From ncoghlan at gmail.com  Wed Jul  6 06:33:35 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 6 Jul 2011 14:33:35 +1000
Subject: [Python-Dev] Compiling Python 3.2 on Cygwin fails
In-Reply-To: <CAD+XWwpa54pUD40cOO8SOwRhZuM25bc-rbKMQzAWea4A=fSdBg@mail.gmail.com>
References: <ipp0u8$q8f$1@news-cypress.fernuni-hagen.de>
	<0a6de4fa-aa64-4485-a6da-72cf1bdad739@m9g2000yqe.googlegroups.com>
	<4e40b036-e232-45d5-be9d-b9019fe4b414@gh5g2000vbb.googlegroups.com>
	<CAF3Xjbwff5jcMhySb812dyMm0EduaqN3A9S7CNphFzMoMTuYLA@mail.gmail.com>
	<CAGGBd_pYc-yT5JrNRv0TqpfEw2=fMWiBTuA39sm6bEtuEn4hiQ@mail.gmail.com>
	<CAD+XWwo_dcrjcQjP3Lf_jijE8LhWmcj6GDyTUqkgEhLd4SAnTw@mail.gmail.com>
	<CAGGBd_p_Y6hv7zyfUC_kHor5DBebXJr34agdH=dAXc0JisAMVw@mail.gmail.com>
	<CAD+XWwpJzmTQSP7UdM+sR=h3wrd_xNjBN3ZVo+NiM_mHVMdj1A@mail.gmail.com>
	<CAGGBd_pfOESg4r_oovbcmMoRKj=NsV_avDML_pp0-gmvQR68Lg@mail.gmail.com>
	<CAD+XWwpa54pUD40cOO8SOwRhZuM25bc-rbKMQzAWea4A=fSdBg@mail.gmail.com>
Message-ID: <CADiSq7ezCukRmd3__X31KW2VwDq43K+skv5e9AaP3r_Q-gBJWg@mail.gmail.com>

On Wed, Jul 6, 2011 at 10:38 AM, Brian Curtin <brian.curtin at gmail.com> wrote:
> On Tue, Jul 5, 2011 at 15:10, Dan Stromberg <drsalists at gmail.com> wrote:
>> Am I correct in assuming that "stable" buildbots are required to be
>> reasonably functional before a release is tagged?
>
> Yep - all green is the goal.

Indeed, that's the main difference between the stable and unstable buildbots.

stable = this should work. If it doesn't, somebody broke something and
the relevant branch should be fixed
unstable = someone cared enough to set up this buildbot, but due to
problems with either the platform in general or the specific machine
it spends a lot of its time red for reasons that aren't the fault of
recent changes to Python

A Cygwin buildbot would start in the latter category then potentially
migrate to stable if it proved itself with green results over a period
of time.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Wed Jul  6 08:49:32 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 6 Jul 2011 16:49:32 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Remove mention of
 medical condition from the test suite.
In-Reply-To: <iuvrf0$vbq$1@dough.gmane.org>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CA+OGgf4B6Qsb0AODsB6CRoMnBiCc5Lu-2Rt_fxxo1JGsO6hRig@mail.gmail.com>
	<87fwmldyyz.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7e9KQ8AN3-_O8RDaYO7mGj0KqB4W1h5MwJosgFU1TrtMQ@mail.gmail.com>
	<iuvrf0$vbq$1@dough.gmane.org>
Message-ID: <CADiSq7fm_m5ymgEn7BfJNGT+neU_cCyWBUUb-dqSjZUq0LOHTg@mail.gmail.com>

On Wed, Jul 6, 2011 at 6:17 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> For this test string, a) I'm not a native speaker and therefore don't know of
> any special treatment this pun deserves

It's not an especially *good* joke, just a very old one that plays on
double meanings of both "see" (as in sight and understanding) and
"saw" (as in sight and a tool). Still, I'd put it in the same category
as the Monty Python quotes we have scattered around the test suite -
if people came across them and didn't realise they were quotes they
might be puzzled, but attempting to retain that Pythonesque sense of
humour is itself part of what makes the Python community what it is.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From stefan_ml at behnel.de  Wed Jul  6 08:51:21 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 06 Jul 2011 08:51:21 +0200
Subject: [Python-Dev] cpython: Remove mention of medical condition from
	the test suite.
In-Reply-To: <iv0s5c$21v$1@dough.gmane.org>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>	<CAP7+vJ+QYb13HLkuR=2=vd8f9u5f5AB_opDuSoC9tbJzBK3YAA@mail.gmail.com>	<CAP7+vJL7j6kCo8cCrgUR2U5XwtKS9KkLTrzex25ZVp8On7tiLA@mail.gmail.com>
	<iv0s5c$21v$1@dough.gmane.org>
Message-ID: <iv10la$pev$1@dough.gmane.org>

Georg Brandl, 06.07.2011 07:35:
> Well, it was stated that even non-joking use of such words can offend
> (the example given was "your argument is blind to (these facts)").
> I consider use in jokes to be more serious, since it's careless use.
> Sorry if I overreacted here.

There's a common saying that a disabled person can't be considered "normal" 
unless he/she can also be called an asshole from time to time.

Personally, I do not consider the pun in question harmful, simply because 
it's so clearly just a play on words that doesn't make much sense, apart 
from having a double meaning.

In general, however, I think it's important to make jokes with and about 
disabled people. That keeps them inside of our society, just like anyone 
else. Treating them as "untouchable" means to single them out.

On that note, if anyone of you ever makes it to the beautiful village of 
Amersfoort (NL), don't miss the "Downey's" caf?. Nice and friendly place to be.

Stefan


From g.brandl at gmx.net  Wed Jul  6 09:08:17 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 06 Jul 2011 09:08:17 +0200
Subject: [Python-Dev] cpython: Remove mention of medical condition from
	the test suite.
In-Reply-To: <iv10la$pev$1@dough.gmane.org>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>	<CAP7+vJ+QYb13HLkuR=2=vd8f9u5f5AB_opDuSoC9tbJzBK3YAA@mail.gmail.com>	<CAP7+vJL7j6kCo8cCrgUR2U5XwtKS9KkLTrzex25ZVp8On7tiLA@mail.gmail.com>
	<iv0s5c$21v$1@dough.gmane.org> <iv10la$pev$1@dough.gmane.org>
Message-ID: <iv11jq$sk8$1@dough.gmane.org>

Am 06.07.2011 08:51, schrieb Stefan Behnel:
> Georg Brandl, 06.07.2011 07:35:
>> Well, it was stated that even non-joking use of such words can offend
>> (the example given was "your argument is blind to (these facts)").
>> I consider use in jokes to be more serious, since it's careless use.
>> Sorry if I overreacted here.
> 
> There's a common saying that a disabled person can't be considered "normal" 
> unless he/she can also be called an asshole from time to time.
> 
> Personally, I do not consider the pun in question harmful, simply because 
> it's so clearly just a play on words that doesn't make much sense, apart 
> from having a double meaning.
> 
> In general, however, I think it's important to make jokes with and about 
> disabled people. That keeps them inside of our society, just like anyone 
> else. Treating them as "untouchable" means to single them out.

I don't think we need to rehash this here (not meaning to pick on you, Stefan);
those who are interested can sign up to the diversity list and read/post to
the original thread here:

http://mail.python.org/mailman/private/diversity/2011-July/002509.html

Georg


From stefan_ml at behnel.de  Wed Jul  6 09:13:58 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 06 Jul 2011 09:13:58 +0200
Subject: [Python-Dev] cpython: Remove mention of medical condition from
	the test suite.
In-Reply-To: <iv11jq$sk8$1@dough.gmane.org>
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>	<CAP7+vJ+QYb13HLkuR=2=vd8f9u5f5AB_opDuSoC9tbJzBK3YAA@mail.gmail.com>	<CAP7+vJL7j6kCo8cCrgUR2U5XwtKS9KkLTrzex25ZVp8On7tiLA@mail.gmail.com>	<iv0s5c$21v$1@dough.gmane.org>
	<iv10la$pev$1@dough.gmane.org> <iv11jq$sk8$1@dough.gmane.org>
Message-ID: <iv11vn$via$1@dough.gmane.org>

Georg Brandl, 06.07.2011 09:08:
> Am 06.07.2011 08:51, schrieb Stefan Behnel:
>> Georg Brandl, 06.07.2011 07:35:
>>> Well, it was stated that even non-joking use of such words can offend
>>> (the example given was "your argument is blind to (these facts)").
>>> I consider use in jokes to be more serious, since it's careless use.
>>> Sorry if I overreacted here.
>>
>> There's a common saying that a disabled person can't be considered "normal"
>> unless he/she can also be called an asshole from time to time.
>>
>> Personally, I do not consider the pun in question harmful, simply because
>> it's so clearly just a play on words that doesn't make much sense, apart
>> from having a double meaning.
>>
>> In general, however, I think it's important to make jokes with and about
>> disabled people. That keeps them inside of our society, just like anyone
>> else. Treating them as "untouchable" means to single them out.
>
> I don't think we need to rehash this here (not meaning to pick on you, Stefan);

Agreed.


> those who are interested can sign up to the diversity list and read/post to
> the original thread here:
>
> http://mail.python.org/mailman/private/diversity/2011-July/002509.html

Speaking of "singling out", I've been wondering more than once why that 
list has a private archive ...

Stefan


From solipsis at pitrou.net  Wed Jul  6 14:12:00 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jul 2011 14:12:00 +0200
Subject: [Python-Dev] cpython: Remove mention of medical condition from
 the test suite.
References: <E1QdQNb-0003Ir-03@dinsdale.python.org>
	<CAP7+vJ+QYb13HLkuR=2=vd8f9u5f5AB_opDuSoC9tbJzBK3YAA@mail.gmail.com>
	<CAP7+vJL7j6kCo8cCrgUR2U5XwtKS9KkLTrzex25ZVp8On7tiLA@mail.gmail.com>
	<iv0s5c$21v$1@dough.gmane.org> <iv10la$pev$1@dough.gmane.org>
	<iv11jq$sk8$1@dough.gmane.org> <iv11vn$via$1@dough.gmane.org>
Message-ID: <20110706141200.4e2309e9@pitrou.net>

On Wed, 06 Jul 2011 09:13:58 +0200
Stefan Behnel <stefan_ml at behnel.de> wrote:
> 
> > those who are interested can sign up to the diversity list and read/post to
> > the original thread here:
> >
> > http://mail.python.org/mailman/private/diversity/2011-July/002509.html
> 
> Speaking of "singling out", I've been wondering more than once why that 
> list has a private archive ...

I suspect the explanation is itself given somewhere in a message inside
the private archive ;)

Regards

Antoine.



From vinay_sajip at yahoo.co.uk  Wed Jul  6 20:31:55 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 6 Jul 2011 18:31:55 +0000 (UTC)
Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs testing!
Message-ID: <loom.20110706T201450-905@post.gmane.org>

The C implementation of the PEP 397-compatible Python Launcher for Windows has
come along nicely in the last few days, and now reached a point where it would
benefit from some testing by interested python-dev members. Points of note:

1. As well as source available on

https://bitbucket.org/vinay.sajip/pylauncher

there are built 32- and 64-bit msi files at

https://bitbucket.org/vinay.sajip/pylauncher/downloads

Please remember that this is beta software. While it appears stable, I've
tested in virtual machines (WinXP 32-bit, Win7 32-bit and 64-bit) which I can
readily restore from backup. There are also msm files which could be used to
e.g. integrate with the Python msi files, if approved, at some later date.

2. On installation, any existing associations are saved, and restored when the
launcher is uninstalled (this is done automatically by Windows Installer).

3. On uninstallation, if there are Pythons installed and no associations, a
dialog pops up listing all the installed Pythons and offering the user the
chance to associate one of the Pythons with the Python extensions. The user
can choose to associate or not, but once they choose an association, then that
association will always be there unless the launcher is reinstalled (in which
case it takes over the association while it's still installed, and restores
the previous one when it's uninstalled).

4. I've tried to cover all of the points in the PEP. There is a test suite -
while this appears to be small (7 tests) the individual shebangs are all
tested, as are the customisable commands etc. However, I'm sure some of you
will break it ;-)

5. I used WiX to build the msm/msi files, but that's only because of increased
familiarity over msilib. The build procedure could switch over to msilib at
some later date. All the other code is just plain C and Win32 APIs (gosh -
takes me back! Window procedures, anyone?). The code builds with Visual Studio
and also Visual Studio Express (C++ edition).

Regards,

Vinay Sajip


From wolfson at gmail.com  Thu Jul  7 01:02:38 2011
From: wolfson at gmail.com (Ben Wolfson)
Date: Wed, 6 Jul 2011 16:02:38 -0700
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
Message-ID: <CAPc-aXmokgdSryR_Lm58WWgRaM-snVUZhvh0aCO=cVLNKYeT-w@mail.gmail.com>

FWIW, new patches have been attached to the bug report
(http://bugs.python.org/issue12014), one of which is intended to bring
behavior in line with the documentation, and the other of which is
intended to implement Greg Ewing's proposal to allow only identifiers
(or integers) in the arg_name, attribute_name, and element_index
sections.

On Fri, Jun 10, 2011 at 2:15 PM, Ben Wolfson <wolfson at gmail.com> wrote:
> Hello,
>
> I'm writing because discussion in a bug report I submitted
> (<http://bugs.python.org/issue12014>) has suggested that, insofar as
> at least part of the issue revolves around the interpretation of PEP
> 3101, that aspect belonged on python-dev. In particular, I was told
> that the PEP, not the documentation, is authoritative. Since I'm the
> one who thinks something is wrong, it seems appropriate for me to be
> the one to bring it up.
>
> Basically, the issue is that the current behavior of str.format is at
> variance at least with the documentation
> <http://docs.python.org/library/string.html#formatstrings>, is almost
> certainly at variance with PEP3101 in one respect, and is in my
> opinion at variance with PEP3101 in another respect as well, regarding
> what characters can be present in what the grammar given in the
> documentation calls an element_index, that is, the bit between the
> square brackets in "{0.attr[idx]}".
>
> Both discovering the current behavior and interpreting the
> documentation are pretty straightforward; interpreting what the PEP
> actually calls for is more vexed. I'll do the first two things first.
> TOC for the remainder:
>
> 1. What does the current implementation do?
> 2. What does the documentation say?
> 3. What does the PEP say? [this part is long, but the PEP is not
> clear, and I wanted to be thorough]
> 4. Who cares?
>
> 1. What does the current implementation do?
>
> Suppose you have this dictionary:
>
> d = {"@": 0,
> ? ? "!": 1,
> ? ? ":": 2,
> ? ? "^": 3,
> ? ? "}": 4,
> ? ? "{": {"}": 5},
> ? ?}
>
> Then the following expressions have the following results:
>
> (a) "{0[@]}".format(d) ? ?--> '0'
> (b) "{0[!]}".format(d) ? ?--> ValueError: Missing ']' in format string
> (c) "{0[:]}".format(d) ? ?--> ValueError: Missing ']' in format string
> (d) "{0[^]}".format(d) ? ?--> '3'
> (e) "{0[}]}".format(d) ? ?--> ValueError: Missing ']' in format string
> (f) "{0[{]}".format(d) ? ?--> ValueError: unmatched '{' in format
> (g) "{0[{][}]}".format(d) --> '5'
>
> Given (e) and (f), I think (g) should be a little surprising, though
> you can probably guess what's going on and it's not hard to see why it
> happens if you look at the source: (e) and (f) fail because
> MarkupIterator_next (in Objects/stringlib/string_format.h) scans
> through the string looking for curly braces, because it treats them as
> semantically significant no matter what context they occur in. So,
> according to MarkupIterator_next, the *first* right curly brace in (e)
> indicates the end of the replacement field, giving "{0[}". In (f), the
> second left curly brace indicates (to MarkupIterator_next) the start
> of a *new* replacement field, and since there's only one right curly
> brace, it complains. In (g), MarkupIterator_next treats the second
> left curly brace as starting a new replacement field and the first
> right curly brace as closing it. However, actually, those braces don't
> define new replacement fields, as indicated by the fact that the whole
> expression treats the element_index fields as just plain old strings.
> (So the current implementation is somewhat schizophrenic, acting at
> one point as if the braces have to be balanced because '{[]}' is a
> replacement field and at another point treating the braces as
> insignificant.)
>
> The explanation for (b) and (c) is that parse_field (same source file)
> treats ':' and '!' ?as indicating the end of the field_name section of
> the replacement field, regardless of whether those characters occur
> within square brackets or not.
>
> So, that's what the current implementation does, in those cases.
>
> 2. What does the documentation say?
>
> The documentation gives a grammar for replacement fields:
>
> """
> replacement_field ::= ?"{" [field_name] ["!" conversion] [":" format_spec] "}"
> field_name ? ? ? ?::= ?arg_name ("." attribute_name | "[" element_index "]")*
> arg_name ? ? ? ? ?::= ?[identifier | integer]
> attribute_name ? ?::= ?identifier
> element_index ? ? ::= ?integer | index_string
> index_string ? ? ?::= ?<any source character except "]"> +
> conversion ? ? ? ?::= ?"r" | "s"
> format_spec ? ? ? ::= ?<described in the next section>
> """
>
> Given this definition of index_string, all of (a)--(g) should be
> legal, and the results should be the strings '0', '1', '2', '3',
> "{'}': 5}", and '5'. There is no need to exclude ':', '!', '}', or '{'
> from the index_string field; allowing them creates no ambiguity,
> because the field is delimited by square brackets.
>
> Tangent: the definition of attribute_name here also does not describe
> the current behavior ("{0. ?;}".format(x) works fine and will call
> getattr(x, " ;")) and the PEP does not require the attribute_name to
> be an identifier; in fact it explicitly states that the attribute_name
> doesn't need to be a valid Python identifier. attribute_name should
> read (to reflect something like actual behavior, anyway) "<any source
> character except '[', '.', ':', '!', '{', or '}'> +". The same goes
> for arg_name (with the integer alternation). Observe:
>
>>>> x = lambda: None
>>>> setattr(x, ']]', 3)
>>>> "{].]]}".format(**{"]":x}) ? ? # (h)
> '3'
>
> One can also presently do this (hence "something like actual behavior"):
>>>> setattr(x, 'f}', 4)
>>>> "{a{s.f}}".format(**{"a{s":x})
> '4'
> But not this:
>>>> "{a{s.func_name}".format(**{"a{s":x})
> as it raises a ValueError, for the same reason as explains (g) above.
>
>
> 3. What does the PEP say?
>
> Well... It's actually hard to tell!
>
> Summary: The PEP does not contain a grammar for replacement fields,
> and is surprisingly nonspecific about what can appear where, at least
> when talking about the part of the replacement field before the format
> specifier. The most reasonable interpretation of the parts of the PEP
> that seem to be relevant favors the documentation, rather than the
> implementation.
>
> This can be separated into two sub-questions.
>
> A. What does the PEP say about ':' and '!'?
>
> It says two things that pertain to element_index fields.
>
> The first is this:
> """
> ? ? ? ? ? ? ? ? ? The rules for parsing an item key are very simple.
> ? ?If it starts with a digit, then it is treated as a number, otherwise
> ? ?it is used as a string.
>
> ? ?Because keys are not quote-delimited, it is not possible to
> ? ?specify arbitrary dictionary keys (e.g., the strings "10" or
> ? ?":-]") from within a format string.
> """
>
> So it notes that some things can't be used:
>
> ?- Because anything composed entirely of digits is treated as a
> number, you can't get a string composed entirely of digits. Clear
> enough.
>
> ?- What's the explanation for the second example, though? Well, you
> can't have a right square bracket in the index_string, so that would
> already mean that you can't do this: "{0[:-]]}".format(...) regardless
> of the whether colons are legal or not. So, although the PEP gives an
> example of a string that can't in the element_index part of a
> replacement field, and that string contains a colon, that string would
> have been disallowed for other reasons anyway.
>
> The second is this:
>
> """
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?The str.format() function will have
> ? ?a minimalist parser which only attempts to figure out when it is
> ? ?"done" with an identifier (by finding a '.' or a ']', or '}',
> ? ?etc.).
> """
>
> This requires some interpretation. For one thing, the contents of an
> element_index aren't identifiers. For another, it's not true that
> you're done with an identifier (or whatever) whenever you see *any* of
> those; it depends on context. When parsing this: "{0[a.b]}" the parser
> should not stop at the '.'; it should keep going until it reaches the
> ']', and that will give the element_index. And when parsing this:
> "{0.f]oo[bar].baz}", it's done with the identifier "foo" when it
> reaches the '[', not when it reaches the second '.', and not when it
> reaches the ']', either (recall (h)). The "minimalist parser" is, I
> take it, one that works like this: when parsing an arg_name you're
> done when you reach a '[', a ':', a '!', a '.', '{', or a '}'; the
> same rules apply when parsing a attribute_name; when parsing an
> element_index you're done when you see a ']'.
>
> Now as regards the curly braces there are some other parts of the PEP
> that perhaps should be taken into account (see below), but I can find
> no justification for excluding ':' and '!' from the element_index
> field; the bit quoted above about having a minimalist parser isn't a
> good justification for that, and it's the only part of the entire PEP
> that comes close to addressing the question.
>
> B. What does it say about '}' and '{'?
>
> It still doesn't say much explicitly. There's the quotation I just
> gave, and then these passages:
>
> """
> ? ?Brace characters ('curly braces') are used to indicate a
> ? ?replacement field within the string:
>
> ? ?[...]
>
> ? ?Braces can be escaped by doubling:
> """
>
> Taken by itself, this would suggest that wherever there's an unescaped
> brace, there's a replacement field. That would mean that the current
> implementation's behavior is correct in (e) and (f) but incorrect in
> (g). However, I think this is a bad interpretation; unescaped braces
> can indicate the presence of a replacement field without that meaning
> that *within* a replacement field braces have that meaning, no matter
> where within the replacement field they occur.
>
> Later in the PEP, talking about this example:
>
> ? ? ? ?"{0:{1}}".format(a, b)
>
> We have this:
>
> """
> ? ?These 'internal' replacement fields can only occur in the format
> ? ?specifier part of the replacement field. ?Internal replacement fields
> ? ?cannot themselves have format specifiers. ?This implies also that
> ? ?replacement fields cannot be nested to arbitrary levels.
>
> ? ?Note that the doubled '}' at the end, which would normally be
> ? ?escaped, is not escaped in this case. ?The reason is because
> ? ?the '{{' and '}}' syntax for escapes is only applied when used
> ? ?*outside* of a format field. ?Within a format field, the brace
> ? ?characters always have their normal meaning.
> """
>
> The claim "within a format field, the brace characters always have
> their normal meaning" might be taken to mean that within a
> *replacement* field, brace characters always indicate the start (or
> end) of a replacement field. But the PEP at this point is clearly
> talking about the formatting section of a replacement field---the part
> that follows the ':', if present. ("Format field" is nowhere defined
> in the PEP, but it seems reasonable to take it to mean "the format
> specifier of a replacement field".) However, it seems most reasonable
> to me to take "normal meaning" to mean "just a brace character".
>
> Note that the present implementation only kinda sorta conforms to the
> PEP in this respect:
>
>
>>>> import datetime
>>>> format(datetime.datetime.now(), "{{%Y")
> '{{2011'
>>>> "{0:{{%{1}}".format(datetime.datetime.now(), 'Y') # (i)
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> ValueError: unmatched '{' in format
>>>> "{0:{{%{1}}}}".format(datetime.datetime.now(), 'Y') # (j)
> '{2011}'
>
> Here the brace characters in (i) and (j) are treated, again in
> MarkupIterator_next, as indicating the start of a replacement field.
> In (i), this leads the function to throw an exception; since they're
> balanced in (j), processing proceeds further, and the doubled braces
> aren't treated as indicating the start or end of a replacement
> field---because they're escaped. ?Given that the format spec part of a
> replacement field can contain further replacement fields, this is, I
> think, correct behavior, but it's not easy to see how it comports with
> the PEP, whose language is not very exact.
>
> The call to the built-in format() bypasses the mechanism that leads to
> these results.
>
> The PEP is very, very nonspecific about the parts of the replacement
> field that precede the format specifier. I don't know what kind of
> discussion surrounded the drawing up of the grammar that appears in
> the documentation, but I think that it, and not the implementation,
> should be followed.
>
> The implementation only works the way it does because of parsing
> shortcuts: it raises ValueErrors for (b) and (c) because it
> generalizes something true of the attribute_name field (encountering a
> ':' or '!' means one has moved on to the format_spec or conversion
> part of the replacement field) to the element_index field. And it
> raises an error for (e) and (f), but lets (g) through, for the reasons
> already mentioned. It is, in that respect, inconsistent; it treats the
> curly brace as having one semantic significance at one point and an
> entirely different significance at another point, so that it does the
> right thing in the case of (g) entirely by accident. There is, I
> think, no way to construe the PEP so that it is reasonable to do what
> the present implementation does in all three cases (if "{" indicates
> the start of a replacement field in (f), it should do so in (g) as
> well); I think it's
> actually pretty difficult to construe the PEP in any way that makes
> what it does in the case of (e) and (f) correct.
>
> 4. Who cares?
>
> Well, I do. (Obviously.) I even have a use case: I want to be able to
> put arbitrary (or as close to arbitrary as possible) strings in the
> element_index field, because I've got some objects that (should!)
> enable me to do this:
>
> p.say("I'm warning you, {e.red.underline[don't touch that!]}")
>
> and have this written ("e" for "effects") to p.out:
>
> I'm warning you, \x1b[31m\x1b[4mdon't touch that!\x1b[0m
>
> I have a way around the square bracket problem, but it would be quite
> burdensome to have to deal with all of !:{} as well; enough that
> I would fall back on something like this:
>
> "I'm warning you, {0}".format(e.red.underline("don't touch that!"))
>
> or some similar interpolation-based strategy, which I think would be a
> shame, because of the way it chops up the string.
>
> But I also think the present behavior is extremely counterintuitive,
> unnecessarily complex, and illogical (even unpythonic!). Isn't it
> bizarre that (g) should work, given what (e) and (f) do? Isn't it
> strange that (b) and (c) should fail, given that there's no real
> reason for them to do so---no ambiguity that has to be avoided? And
> something's gotta give; the documentation and the implementation do
> not correspond.
>
> Beyond the counterintuitiveness of the present implementation, it is
> also, I think, self-inconsistent. (e) and (f) fail because the
> interior brace is treated as starting or ending a replacement field,
> even though interior replacement fields aren't allowed in that part of
> a replacement field. (g) succeeds because the braces are balanced:
> they are treated at one point as if they were part of a replacement
> field, and at another (correctly) as if they are not. But this makes
> the failure of (e) and (f) unaccountable. It would not have been
> impossible for the PEP to allow interior replacement fields anywhere,
> and not just in the format spec, in which case we might have had this:
>
> (g') "{0[{][}]}".format(range(10), **{'][':4}) --> '3'
> or this:
> (g'') "{0[{][}]}".format({'4':3}, **{'][':4}) --> '3'
> or something with that general flavor.
>
> As far as I can tell, the only way to consistently maintain that (e)
> and (f) should fail requires that one take (g') or (g'') to be
> correct: either the interior braces signal replacement fields (hence
> must be balanced) or they don't (or they're escaped).
>
> Regarding the documentation, it could of course be rendered correct by
> changing *it*, rather than the implementation. But wouldn't it be
> tremendously weird to have to explain that, in the part of the
> replacement field preceding the conversion, you can't employ any curly
> braces, unless they're balanced, and you can't employ ':' or '!' at
> all, even though they have no semantic significance? So these are out:
>
> {0[{].foo}
> {0[}{}]}
> {0[a:b]}
>
> But these are in:
>
> {0[{}{}]}
> {0[{{].foo.}}} ?(k)
>
> ((k) does work, if you give it an object with the right structure,
> though it probably should not.)
>
> And, moreover, someone would then have to actually change the
> documentation, whereas there's a patch already, attached to the bug
> report linked way up at the top of this email, that makes (a)--(g) all
> work, leaves (i) and (j) as they are, and has the welcome side-effect
> of making (k) *not* work (if any code anywhere relies on (k) or things
> like it working, I will be very surprised: anyway the fact that (k)
> works is, technically, undocumented). It is also quite simple. It
> doesn't effect the built-in format(), because the built-in format() is
> concerned only with the format-specifier part of the replacement field
> and not all the stuff that comes before that, telling str.format
> *what* object to format.
>
> Thanks for reading,
> --
> Ben Wolfson
> "Human kind has used its intelligence to vary the flavour of drinks,
> which may be sweet, aromatic, fermented or spirit-based. ... Family
> and social life also offer numerous other occasions to consume drinks
> for pleasure." [Larousse, "Drink" entry]
>



-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

From victor.stinner at haypocalc.com  Thu Jul  7 02:51:55 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 02:51:55 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
Message-ID: <1309999915.2958.8.camel@marge>

Hi,

Last may, I proposed to deprecate open() function, StreamWriter and
StreamReader classes of the codecs module. I accepted to keep open()
after the discussion on python-dev. Here is a more complete proposition
as a PEP. It is a draft and I expect a lot of comments :)

Victor

-----------------------

PEP: xxx
Title: Deprecate codecs.StreamReader and codecs.StreamWriter
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-May-2011
Python-Version: 3.3


Abstract
========

io.TextIOWrapper and codecs.StreamReaderWriter offer the same API
[#f1]_. TextIOWrapper has more features and is faster than
StreamReaderWriter. Duplicate code means that bugs should be fixed
twice and that we may have subtle differences between the two
implementations.

The codecs modules was introduced in Python 2.0, see the PEP 100. The
io module was introduced in Python 2.6 and 3.0 (see the PEP 3116), and
reimplemented in C in Python 2.7 and 3.1.


Motivation
==========

When the Python I/O model was updated for 3.0, the concept of a
"stream-with-known-encoding" was introduced in the form of
io.TextIOWrapper. As this class is critical to the performance of
text-based I/O in Python 3, this module has an optimised C version
which is used by CPython by default. Many corner cases in handling
buffering, stateful codecs and universal newlines have been dealt with
since the release of Python 3.0.

This new interface overlaps heavily with the legacy
codecs.StreamReader, codecs.StreamWriter and codecs.StreamReaderWriter
interfaces that were part of the original codec interface design in
PEP 100. These interfaces are organised around the principle of an
encoding with an associated stream (i.e. the reverse of arrangement in
the io module), so the original PEP 100 design required that codec
writers provide appropriate StreamReader and StreamWriter
implementations in addition to the core codec encode() and decode()
methods. This places a heavy burden on codec authors providing these
specialised implementations to correctly handle many of the corner
cases that have now been dealt with by io.TextIOWrapper. While deeper
integration between the codec and the stream allows for additional
optimisations in theory, these optimisations have in practice either
not been carried out and else the associated code duplication means
that the corner cases that have been fixed in io.TextIOWrapper are
still not handled correctly in the various StreamReader and
StreamWriter implementations.

Accordingly, this PEP proposes that:

* codecs.open() be updated to delegate to the builtin open() in Python
  3.3;
* the legacy codecs.Stream* interfaces, including the streamreader and
  streamwriter attributes of codecs.CodecInfo be deprecated in Python
  3.3 and removed in Python 3.4.


Rationale
=========

StreamReader and StreamWriter issues
''''''''''''''''''''''''''''''''''''

 * StreamReader is unable to translate newlines.
 * StreamReaderWriter handles reads using StreamReader and writes
   using StreamWriter. These two classes may be inconsistent. To stay
   consistent, flush() must be called after each write which slows
   down interlaced read-write.
 * StreamWriter doesn't support "line buffering" (flush if the input
   text contains a newline).
 * StreamReader classes of the CJK encodings (e.g. GB18030) don't
   support universal newlines, only UNIX newlines ('\\n').
 * StreamReader and StreamWriter are stateful codecs but don't expose
   functions to control their state (getstate() or setstate()). Each
   codec has to implement corner cases, see "Issue with stateful
   codecs".
 * StreamReader and StreamWriter are very similar to IncrementalReader
   and IncrementalEncoder, some code is duplicated for stateful codecs
   (e.g. UTF-16).
 * Each codec has to reimplement its own StreamReader and StreamWriter
   class, even if it's trivial (just call the encoder/decoder).
 * codecs.open(filename, "r") creates a io.TextIOWrapper object.
 * No codec implements an optimized method in StreamReader or
   StreamWriter based on the specificities of the codec.


TextIOWrapper features
''''''''''''''''''''''

 * TextIOWrapper supports any kind of newline, including translating
   newlines (to UNIX newlines), to read and write.
 * TextIOWrapper reuses incremental encoders and decoders (no
   duplication of code).
 * The io module (TextIOWrapper) is faster than the codecs module
   (StreamReader). It is implemented in C, whereas codecs is
   implemented in Python.
 * TextIOWrapper has a readahead algorithm which speeds up small
   reads: read character by character or line by line (io is 10x
   through 25x faster than codecs on these operations).
 * TextIOWrapper has a write buffer.
 * TextIOWrapper.tell() is optimized.
 * TextIOWrapper supports random access (read+write) using a single
   class which permit to optimize interlaced read-write (but no such
   optimization is implemented).


Possible improvements of StreamReader and StreamWriter
''''''''''''''''''''''''''''''''''''''''''''''''''''''

It would be possible to add functions to StreamReader and StreamWriter
to give access to the state of codec. And so it would be possible fix
issues with stateful codecs in a base class instead of having to fix
them is each stateful StreamReader and StreamWriter classes.

It would be possible to change StreamReader and StreamWriter to make
them use IncrementalDecoder and IncrementalEncoder.

A codec can implement variants which are optimized for the specific
encoding or intercept certain stream methods to add functionality or
improve the encoding/decoding performance. TextIOWrapper cannot
implement such optimization, but TextIOWrapper uses incremental
encoders and decoders and uses read and write buffers, so the overhead
of incomplete inputs is low or nul.

A lot more could be done for other variable length encoding codecs,
e.g. UTF-8, since these often have problems near the end of a read due
to missing bytes. The UTF-32-BE/LE codecs could simply multiply the
character position by 4 to get the byte position.


Usage of StreamReader and StreamWriter
''''''''''''''''''''''''''''''''''''''

These classes are rarely used directly, but indirectly using
codecs.open(). They are not used in Python 3 standard library (except
in the codecs module).

Some projects implement their own codec with StreamReader and
StreamWriter, but don't use these classes.


Backwards Compatibility
=======================

Keep the public API, codecs.open
''''''''''''''''''''''''''''''''

codecs.open() can be replaced by the builtin open() function. open()
has a similar API but has also more options.

codecs.open() was the only way to open a text file in Unicode mode
until Python 2.6. Many Python 2 programs uses this function. Removing
codecs.open() implies more work to port programs from Python 2 to
Python 3, especially projets using the same code base for the two
Python versions (without using 2to3 program).

codecs.open() is kept for backward compatibility with Python 2.


Deprecate StreamReader and StreamWriter
'''''''''''''''''''''''''''''''''''''''

Instanciate StreamReader or StreamWriter must raise a
DeprecationWarning in Python 3.3. Implement a subclass don't raise a
DeprecationWarning.

codecs.open() will be changed to reuse the builtin open() function
(TextIOWrapper).

EncodedFile(), StreamRandom, StreamReader, StreamReaderWriter and
StreamWriter will be removed in Python 3.4.


Issue with stateful codecs
==========================

It is difficult to use correctly a stateful codec with a stream. Some
cases are supported by the codecs module, while io has no more known
bug related to stateful codecs. The main difference between the codecs
and the io module is that bugs have to be fixed in StreamReader and/or
StreamWriter classes of each codec for the codecs module, whereas bugs
can be fixed only once in io.TextIOWrapper. Here are some examples of
issues with stateful codecs.

Stateful codecs
'''''''''''''''

Python supports the following stateful codecs:

 * cp932
 * cp949
 * cp950
 * euc_jis_2004
 * euc_jisx2003
 * euc_jp
 * euc_kr
 * gb18030
 * gbk
 * hz
 * iso2022_jp
 * iso2022_jp_1
 * iso2022_jp_2
 * iso2022_jp_2004
 * iso2022_jp_3
 * iso2022_jp_ext
 * iso2022_kr
 * shift_jis
 * shift_jis_2004
 * shift_jisx0213
 * utf_8_sig
 * utf_16
 * utf_32

Read and seek(0)
''''''''''''''''

::

    with open(filename, 'w', encoding='utf_16') as f:
        f.write('abc')
        f.write('def')
        f.seek(0)
        assert f.read() == 'abcdef'
        f.seek(0)
        assert f.read() == 'abcdef'

The io and codecs modules support this usecase correctly.

Write, seek(0) and seek(n)
''''''''''''''''''''''''''

::

    with open(filename, 'w', encoding='utf_16') as f:
        f.write('abc')
        pos = f.tell()
    with open(filename, 'r+', encoding='utf_16') as f:
        f.seek(pos)
        f.write('def')
        f.seek(0)
        f.write('###')
    with open(filename, 'r', encoding='utf_16') as f:
        assert f.read() == '###def'

The io module supports this usecase, whereas codecs fails because it
writes a new BOM on the second write.

Append mode
'''''''''''

::

    with open(filename, 'w', encoding='utf_16') as f:
        f.write('abc')
    with open(filename, 'a', encoding='utf_16') as f:
        f.write('def')
    with open(filename, 'r', encoding='utf_16') as f:
        assert f.read() == 'abcdef'

The io module supports this usecase, whereas codecs fails because it
writes a new BOM on the second write.


Links
=====

 * `PEP 100: Python Unicode Integration
   <http://www.python.org/dev/peps/pep-0100/>`_
 * `PEP 3116 <http://www.python.org/dev/peps/pep-3116/>`_
 * `Issue #8796: Deprecate codecs.open()
   <http://bugs.python.org/issue8796>`_
 * `[python-dev] Deprecate codecs.open() and StreamWriter/StreamReader
   <http://mail.python.org/pipermail/python-dev/2011-May/111591.html>`_


Copyright
=========

This document has been placed in the public domain.


Footnotes
=========

.. [#f1] StreamReaderWriter has two more attributes than
         TextIOWrapper, reader and writer.


From benjamin at python.org  Thu Jul  7 03:16:39 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 6 Jul 2011 20:16:39 -0500
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <1309999915.2958.8.camel@marge>
References: <1309999915.2958.8.camel@marge>
Message-ID: <CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>

2011/7/6 Victor Stinner <victor.stinner at haypocalc.com>:
> codecs.open() will be changed to reuse the builtin open() function
> (TextIOWrapper).

This doesn't strike me as particularly backwards compatible, since
you've just enumerated the differences between StreamWriter/Reader and
TextIOWrapper.



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Thu Jul  7 05:26:20 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 7 Jul 2011 13:26:20 +1000
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
Message-ID: <CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>

On Thu, Jul 7, 2011 at 11:16 AM, Benjamin Peterson <benjamin at python.org> wrote:
> 2011/7/6 Victor Stinner <victor.stinner at haypocalc.com>:
>> codecs.open() will be changed to reuse the builtin open() function
>> (TextIOWrapper).
>
> This doesn't strike me as particularly backwards compatible, since
> you've just enumerated the differences between StreamWriter/Reader and
> TextIOWrapper.

The API of the resulting object is the same (i.e. they're file-like
objects). The behavioural differences are due to cases where the
codec-specific classes are currently broken.

Victor, could you please check this into the PEPs repo? It's easier to
reference once it has a real number.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From benjamin at python.org  Thu Jul  7 05:51:52 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 6 Jul 2011 22:51:52 -0500
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
Message-ID: <CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>

2011/7/6 Nick Coghlan <ncoghlan at gmail.com>:
> On Thu, Jul 7, 2011 at 11:16 AM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2011/7/6 Victor Stinner <victor.stinner at haypocalc.com>:
>>> codecs.open() will be changed to reuse the builtin open() function
>>> (TextIOWrapper).
>>
>> This doesn't strike me as particularly backwards compatible, since
>> you've just enumerated the differences between StreamWriter/Reader and
>> TextIOWrapper.
>
> The API of the resulting object is the same (i.e. they're file-like
> objects). The behavioural differences are due to cases where the
> codec-specific classes are currently broken.

Yes, but as we all know too well, people are surely relying on
whatever behavior there is, broken or not.



-- 
Regards,
Benjamin

From vinay_sajip at yahoo.co.uk  Thu Jul  7 08:53:50 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 7 Jul 2011 06:53:50 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Draft_PEP=3A_Deprecate_codecs=2EStreamRead?=
	=?utf-8?q?er_and=09codecs=2EStreamWriter?=
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
Message-ID: <loom.20110707T084728-452@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:

> 
> 2011/7/6 Nick Coghlan <ncoghlan <at> gmail.com>:

> > The API of the resulting object is the same (i.e. they're file-like
> > objects). The behavioural differences are due to cases where the
> > codec-specific classes are currently broken.
> 
> Yes, but as we all know too well, people are surely relying on
> whatever behavior there is, broken or not.
> 

There's also the fact that code which currently runs under 2.x and 3.x would
stop working if codecs.StreamReader/StreamWriter were to go away. Of course, if
the codecs interfaces were re-implemented using io module code, the only
portability issues would be because of people relying on broken aspects of the
existing codecs code - which is unlikely to be all (or even most) of the people
using codecs.StreamReader/StreamWriter.

Regards,

Vinay Sajip


From mal at egenix.com  Thu Jul  7 10:07:38 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 07 Jul 2011 10:07:38 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader
	and	codecs.StreamWriter
In-Reply-To: <1309999915.2958.8.camel@marge>
References: <1309999915.2958.8.camel@marge>
Message-ID: <4E15694A.9070002@egenix.com>

Victor Stinner wrote:
> Hi,
> 
> Last may, I proposed to deprecate open() function, StreamWriter and
> StreamReader classes of the codecs module. I accepted to keep open()
> after the discussion on python-dev. Here is a more complete proposition
> as a PEP. It is a draft and I expect a lot of comments :)

The PEP's arguments for deprecating two essential codec design
components are very one sided, by comparing "issues" to "features".

Please add all the comments I've made on the subject to the PEP.
The most important one missing is the fact and major difference
that TextIOWrapper does not work on a per codec basis, but
only on a per stream basis.

By removing the StreamReader and StreamWriter API parts of the
codec design, you essentially make it impossible to add
per codec variations and optimizations that require full access
to the stream interface.

A mentioned before, many improvements are possible and lots of those
can build on TextIOWrapper and the incremental codec parts.

That said, I'm not really up for a longer discussion on this. We've
already had the discussion and decided against removing those
parts of the codec API.

Redirecting codecs.open() to open() should be investigated.

For the issues you mention in the PEP, please open tickets
or add ticket references to the PEP.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 07 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


> Victor
> 
> -----------------------
> 
> PEP: xxx
> Title: Deprecate codecs.StreamReader and codecs.StreamWriter
> Version: $Revision$
> Last-Modified: $Date$
> Author: Victor Stinner
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 28-May-2011
> Python-Version: 3.3
> 
> 
> Abstract
> ========
> 
> io.TextIOWrapper and codecs.StreamReaderWriter offer the same API
> [#f1]_. TextIOWrapper has more features and is faster than
> StreamReaderWriter. Duplicate code means that bugs should be fixed
> twice and that we may have subtle differences between the two
> implementations.
> 
> The codecs modules was introduced in Python 2.0, see the PEP 100. The
> io module was introduced in Python 2.6 and 3.0 (see the PEP 3116), and
> reimplemented in C in Python 2.7 and 3.1.
> 
> 
> Motivation
> ==========
> 
> When the Python I/O model was updated for 3.0, the concept of a
> "stream-with-known-encoding" was introduced in the form of
> io.TextIOWrapper. As this class is critical to the performance of
> text-based I/O in Python 3, this module has an optimised C version
> which is used by CPython by default. Many corner cases in handling
> buffering, stateful codecs and universal newlines have been dealt with
> since the release of Python 3.0.
> 
> This new interface overlaps heavily with the legacy
> codecs.StreamReader, codecs.StreamWriter and codecs.StreamReaderWriter
> interfaces that were part of the original codec interface design in
> PEP 100. These interfaces are organised around the principle of an
> encoding with an associated stream (i.e. the reverse of arrangement in
> the io module), so the original PEP 100 design required that codec
> writers provide appropriate StreamReader and StreamWriter
> implementations in addition to the core codec encode() and decode()
> methods. This places a heavy burden on codec authors providing these
> specialised implementations to correctly handle many of the corner
> cases that have now been dealt with by io.TextIOWrapper. While deeper
> integration between the codec and the stream allows for additional
> optimisations in theory, these optimisations have in practice either
> not been carried out and else the associated code duplication means
> that the corner cases that have been fixed in io.TextIOWrapper are
> still not handled correctly in the various StreamReader and
> StreamWriter implementations.
> 
> Accordingly, this PEP proposes that:
> 
> * codecs.open() be updated to delegate to the builtin open() in Python
>   3.3;
> * the legacy codecs.Stream* interfaces, including the streamreader and
>   streamwriter attributes of codecs.CodecInfo be deprecated in Python
>   3.3 and removed in Python 3.4.
> 
> 
> Rationale
> =========
> 
> StreamReader and StreamWriter issues
> ''''''''''''''''''''''''''''''''''''
> 
>  * StreamReader is unable to translate newlines.
>  * StreamReaderWriter handles reads using StreamReader and writes
>    using StreamWriter. These two classes may be inconsistent. To stay
>    consistent, flush() must be called after each write which slows
>    down interlaced read-write.
>  * StreamWriter doesn't support "line buffering" (flush if the input
>    text contains a newline).
>  * StreamReader classes of the CJK encodings (e.g. GB18030) don't
>    support universal newlines, only UNIX newlines ('\\n').
>  * StreamReader and StreamWriter are stateful codecs but don't expose
>    functions to control their state (getstate() or setstate()). Each
>    codec has to implement corner cases, see "Issue with stateful
>    codecs".
>  * StreamReader and StreamWriter are very similar to IncrementalReader
>    and IncrementalEncoder, some code is duplicated for stateful codecs
>    (e.g. UTF-16).
>  * Each codec has to reimplement its own StreamReader and StreamWriter
>    class, even if it's trivial (just call the encoder/decoder).
>  * codecs.open(filename, "r") creates a io.TextIOWrapper object.
>  * No codec implements an optimized method in StreamReader or
>    StreamWriter based on the specificities of the codec.
> 
> 
> TextIOWrapper features
> ''''''''''''''''''''''
> 
>  * TextIOWrapper supports any kind of newline, including translating
>    newlines (to UNIX newlines), to read and write.
>  * TextIOWrapper reuses incremental encoders and decoders (no
>    duplication of code).
>  * The io module (TextIOWrapper) is faster than the codecs module
>    (StreamReader). It is implemented in C, whereas codecs is
>    implemented in Python.
>  * TextIOWrapper has a readahead algorithm which speeds up small
>    reads: read character by character or line by line (io is 10x
>    through 25x faster than codecs on these operations).
>  * TextIOWrapper has a write buffer.
>  * TextIOWrapper.tell() is optimized.
>  * TextIOWrapper supports random access (read+write) using a single
>    class which permit to optimize interlaced read-write (but no such
>    optimization is implemented).
> 
> 
> Possible improvements of StreamReader and StreamWriter
> ''''''''''''''''''''''''''''''''''''''''''''''''''''''
> 
> It would be possible to add functions to StreamReader and StreamWriter
> to give access to the state of codec. And so it would be possible fix
> issues with stateful codecs in a base class instead of having to fix
> them is each stateful StreamReader and StreamWriter classes.
> 
> It would be possible to change StreamReader and StreamWriter to make
> them use IncrementalDecoder and IncrementalEncoder.
> 
> A codec can implement variants which are optimized for the specific
> encoding or intercept certain stream methods to add functionality or
> improve the encoding/decoding performance. TextIOWrapper cannot
> implement such optimization, but TextIOWrapper uses incremental
> encoders and decoders and uses read and write buffers, so the overhead
> of incomplete inputs is low or nul.
> 
> A lot more could be done for other variable length encoding codecs,
> e.g. UTF-8, since these often have problems near the end of a read due
> to missing bytes. The UTF-32-BE/LE codecs could simply multiply the
> character position by 4 to get the byte position.
> 
> 
> Usage of StreamReader and StreamWriter
> ''''''''''''''''''''''''''''''''''''''
> 
> These classes are rarely used directly, but indirectly using
> codecs.open(). They are not used in Python 3 standard library (except
> in the codecs module).
> 
> Some projects implement their own codec with StreamReader and
> StreamWriter, but don't use these classes.
> 
> 
> Backwards Compatibility
> =======================
> 
> Keep the public API, codecs.open
> ''''''''''''''''''''''''''''''''
> 
> codecs.open() can be replaced by the builtin open() function. open()
> has a similar API but has also more options.
> 
> codecs.open() was the only way to open a text file in Unicode mode
> until Python 2.6. Many Python 2 programs uses this function. Removing
> codecs.open() implies more work to port programs from Python 2 to
> Python 3, especially projets using the same code base for the two
> Python versions (without using 2to3 program).
> 
> codecs.open() is kept for backward compatibility with Python 2.
> 
> 
> Deprecate StreamReader and StreamWriter
> '''''''''''''''''''''''''''''''''''''''
> 
> Instanciate StreamReader or StreamWriter must raise a
> DeprecationWarning in Python 3.3. Implement a subclass don't raise a
> DeprecationWarning.
> 
> codecs.open() will be changed to reuse the builtin open() function
> (TextIOWrapper).
> 
> EncodedFile(), StreamRandom, StreamReader, StreamReaderWriter and
> StreamWriter will be removed in Python 3.4.
> 
> 
> Issue with stateful codecs
> ==========================
> 
> It is difficult to use correctly a stateful codec with a stream. Some
> cases are supported by the codecs module, while io has no more known
> bug related to stateful codecs. The main difference between the codecs
> and the io module is that bugs have to be fixed in StreamReader and/or
> StreamWriter classes of each codec for the codecs module, whereas bugs
> can be fixed only once in io.TextIOWrapper. Here are some examples of
> issues with stateful codecs.
> 
> Stateful codecs
> '''''''''''''''
> 
> Python supports the following stateful codecs:
> 
>  * cp932
>  * cp949
>  * cp950
>  * euc_jis_2004
>  * euc_jisx2003
>  * euc_jp
>  * euc_kr
>  * gb18030
>  * gbk
>  * hz
>  * iso2022_jp
>  * iso2022_jp_1
>  * iso2022_jp_2
>  * iso2022_jp_2004
>  * iso2022_jp_3
>  * iso2022_jp_ext
>  * iso2022_kr
>  * shift_jis
>  * shift_jis_2004
>  * shift_jisx0213
>  * utf_8_sig
>  * utf_16
>  * utf_32
> 
> Read and seek(0)
> ''''''''''''''''
> 
> ::
> 
>     with open(filename, 'w', encoding='utf_16') as f:
>         f.write('abc')
>         f.write('def')
>         f.seek(0)
>         assert f.read() == 'abcdef'
>         f.seek(0)
>         assert f.read() == 'abcdef'
> 
> The io and codecs modules support this usecase correctly.
> 
> Write, seek(0) and seek(n)
> ''''''''''''''''''''''''''
> 
> ::
> 
>     with open(filename, 'w', encoding='utf_16') as f:
>         f.write('abc')
>         pos = f.tell()
>     with open(filename, 'r+', encoding='utf_16') as f:
>         f.seek(pos)
>         f.write('def')
>         f.seek(0)
>         f.write('###')
>     with open(filename, 'r', encoding='utf_16') as f:
>         assert f.read() == '###def'
> 
> The io module supports this usecase, whereas codecs fails because it
> writes a new BOM on the second write.
> 
> Append mode
> '''''''''''
> 
> ::
> 
>     with open(filename, 'w', encoding='utf_16') as f:
>         f.write('abc')
>     with open(filename, 'a', encoding='utf_16') as f:
>         f.write('def')
>     with open(filename, 'r', encoding='utf_16') as f:
>         assert f.read() == 'abcdef'
> 
> The io module supports this usecase, whereas codecs fails because it
> writes a new BOM on the second write.
> 
> 
> Links
> =====
> 
>  * `PEP 100: Python Unicode Integration
>    <http://www.python.org/dev/peps/pep-0100/>`_
>  * `PEP 3116 <http://www.python.org/dev/peps/pep-3116/>`_
>  * `Issue #8796: Deprecate codecs.open()
>    <http://bugs.python.org/issue8796>`_
>  * `[python-dev] Deprecate codecs.open() and StreamWriter/StreamReader
>    <http://mail.python.org/pipermail/python-dev/2011-May/111591.html>`_
> 
> 
> Copyright
> =========
> 
> This document has been placed in the public domain.
> 
> 
> Footnotes
> =========
> 
> .. [#f1] StreamReaderWriter has two more attributes than
>          TextIOWrapper, reader and writer.
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

From ncoghlan at gmail.com  Thu Jul  7 10:29:26 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 7 Jul 2011 18:29:26 +1000
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
Message-ID: <CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>

On Thu, Jul 7, 2011 at 1:51 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2011/7/6 Nick Coghlan <ncoghlan at gmail.com>:
>> On Thu, Jul 7, 2011 at 11:16 AM, Benjamin Peterson <benjamin at python.org> wrote:
>>> 2011/7/6 Victor Stinner <victor.stinner at haypocalc.com>:
>>>> codecs.open() will be changed to reuse the builtin open() function
>>>> (TextIOWrapper).
>>>
>>> This doesn't strike me as particularly backwards compatible, since
>>> you've just enumerated the differences between StreamWriter/Reader and
>>> TextIOWrapper.
>>
>> The API of the resulting object is the same (i.e. they're file-like
>> objects). The behavioural differences are due to cases where the
>> codec-specific classes are currently broken.
>
> Yes, but as we all know too well, people are surely relying on
> whatever behavior there is, broken or not.

True, but that's why changes like this are always a judgement call -
is the gain in correctness worth the risk of breakage? We sometimes
break workarounds when we fix bugs, too. From the discussion last time
around, that particular change wasn't very controversial, which is why
it is already in the 3.3 development tree.

Unless somebody steps forward to fix them, the Stream* classes have to
go (albeit with a suitable period of deprecation). They're *actively
harmful* in their current state, so retaining the status quo is not a
viable option in this case.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Thu Jul  7 10:49:22 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 7 Jul 2011 08:49:22 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Draft_PEP=3A_Deprecate_codecs=2EStreamRead?=
	=?utf-8?q?er_and=09codecs=2EStreamWriter?=
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>
Message-ID: <loom.20110707T104519-937@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Unless somebody steps forward to fix them, the Stream* classes have to
> go (albeit with a suitable period of deprecation). They're *actively
> harmful* in their current state, so retaining the status quo is not a
> viable option in this case.

I can understand that there might be specific issues with them, but isn't
"actively harmful" a little strong? I don't see who is being actively harmed by
them, nor how.

Regards,

Vinay Sajip




From ncoghlan at gmail.com  Thu Jul  7 11:00:48 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 7 Jul 2011 19:00:48 +1000
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <loom.20110707T104519-937@post.gmane.org>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>
	<loom.20110707T104519-937@post.gmane.org>
Message-ID: <CADiSq7fCjf8HVQkKO_LQGD=NDVGq0tgJarwJv+CB_fy9ZnS1=Q@mail.gmail.com>

On Thu, Jul 7, 2011 at 6:49 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>> Unless somebody steps forward to fix them, the Stream* classes have to
>> go (albeit with a suitable period of deprecation). They're *actively
>> harmful* in their current state, so retaining the status quo is not a
>> viable option in this case.
>
> I can understand that there might be specific issues with them, but isn't
> "actively harmful" a little strong? I don't see who is being actively harmed by
> them, nor how.

Anyone forward porting codecs.open based code will get subpar IO in
Python 3 *because* they're trying to do the right thing in Python 2.
That's actively harmful in my book.

Codec writers are also told to implement utterly unnecessary
functionality just because PEP 100 says so. Significantly less common,
but still harmful.

The bare minimum change needed is for codecs.open() to do the right
thing in Py3k and be a wrapper around builtin open() and the main IO
stack. Once that happens, the legacy Stream* APIs become redundant
cruft that should be deprecated (although that part is significantly
less important than fixing codecs.open() itself)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Thu Jul  7 12:22:51 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 12:22:51 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
Message-ID: <4E1588FB.2050403@haypocalc.com>

Le 07/07/2011 03:16, Benjamin Peterson a ?crit :
> 2011/7/6 Victor Stinner<victor.stinner at haypocalc.com>:
>> codecs.open() will be changed to reuse the builtin open() function
>> (TextIOWrapper).
> This doesn't strike me as particularly backwards compatible, since
> you've just enumerated the differences between StreamWriter/Reader and
> TextIOWrapper.
Which kind of differences are you thinking about? I only listed two 
attributes specific to StreamReaderWriter (.reader and .writer). You 
mean that these attributes are used? There are maybe other subtle 
differences between Stream* and TextIOWrapper, but I don't think that 
anyone relies on them. Should I try to list all differences in the PEP?

If I understood correctly the previous discussion, an important point is 
to be able to write code Python 2 which "just works" on Python 3 without 
using 2to3. If you don't rely on the subtle implementation details of 
Stream*, it's possible (to use codecs.open in Python 3, even if 
codecs.open is implemented to reuse TextIOWrapper via open). If you rely 
on the differences, I bet that it is easy to not use these differences 
(and continue to be compatible with Python 2). For example, you can 
replace f.reader.read() by f.read(), it's just the same.

The two classical usages of codecs.open() (of text files) are:

- read the whole file content
- read the file line by line

For this two usecases, the API is exactly the same. Using 
f=codecs.open() or f=open() (in Python 3, or f=io.open() in Python 2), 
you can use:

- for line in f: ...
- while True: line = f.readline(); if not line: break; ...
- f.read()

I'm not saying that my PEP doesn't break the compatibility, it *does* 
break the backward compatibility. That's why we need a PEP. That's why 
there is a section called "Backwards Compatibility" in the PEP. I'm 
trying to say that I bet that nobody will notice.

The most impacting change will be (at the end) the removal of the 
StreamReader/StreamWriter API. If a program uses directly these classes, 
it will have to be updated to use TextIOWrapper (or codecs.open() or 
open() maybe).

I wrote in a previous email:

"I did search for usage of these classes on the Internet, and except 
projects
implementing their own codecs (and so implement their
StreamReader/StreamWriter classes, even if they don't use it), I only found
one project using directly StreamReader: pygment (*). I searched 
quickly, so
don't trust these results :-) StreamReader & friends are used indirectly
through codecs.open(). My patch changes codecs.open() to make it reuse open
(io.TextIOWrapper), so the deprecation of StreamReader would not be 
noticed by
most users.

(*) I also found Sphinx, but I was wrong: it doesn't use StreamReader, 
it just
has a full copy of the UTF-8-SIG codec which has a StreamReader class. I 
don't
think that the class is used."

Victor

From victor.stinner at haypocalc.com  Thu Jul  7 12:26:06 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 12:26:06 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
References: <1309999915.2958.8.camel@marge>	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
Message-ID: <4E1589BE.9040508@haypocalc.com>

Le 07/07/2011 05:26, Nick Coghlan a ?crit :
> Victor, could you please check this into the PEPs repo? It's easier to
> reference once it has a real number.
How do I upload it? Should I contact a PEP editor? How?

Victor

From victor.stinner at haypocalc.com  Thu Jul  7 12:32:32 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 12:32:32 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader
	and	codecs.StreamWriter
In-Reply-To: <4E15694A.9070002@egenix.com>
References: <1309999915.2958.8.camel@marge> <4E15694A.9070002@egenix.com>
Message-ID: <4E158B40.6010500@haypocalc.com>

Le 07/07/2011 10:07, M.-A. Lemburg a ?crit :
> The PEP's arguments for deprecating two essential codec design
> components are very one sided, by comparing "issues" to "features".
Yes, please help me to write an unbiased PEP. I don't know which tool is 
more appropriate to write a PEP with many authors.

Can I upload it to the peps repository? According to the PEP 1, only a 
PEP editor can do that.
> Please add all the comments I've made on the subject to the PEP.
I tried to incorporate all of your comments, but because the discussion 
on the bug tracker and on python-dev was long, I missed maybe some 
(important) points. Sorry about that, and please tell me which points 
should be added to the PEP.
> The most important one missing is the fact and major difference
> that TextIOWrapper does not work on a per codec basis, but
> only on a per stream basis.
Yeah, it's not clear in the PEP, I should detail this point.
> By removing the StreamReader and StreamWriter API parts of the
> codec design, you essentially make it impossible to add
> per codec variations and optimizations that require full access
> to the stream interface.
>
> A mentioned before, many improvements are possible and lots of those
> can build on TextIOWrapper and the incremental codec parts.
I wrote that in the "Possible improvements of StreamReader and 
StreamWriter" section:

"A codec can implement variants which are optimized for the specific 
encoding ..."
and
"It would be possible to change StreamReader and StreamWriter to make 
them use IncrementalDecoder and IncrementalEncoder."
> For the issues you mention in the PEP, please open tickets
> or add ticket references to the PEP.
Ok, I will do that. There are other Stream* issues, a recent example:
http://bugs.python.org/issue12508

Victor

From vinay_sajip at yahoo.co.uk  Thu Jul  7 12:53:56 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 7 Jul 2011 10:53:56 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Draft_PEP=3A_Deprecate_codecs=2EStreamRead?=
	=?utf-8?q?er_and=09codecs=2EStreamWriter?=
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>
	<loom.20110707T104519-937@post.gmane.org>
	<CADiSq7fCjf8HVQkKO_LQGD=NDVGq0tgJarwJv+CB_fy9ZnS1=Q@mail.gmail.com>
Message-ID: <loom.20110707T124219-705@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Anyone forward porting codecs.open based code will get subpar IO in
> Python 3 *because* they're trying to do the right thing in Python 2.
> That's actively harmful in my book.

I see. Presumably if they're doing a porting exercise, then it's easy enough for
them to convert codecs.open() calls to open(), if they don't want the
performance to be sub-optimal. But I thought the main thrust of this was about
deprecation of the Stream* classes, not open() vs. codecs.open().
 
> Codec writers are also told to implement utterly unnecessary
> functionality just because PEP 100 says so. Significantly less common,
> but still harmful.

Presumably only an issue for anyone writing new codecs for 2.x - I'm not sure
how many cases that'd be.

> The bare minimum change needed is for codecs.open() to do the right
> thing in Py3k and be a wrapper around builtin open() and the main IO
> stack. Once that happens, the legacy Stream* APIs become redundant
> cruft that should be deprecated (although that part is significantly
> less important than fixing codecs.open() itself)

I've no issue with telling people to use open() rather than codecs.open() when
moving code from 2.x to 3.x. But in 2.x, is there any other API which allows you
to wrap arbitrary streams? If not, then ISTM that removing the Stream* classes
would give 2.x->3.x porting projects more trouble than codecs.open() -> open().

Regards,

Vinay Sajip





From barry at python.org  Thu Jul  7 13:12:27 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 7 Jul 2011 07:12:27 -0400
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
 codecs.StreamWriter
In-Reply-To: <4E1589BE.9040508@haypocalc.com>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<4E1589BE.9040508@haypocalc.com>
Message-ID: <20110707071227.7ea79618@resist.wooz.org>

On Jul 07, 2011, at 12:26 PM, Victor Stinner wrote:

>Le 07/07/2011 05:26, Nick Coghlan a ?crit :
>> Victor, could you please check this into the PEPs repo? It's easier to
>> reference once it has a real number.
>How do I upload it? Should I contact a PEP editor? How?

Email peps at python.org

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110707/81884ffc/attachment.pgp>

From ncoghlan at gmail.com  Thu Jul  7 13:43:24 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 7 Jul 2011 21:43:24 +1000
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <20110707071227.7ea79618@resist.wooz.org>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<4E1589BE.9040508@haypocalc.com>
	<20110707071227.7ea79618@resist.wooz.org>
Message-ID: <CADiSq7cbXLmXx4DSn4z=OG8KkZUQykNEXQKPpz7rNuUjib7=8g@mail.gmail.com>

On Thu, Jul 7, 2011 at 9:12 PM, Barry Warsaw <barry at python.org> wrote:
> On Jul 07, 2011, at 12:26 PM, Victor Stinner wrote:
>
>>Le 07/07/2011 05:26, Nick Coghlan a ?crit :
>>> Victor, could you please check this into the PEPs repo? It's easier to
>>> reference once it has a real number.
>>How do I upload it? Should I contact a PEP editor? How?
>
> Email peps at python.org

Or just check it in to hg.python.org/peps (claiming the next number in
sequence - 400 at the time of writing this email). I asked if that
approach was OK quite some time ago and David said yes - PEP 1 is
written the way it is because not everyone that writes a PEP has
commit privileges for the python.org repos.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Thu Jul  7 14:08:45 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 7 Jul 2011 22:08:45 +1000
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <loom.20110707T124219-705@post.gmane.org>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>
	<loom.20110707T104519-937@post.gmane.org>
	<CADiSq7fCjf8HVQkKO_LQGD=NDVGq0tgJarwJv+CB_fy9ZnS1=Q@mail.gmail.com>
	<loom.20110707T124219-705@post.gmane.org>
Message-ID: <CADiSq7fUkE9E1nv33mriavq6++9i3hGE0q_0cKNLcO1VLd2JgA@mail.gmail.com>

On Thu, Jul 7, 2011 at 8:53 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> I've no issue with telling people to use open() rather than codecs.open() when
> moving code from 2.x to 3.x. But in 2.x, is there any other API which allows you
> to wrap arbitrary streams? If not, then ISTM that removing the Stream* classes
> would give 2.x->3.x porting projects more trouble than codecs.open() -> open().

No, using the io module is a far more robust way to wrap arbitrary
streams than using the codecs module.

It's unfortunate that nobody pointed out the redundancy when PEP 3116
was discussed and implemented, as I expect PEP 100 would have been
updated and the Stream* APIs would have been either reused or
officially jettisoned as part of the Py3k migration.

However, we're now in a situation where we have:

1. A robust Unicode capable IO implementation (the io module, based on
PEP 3116) that is available in both 2.x and 3.x that is designed to
minimise the amount of work involved in writing new codecs
2. A legacy IO implementation (the codecs module) that is available in
both 2.x and 3.x, but requires additional work on the part of codec
authors and isn't as robust as the PEP 3116 implementation

So the options are:

A. Bring the codecs module IO implementation up to the standard of the
io module implementation (less the C acceleration) and maintain the
requirement that codec authors provide StreamReader and StreamWriter
implementations.

B. Retain the full codecs module API, but reimplement relevant parts
in terms of the io module.

C. Deprecate the codecs.Stream* interfaces and make codecs.open() a
simple wrapper around the builtin open() function. Formally drop the
requirement that codec authors provide StreamReader/Writer instances
(since they are not used by the core IO implementation)

Currently, nobody has stepped forward to do the work of maintaining
the codecs IO implementation independently of the io module, so the
only two options seriously on the table are B and C. That may change
if someone actually goes through and *fixes* all the error cases that
are handled correctly by the io module but not by the codecs module
and credibly promises to continue to do so for at least the life of
3.3.

A 2to3 fixer that simply changes "codecs.open" to "open" is not
viable, as the function signatures are not compatible (the buffering
parameter appears in a different location):
    codecs.open(): open(filename, mode='rb', encoding=None,
errors='strict', buffering=1)
    3.x builtin open(): open(file, mode='r', buffering=-1,
encoding=None, errors=None, newline=None, closefd=True)

Now, the backported io module does make it possible to write correct
code as far back as 2.6 that can be forward ported cleanly to 3.x
without requiring code modifications. However, it would be nice to
transparently upgrade code that uses codecs.open to the core IO
implementation in 3.x. For people new to Python, the parallel (and
currently deficient) alternative IO implementation also qualifies at
the very least as an attractive nuisance.

Now, it may be that this PEP runs afoul of Guido's stated preference
not to introduce any more backwards incompatibilities between 2.x and
3.x that aren't absolutely essential. In that case, it may be
reasonable to add an option D to the mix, where we just add
documentation notes telling people not to use the affected codecs
module APIs and officially declare that bug reports on those APIs will
be handled with "don't use these, use the io module instead", as that
would also deal with the maintenance problem. It's pretty ugly from an
end user's point of view, though.

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Thu Jul  7 14:11:14 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 14:11:14 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader
	and	codecs.StreamWriter
In-Reply-To: <CADiSq7cbXLmXx4DSn4z=OG8KkZUQykNEXQKPpz7rNuUjib7=8g@mail.gmail.com>
References: <1309999915.2958.8.camel@marge>	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>	<4E1589BE.9040508@haypocalc.com>	<20110707071227.7ea79618@resist.wooz.org>
	<CADiSq7cbXLmXx4DSn4z=OG8KkZUQykNEXQKPpz7rNuUjib7=8g@mail.gmail.com>
Message-ID: <4E15A262.20100@haypocalc.com>

Le 07/07/2011 13:43, Nick Coghlan a ?crit :
> Or just check it in to hg.python.org/peps (claiming the next number in
> sequence - 400 at the time of writing this email). I asked if that
> approach was OK quite some time ago and David said yes - PEP 1 is
> written the way it is because not everyone that writes a PEP has
> commit privileges for the python.org repos.
Ok, done:
http://www.python.org/dev/peps/pep-0400/
http://hg.python.org/peps/file/tip/pep-0400.txt

I started to include Marc-Andre's suggestions.

Victor

From gklein at xs4all.nl  Thu Jul  7 14:59:21 2011
From: gklein at xs4all.nl (Gertjan Klein)
Date: Thu, 07 Jul 2011 14:59:21 +0200
Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs
	testing!
References: <loom.20110706T201450-905@post.gmane.org>
Message-ID: <6rab1717d1t3prr464dfcgofebp7mr59js@4ax.com>

Vinay Sajip wrote:

>The C implementation of the PEP 397-compatible Python Launcher for Windows has
>come along nicely in the last few days, and now reached a point where it would
>benefit from some testing by interested python-dev members.

I've gotten the sources from:

>https://bitbucket.org/vinay.sajip/pylauncher

GUILauncher and CLILauncher refuse to build with Visual C++ 2008 Express
Edition (using Launchers.sln):

.\CLILauncher.rc(97) : error RC2135 : file not found:
C:\Users\Vinay\Projects\Launchers\launcher.ico

This is on Windows XP Pro, 32 bit.

There are a few compilation warnings as well:

.\launcher.c(59) : warning C4996: '_wgetenv': This function or variable
may be unsafe. Consider using _wdupenv_s instead. To disable
deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

Associator builds (with the same warning displayed above). When checking
it's dependencies, I see it depends on MSVCR90.DLL. You've mentioned
that you use only "plain C and Win32 APIs"; would it be possible to
remove this dependency? That would make it possible to copy the
executable to a directory on the PATH, without having to worry about
installing the C(++) runtime.

Regards,
Gertjan.



From victor.stinner at haypocalc.com  Thu Jul  7 15:26:11 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 15:26:11 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <loom.20110707T124219-705@post.gmane.org>
References: <1309999915.2958.8.camel@marge>	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>	<CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>	<loom.20110707T104519-937@post.gmane.org>	<CADiSq7fCjf8HVQkKO_LQGD=NDVGq0tgJarwJv+CB_fy9ZnS1=Q@mail.gmail.com>
	<loom.20110707T124219-705@post.gmane.org>
Message-ID: <4E15B3F3.8010506@haypocalc.com>

Le 07/07/2011 12:53, Vinay Sajip a ?crit :
> I've no issue with telling people to use open() rather than codecs.open() when
> moving code from 2.x to 3.x. But in 2.x, is there any other API which allows you
> to wrap arbitrary streams?
Yes, io.TextIOWrapper.

Victor

From p.f.moore at gmail.com  Thu Jul  7 16:10:50 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 7 Jul 2011 15:10:50 +0100
Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs
	testing!
In-Reply-To: <loom.20110706T201450-905@post.gmane.org>
References: <loom.20110706T201450-905@post.gmane.org>
Message-ID: <CACac1F-_E=-yRChjoyQknre37c5y40+fmpJVF4oi-6Ao68T=DQ@mail.gmail.com>

On 6 July 2011 19:31, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> The C implementation of the PEP 397-compatible Python Launcher for Windows has
> come along nicely in the last few days, and now reached a point where it would
> benefit from some testing by interested python-dev members. Points of note:
>
> 1. As well as source available on
>
> https://bitbucket.org/vinay.sajip/pylauncher
>
> there are built 32- and 64-bit msi files at
>
> https://bitbucket.org/vinay.sajip/pylauncher/downloads

I just tried to install launcher.msi, and it seems to have done nothing.

This is on Windows XP SP3 with Python 2.7 installed. It's a corporate
build so there may be funny security limits hindering things, but I
have admin rights and normally don't have problems.

msiexec /i launcher.msi /l* launcher.log gives the following log file.
I can't see any errors in there, but all those "return code 1" entries
look odd (surely 0 is success and 1 failure as with OS commands?)

Paul.

=== Logging started: 07/07/2011  15:01:42 ===
Action 15:01:42: INSTALL.
Action start 15:01:42: INSTALL.
Action 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92.
Action start 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92.
Action ended 15:01:42:
ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92. Return value
1.
Action 15:01:42: ValidateProductID.
Action start 15:01:42: ValidateProductID.
Action ended 15:01:42: ValidateProductID. Return value 1.
Action 15:01:42: CostInitialize. Computing space requirements
Action start 15:01:42: CostInitialize.
Action ended 15:01:42: CostInitialize. Return value 1.
Action 15:01:42: FileCost. Computing space requirements
Action start 15:01:42: FileCost.
Action ended 15:01:42: FileCost. Return value 1.
Action 15:01:42: CostFinalize. Computing space requirements
Action start 15:01:42: CostFinalize.
Action ended 15:01:42: CostFinalize. Return value 1.
Action 15:01:42: ExecuteAction.
Action start 15:01:42: ExecuteAction.
Action start 15:01:42: INSTALL.
Action start 15:01:42: ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92.
Action ended 15:01:42:
ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92. Return value
1.
Action start 15:01:42: ValidateProductID.
Action ended 15:01:42: ValidateProductID. Return value 1.
Action start 15:01:42: CostInitialize.
Action ended 15:01:42: CostInitialize. Return value 1.
Action start 15:01:42: FileCost.
Action ended 15:01:42: FileCost. Return value 1.
Action start 15:01:42: CostFinalize.
Action ended 15:01:42: CostFinalize. Return value 1.
Action start 15:01:42: InstallValidate.
Action ended 15:01:42: InstallValidate. Return value 1.
Action start 15:01:42: InstallInitialize.
Action ended 15:01:42: InstallInitialize. Return value 1.
Action start 15:01:42: ProcessComponents.
Action ended 15:01:42: ProcessComponents. Return value 1.
Action start 15:01:42: UnpublishFeatures.
Action ended 15:01:42: UnpublishFeatures. Return value 1.
Action start 15:01:42: RemoveRegistryValues.
Action ended 15:01:42: RemoveRegistryValues. Return value 1.
Action start 15:01:42: RemoveFiles.
Action ended 15:01:42: RemoveFiles. Return value 0.
Action start 15:01:42: InstallFiles.
Action ended 15:01:42: InstallFiles. Return value 1.
Action start 15:01:42: WriteRegistryValues.
Action ended 15:01:42: WriteRegistryValues. Return value 1.
Action start 15:01:42: RegisterUser.
Action ended 15:01:42: RegisterUser. Return value 0.
Action start 15:01:42: RegisterProduct.
Action ended 15:01:42: RegisterProduct. Return value 1.
Action start 15:01:42: PublishFeatures.
Action ended 15:01:42: PublishFeatures. Return value 1.
Action start 15:01:42: PublishProduct.
Action ended 15:01:42: PublishProduct. Return value 1.
Action start 15:01:42: InstallFinalize.
Action ended 15:01:43: InstallFinalize. Return value 1.
Action ended 15:01:43: INSTALL. Return value 1.
Property(S): TARGETDIR = D:\
Property(S): SourceDir = D:\Downloads\
Property(S): Manufacturer = Vinay Sajip
Property(S): ProductCode = {298B5D62-1287-427F-B8D9-B44D605F8F6B}
Property(S): ProductLanguage = 1033
Property(S): ProductName = Python Launcher
Property(S): ProductVersion = 1.0.0.0
Property(S): UpgradeCode = {36B0A82E-0B4E-47CD-895B-FD4EC726B3AC}
Property(S): ARPPRODUCTICON = arpicon
Property(S): ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92 =
C:\Program Files\
Property(S): ProgDir.9CA91B9B_69A1_494B_A200_44AA4D54BC92 = C:\Program
Files\Python Launcher\
Property(S): PackageCode = {C9DC0563-F67F-4F32-A0B1-1D77B10AD2AF}
Property(S): ProductState = 5
Property(S): ProductToBeRegistered = 1
Property(S): CURRENTDIRECTORY = D:\Data
Property(S): CLIENTUILEVEL = 0
Property(S): CLIENTPROCESSID = 5636
Property(S): PRODUCTLANGUAGE = 1033
Property(S): USERNAME = Atos Origin
Property(S): COMPANYNAME = Atos Origin
Property(S): ROOTDRIVE = D:\
Property(S): EXECUTEACTION = INSTALL
Property(S): ACTION = INSTALL
Property(S): INSTALLLEVEL = 1
Property(S): SECONDSEQUENCE = 1
Property(S): _MSI_FEATURE_SELECTION = _NONE_
Property(S): VersionDatabase = 200
Property(S): VersionMsi = 3.01
Property(S): VersionNT = 501
Property(S): WindowsBuild = 2600
Property(S): ServicePackLevel = 3
Property(S): ServicePackLevelMinor = 0
Property(S): MsiNTProductType = 1
Property(S): WindowsFolder = C:\WINDOWS\
Property(S): WindowsVolume = C:\
Property(S): SystemFolder = C:\WINDOWS\system32\
Property(S): System16Folder = C:\WINDOWS\system\
Property(S): RemoteAdminTS = 1
Property(S): TempFolder = C:\TEMP\
Property(S): ProgramFilesFolder = C:\Program Files\
Property(S): CommonFilesFolder = C:\Program Files\Common Files\
Property(S): AppDataFolder = D:\Documents and Settings\UK03306\Application Data\
Property(S): FavoritesFolder = D:\Documents and Settings\UK03306\Favorites\
Property(S): NetHoodFolder = D:\Documents and Settings\UK03306\NetHood\
Property(S): PersonalFolder = D:\Documents and Settings\UK03306\My Documents\
Property(S): PrintHoodFolder = D:\Documents and Settings\UK03306\PrintHood\
Property(S): RecentFolder = D:\Documents and Settings\UK03306\Recent\
Property(S): SendToFolder = D:\Documents and Settings\UK03306\SendTo\
Property(S): TemplateFolder = D:\Documents and Settings\UK03306\Templates\
Property(S): CommonAppDataFolder = D:\Documents and Settings\All
Users\Application Data\
Property(S): LocalAppDataFolder = D:\Documents and
Settings\UK03306\Local Settings\Application Data\
Property(S): MyPicturesFolder = D:\Documents and Settings\UK03306\My
Documents\My Pictures\
Property(S): AdminToolsFolder = D:\Documents and
Settings\UK03306\Start Menu\Programs\Administrative Tools\
Property(S): StartupFolder = D:\Documents and Settings\UK03306\Start
Menu\Programs\Startup\
Property(S): ProgramMenuFolder = D:\Documents and
Settings\UK03306\Start Menu\Programs\
Property(S): StartMenuFolder = D:\Documents and Settings\UK03306\Start Menu\
Property(S): DesktopFolder = D:\Documents and Settings\UK03306\Desktop\
Property(S): FontsFolder = C:\WINDOWS\Fonts\
Property(S): GPTSupport = 1
Property(S): OLEAdvtSupport = 1
Property(S): ShellAdvtSupport = 1
Property(S): Intel = 6
Property(S): PhysicalMemory = 2991
Property(S): VirtualMemory = 3325
Property(S): AdminUser = 1
Property(S): LogonUser = UK03306
Property(S): UserSID = S-1-5-21-1957994488-1604221776-515967899-4426
Property(S): UserLanguageID = 2057
Property(S): ComputerName = UKCNU1131P6N
Property(S): SystemLanguageID = 2057
Property(S): ScreenX = 1280
Property(S): ScreenY = 1024
Property(S): CaptionHeight = 26
Property(S): BorderTop = 1
Property(S): BorderSide = 1
Property(S): TextHeight = 16
Property(S): ColorBits = 32
Property(S): TTCSupport = 1
Property(S): Time = 15:01:43
Property(S): Date = 07/07/2011
Property(S): MsiNetAssemblySupport = 2.0.50727.3053
Property(S): MsiWin32AssemblySupport = 5.1.2600.5512
Property(S): RedirectedDllSupport = 2
Property(S): Privileged = 1
Property(S): Installed = 00:00:00
Property(S): DATABASE = C:\WINDOWS\Installer\153760c.msi
Property(S): OriginalDatabase = D:\Downloads\launcher.msi
Property(S): UILevel = 5
Property(S): CostingComplete = 1
Property(S): OutOfDiskSpace = 0
Property(S): OutOfNoRbDiskSpace = 0
Property(S): PrimaryVolumeSpaceAvailable = 0
Property(S): PrimaryVolumeSpaceRequired = 0
Property(S): PrimaryVolumeSpaceRemaining = 0
Property(S): SOURCEDIR = D:\Downloads\
Property(S): SourcedirProduct = {298B5D62-1287-427F-B8D9-B44D605F8F6B}
Action ended 15:01:43: ExecuteAction. Return value 1.
Action ended 15:01:43: INSTALL. Return value 1.
Property(C): TARGETDIR = D:\
Property(C): Manufacturer = Vinay Sajip
Property(C): ProductCode = {298B5D62-1287-427F-B8D9-B44D605F8F6B}
Property(C): ProductLanguage = 1033
Property(C): ProductName = Python Launcher
Property(C): ProductVersion = 1.0.0.0
Property(C): UpgradeCode = {36B0A82E-0B4E-47CD-895B-FD4EC726B3AC}
Property(C): ARPPRODUCTICON = arpicon
Property(C): ProgramFilesFolder.9CA91B9B_69A1_494B_A200_44AA4D54BC92 =
C:\Program Files\
Property(C): ProgDir.9CA91B9B_69A1_494B_A200_44AA4D54BC92 = C:\Program
Files\Python Launcher\
Property(C): PackageCode = {C9DC0563-F67F-4F32-A0B1-1D77B10AD2AF}
Property(C): ProductState = 5
Property(C): ProductToBeRegistered = 1
Property(C): CURRENTDIRECTORY = D:\Data
Property(C): CLIENTUILEVEL = 0
Property(C): CLIENTPROCESSID = 5636
Property(C): PRODUCTLANGUAGE = 1033
Property(C): VersionDatabase = 200
Property(C): VersionMsi = 3.01
Property(C): VersionNT = 501
Property(C): WindowsBuild = 2600
Property(C): ServicePackLevel = 3
Property(C): ServicePackLevelMinor = 0
Property(C): MsiNTProductType = 1
Property(C): WindowsFolder = C:\WINDOWS\
Property(C): WindowsVolume = C:\
Property(C): SystemFolder = C:\WINDOWS\system32\
Property(C): System16Folder = C:\WINDOWS\system\
Property(C): RemoteAdminTS = 1
Property(C): TempFolder = C:\TEMP\
Property(C): ProgramFilesFolder = C:\Program Files\
Property(C): CommonFilesFolder = C:\Program Files\Common Files\
Property(C): AppDataFolder = D:\Documents and Settings\UK03306\Application Data\
Property(C): FavoritesFolder = D:\Documents and Settings\UK03306\Favorites\
Property(C): NetHoodFolder = D:\Documents and Settings\UK03306\NetHood\
Property(C): PersonalFolder = D:\Documents and Settings\UK03306\My Documents\
Property(C): PrintHoodFolder = D:\Documents and Settings\UK03306\PrintHood\
Property(C): RecentFolder = D:\Documents and Settings\UK03306\Recent\
Property(C): SendToFolder = D:\Documents and Settings\UK03306\SendTo\
Property(C): TemplateFolder = D:\Documents and Settings\UK03306\Templates\
Property(C): CommonAppDataFolder = D:\Documents and Settings\All
Users\Application Data\
Property(C): LocalAppDataFolder = D:\Documents and
Settings\UK03306\Local Settings\Application Data\
Property(C): MyPicturesFolder = D:\Documents and Settings\UK03306\My
Documents\My Pictures\
Property(C): AdminToolsFolder = D:\Documents and
Settings\UK03306\Start Menu\Programs\Administrative Tools\
Property(C): StartupFolder = D:\Documents and Settings\UK03306\Start
Menu\Programs\Startup\
Property(C): ProgramMenuFolder = D:\Documents and
Settings\UK03306\Start Menu\Programs\
Property(C): StartMenuFolder = D:\Documents and Settings\UK03306\Start Menu\
Property(C): DesktopFolder = D:\Documents and Settings\UK03306\Desktop\
Property(C): FontsFolder = C:\WINDOWS\Fonts\
Property(C): GPTSupport = 1
Property(C): OLEAdvtSupport = 1
Property(C): ShellAdvtSupport = 1
Property(C): Intel = 6
Property(C): PhysicalMemory = 2991
Property(C): VirtualMemory = 3325
Property(C): AdminUser = 1
Property(C): LogonUser = UK03306
Property(C): UserSID = S-1-5-21-1957994488-1604221776-515967899-4426
Property(C): UserLanguageID = 2057
Property(C): ComputerName = UKCNU1131P6N
Property(C): SystemLanguageID = 2057
Property(C): ScreenX = 1280
Property(C): ScreenY = 1024
Property(C): CaptionHeight = 26
Property(C): BorderTop = 1
Property(C): BorderSide = 1
Property(C): TextHeight = 16
Property(C): ColorBits = 32
Property(C): TTCSupport = 1
Property(C): Time = 15:01:43
Property(C): Date = 07/07/2011
Property(C): MsiNetAssemblySupport = 2.0.50727.3053
Property(C): MsiWin32AssemblySupport = 5.1.2600.5512
Property(C): RedirectedDllSupport = 2
Property(C): Privileged = 1
Property(C): USERNAME = Atos Origin
Property(C): COMPANYNAME = Atos Origin
Property(C): Installed = 00:00:00
Property(C): DATABASE = C:\WINDOWS\Installer\153760c.msi
Property(C): OriginalDatabase = D:\Downloads\launcher.msi
Property(C): VersionHandler = 3.01
Property(C): ROOTDRIVE = D:\
Property(C): EXECUTEACTION = INSTALL
Property(C): ACTION = INSTALL
Property(C): UILevel = 5
Property(C): CostingComplete = 0
Property(C): OutOfDiskSpace = 0
Property(C): OutOfNoRbDiskSpace = 0
Property(C): PrimaryVolumeSpaceAvailable = 0
Property(C): PrimaryVolumeSpaceRequired = 0
Property(C): PrimaryVolumeSpaceRemaining = 0
Property(C): INSTALLLEVEL = 1
=== Logging stopped: 07/07/2011  15:01:43 ===
MSI (c) (04:38) [15:01:43:087]: Product: Python Launcher --
Configuration completed successfully.

From victor.stinner at haypocalc.com  Thu Jul  7 16:27:14 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 16:27:14 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader
	and	codecs.StreamWriter
In-Reply-To: <CADiSq7fUkE9E1nv33mriavq6++9i3hGE0q_0cKNLcO1VLd2JgA@mail.gmail.com>
References: <1309999915.2958.8.camel@marge>	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>	<CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>	<loom.20110707T104519-937@post.gmane.org>	<CADiSq7fCjf8HVQkKO_LQGD=NDVGq0tgJarwJv+CB_fy9ZnS1=Q@mail.gmail.com>	<loom.20110707T124219-705@post.gmane.org>
	<CADiSq7fUkE9E1nv33mriavq6++9i3hGE0q_0cKNLcO1VLd2JgA@mail.gmail.com>
Message-ID: <4E15C242.20902@haypocalc.com>


> B. Retain the full codecs module API, but reimplement relevant parts
> in terms of the io module.
This solution would not break backward compatibility, or less than my 
PEP. I didn't try to implement this solution. It should be possible for 
StreamReader (-> TextIOWrapper), StreamWriter (-> TextIOWrapper) and 
StreamReaderWriter (-> TextIOWrapper), but not for EncodedFile (by the 
why, who use this horrible class? :-)).

I would prefer solution C to have only one obvious way to read-write 
text files in Python 3(.4).
> A 2to3 fixer that simply changes "codecs.open" to "open" is not
> viable, as the function signatures are not compatible (the buffering
> parameter appears in a different location):
>      codecs.open(): open(filename, mode='rb', encoding=None,
> errors='strict', buffering=1)
>      3.x builtin open(): open(file, mode='r', buffering=-1,
> encoding=None, errors=None, newline=None, closefd=True)
What? The prototypes are very close, you just to have to invert some 
arguments. Why do you think that we cannot implement that?
> Now, the backported io module does make it possible to write correct
> code as far back as 2.6 that can be forward ported cleanly to 3.x
> without requiring code modifications. However, it would be nice to
> transparently upgrade code that uses codecs.open to the core IO
> implementation in 3.x. For people new to Python, the parallel (and
> currently deficient) alternative IO implementation also qualifies at
> the very least as an attractive nuisance.
Use codecs.open() if you would like to support Python 3 and Python 2 
older than 2.6.

If you don't care of Python 2.5, use directly the io module (you just 
have to know that it is slow in Python 2.6).

Victor

From solipsis at pitrou.net  Thu Jul  7 16:28:07 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jul 2011 16:28:07 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
 codecs.StreamWriter
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<loom.20110707T084728-452@post.gmane.org>
Message-ID: <20110707162807.7c5932aa@pitrou.net>

On Thu, 7 Jul 2011 06:53:50 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Benjamin Peterson <benjamin <at> python.org> writes:
> 
> > 
> > 2011/7/6 Nick Coghlan <ncoghlan <at> gmail.com>:
> 
> > > The API of the resulting object is the same (i.e. they're file-like
> > > objects). The behavioural differences are due to cases where the
> > > codec-specific classes are currently broken.
> > 
> > Yes, but as we all know too well, people are surely relying on
> > whatever behavior there is, broken or not.
> 
> There's also the fact that code which currently runs under 2.x and 3.x would
> stop working if codecs.StreamReader/StreamWriter were to go away.

That's a fact of life for any deprecation. But it only stops working
*after* the deprecation period has expired. And deprecated stuff can
actually stay in for a long time, depending on its popularity.

The main point of the PEP, IMO, is actually the deprecation itself. By
deprecating, we signal that something isn't actively maintained
anymore, and that a (allegedly better) alternative is available.
I think that's a very reasonable thing to do, regardless of whether or
not the "thing" actually gets removed in a later version.

Regards

Antoine.



From solipsis at pitrou.net  Thu Jul  7 16:31:37 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jul 2011 16:31:37 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
 codecs.StreamWriter
References: <1309999915.2958.8.camel@marge>
	<4E15694A.9070002@egenix.com>
Message-ID: <20110707163137.38d88c33@pitrou.net>

On Thu, 07 Jul 2011 10:07:38 +0200
"M.-A. Lemburg" <mal at egenix.com> wrote:
> 
> That said, I'm not really up for a longer discussion on this. We've
> already had the discussion and decided against removing those
> parts of the codec API.

I don't remember any such decision. We decided against unilateraly
removing them without a PEP, which is quite different.
Now we have a PEP and the matter can be discussed using the appropriate
procedure.

Regards

Antoine.



From p.f.moore at gmail.com  Thu Jul  7 16:41:46 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 7 Jul 2011 15:41:46 +0100
Subject: [Python-Dev] Python Launcher for Windows (PEP 397) needs
	testing!
In-Reply-To: <1310048670.36941.YahooMailRC@web25808.mail.ukl.yahoo.com>
References: <loom.20110706T201450-905@post.gmane.org>
	<CACac1F-_E=-yRChjoyQknre37c5y40+fmpJVF4oi-6Ao68T=DQ@mail.gmail.com>
	<1310048670.36941.YahooMailRC@web25808.mail.ukl.yahoo.com>
Message-ID: <CACac1F9S4N8MWQXZz_8YFPpWxLshT+Xsw3-eQN8HYJBpDHJPJw@mail.gmail.com>

On 7 July 2011 15:24, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Hi Paul,
>
> Thanks for trying it out. If it installs successfully, nothing will appear to
> happen, Unix-style :-)
>
> There should be files installed in c:\Program Files\Python Launcher:
>
> py.exe, pyw.exe, py.ini
>
> and there should be registry entries in HKEY_CLASSES_ROOT\Python.File,
> Python.NoConFile and Python.CompiledFile which point to those installed files.
>
> Can you confirm if this is the case?

Ah, yes. This is what has happened.

In that case, some points:

1. A "silent by default" installer like this is not very usual in the
Windows world, I'd have expected a confirmation dialog at least. For
silent installs, msiexec /silent is available.

2. I thought the idea from the PEP was to put py.exe/pyw.exe into a
"system" location which is already on the PATH (likely
windows\system32, or whatever the 64-bit equivalent is). That way, py
or py -3 and similar can be used to launch the interactive
interpreter.

3. Given that you're not installing in system32, you should add the
install dir to PATH (and remove it on uninstall :-)) That's definitely
a second-best option, though, so I'd rather you didn't, but installed
in system32 instead :-)

4. If you embed both of the py.ico and pyc.ico files into the launcher
exes, you wouldn't need to have them as separate files (I see py.ico
is embedded already) and so the footprint becomes 2 exes and an ini
file. That might be a bit more palatable for people who are
uncomfortable with installing into somewhere like system32.

Otherwise it looks great.

> P.S. I didn't copy the list, as if there is some problem which requires going
> back and forth a bit, it would be OT for the list.

Agreed. I've added the list back in as the above is really design
feedback, and so would benefit from comments from the list.

Paul.

From vinay_sajip at yahoo.co.uk  Thu Jul  7 17:07:04 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 7 Jul 2011 15:07:04 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Python_Launcher_for_Windows_=28PEP_397=29_?=
	=?utf-8?q?needs=09testing!?=
References: <loom.20110706T201450-905@post.gmane.org>
	<6rab1717d1t3prr464dfcgofebp7mr59js@4ax.com>
Message-ID: <loom.20110707T170251-989@post.gmane.org>

Hi Gertjan,

Thanks for trying it.

> .\CLILauncher.rc(97) : error RC2135 : file not found:
> C:\Users\Vinay\Projects\Launchers\launcher.ico

Somewhere there's an absolute path where it should be relative - I'll get on it.

> There are a few compilation warnings as well:
> 
> .\launcher.c(59) : warning C4996: '_wgetenv': This function or variable
> may be unsafe. Consider using _wdupenv_s instead. To disable
> deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

This is a security related warning which I think can be ignored - we only ever
read environment values, never write them, so making a private copy via _wdupenv
isn't really needed IMO. I don't disable the warning because it would disable
all security-related warnings.
 
> Associator builds (with the same warning displayed above). When checking
> it's dependencies, I see it depends on MSVCR90.DLL. You've mentioned
> that you use only "plain C and Win32 APIs"; would it be possible to
> remove this dependency? That would make it possible to copy the
> executable to a directory on the PATH, without having to worry about
> installing the C(++) runtime.

I noticed this too, and later builds of associator have the library linked in
statically.

If you get any more issues, you can post them on the BitBucket issue tracker -
they are probably OT for here, unless design/PEP related.

Regards,

Vinay Sajip




From vinay_sajip at yahoo.co.uk  Thu Jul  7 17:19:26 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 7 Jul 2011 15:19:26 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Python_Launcher_for_Windows_=28PEP_397=29_?=
	=?utf-8?q?needs=09testing!?=
References: <loom.20110706T201450-905@post.gmane.org>
	<CACac1F-_E=-yRChjoyQknre37c5y40+fmpJVF4oi-6Ao68T=DQ@mail.gmail.com>
	<1310048670.36941.YahooMailRC@web25808.mail.ukl.yahoo.com>
	<CACac1F9S4N8MWQXZz_8YFPpWxLshT+Xsw3-eQN8HYJBpDHJPJw@mail.gmail.com>
Message-ID: <loom.20110707T170720-263@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> In that case, some points:
> 
> 1. A "silent by default" installer like this is not very usual in the
> Windows world, I'd have expected a confirmation dialog at least. For
> silent installs, msiexec /silent is available.

Agreed, a "success" message is probably a good idea for the standalone launcher.
The installer is a bit tentative because (a) the PEP is still
under discussion, and (b) I don't know exactly how launcher installation
will work as part of Python installation. 
 
> 2. I thought the idea from the PEP was to put py.exe/pyw.exe into a
> "system" location which is already on the PATH (likely
> windows\system32, or whatever the 64-bit equivalent is). That way, py
> or py -3 and similar can be used to launch the interactive
> interpreter.

This can always be changed in the installer - the PEP says install in System32
"if possible", and I'm not yet sure of all the security/permissions implications
of that. The current location should be usable for test purposes, even if you
have to manually add to the path for now.

> 3. Given that you're not installing in system32, you should add the
> install dir to PATH (and remove it on uninstall ) That's definitely
> a second-best option, though, so I'd rather you didn't, but installed
> in system32 instead 

I'll look at changing the installer builds to do this.

> 4. If you embed both of the py.ico and pyc.ico files into the launcher
> exes, you wouldn't need to have them as separate files (I see py.ico
> is embedded already) and so the footprint becomes 2 exes and an ini
> file. That might be a bit more palatable for people who are
> uncomfortable with installing into somewhere like system32.
> 
> Otherwise it looks great.
> 

Thanks for the feedback. Please log any implementation-related issues on the
BitBucket tracker.

Regards,

Vinay Sajip


From solipsis at pitrou.net  Thu Jul  7 18:22:43 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jul 2011 18:22:43 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
 codecs.StreamWriter
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<CADiSq7cbgVwBBkt98Em3zWQgVaZHyWYc5T24FfRgX0j1rVyUVw@mail.gmail.com>
	<loom.20110707T104519-937@post.gmane.org>
	<CADiSq7fCjf8HVQkKO_LQGD=NDVGq0tgJarwJv+CB_fy9ZnS1=Q@mail.gmail.com>
	<loom.20110707T124219-705@post.gmane.org>
	<CADiSq7fUkE9E1nv33mriavq6++9i3hGE0q_0cKNLcO1VLd2JgA@mail.gmail.com>
Message-ID: <20110707182243.6037f5fd@pitrou.net>

On Thu, 7 Jul 2011 22:08:45 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> Currently, nobody has stepped forward to do the work of maintaining
> the codecs IO implementation independently of the io module, so the
> only two options seriously on the table are B and C.

Since nobody has stepped up to implement option B either, I think the
only option seriously on the table is C.

> Now, it may be that this PEP runs afoul of Guido's stated preference
> not to introduce any more backwards incompatibilities between 2.x and
> 3.x that aren't absolutely essential.

Well, a deprecation isn't an incompatibility in itself.
Especially when deprecation warnings are hidden by default.

Regards

Antoine.



From widebandwidth at gmail.com  Thu Jul  7 18:57:59 2011
From: widebandwidth at gmail.com (high bandwidth)
Date: Thu, 7 Jul 2011 12:57:59 -0400
Subject: [Python-Dev] making socket.getaddrinfo use cached dns
Message-ID: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>

I use cached dns lookups with pdnsd on my ubuntu machine to speed up web
access as regular lookups can take 15-30 seconds. However, python's
mechanize and urllib etc use socket.getaddrinfo, which seems not to be using
dns cacheing or taking a long time because of ipv6 lookups. In either case,
I subsequent access to the same site to be fast and not require lengthy
calls to getaddrinfo. How can I get python to correctly use cached dns
lookups and ipv4 only (at least in those cases where it is appropriate).

As mentioned here:
http://stackoverflow.com/questions/2014534/force-python-mechanize-urllib2-to-only-use-a-requests/2035686#2035686
It seems that many python libraries are hardcoded to look for both ipv6 and
ipv4. Since ipv6 lookups are extremely slow at least for some users, perhaps
the devs should carry over these optional arguments to higher level
libraries like mechanize, urllib, httplib etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110707/aa5fdb96/attachment.html>

From tjreedy at udel.edu  Thu Jul  7 19:33:17 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 07 Jul 2011 10:33:17 -0700
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <20110707162807.7c5932aa@pitrou.net>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<loom.20110707T084728-452@post.gmane.org>
	<20110707162807.7c5932aa@pitrou.net>
Message-ID: <iv4qih$doe$1@dough.gmane.org>

On 7/7/2011 7:28 AM, Antoine Pitrou wrote:

> The main point of the PEP, IMO, is actually the deprecation itself. By
> deprecating, we signal that something isn't actively maintained
> anymore, and that a (allegedly better) alternative is available.
> I think that's a very reasonable thing to do, regardless of whether or
> not the "thing" actually gets removed in a later version.

Yes, the final decision could be deprecate now, remove in 4.0, as 
happened during the 2.x series.




From phd at phdru.name  Thu Jul  7 19:59:01 2011
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 7 Jul 2011 21:59:01 +0400
Subject: [Python-Dev] making socket.getaddrinfo use cached dns
In-Reply-To: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>
References: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>
Message-ID: <20110707175901.GA28561@iskra.aviel.ru>

Hello.

   We are sorry but we cannot help you. This mailing list is to work on
developing Python (adding new features to Python itself and fixing bugs);
if you're having problems learning, understanding or using Python, please
find another forum. Probably python-list/comp.lang.python mailing list/news
group is the best place; there are Python developers who participate in it;
you may get a faster, and probably more complete, answer there. See
http://www.python.org/community/ for other lists/news groups/fora. Thank
you for understanding.

On Thu, Jul 07, 2011 at 12:57:59PM -0400, high bandwidth wrote:
> I use cached dns lookups with pdnsd on my ubuntu machine to speed up web
> access as regular lookups can take 15-30 seconds. However, python's
> mechanize and urllib etc use socket.getaddrinfo, which seems not to be using
> dns cacheing or taking a long time because of ipv6 lookups. In either case,
> I subsequent access to the same site to be fast and not require lengthy
> calls to getaddrinfo. How can I get python to correctly use cached dns
> lookups and ipv4 only (at least in those cases where it is appropriate).
> 
> As mentioned here:
> http://stackoverflow.com/questions/2014534/force-python-mechanize-urllib2-to-only-use-a-requests/2035686#2035686
> It seems that many python libraries are hardcoded to look for both ipv6 and
> ipv4. Since ipv6 lookups are extremely slow at least for some users, perhaps
> the devs should carry over these optional arguments to higher level
> libraries like mechanize, urllib, httplib etc.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From victor.stinner at haypocalc.com  Thu Jul  7 20:43:51 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 07 Jul 2011 20:43:51 +0200
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader
	and	codecs.StreamWriter
In-Reply-To: <iv4qih$doe$1@dough.gmane.org>
References: <1309999915.2958.8.camel@marge>	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>	<loom.20110707T084728-452@post.gmane.org>	<20110707162807.7c5932aa@pitrou.net>
	<iv4qih$doe$1@dough.gmane.org>
Message-ID: <4E15FE67.8020601@haypocalc.com>

Le 07/07/2011 19:33, Terry Reedy a ?crit :
> On 7/7/2011 7:28 AM, Antoine Pitrou wrote:
>
>> The main point of the PEP, IMO, is actually the deprecation itself. By
>> deprecating, we signal that something isn't actively maintained
>> anymore, and that a (allegedly better) alternative is available.
>> I think that's a very reasonable thing to do, regardless of whether or
>> not the "thing" actually gets removed in a later version.
>
> Yes, the final decision could be deprecate now, remove in 4.0, as 
> happened during the 2.x series.

Python 4? Are you serious?

Victor

From brett at python.org  Thu Jul  7 21:59:41 2011
From: brett at python.org (Brett Cannon)
Date: Thu, 7 Jul 2011 12:59:41 -0700
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <4E15FE67.8020601@haypocalc.com>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<loom.20110707T084728-452@post.gmane.org>
	<20110707162807.7c5932aa@pitrou.net>
	<iv4qih$doe$1@dough.gmane.org> <4E15FE67.8020601@haypocalc.com>
Message-ID: <CAP1=2W7Xx4DKP0h8u=uxaA=RgUjUfX=jZYGkGeDNOS+RjCdqGw@mail.gmail.com>

On Thu, Jul 7, 2011 at 11:43, Victor Stinner
<victor.stinner at haypocalc.com>wrote:

> Le 07/07/2011 19:33, Terry Reedy a ?crit :
>
>  On 7/7/2011 7:28 AM, Antoine Pitrou wrote:
>>
>>  The main point of the PEP, IMO, is actually the deprecation itself. By
>>> deprecating, we signal that something isn't actively maintained
>>> anymore, and that a (allegedly better) alternative is available.
>>> I think that's a very reasonable thing to do, regardless of whether or
>>> not the "thing" actually gets removed in a later version.
>>>
>>
>> Yes, the final decision could be deprecate now, remove in 4.0, as happened
>> during the 2.x series.
>>
>
> Python 4? Are you serious?
>


Yes he is, as are others who would support that position (not me; I prefer
two releases of pending deprecation, one release deprecation, then removal).
When I was organizing the stdlib reorg, one viewpoint that came up was to
never actually remove module code but simply deprecate it so that that those
who care to use the module can continue to do so, but otherwise let it
bit-rot so that pre-existing code does not necessarily break.

-Brett



>
> Victor
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> brett%40python.org<http://mail.python.org/mailman/options/python-dev/brett%40python.org>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110707/49c1354f/attachment-0001.html>

From ncoghlan at gmail.com  Fri Jul  8 03:16:46 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 8 Jul 2011 11:16:46 +1000
Subject: [Python-Dev] Draft PEP: Deprecate codecs.StreamReader and
	codecs.StreamWriter
In-Reply-To: <4E15FE67.8020601@haypocalc.com>
References: <1309999915.2958.8.camel@marge>
	<CAPZV6o-xkgR45y-WFZ2LEo3EKhzNiQr0JT82+1wGj6TCOJdv+g@mail.gmail.com>
	<CADiSq7eOS0UE2xBoBJtRkfpg7OejKSwKEn7j6-v0YW0eSNjSzw@mail.gmail.com>
	<CAPZV6o9m7-zmtEepysYbR7TttgDTsnvt4hrg2gxE+pWDz3=CxA@mail.gmail.com>
	<loom.20110707T084728-452@post.gmane.org>
	<20110707162807.7c5932aa@pitrou.net> <iv4qih$doe$1@dough.gmane.org>
	<4E15FE67.8020601@haypocalc.com>
Message-ID: <CADiSq7cvjuWb1HwM1cpmgFTmgw2RNhFqUjftEj4V7KXtuQVz6w@mail.gmail.com>

On Fri, Jul 8, 2011 at 4:43 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Le 07/07/2011 19:33, Terry Reedy a ?crit :
>>
>> On 7/7/2011 7:28 AM, Antoine Pitrou wrote:
>>
>>> The main point of the PEP, IMO, is actually the deprecation itself. By
>>> deprecating, we signal that something isn't actively maintained
>>> anymore, and that a (allegedly better) alternative is available.
>>> I think that's a very reasonable thing to do, regardless of whether or
>>> not the "thing" actually gets removed in a later version.
>>
>> Yes, the final decision could be deprecate now, remove in 4.0, as happened
>> during the 2.x series.
>
> Python 4? Are you serious?

Py3k was a mythological "some time in the dim distant future" target
for backwards incompatible changes for a long time before it became a
real project that people were working on actually building. Py4k is
now a similarly mythological beast :)

However, like Brett, I don't think it's actually needed in this
particular case. Deprecation in 3.3, removal in 3.5 is a time frame
completely in line with the desire to avoid a repeat of the
PyCObject/PyCapsule related incompatibility problems.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From sylvain.thenault at logilab.fr  Fri Jul  8 15:51:19 2011
From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=)
Date: Fri, 8 Jul 2011 15:51:19 +0200
Subject: [Python-Dev] status of absolute_import w/ python 2.7
Message-ID: <20110708135119.GD2114@lupus.logilab.fr>

Hi there,

the documentation state that absolute_import feature is the default
behaviour with python 2.7, though it seems that it behave differently
with the __future__ import :

$ cat package/__init__.py

import subpackage

$ python2.7
Python 2.7.1+ (default, Apr 20 2011, 22:33:39) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import package
>>> 

$ cat package/__init__.py

from __future__ import absolute_import
import subpackage

$ python2.7
Python 2.7.1+ (default, Apr 20 2011, 22:33:39) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import package
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "package/__init__.py", line 23, in <module>
    import subpackage
ImportError: No module named subpackage


Maybe the doc should be fixed ?
-- 
Sylvain Th?nault                               LOGILAB, Paris (France)
Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
CubicWeb, the semantic web framework:    http://www.cubicweb.org


From status at bugs.python.org  Fri Jul  8 18:07:23 2011
From: status at bugs.python.org (Python tracker)
Date: Fri,  8 Jul 2011 18:07:23 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110708160723.22E3F1DEE9@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-07-01 - 2011-07-08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2857 ( +7)
  closed 21444 (+45)
  total  24301 (+52)

Open issues with patches: 1243 


Issues opened (33)
==================

#10898: posixmodule.c redefines FSTAT
http://bugs.python.org/issue10898  reopened by alanh

#12472: Build failure on IRIX
http://bugs.python.org/issue12472  opened by kais58

#12476: ctypes: need example how to pass raw data from Python
http://bugs.python.org/issue12476  opened by techtonik

#12478: Possible error in HTTPErrorProcessor documentation
http://bugs.python.org/issue12478  opened by sandro.tosi

#12479: Add HTTPErrorProcessor class definition
http://bugs.python.org/issue12479  opened by sandro.tosi

#12480: urllib2 doesn't use proxy (fieddler2), configed the proxy with
http://bugs.python.org/issue12480  opened by Or.Wilder

#12483: CThunkObject_dealloc should call PyObject_GC_UnTrack?
http://bugs.python.org/issue12483  opened by rfk

#12484: The Py_InitModule functions no longer exist, but remain in the
http://bugs.python.org/issue12484  opened by alejolp

#12485: textwrap.wrap: new argument for more pleasing output
http://bugs.python.org/issue12485  opened by parent5446

#12486: tokenize module should have a unicode API
http://bugs.python.org/issue12486  opened by Devin Jeanpierre

#12488: multiprocessing.Connection does not communicate pipe closure b
http://bugs.python.org/issue12488  opened by lcampagn

#12489: email.errors.HeaderParseError if base64url is used
http://bugs.python.org/issue12489  opened by guettli

#12491: Update glossary documentation for the term 'attribute'
http://bugs.python.org/issue12491  opened by orsenthil

#12494: subprocess: check_output() doesn't close pipes on error
http://bugs.python.org/issue12494  opened by haypo

#12495: Rewrite InterProcessSignalTests
http://bugs.python.org/issue12495  opened by haypo

#12498: asyncore.dispatcher_with_send, disconnection problem + miss-co
http://bugs.python.org/issue12498  opened by Fran??ois-Xavier.Bourlet

#12499: textwrap.wrap: add control for fonts with different character 
http://bugs.python.org/issue12499  opened by parent5446

#12500: Skip test_ssl.test_connect_ex() on connection error
http://bugs.python.org/issue12500  opened by haypo

#12502: 100% cpu usage when using asyncore with UNIX socket
http://bugs.python.org/issue12502  opened by Alexey.Agapitov

#12503: "with" statement error message is more confusing in Py2.7
http://bugs.python.org/issue12503  opened by Ismael.Garrido

#12506: NIS module cant handle multiple NIS map entries for the same G
http://bugs.python.org/issue12506  opened by bjorn.lofdahl

#12507: tkSimpleDialog problem
http://bugs.python.org/issue12507  opened by rzn8tr

#12508: Codecs Anomaly
http://bugs.python.org/issue12508  opened by spatz123

#12509: test_gdb fails on debug build when builddir != srcdir
http://bugs.python.org/issue12509  opened by dmalcolm

#12510: IDLE get_the_calltip mishandles raw strings
http://bugs.python.org/issue12510  opened by Roy.Fox

#12512: codecs: StreamWriter issues with stateful codecs after a seek 
http://bugs.python.org/issue12512  opened by haypo

#12513: codec.StreamReaderWriter: issues with interlaced read-write
http://bugs.python.org/issue12513  opened by haypo

#12514: timeit disables garbage collection if timed code raises an exc
http://bugs.python.org/issue12514  opened by Gareth.Rees

#12515: email modifies the message structure when the parsed email is 
http://bugs.python.org/issue12515  opened by xavierd

#12516: imghdr.what should take one argument
http://bugs.python.org/issue12516  opened by jfinkels

#12517: Large file support on Windows: sizeof(off_t) is 32 bits
http://bugs.python.org/issue12517  opened by haypo

#12518: In string.Template it's impossible to transform delimiter in t
http://bugs.python.org/issue12518  opened by py.user

#12519: Call next version 3.3.0
http://bugs.python.org/issue12519  opened by eric.araujo



Most recent 15 issues with no replies (15)
==========================================

#12519: Call next version 3.3.0
http://bugs.python.org/issue12519

#12518: In string.Template it's impossible to transform delimiter in t
http://bugs.python.org/issue12518

#12516: imghdr.what should take one argument
http://bugs.python.org/issue12516

#12515: email modifies the message structure when the parsed email is 
http://bugs.python.org/issue12515

#12513: codec.StreamReaderWriter: issues with interlaced read-write
http://bugs.python.org/issue12513

#12510: IDLE get_the_calltip mishandles raw strings
http://bugs.python.org/issue12510

#12509: test_gdb fails on debug build when builddir != srcdir
http://bugs.python.org/issue12509

#12506: NIS module cant handle multiple NIS map entries for the same G
http://bugs.python.org/issue12506

#12498: asyncore.dispatcher_with_send, disconnection problem + miss-co
http://bugs.python.org/issue12498

#12495: Rewrite InterProcessSignalTests
http://bugs.python.org/issue12495

#12491: Update glossary documentation for the term 'attribute'
http://bugs.python.org/issue12491

#12486: tokenize module should have a unicode API
http://bugs.python.org/issue12486

#12483: CThunkObject_dealloc should call PyObject_GC_UnTrack?
http://bugs.python.org/issue12483

#12479: Add HTTPErrorProcessor class definition
http://bugs.python.org/issue12479

#12478: Possible error in HTTPErrorProcessor documentation
http://bugs.python.org/issue12478



Most recent 15 issues waiting for review (15)
=============================================

#12517: Large file support on Windows: sizeof(off_t) is 32 bits
http://bugs.python.org/issue12517

#12516: imghdr.what should take one argument
http://bugs.python.org/issue12516

#12514: timeit disables garbage collection if timed code raises an exc
http://bugs.python.org/issue12514

#12509: test_gdb fails on debug build when builddir != srcdir
http://bugs.python.org/issue12509

#12502: 100% cpu usage when using asyncore with UNIX socket
http://bugs.python.org/issue12502

#12500: Skip test_ssl.test_connect_ex() on connection error
http://bugs.python.org/issue12500

#12499: textwrap.wrap: add control for fonts with different character 
http://bugs.python.org/issue12499

#12494: subprocess: check_output() doesn't close pipes on error
http://bugs.python.org/issue12494

#12485: textwrap.wrap: new argument for more pleasing output
http://bugs.python.org/issue12485

#12483: CThunkObject_dealloc should call PyObject_GC_UnTrack?
http://bugs.python.org/issue12483

#12479: Add HTTPErrorProcessor class definition
http://bugs.python.org/issue12479

#12454: mailbox: use ASCII to read/write .mh_sequences files
http://bugs.python.org/issue12454

#12452: reuse plistlib in sysconfig; deprecation process in plistlib
http://bugs.python.org/issue12452

#12451: open: avoid the locale encoding when possible
http://bugs.python.org/issue12451

#12448: smtplib's __main__ doesn't flush when prompting
http://bugs.python.org/issue12448



Top 10 most discussed issues (10)
=================================

#10181: Problems with Py_buffer management in memoryobject.c (and else
http://bugs.python.org/issue10181  35 msgs

#6721: Locks in python standard library should be sanitized on fork
http://bugs.python.org/issue6721  10 msgs

#8716: test_tk/test_tkk_guionly fails on OS X if run from buildbot sl
http://bugs.python.org/issue8716   9 msgs

#10898: posixmodule.c redefines FSTAT
http://bugs.python.org/issue10898   8 msgs

#12167: test_packaging reference leak
http://bugs.python.org/issue12167   7 msgs

#12411: cgi.parse_multipart is broken on 3.x
http://bugs.python.org/issue12411   6 msgs

#12488: multiprocessing.Connection does not communicate pipe closure b
http://bugs.python.org/issue12488   6 msgs

#10117: Tools/scripts/reindent.py fails on non-UTF-8 encodings
http://bugs.python.org/issue10117   5 msgs

#10883: urllib: socket is not closed explicitly
http://bugs.python.org/issue10883   5 msgs

#11732: Skip decorator for tests requiring manual intervention on Wind
http://bugs.python.org/issue11732   5 msgs



Issues closed (45)
==================

#4673: Distutils should provide an uninstall command
http://bugs.python.org/issue4673  closed by eric.araujo

#6234: cgi.FieldStorage is broken when given POST data
http://bugs.python.org/issue6234  closed by eric.araujo

#7123: Multiprocess Process does not always exit when run from a thre
http://bugs.python.org/issue7123  closed by neologix

#9015: f.write(s) for s > 2GB hangs in win64 (and win32?)
http://bugs.python.org/issue9015  closed by haypo

#9611: FileIO not 64-bit safe under Windows
http://bugs.python.org/issue9611  closed by haypo

#9642: #ifdef and mbcs: don't check for	defined(HAVE_USABLE_WCHAR_T)
http://bugs.python.org/issue9642  closed by haypo

#10403: Use "member" consistently
http://bugs.python.org/issue10403  closed by python-dev

#11512: adding test suite for cgitb
http://bugs.python.org/issue11512  closed by brian.curtin

#11859: test_interrupted_write_text() of test_io failed of Python 3.3 
http://bugs.python.org/issue11859  closed by haypo

#12016: Wrong behavior for '\xff\n'.decode('gb2312', 'ignore')
http://bugs.python.org/issue12016  closed by haypo

#12147: smtplib.send_message does not implement corectly rfc 2822
http://bugs.python.org/issue12147  closed by r.david.murray

#12169: Factor out common code for d2 commands register, upload and up
http://bugs.python.org/issue12169  closed by eric.araujo

#12198: zipfile.py:1047: DeprecationWarning: 'H' format requires 0 <= 
http://bugs.python.org/issue12198  closed by petri.lehtinen

#12291: file written using marshal in 3.2 can be read by 2.7, but not 
http://bugs.python.org/issue12291  closed by python-dev

#12352: multiprocessing.Value() hangs
http://bugs.python.org/issue12352  closed by neologix

#12391: packaging install fails to clean up temp files
http://bugs.python.org/issue12391  closed by python-dev

#12412: non defined representation for pwd.struct_passwd
http://bugs.python.org/issue12412  closed by eric.araujo

#12423: signal handler doesn't handle SIGABRT from os.abort
http://bugs.python.org/issue12423  closed by haypo

#12426: packaging.tests.test_command_install_dist.InstallTestCase fail
http://bugs.python.org/issue12426  closed by haypo

#12429: test_io.check_interrupted_write() sporadic failures on FreeBSD
http://bugs.python.org/issue12429  closed by haypo

#12438: IDLE problem displaying warning message
http://bugs.python.org/issue12438  closed by python-dev

#12456: Hangs in concurrent.futures
http://bugs.python.org/issue12456  closed by pitrou

#12459: time.sleep(-1.0) behaviour
http://bugs.python.org/issue12459  closed by haypo

#12465: gc.get_referents can be used to crash Python
http://bugs.python.org/issue12465  closed by georg.brandl

#12467: test_threading.test_6_daemon_threads(): Assertion failed: PyUn
http://bugs.python.org/issue12467  closed by haypo

#12468: longjmp causes uninitialized stack frame
http://bugs.python.org/issue12468  closed by neologix

#12469: test_faulthandler failures on FreeBSD 6
http://bugs.python.org/issue12469  closed by haypo

#12470: Fix cut&paste typo in test_shutil
http://bugs.python.org/issue12470  closed by orsenthil

#12471: wrong TypeError message on '%i' formatting
http://bugs.python.org/issue12471  closed by python-dev

#12473: factory func of collections.defaultdict should receive the "mi
http://bugs.python.org/issue12473  closed by amaury.forgeotdarc

#12474: Invalid read in symtable.c
http://bugs.python.org/issue12474  closed by python-dev

#12475: Generator bug allows you to chain arbitrary tracebacks to the 
http://bugs.python.org/issue12475  closed by python-dev

#12477: input() does'nt strip CR
http://bugs.python.org/issue12477  closed by georg.brandl

#12481: test_faulthandler: popups on Windows/Debug/x64
http://bugs.python.org/issue12481  closed by haypo

#12482: input() not working correctly on Mac OS X
http://bugs.python.org/issue12482  closed by ned.deily

#12487: urllib2.urlopen() returns object missing context manager
http://bugs.python.org/issue12487  closed by orsenthil

#12490: Documentation for itertools.chain.from_iterable is incorrect
http://bugs.python.org/issue12490  closed by RodolphoEckhardt

#12492: Inconsistent Python find() behavior
http://bugs.python.org/issue12492  closed by georg.brandl

#12493: subprocess: Popen.communicate() doesn't handle EINTR in some c
http://bugs.python.org/issue12493  closed by haypo

#12496: test_ssl test_connect_capath fails with "certificate verify fa
http://bugs.python.org/issue12496  closed by ned.deily

#12497: test_codecmaps_* skipped - Could not retrieve *
http://bugs.python.org/issue12497  closed by ned.deily

#12501: callable(): remove/amend the deprecation warning in Python 2.7
http://bugs.python.org/issue12501  closed by python-dev

#12504: packaging: fix database to stop skipping uninstall tests on wi
http://bugs.python.org/issue12504  closed by eric.araujo

#12505: python interpreter not handle wildards properly
http://bugs.python.org/issue12505  closed by brian.curtin

#12511: class/instance xyz has no attribute '_abc'
http://bugs.python.org/issue12511  closed by Andreas.Hasenkopf

From mail at kerrickstaley.com  Sun Jul 10 02:45:04 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Sat, 9 Jul 2011 19:45:04 -0500
Subject: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink
	upstream
In-Reply-To: <4D7E88BB.9030109@voidspace.org.uk>
References: <AANLkTikTfD9MRnonUeyA1MhBh4_EhR7SaOiodRq_X6ry@mail.gmail.com>
	<20110303211140.1e2280fd@neurotica.wooz.org>
	<20110304040954.GE3275@unaka.lan>
	<AANLkTi=dBKryU3JUGp1jA-yWbNz+vooDTN2FdjhvLt3o@mail.gmail.com>
	<AANLkTin5iDu6ybOJrMrs_bfiVO89BRx5mBStkoVzJdSb@mail.gmail.com>
	<20110304135723.F3307249609@kimball.webabinitio.net>
	<20110304101057.6b16622e@neurotica.wooz.org>
	<AANLkTimGmx-96U_x8X=+kF3Pv6HC=sdzWQV011kM1E3T@mail.gmail.com>
	<20110304133127.554716a0@neurotica.wooz.org>
	<AANLkTinsb915rQsj8EzJFR2+EdbG5z=wMrY9xqeV1HWV@mail.gmail.com>
	<4D7157EC.5090709@v.loewis.de> <4D718816.20304@skippinet.com.au>
	<AANLkTikiGWh896wsAJAr_H6VUoW3zUK5GDrNPMLKUati@mail.gmail.com>
	<4D723C90.6090307@voidspace.org.uk> <4D72F267.3050204@gmail.com>
	<AANLkTinL9juZGBTbfiAOs_Ls=bxyHkcRTYS4AyAeKJNi@mail.gmail.com>
	<4D743248.9060705@gmail.com> <4D754123.7070703@voidspace.org.uk>
	<4D756FCA.4070509@canterbury.ac.nz> <4D75718B.7050908@voidspace.org.uk>
	<4D759568.9000508@g.nevcal.com> <4D7E88BB.9030109@voidspace.org.uk>
Message-ID: <CANaWP3zu7cENB7OVNhq=q4CFLyPN+RyurZrf=zqf4D5P0VRvzQ@mail.gmail.com>

Sorry that I dropped the ball on this. I'd still like to see this get
implemented, but I got distracted with school and forgot about it.
Updates I have made to the PEP will be sent as a patch immediately
after this email.

Here's a summary of what was happenening when we left off:
* The draft SVN version from March 4 was pretty much complete.

* Some were concerned about addressing Windows, but it was generally
agreed to leave the Windows issue to another PEP. PEP 397 was started
on March 15 to address the Windows side of the issue. PEP 397
recommends that the Windows Python launcher read the shebang and use
it to determine which Python version to use; this allows one syntax
for both operating systems that is compatible with the current PEP 394
recommendation.

* There were concerns from Ned Deily about the naming of other
binaries such as idle, pydoc, and python-config; these need to be
created as idle2, pydoc2, and python2-config, with links created at
the locations of the original binaries.

* There were concerns from Glenn Linderman that the shebang line
doesn't encode enough information to flexibly handle Windows launching
(or even launching in general).

====
Here are my thoughts:
* For Ned's comments, I agree. Although the issue isn't as large with
these programs, there's no reason we can't handle them in the same
way. I updated the PEP.

* For Glenn's comments, I think the method you propose adds too much
complexity. Regardless, if the #@ syntax is implemented, it can be
described in PEP 397; it won't have an impact on the contents of this
PEP. I think, though, that having multiple syntaxes may cause many
scripts to be incompatible with multiple platforms when they don't
have to be, since Unix coders will rarely add a #@ line, and Windows
coders will likely forget the #! line.

Also, #@ really ignores the purpose of a shebang: shebangs simply
indicate an interpreter that works with the script; the shebang allows
users to run arbitrary scripts without worrying about which
interpreter they should specify. There's no reason that a script
should use one interpreter on Unix, but be incompatible with that
interpreter on Windows yet compatible with another. The name of the
Unix binary is enough to determine the implementation and version of
the interpreter to be used on Unix, and a Windows launcher should
always invoke the same implementation/version on Windows (and this
won't require hard-coding anything into the launcher if done right).
If you want the script to run with a specific interpreter and version,
possibly contingent on which operating system you're running the
script under, then you can just invoke the interpreter by name with
the script as an argument (e.g. python3 myprogram.py).

TL;DR: shebangs encode a default implementation/version, and if you
need something special, you can just manually run python3 myprogram.py
or use a .bat file.

====
Also, I updated the PEP with the clarification that commands like
python3 should be hard links (because they'll be invoked from code and
are more efficient; also, hard links are just as flexible as symlinks
here), while commands like python should be soft links (because this
makes it clear to sysadmins that they can be "switched", and it's
needed for flexibility if python3 changes). This really doesn't
matter, but can we keep it this way unless there are serious
objections?

Regards,
Kerrick Staley

From mail at kerrickstaley.com  Sun Jul 10 02:46:09 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Sat, 9 Jul 2011 19:46:09 -0500
Subject: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink
	upstream
In-Reply-To: <CANaWP3zu7cENB7OVNhq=q4CFLyPN+RyurZrf=zqf4D5P0VRvzQ@mail.gmail.com>
References: <AANLkTikTfD9MRnonUeyA1MhBh4_EhR7SaOiodRq_X6ry@mail.gmail.com>
	<20110303211140.1e2280fd@neurotica.wooz.org>
	<20110304040954.GE3275@unaka.lan>
	<AANLkTi=dBKryU3JUGp1jA-yWbNz+vooDTN2FdjhvLt3o@mail.gmail.com>
	<AANLkTin5iDu6ybOJrMrs_bfiVO89BRx5mBStkoVzJdSb@mail.gmail.com>
	<20110304135723.F3307249609@kimball.webabinitio.net>
	<20110304101057.6b16622e@neurotica.wooz.org>
	<AANLkTimGmx-96U_x8X=+kF3Pv6HC=sdzWQV011kM1E3T@mail.gmail.com>
	<20110304133127.554716a0@neurotica.wooz.org>
	<AANLkTinsb915rQsj8EzJFR2+EdbG5z=wMrY9xqeV1HWV@mail.gmail.com>
	<4D7157EC.5090709@v.loewis.de> <4D718816.20304@skippinet.com.au>
	<AANLkTikiGWh896wsAJAr_H6VUoW3zUK5GDrNPMLKUati@mail.gmail.com>
	<4D723C90.6090307@voidspace.org.uk> <4D72F267.3050204@gmail.com>
	<AANLkTinL9juZGBTbfiAOs_Ls=bxyHkcRTYS4AyAeKJNi@mail.gmail.com>
	<4D743248.9060705@gmail.com> <4D754123.7070703@voidspace.org.uk>
	<4D756FCA.4070509@canterbury.ac.nz> <4D75718B.7050908@voidspace.org.uk>
	<4D759568.9000508@g.nevcal.com> <4D7E88BB.9030109@voidspace.org.uk>
	<CANaWP3zu7cENB7OVNhq=q4CFLyPN+RyurZrf=zqf4D5P0VRvzQ@mail.gmail.com>
Message-ID: <CANaWP3yMQnYw+NdH5-zYN-52epMbLNVBYz9mYCQkX74i450WHA@mail.gmail.com>

$ svn diff
Index: pep-0394.txt
===================================================================
--- pep-0394.txt        (revision 88860)
+++ pep-0394.txt        (working copy)
@@ -1,5 +1,5 @@
 PEP: 394
-Title: The "python" command on Unix-Like Systems
+Title: The "python" Command on Unix-Like Systems
 Version: $Revision$
 Last-Modified: $Date$
 Author: Kerrick Staley <mail at kerrickstaley.com>,
@@ -53,10 +53,14 @@
 * When reinvoking the interpreter from a Python script, querying
   ``sys.executable`` to avoid hardcoded assumptions regarding the
   interpreter location remains the preferred approach.
+* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0
+should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``,
+with the original commands invoking these binaries by default, but possibly
+invoking the Python 3.0 versions instead.

 These recommendations are the outcome of the relevant python-dev discussion in
-March 2011 [1] (NOTE: More accurately, they will be such once that "Draft"
-status disappears from the PEP header, it has been moved into the "Other
+March to July 2011 [1] (NOTE: More accurately, they will be such once that
+"Draft" status disappears from the PEP header, it has been moved into
the "Other
 Informational PEP" section in PEP 0 and this note has been deleted)


@@ -144,11 +148,16 @@

 While technically a new feature, the ``make install`` command and the Mac OS
 X installer in the 2.7 version of CPython will be adjusted to create the
-new ``python2`` command in addition to the existing ``python`` and
-``python2.7`` commands. This feature will first appear in CPython 2.7.2.
+``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries,
+with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as hard links
+to the respective binaries, and ``python``, ``idle``, ``pydoc``, and
+``python-config`` as symbolic links to the respective hard links.  This feature
+will first appear in CPython 2.7.2.

-The ``make install`` command in the CPython 3.x series will continue to
-install only the ``python3`` symlink for the foreseeable future.
+The ``make install`` command in the CPython 3.x series will similarly install
+the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries
+(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and
+``python3-config`` as hard links.


 Impact on PYTHON* Environment Variables
@@ -166,27 +175,12 @@
 Exclusions of MS Windows
 ========================

-This PEP deliberately excludes any proposals relating to Microsoft Windows.
-The use of parallel installs on Windows suffers from numerous issues,
-including the "last installed wins" behaviour for handling of file
-associations, a lack of universal robust symlink support for easy aliasing of
-commands, the fact that the Python executable is not available on ``PATH`` by
-default, the fact that the ``python.exe`` and ``pythonw.exe`` names are
-used for both Python 2 and Python 3 binaries and the lack of distinction
-between the different Python versions when the Start menu shortcuts are
-divorced from their containing folders (e.g. when they appear in the
-"Recently Used" list.
+This PEP deliberately excludes any proposals relating to Microsoft Windows:
+devising an equivalent solution for Windows was deemed too complex to handle
+here. PEP 397 and the related discussion on the python-dev mailing list address
+this issue.

-While these questions are well worth addressing, they do not have easy
-answers. The authors of this particular PEP aren't inclined to even begin
-trying to answer them, but anyone that wants to tackle them should feel free
-to start working on their own PEP.

-Note that, while the topic has been excluded from this PEP, there is plenty of
-material in the linked python-dev discussion that may be useful in the design
-and implementation of a Windows-specific solution.
-
-
 References
 ==========

From riscutiavlad at gmail.com  Sun Jul 10 21:54:46 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Sun, 10 Jul 2011 12:54:46 -0700
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <BANLkTi=QEvc8sMyEaLJfagYMPoAY0Nvftw@mail.gmail.com>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
	<4E067121.9070802@canterbury.ac.nz>
	<BANLkTi=QEvc8sMyEaLJfagYMPoAY0Nvftw@mail.gmail.com>
Message-ID: <CAJ-9HZ2m4QQ+QViHYORt+aDPZxJkP_3i0D1Xw5aCH6+9-czteg@mail.gmail.com>

Opened http://bugs.python.org/issue12528 to track this.

Thank you,
Vlad

On Sun, Jun 26, 2011 at 5:48 PM, Vlad Riscutia <riscutiavlad at gmail.com>wrote:

> Well it's not really layout, because alignment is handled by pack option.
> It is how the field gets allocated. At this point I believe it will be more
> complex to come up with custom allocation option, precisely because it's up
> to each compiler to allocate the structure. Such flexibility will add a lot
> of complexity and it might not payoff. This is debatable, but I would go
> with a 3 option strategy at this point - native, GCC, MSVC and if we
> actually find interop issues with other compilers we can look into custom
> allocation.
>
> I will try to refactor the C code to more easily accommodate a mode option
> (while leaving behavior the same for now), then we can decide how the
> library interface should look like.
>
> Thank you,
> Vlad
>
> On Sat, Jun 25, 2011 at 4:37 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:
>
>> Vlad Riscutia wrote:
>>
>>> Longer term though, I think it would be better to add a property on the
>>> Structure class for configurable allocation strategy, for example Native
>>> (default), GCC, MSVC
>>>
>>
>> It could also be good to have a mode which lets you specify
>> *exactly* how the bits are laid out, independent of any
>> particular compiler.
>>
>> --
>> Greg
>>
>> ______________________________**_________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
>> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
>> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110710/26c85717/attachment.html>

From georg at python.org  Mon Jul 11 07:23:32 2011
From: georg at python.org (Georg Brandl)
Date: Mon, 11 Jul 2011 07:23:32 +0200
Subject: [Python-Dev] [RELEASED] Python 3.2.1
Message-ID: <4E1A88D4.5070704@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I am pleased to announce the
final release of Python 3.2.1.

Python 3.2.1 is the first bugfix release for Python 3.2, fixing over 120
bugs and regressions in Python 3.2.

For an extensive list of changes and features in the 3.2 line, see

    http://docs.python.org/3.2/whatsnew/3.2.html

To download Python 3.2.1 visit:

    http://www.python.org/download/releases/3.2.1/

This is a final release: Please report any bugs you may notice to:

    http://bugs.python.org/


Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.2's contributors)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)

iEYEARECAAYFAk4aiNMACgkQN9GcIYhpnLDofwCglfgDQ1/B/TxxwfqtDxK13ksz
micAn0CVWmNNaYE2a6z0N7+Dz+hCZSj1
=7Mia
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Mon Jul 11 09:47:01 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jul 2011 09:47:01 +0200
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
Message-ID: <4E1AAA75.7010903@v.loewis.de>

Am 25.06.2011 18:33, schrieb Vlad Riscutia:
> I would like to hear some opinions on this.

Sounds fine to me in principle. Warnings can be added to the
documentation at any time; please submit a patch to the tracker.
As for the API change - make sure you post your proposed API
to the list before spending time to implement it.

Regards,
Martin

From mfoord at python.org  Mon Jul 11 16:36:49 2011
From: mfoord at python.org (Michael Foord)
Date: Mon, 11 Jul 2011 15:36:49 +0100
Subject: [Python-Dev] Fwd: Dead link to documentation on Python main page
In-Reply-To: <2EF26144-B59B-4EF1-9F65-AB623C421CE2@59bits.nl>
References: <2EF26144-B59B-4EF1-9F65-AB623C421CE2@59bits.nl>
Message-ID: <4E1B0A81.9040702@python.org>



-------- Original Message --------
Subject: 	Dead link to documentation on Python main page
Date: 	Mon, 11 Jul 2011 09:27:57 -0500
From: 	Ren? van Oostrum <rene at 59bits.nl>
To: 	webmaster at python.org



Hi,

http://www.python.org ->  Quick Links (3.2.1), Documentation points to http://http://docs.python.org/3.2.1/, which does not exist.

Best regards,
Ren? van Oostrum

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110711/c0ef0391/attachment.html>

From riscutiavlad at gmail.com  Mon Jul 11 17:59:08 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Mon, 11 Jul 2011 08:59:08 -0700
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <4E1AAA75.7010903@v.loewis.de>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
	<4E1AAA75.7010903@v.loewis.de>
Message-ID: <CAJ-9HZ3+Cpr50gYy7aqG0DcSuawpBLEnRMfE-eJauynT=ZZxFg@mail.gmail.com>

Actually I already attached patch implementing everything to the issue (not
too much time spent :)). I'm hoping someone can review.

Thank you,
Vlad

On Mon, Jul 11, 2011 at 12:47 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> Am 25.06.2011 18:33, schrieb Vlad Riscutia:
> > I would like to hear some opinions on this.
>
> Sounds fine to me in principle. Warnings can be added to the
> documentation at any time; please submit a patch to the tracker.
> As for the API change - make sure you post your proposed API
> to the list before spending time to implement it.
>
> Regards,
> Martin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110711/1f02629c/attachment.html>

From merwok at netwok.org  Tue Jul 12 15:30:30 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 12 Jul 2011 15:30:30 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Fix syntax in packaging
 docs and update suspicious ignore file.
In-Reply-To: <E1QfU8z-0005FH-6H@dinsdale.python.org>
References: <E1QfU8z-0005FH-6H@dinsdale.python.org>
Message-ID: <4E1C4C76.8040504@netwok.org>

> changeset:   71283:3a4b983dd70b
> user:        Georg Brandl <georg at python.org>
> summary:
>   Fix syntax in packaging docs and update suspicious ignore file.
> 
> files:
>   Doc/library/packaging.compiler.rst   |   2 +-
>   Doc/packaging/builtdist.rst          |   2 +-
>   Doc/packaging/commandref.rst         |   2 +-

Thanks for the fixes, Georg.  I fix problems when I get warnings, but
these fell through.  Maybe you used a special command line to find those?

>   Doc/tools/sphinxext/susp-ignored.csv |  27 ++++++++++++++-

I?ve always wondered about that file?s role, and I?ve finally found
answers in Doc/tools/sphinxext/suspicious.py.  Should we update this
file when we get suspicious output?  Does using ?.. block-code:: none?
prevent program output from appearing as suspicious?

Regards

From g.brandl at gmx.net  Tue Jul 12 19:59:50 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 12 Jul 2011 19:59:50 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Fix syntax in packaging
 docs and update suspicious ignore file.
In-Reply-To: <4E1C4C76.8040504@netwok.org>
References: <E1QfU8z-0005FH-6H@dinsdale.python.org>
	<4E1C4C76.8040504@netwok.org>
Message-ID: <ivi21e$6qf$1@dough.gmane.org>

Am 12.07.2011 15:30, schrieb ?ric Araujo:
>> changeset:   71283:3a4b983dd70b
>> user:        Georg Brandl <georg at python.org>
>> summary:
>>   Fix syntax in packaging docs and update suspicious ignore file.
>> 
>> files:
>>   Doc/library/packaging.compiler.rst   |   2 +-
>>   Doc/packaging/builtdist.rst          |   2 +-
>>   Doc/packaging/commandref.rst         |   2 +-
> 
> Thanks for the fixes, Georg.  I fix problems when I get warnings, but
> these fell through.  Maybe you used a special command line to find those?

Yes: "make suspicious" in the Doc directory invokes the special builder that
looks for things that look like reST markup in the output, where it shouldn't
occur.  This is a routine step in PEP 101, so it will be done before every
release.

>>   Doc/tools/sphinxext/susp-ignored.csv |  27 ++++++++++++++-
> 
> I?ve always wondered about that file?s role, and I?ve finally found
> answers in Doc/tools/sphinxext/suspicious.py.  Should we update this
> file when we get suspicious output?  Does using ?.. block-code:: none?
> prevent program output from appearing as suspicious?

No (and neither do normal code blocks).  That is one thing that should be
improved in the builder: the content of code blocks ("literal" nodes in
docutils) shouldn't be checked.

Georg


From brett at python.org  Wed Jul 13 05:41:28 2011
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jul 2011 20:41:28 -0700
Subject: [Python-Dev] status of absolute_import w/ python 2.7
In-Reply-To: <20110708135119.GD2114@lupus.logilab.fr>
References: <20110708135119.GD2114@lupus.logilab.fr>
Message-ID: <CAP1=2W5mvMpA7kGeSFYxJqs2--ZpdBNndZP+eaw8zHd+k-TN3w@mail.gmail.com>

On Fri, Jul 8, 2011 at 06:51, Sylvain Th?nault
<sylvain.thenault at logilab.fr>wrote:

> Hi there,
>
> the documentation state that absolute_import feature is the default
> behaviour with python 2.7, though it seems that it behave differently
> with the __future__ import :
>
> $ cat package/__init__.py
>
> import subpackage
>
> $ python2.7
> Python 2.7.1+ (default, Apr 20 2011, 22:33:39)
> [GCC 4.5.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import package
> >>>
>
> $ cat package/__init__.py
>
> from __future__ import absolute_import
> import subpackage
>
> $ python2.7
> Python 2.7.1+ (default, Apr 20 2011, 22:33:39)
> [GCC 4.5.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import package
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "package/__init__.py", line 23, in <module>
>    import subpackage
> ImportError: No module named subpackage
>

So are you claiming that the import of 'package' w/o the __future__
statement actually succeeds even though there is no package.subpackage
module? Obviously that would be a flat-out bug, but I just double-checked my
sanity and that does nto work with a CPython 2.7 checkout.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110712/164e3dd0/attachment.html>

From ncoghlan at gmail.com  Wed Jul 13 06:40:50 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jul 2011 14:40:50 +1000
Subject: [Python-Dev] status of absolute_import w/ python 2.7
In-Reply-To: <CAP1=2W5mvMpA7kGeSFYxJqs2--ZpdBNndZP+eaw8zHd+k-TN3w@mail.gmail.com>
References: <20110708135119.GD2114@lupus.logilab.fr>
	<CAP1=2W5mvMpA7kGeSFYxJqs2--ZpdBNndZP+eaw8zHd+k-TN3w@mail.gmail.com>
Message-ID: <CADiSq7dtKSt13z4c=CuVaRqCF504rb4eGRwBJ3nphEBAvgSjJw@mail.gmail.com>

On Wed, Jul 13, 2011 at 1:41 PM, Brett Cannon <brett at python.org> wrote:
> So are you claiming that the import of 'package' w/o the __future__
> statement actually succeeds even though there is no package.subpackage
> module? Obviously that would be a flat-out bug, but I just double-checked my
> sanity and that does nto work with a CPython 2.7 checkout.

No, the problem is that __future__.absolute_import claims to be the
default behaviour in 2.7, but this does not appear to actually be the
case - it still tries the implicit relative import. E.g, given the
following setup:

cwd
  /package
    /__init__.py
    /submodule.py
    /submodule2.py

an "import submodule" in __init__.py or submodule2.py will succeed.

Now, the what's new for 2.7 doesn't actually *say* we made that change
and I can't find any evidence for it in NEWS either, so I think the
bug is actually in the __future__ module (and docs:
http://docs.python.org/library/__future__).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ulrich.eckhardt at dominolaser.com  Wed Jul 13 10:25:46 2011
From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt)
Date: Wed, 13 Jul 2011 10:25:46 +0200
Subject: [Python-Dev] [Python-checkins] cpython: no one passes NULL here
	(or should anyway)
In-Reply-To: <CAPZV6o9Jsp_eFe-53UgGzaxdJexdYrgbNok1vr8WcxgU5kW_LQ@mail.gmail.com>
References: <E1QdUkL-00046L-To@dinsdale.python.org>
	<CADiSq7cQi7qiCKZq=KuHoscWGiEeqV_hUgPmxXwxcU8+G9bA8Q@mail.gmail.com>
	<CAPZV6o9Jsp_eFe-53UgGzaxdJexdYrgbNok1vr8WcxgU5kW_LQ@mail.gmail.com>
Message-ID: <201107131025.46685.ulrich.eckhardt@dominolaser.com>

On Monday 04 July 2011, Benjamin Peterson wrote:
> If someone's static analysis tool starts complaining about it, I'd be
> happy to consider adding an assert...

Here is one! To me an assertion is what actually documents that you don't 
expect NULL in this context and that the missing check is not an oversight.

+1 for assert()

Uli
**************************************************************************************
Domino Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at http://www.dominolaser.com
**************************************************************************************
Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden.
E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Domino Laser GmbH ist f?r diese Folgen nicht verantwortlich.
**************************************************************************************


From barry at python.org  Wed Jul 13 15:31:25 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 13 Jul 2011 09:31:25 -0400
Subject: [Python-Dev] status of absolute_import w/ python 2.7
In-Reply-To: <CADiSq7dtKSt13z4c=CuVaRqCF504rb4eGRwBJ3nphEBAvgSjJw@mail.gmail.com>
References: <20110708135119.GD2114@lupus.logilab.fr>
	<CAP1=2W5mvMpA7kGeSFYxJqs2--ZpdBNndZP+eaw8zHd+k-TN3w@mail.gmail.com>
	<CADiSq7dtKSt13z4c=CuVaRqCF504rb4eGRwBJ3nphEBAvgSjJw@mail.gmail.com>
Message-ID: <20110713093125.247ed4ea@resist.wooz.org>

On Jul 13, 2011, at 02:40 PM, Nick Coghlan wrote:

>Now, the what's new for 2.7 doesn't actually *say* we made that change
>and I can't find any evidence for it in NEWS either, so I think the
>bug is actually in the __future__ module (and docs:
>http://docs.python.org/library/__future__).

I think that's right.  The change was not made for 2.7.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110713/8c1b4cfa/attachment.pgp>

From merwok at netwok.org  Wed Jul 13 16:34:54 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 13 Jul 2011 16:34:54 +0200
Subject: [Python-Dev] status of absolute_import w/ python 2.7
In-Reply-To: <CADiSq7dtKSt13z4c=CuVaRqCF504rb4eGRwBJ3nphEBAvgSjJw@mail.gmail.com>
References: <20110708135119.GD2114@lupus.logilab.fr>	<CAP1=2W5mvMpA7kGeSFYxJqs2--ZpdBNndZP+eaw8zHd+k-TN3w@mail.gmail.com>
	<CADiSq7dtKSt13z4c=CuVaRqCF504rb4eGRwBJ3nphEBAvgSjJw@mail.gmail.com>
Message-ID: <4E1DAD0E.8020101@netwok.org>

Le 13/07/2011 06:40, Nick Coghlan a ?crit :
> Now, the what's new for 2.7 doesn't actually *say* we made that change
> and I can't find any evidence for it in NEWS either, so I think the
> bug is actually in the __future__ module (and docs:
> http://docs.python.org/library/__future__).

I seemed to recall the change was done in 2.6, but I found only that:

> C API: the PyImport_Import() and PyImport_ImportModule() functions
> now default to absolute imports, not relative imports. This will
> affect C extensions that import other modules.

http://docs.python.org/dev/whatsnew/2.6#porting-to-python-2-6

Regards

From thinke365 at gmail.com  Thu Jul 14 03:02:01 2011
From: thinke365 at gmail.com (smith jack)
Date: Thu, 14 Jul 2011 09:02:01 +0800
Subject: [Python-Dev] Failed to install PIL on windows,
 may have something to do with python implementaton
Message-ID: <CAN1Fwxc=zR+ChuYBcfoKvcs=p6ftR9FYk=mmH6mQvS1LDYz97Q@mail.gmail.com>

I am a programmer with more than two years of python programming
experience, often the debugging process may led me to the details of
some details of python implementation, this is exciting : )

Now i face a problem, that is i want to install PIL on windows, but
failed, the error messages tells that the python installation cannot
find python information in the windows register, lol, this is true,
since i got a zipped python and extract it the a directory, not using
a msi file to install python.

So how should i do in order to install PIL for my python environment?
I find that PIL installation file is a zipped file with execution
capability  (is it of 7z format?)  Now i can open this exe file using
7z, and there are lots of .py files there, with directory somewhat as
follows:

PLATLIB
    PIL\*.py
    PIL.pth   (a plaintext file with content "PIL")

SCRIPTS
    *.py

so can i just extrat the py files to my python installation directory?
then will PIL work correctly? (should i extract PIL\*.py to
python_24\Lib\site-packages, and extract *.py to python_24\Scripts?)

Will this work?  since this have something to do with the
implementation of Python,   i come to this maillist and ask for
help.....  Hope i can receive some suggestions, Thank you :)

From rdmurray at bitdance.com  Thu Jul 14 03:35:10 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 13 Jul 2011 21:35:10 -0400
Subject: [Python-Dev] Failed to install PIL on windows,
	may have something to do with python implementaton
In-Reply-To: <CAN1Fwxc=zR+ChuYBcfoKvcs=p6ftR9FYk=mmH6mQvS1LDYz97Q@mail.gmail.com>
References: <CAN1Fwxc=zR+ChuYBcfoKvcs=p6ftR9FYk=mmH6mQvS1LDYz97Q@mail.gmail.com>
Message-ID: <20110714013511.BDF04B14005@webabinitio.net>

On Thu, 14 Jul 2011 09:02:01 +0800, smith jack <thinke365 at gmail.com> wrote:
> Will this work?  since this have something to do with the
> implementation of Python,   i come to this maillist and ask for
> help.....  Hope i can receive some suggestions, Thank you :)

Yes, but this list is about the *development* of Python.  For help on
topic like this, which are about using Python, you'll have better luck
getting an answer if you post to python-list.  There are also probably
a lot more people with experience using Python on windows on that list
than there are here.  Since we release Python as an MSI, we're not too
likely to know how to work with it on Windows if you don't install it
from the MSI.

--
R. David Murray           http://www.bitdance.com

From ethan at stoneleaf.us  Thu Jul 14 07:41:05 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 13 Jul 2011 22:41:05 -0700
Subject: [Python-Dev] Failed to install PIL on windows,
 may have something to do with python implementaton
In-Reply-To: <CAN1Fwxc=zR+ChuYBcfoKvcs=p6ftR9FYk=mmH6mQvS1LDYz97Q@mail.gmail.com>
References: <CAN1Fwxc=zR+ChuYBcfoKvcs=p6ftR9FYk=mmH6mQvS1LDYz97Q@mail.gmail.com>
Message-ID: <4E1E8171.1040809@stoneleaf.us>

smith jack wrote:
> i want to install PIL on windows, but failed

Python-Dev is for discussion of developing the next release of Python. 
This question should go to python-list, as your last question did.

Good luck.

~Ethan~

From g.brandl at gmx.net  Thu Jul 14 13:33:32 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 14 Jul 2011 13:33:32 +0200
Subject: [Python-Dev] cpython: Issue #12550: regrtest displays the
 Python traceback on SIGALRM or SIGUSR1
In-Reply-To: <E1Qh7JO-0008V4-U2@dinsdale.python.org>
References: <E1Qh7JO-0008V4-U2@dinsdale.python.org>
Message-ID: <ivmk55$8tu$1@dough.gmane.org>

Would it make sense to not propagate the signal in one case (e.g. SIGUSR1),
i.e. just display the traceback in this case?

Georg

Am 13.07.2011 23:49, schrieb victor.stinner:
> http://hg.python.org/cpython/rev/30f91fbfc8b3
> changeset:   71315:30f91fbfc8b3
> user:        Victor Stinner <victor.stinner at haypocalc.com>
> date:        Wed Jul 13 23:47:21 2011 +0200
> summary:
>   Issue #12550: regrtest displays the Python traceback on SIGALRM or SIGUSR1
> 
> files:
>   Lib/test/regrtest.py |  12 +++++++++++-
>   1 files changed, 11 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
> --- a/Lib/test/regrtest.py
> +++ b/Lib/test/regrtest.py
> @@ -172,6 +172,7 @@
>  import platform
>  import random
>  import re
> +import signal
>  import sys
>  import sysconfig
>  import tempfile
> @@ -266,9 +267,18 @@
>      on the command line.
>      """
>  
> -    # Display the Python traceback fatal errors (e.g. segfault)
> +    # Display the Python traceback on fatal errors (e.g. segfault)
>      faulthandler.enable(all_threads=True)
>  
> +    # Display the Python traceback on SIGALRM or SIGUSR1 signal
> +    signals = []
> +    if hasattr(signal, 'SIGALRM'):
> +        signals.append(signal.SIGALRM)
> +    if hasattr(signal, 'SIGUSR1'):
> +        signals.append(signal.SIGUSR1)
> +    for signum in signals:
> +        faulthandler.register(signum, chain=True)
> +
>      replace_stdout()
>  
>      support.record_original_stdout(sys.stdout)
> 
> 
> 
> 
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins



From ezio.melotti at gmail.com  Thu Jul 14 14:57:48 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Thu, 14 Jul 2011 15:57:48 +0300
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Merge with 3.2.
In-Reply-To: <E1QhLTp-0007zF-Vy@dinsdale.python.org>
References: <E1QhLTp-0007zF-Vy@dinsdale.python.org>
Message-ID: <4E1EE7CC.7070906@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110714/7bd18ccf/attachment.html>

From solipsis at pitrou.net  Thu Jul 14 16:42:10 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 14 Jul 2011 16:42:10 +0200
Subject: [Python-Dev] PEP 3151 implementation
Message-ID: <20110714164210.4b9fc93f@pitrou.net>


Hello,

Just a quick note that I have a working PEP 3151 implementation at
http://hg.python.org/features/pep-3151/, in named branch "pep-3151".
The implementation is (should be) complete on the feature side, so you
can play with it right now. It still lacks a deprecation mechanism for
old names.

It is tracked in http://bugs.python.org/issue12555.

Regards

Antoine.



From benjamin at python.org  Thu Jul 14 19:55:08 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 14 Jul 2011 12:55:08 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.1): Issue #12502:
 asyncore: fix polling loop with AF_UNIX sockets.
In-Reply-To: <E1QhQ6v-0002Ol-S7@dinsdale.python.org>
References: <E1QhQ6v-0002Ol-S7@dinsdale.python.org>
Message-ID: <CAPZV6o8L7Pgn=5Ax=XRt4anRh8gMzUbMMy8cDg+-kODEAZ6vuw@mail.gmail.com>

2011/7/14 charles-francois.natali <python-checkins at python.org>:
> http://hg.python.org/cpython/rev/42ec507815d2
> changeset: ? 71334:42ec507815d2
> branch: ? ? ?3.1
> parent: ? ? ?71126:0d4ca1e77205
> user: ? ? ? ?Charles-Fran?ois Natali <neologix at free.fr>
> date: ? ? ? ?Thu Jul 14 19:53:38 2011 +0200
> summary:
> ?Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.

Please do not apply bugfixes to 3.1.



-- 
Regards,
Benjamin

From cf.natali at gmail.com  Thu Jul 14 20:05:51 2011
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Thu, 14 Jul 2011 20:05:51 +0200
Subject: [Python-Dev] error upon Mercurial commit
Message-ID: <CAH_1eM2=2A29A9=jdKgDEqFDPmWjAMiYuTD5uG56sShCTCri0g@mail.gmail.com>

Hello,

I just committed a patch, and got the following error:

"""
$ hg push
pushing to ssh://hg at hg.python.org/cpython
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 2 changes to 2 files
remote: change(s) NOT sent, something went wrong:
remote: [Failure instance: Traceback from remote host -- Traceback unavailable
remote: ]
remote: sent email to roundup at report at bugs.python.org
remote: notified python-checkins at python.org of incoming changeset ca077f2672e3
"""

But the change seems to have been committed successfully.
I get this on every branch.
Am I the only one in this case? Any idea of what's going on here?

From g.brandl at gmx.net  Thu Jul 14 21:56:32 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 14 Jul 2011 21:56:32 +0200
Subject: [Python-Dev] error upon Mercurial commit
In-Reply-To: <CAH_1eM2=2A29A9=jdKgDEqFDPmWjAMiYuTD5uG56sShCTCri0g@mail.gmail.com>
References: <CAH_1eM2=2A29A9=jdKgDEqFDPmWjAMiYuTD5uG56sShCTCri0g@mail.gmail.com>
Message-ID: <ivnhka$9om$1@dough.gmane.org>

Am 14.07.2011 20:05, schrieb Charles-Fran?ois Natali:
> Hello,
> 
> I just committed a patch, and got the following error:
> 
> """
> $ hg push
> pushing to ssh://hg at hg.python.org/cpython
> searching for changes
> remote: adding changesets
> remote: adding manifests
> remote: adding file changes
> remote: added 1 changesets with 2 changes to 2 files
> remote: change(s) NOT sent, something went wrong:
> remote: [Failure instance: Traceback from remote host -- Traceback unavailable
> remote: ]
> remote: sent email to roundup at report at bugs.python.org
> remote: notified python-checkins at python.org of incoming changeset ca077f2672e3
> """
> 
> But the change seems to have been committed successfully.
> I get this on every branch.
> Am I the only one in this case? Any idea of what's going on here?

That is just output from the buildbot hook.  The changes have been pushed
(not committed, that is done locally on your computer), just that hook
failed to execute (I don't know why.)

Georg


From victor.stinner at haypocalc.com  Thu Jul 14 22:03:25 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 14 Jul 2011 22:03:25 +0200
Subject: [Python-Dev] cpython: Issue #12550: regrtest displays the
 Python traceback on SIGALRM or SIGUSR1
In-Reply-To: <ivmk55$8tu$1@dough.gmane.org>
References: <E1Qh7JO-0008V4-U2@dinsdale.python.org>
	<ivmk55$8tu$1@dough.gmane.org>
Message-ID: <4E1F4B8D.8030508@haypocalc.com>

Le 14/07/2011 13:33, Georg Brandl a ?crit :
> Would it make sense to not propagate the signal in one case (e.g. SIGUSR1),
> i.e. just display the traceback in this case?
>    
I opened this issue for buildbots. If the test suite doesn't fail 
anymore because of a SIGALRM or SIGUSR1, we may miss a bug in a test, or 
a real bug in a module. I don't think that anyone will read the output 
of all builds.

I patched faulthandler for this issue: I added a chain argument to 
faulthandler.register() to call also the previous signal handler. I 
prefer to call the original signal handler to not change the current 
behaviour of the test suite.

Victor


From victor.stinner at haypocalc.com  Thu Jul 14 22:33:09 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 14 Jul 2011 22:33:09 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Merge with 3.2.
In-Reply-To: <4E1EE7CC.7070906@gmail.com>
References: <E1QhLTp-0007zF-Vy@dinsdale.python.org>
	<4E1EE7CC.7070906@gmail.com>
Message-ID: <4E1F5285.7020906@haypocalc.com>

Done:

changeset:   71337:66e519792e4c
tag:         tip
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Jul 14 22:28:36 2011 +0200
files:       Lib/cgi.py Lib/test/test_cgi.py Misc/NEWS
description:
Add cgi.closelog() function to close the log file


Le 14/07/2011 14:57, Ezio Melotti a ?crit :
>
>> diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py
>> --- a/Lib/test/test_cgi.py
>> +++ b/Lib/test/test_cgi.py
>> @@ -155,7 +155,13 @@
>>               cgi.logfp = None
>>               cgi.logfile = "/dev/null"
>>               cgi.initlog("%s", "Testing log 3")
>> -            self.addCleanup(cgi.logfp.close)
>> +            def log_cleanup():
>> +                """Restore the global state of the log vars."""
>> +                cgi.logfile = ''
>> +                cgi.logfp.close()
>> +                cgi.logfp = None
>> +                cgi.log = cgi.initlog
>
>
> It was suggested (on #python-dev) to move this function to the cgi 
> module itself, but since I'm not familiar with it I just added it here 
> in order to fix a failure in the test.
>
> The cgi module has two global vars (logfile and logfp) and a global 
> function (log) that is initialized to initlog and then reassigned to 
> either dolog or nolog (a dummy function that does nothing) in initlog 
> itself[0].
>
> If someone thinks the log_cleanup function should be moved to the 
> cgi.py module and/or the code be refactored a bit, he can do it or 
> open an issue.
>
> [0]: http://hg.python.org/cpython/file/ac1c3291a689/Lib/cgi.py#l50
>
> Best Regards,



From status at bugs.python.org  Fri Jul 15 18:07:26 2011
From: status at bugs.python.org (Python tracker)
Date: Fri, 15 Jul 2011 18:07:26 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110715160726.E2CC81CFD5@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-07-08 - 2011-07-15)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2880 (+23)
  closed 21474 (+30)
  total  24354 (+53)

Open issues with patches: 1254 


Issues opened (36)
==================

#2506: Add mechanism to disable optimizations
http://bugs.python.org/issue2506  reopened by exarkun

#6755: Patch: new method get_wch for ncurses bindings: accept wide ch
http://bugs.python.org/issue6755  reopened by pitrou

#10403: Use "member" consistently
http://bugs.python.org/issue10403  reopened by eric.araujo

#12417: Inappropriate copyright on profile files
http://bugs.python.org/issue12417  reopened by eric.araujo

#12520: spurious output in test_warnings
http://bugs.python.org/issue12520  opened by pitrou

#12521: IDLE 3.2 crashes on Mac OS 10.6 with ActiveState Tcl/Tk 8.5
http://bugs.python.org/issue12521  opened by almccann

#12523: 'str' object has no attribute 'more' [/usr/lib/python3.2/async
http://bugs.python.org/issue12523  opened by Gry

#12524: change httplib docs POST example
http://bugs.python.org/issue12524  opened by georg.brandl

#12526: packaging.pypi.Crawler and resulting objects have a circular A
http://bugs.python.org/issue12526  opened by michael.mulich

#12527: assertRaisesRegex doc'd with 'msg' arg, but it's not implement
http://bugs.python.org/issue12527  opened by Brian.Jones

#12528: Implement configurable bitfield allocation strategy
http://bugs.python.org/issue12528  opened by vladris

#12529: cgi.parse_header fails on double quotes and semicolons
http://bugs.python.org/issue12529  opened by Ben.Darnell

#12531: documentation index entries for * and **
http://bugs.python.org/issue12531  opened by petere

#12532: PyPI server index names with '/' in them
http://bugs.python.org/issue12532  opened by michael.mulich

#12533: python-celementtree prevents me from running python develop.py
http://bugs.python.org/issue12533  opened by Kristoffer.Grundstr??m

#12535: Chained tracebacks are confusing because the first traceback i
http://bugs.python.org/issue12535  opened by r.david.murray

#12537: mailbox's _become_message is very fragile
http://bugs.python.org/issue12537  opened by r.david.murray

#12540: "Restart Shell" command leaves pythonw.exe processes running
http://bugs.python.org/issue12540  opened by Peter.Caven

#12541: Accepting Badly formed headers in urllib HTTPBasicAuth
http://bugs.python.org/issue12541  opened by Alex.Leon

#12542: Remove duplicates of cmp_to_key used in tests
http://bugs.python.org/issue12542  opened by eric.araujo

#12545: Incorrect handling of return codes in the posix_lseek function
http://bugs.python.org/issue12545  opened by Kuberan.Naganathan

#12546: str.format cannot fill with \x00 char
http://bugs.python.org/issue12546  opened by Gavin.Andresen

#12551: Provide data for TLS channel binding
http://bugs.python.org/issue12551  opened by Jajcus

#12553: email should default to 8bit CTE unless policy.must_be_7bit is
http://bugs.python.org/issue12553  opened by r.david.murray

#12555: PEP 3151 implementation
http://bugs.python.org/issue12555  opened by pitrou

#12556: Disable size checks in mmap.mmap()
http://bugs.python.org/issue12556  opened by superbobry

#12558: Locale-dependent crash for float width argument to Tkinter wid
http://bugs.python.org/issue12558  opened by hans.bering

#12559: gzip.open() needs an optional encoding argument
http://bugs.python.org/issue12559  opened by rhettinger

#12560: libpython.so not built on OpenBSD
http://bugs.python.org/issue12560  opened by stsp

#12561: Compiler workaround for wide string constants in Modules/getpa
http://bugs.python.org/issue12561  opened by jschneid

#12562: calling mmap twice fails on Windows
http://bugs.python.org/issue12562  opened by zolnie

#12567: curses implementation of Unicode is wrong in Python 3
http://bugs.python.org/issue12567  opened by haypo

#12568: Add functions to get the width in columns of a character
http://bugs.python.org/issue12568  opened by haypo

#12570: BaseHTTPServer.shutdown() locks if the last request 404'd
http://bugs.python.org/issue12570  opened by Philip.Horger

#12571: Python built on Linux 3.0 doesn't include DLFCN
http://bugs.python.org/issue12571  opened by djc

#12572: HP/UX compiler workarounds
http://bugs.python.org/issue12572  opened by jschneid



Most recent 15 issues with no replies (15)
==========================================

#12568: Add functions to get the width in columns of a character
http://bugs.python.org/issue12568

#12562: calling mmap twice fails on Windows
http://bugs.python.org/issue12562

#12560: libpython.so not built on OpenBSD
http://bugs.python.org/issue12560

#12556: Disable size checks in mmap.mmap()
http://bugs.python.org/issue12556

#12553: email should default to 8bit CTE unless policy.must_be_7bit is
http://bugs.python.org/issue12553

#12542: Remove duplicates of cmp_to_key used in tests
http://bugs.python.org/issue12542

#12541: Accepting Badly formed headers in urllib HTTPBasicAuth
http://bugs.python.org/issue12541

#12540: "Restart Shell" command leaves pythonw.exe processes running
http://bugs.python.org/issue12540

#12532: PyPI server index names with '/' in them
http://bugs.python.org/issue12532

#12531: documentation index entries for * and **
http://bugs.python.org/issue12531

#12526: packaging.pypi.Crawler and resulting objects have a circular A
http://bugs.python.org/issue12526

#12520: spurious output in test_warnings
http://bugs.python.org/issue12520

#12515: email modifies the message structure when the parsed email is 
http://bugs.python.org/issue12515

#12513: codec.StreamReaderWriter: issues with interlaced read-write
http://bugs.python.org/issue12513

#12506: NIS module cant handle multiple NIS map entries for the same G
http://bugs.python.org/issue12506



Most recent 15 issues waiting for review (15)
=============================================

#12572: HP/UX compiler workarounds
http://bugs.python.org/issue12572

#12567: curses implementation of Unicode is wrong in Python 3
http://bugs.python.org/issue12567

#12561: Compiler workaround for wide string constants in Modules/getpa
http://bugs.python.org/issue12561

#12560: libpython.so not built on OpenBSD
http://bugs.python.org/issue12560

#12559: gzip.open() needs an optional encoding argument
http://bugs.python.org/issue12559

#12555: PEP 3151 implementation
http://bugs.python.org/issue12555

#12551: Provide data for TLS channel binding
http://bugs.python.org/issue12551

#12546: str.format cannot fill with \x00 char
http://bugs.python.org/issue12546

#12542: Remove duplicates of cmp_to_key used in tests
http://bugs.python.org/issue12542

#12531: documentation index entries for * and **
http://bugs.python.org/issue12531

#12528: Implement configurable bitfield allocation strategy
http://bugs.python.org/issue12528

#12517: Large file support on Windows: sizeof(off_t) is 32 bits
http://bugs.python.org/issue12517

#12516: imghdr.what should take one argument
http://bugs.python.org/issue12516

#12514: timeit disables garbage collection if timed code raises an exc
http://bugs.python.org/issue12514

#12509: test_gdb fails on debug build when builddir != srcdir
http://bugs.python.org/issue12509



Top 10 most discussed issues (10)
=================================

#8668: Packaging: add a 'develop' command
http://bugs.python.org/issue8668  14 msgs

#3177: Add shutil.open
http://bugs.python.org/issue3177   9 msgs

#6755: Patch: new method get_wch for ncurses bindings: accept wide ch
http://bugs.python.org/issue6755   9 msgs

#12279: Add build_distinfo command to packaging
http://bugs.python.org/issue12279   8 msgs

#12545: Incorrect handling of return codes in the posix_lseek function
http://bugs.python.org/issue12545   8 msgs

#12551: Provide data for TLS channel binding
http://bugs.python.org/issue12551   8 msgs

#5999: compile error on HP-UX 11.22 ia64 - 'mbstate_t' is used	as a t
http://bugs.python.org/issue5999   7 msgs

#12326: Linux 3: tests should avoid using sys.platform == 'linux2'
http://bugs.python.org/issue12326   7 msgs

#12449: Add accelerator "F" to button "Finish" in all MSI installers m
http://bugs.python.org/issue12449   6 msgs

#12572: HP/UX compiler workarounds
http://bugs.python.org/issue12572   6 msgs



Issues closed (31)
==================

#3565: array documentation, method names not 3.x-compliant
http://bugs.python.org/issue3565  closed by pitrou

#4376: Nested ctypes 'BigEndianStructure' fails
http://bugs.python.org/issue4376  closed by python-dev

#10647: scrollbar crash in non-US locale format settings
http://bugs.python.org/issue10647  closed by hfischer

#11863: Enforce PEP 11 - remove support for legacy systems
http://bugs.python.org/issue11863  closed by pitrou

#12149: Segfault in _PyObject_GenericGetAttrWithDict
http://bugs.python.org/issue12149  closed by pitrou

#12440: test_ssl.test_options() failure on Snow Leopard: can't clear o
http://bugs.python.org/issue12440  closed by pitrou

#12452: deprecation process in plistlib
http://bugs.python.org/issue12452  closed by eric.araujo

#12491: Update glossary documentation for the term 'attribute'
http://bugs.python.org/issue12491  closed by rhettinger

#12502: 100% cpu usage when using asyncore with UNIX socket
http://bugs.python.org/issue12502  closed by neologix

#12519: Call next version 3.3.0
http://bugs.python.org/issue12519  closed by eric.araujo

#12522: Implement `os.startfile` under Linux and Mac
http://bugs.python.org/issue12522  closed by rosslagerwall

#12525: Unable to run a thread
http://bugs.python.org/issue12525  closed by eric.smith

#12530: cpython 3.3, __class__ missing.
http://bugs.python.org/issue12530  closed by benjamin.peterson

#12534: Tkinter doesn't support property attributes
http://bugs.python.org/issue12534  closed by r.david.murray

#12536: lib2to3 BaseFix class creates un-needed loggers leading to ref
http://bugs.python.org/issue12536  closed by python-dev

#12538: Extending int class
http://bugs.python.org/issue12538  closed by r.david.murray

#12539: multiprocessing.Event.wait(n) doesn't time out properly
http://bugs.python.org/issue12539  closed by neologix

#12543: `issubclass(collections.deque, collections.Sequence) == False`
http://bugs.python.org/issue12543  closed by rhettinger

#12544: Avoid using a pseudo-dict for _type_equality_funcs in unittest
http://bugs.python.org/issue12544  closed by benjamin.peterson

#12547: whatsnew/3.3: error in example about nntplib
http://bugs.python.org/issue12547  closed by ezio.melotti

#12548: Add suport native Functor
http://bugs.python.org/issue12548  closed by ezio.melotti

#12549: test_platform test_mac_ver fails on Darwin x86_64
http://bugs.python.org/issue12549  closed by ned.deily

#12550: regrtest: register SIGALRM signal using faulthandler
http://bugs.python.org/issue12550  closed by haypo

#12552: email.MIMEText overide BASE64 for utf8 charset
http://bugs.python.org/issue12552  closed by r.david.murray

#12554: Failed imports clean up module, but not sub modules
http://bugs.python.org/issue12554  closed by r.david.murray

#12557: Crash idle on mac
http://bugs.python.org/issue12557  closed by r.david.murray

#12563: Check out my about.me profile!
http://bugs.python.org/issue12563  closed by r.david.murray

#12564: Check out my about.me profile!
http://bugs.python.org/issue12564  closed by r.david.murray

#12565: Check out my about.me profile!
http://bugs.python.org/issue12565  closed by r.david.murray

#12566: Check out my about.me profile!
http://bugs.python.org/issue12566  closed by r.david.murray

#12569: sqlite3 segfaults and bus errors when given certain unicode st
http://bugs.python.org/issue12569  closed by ned.deily

From lekmalek at gmail.com  Sat Jul 16 10:33:22 2011
From: lekmalek at gmail.com (lekmalek)
Date: Sat, 16 Jul 2011 10:33:22 +0200
Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any
 callable object - request commiter
Message-ID: <20110716103322.330f805c@vimes>

Hello all,

Can any of you core devs have a look at
http://bugs.python.org/issue10271. It seems Brett is really busy right
now and this uncontroversial (AFAICT) one liner only needs someone to
review it and commit it. The pb is, it's holding me back a little bit,
and I really would like to have it in the next 3.2 release if possible.

Thanks for your help,

lekma

From fijall at gmail.com  Sat Jul 16 16:12:45 2011
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 16 Jul 2011 16:12:45 +0200
Subject: [Python-Dev] making socket.getaddrinfo use cached dns
In-Reply-To: <20110707175901.GA28561@iskra.aviel.ru>
References: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>
	<20110707175901.GA28561@iskra.aviel.ru>
Message-ID: <CAK5idxSD=7V46=hyaqabWqn1v3VU5qioGaeLgWEpvr0QKa849Q@mail.gmail.com>

On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman <phd at phdru.name> wrote:
> Hello.
>
> ? We are sorry but we cannot help you. This mailing list is to work on
> developing Python (adding new features to Python itself and fixing bugs);

Well, it seems this post is about adding a new feature isn't it?

Cheers,
fijal

From phd at phdru.name  Sat Jul 16 16:50:22 2011
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 16 Jul 2011 18:50:22 +0400
Subject: [Python-Dev] making socket.getaddrinfo use cached dns
In-Reply-To: <CAK5idxSD=7V46=hyaqabWqn1v3VU5qioGaeLgWEpvr0QKa849Q@mail.gmail.com>
References: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>
	<20110707175901.GA28561@iskra.aviel.ru>
	<CAK5idxSD=7V46=hyaqabWqn1v3VU5qioGaeLgWEpvr0QKa849Q@mail.gmail.com>
Message-ID: <20110716145022.GD30480@iskra.aviel.ru>

On Sat, Jul 16, 2011 at 04:12:45PM +0200, Maciej Fijalkowski wrote:
> On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman <phd at phdru.name> wrote:
> > ? We are sorry but we cannot help you. This mailing list is to work on
> > developing Python (adding new features to Python itself and fixing bugs);
> 
> Well, it seems this post is about adding a new feature isn't it?

   I don't think so. The original post is about problems with pdnsd -
could be just a local configuration problem, and has nothing with
Python. The original post is about rolling back getaddrinfo and
returning to gethostby* - certainly not a new feature. That's how I
understand the original post.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From ncoghlan at gmail.com  Sat Jul 16 16:59:17 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jul 2011 00:59:17 +1000
Subject: [Python-Dev] making socket.getaddrinfo use cached dns
In-Reply-To: <CAK5idxSD=7V46=hyaqabWqn1v3VU5qioGaeLgWEpvr0QKa849Q@mail.gmail.com>
References: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>
	<20110707175901.GA28561@iskra.aviel.ru>
	<CAK5idxSD=7V46=hyaqabWqn1v3VU5qioGaeLgWEpvr0QKa849Q@mail.gmail.com>
Message-ID: <CADiSq7dBZoTUz8PCR6tpoHk6DGsx+qBZ1KwToJdibfm8xOXMbw@mail.gmail.com>

On Sun, Jul 17, 2011 at 12:12 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman <phd at phdru.name> wrote:
>> Hello.
>>
>> ? We are sorry but we cannot help you. This mailing list is to work on
>> developing Python (adding new features to Python itself and fixing bugs);
>
> Well, it seems this post is about adding a new feature isn't it?

Not really - the key question was "How can I get python to correctly
use cached dns lookups and ipv4 only (at least in those cases where it
is appropriate)." This isn't the place to ask that question
(particularly since it's the wrong question - the real question is why
the IPv6 lookups are taking so long. Since we just call into the C
standard library for name resolution, whether it's slow or fast is an
OS configuration problem).

The latter part (very indirectly) made a feature suggestion via the
reference off to the SO question. However, hardcoding the *app* to be
IPv4 only really isn't a good workaround for IPv6 resolution taking a
long time to fail at the OS level. Exposing those flags would
encourage people to do exactly that, and that would be a *really* bad
idea (it's unfortunate enough that PEP 3144 stalled, or we might have
had better support for manipulating IPv6 addresses in the standard
library by now. We really shouldn't make things even worse by making
it easy for developers with broken IPv6 setups to switch off IPv6
support entirely).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From solipsis at pitrou.net  Sat Jul 16 17:26:07 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 16 Jul 2011 17:26:07 +0200
Subject: [Python-Dev] making socket.getaddrinfo use cached dns
References: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>
	<20110707175901.GA28561@iskra.aviel.ru>
	<CAK5idxSD=7V46=hyaqabWqn1v3VU5qioGaeLgWEpvr0QKa849Q@mail.gmail.com>
	<CADiSq7dBZoTUz8PCR6tpoHk6DGsx+qBZ1KwToJdibfm8xOXMbw@mail.gmail.com>
Message-ID: <20110716172607.5133294f@pitrou.net>

On Sun, 17 Jul 2011 00:59:17 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Exposing those flags would
> encourage people to do exactly that, and that would be a *really* bad
> idea

Making DNS resolution configurable (for example by allowing the user to
supply their own resolution function) in the stdlib's network APIs
doesn't sound like a really bad idea to me. Of course, a "no_ipv6" flag
would scale poorly and would therefore be a bad API.

> We really shouldn't make things even worse by making
> it easy for developers with broken IPv6 setups to switch off IPv6
> support entirely).

I don't think moralist arguments should have a weight when deciding
which features we add. If developers want to introduce bugs or
limitations in their software they will always be able to do it.

Regards

Antoine.



From guido at python.org  Sat Jul 16 17:46:16 2011
From: guido at python.org (Guido van Rossum)
Date: Sat, 16 Jul 2011 08:46:16 -0700
Subject: [Python-Dev] making socket.getaddrinfo use cached dns
In-Reply-To: <20110716172607.5133294f@pitrou.net>
References: <CALxN-ye+_a1q5P8a2Z7AucHs06qYg6oA9j-2ziAhhDorxuBKOw@mail.gmail.com>
	<20110707175901.GA28561@iskra.aviel.ru>
	<CAK5idxSD=7V46=hyaqabWqn1v3VU5qioGaeLgWEpvr0QKa849Q@mail.gmail.com>
	<CADiSq7dBZoTUz8PCR6tpoHk6DGsx+qBZ1KwToJdibfm8xOXMbw@mail.gmail.com>
	<20110716172607.5133294f@pitrou.net>
Message-ID: <CAP7+vJKu=E6E4gur4yK8ogTubSxJptoJrYv2PFGym43vSQ=Ftg@mail.gmail.com>

On Sat, Jul 16, 2011 at 8:26 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I don't think moralist arguments should have a weight when deciding
> which features we add. If developers want to introduce bugs or
> limitations in their software they will always be able to do it.

Actually when designing language features or APIs, what you call
"moralist arguments" take place all the time. Personally I don't think
there's anything "moral" about wanting to design an API that reduces
common mistakes, and API design should always take expected behavior
of programmers into account. Experienced developers have a huge store
of information about that in their head.

Anyway, even before the word "moralist" was used this thread would
have been better on python-ideas.

-- 
--Guido van Rossum (python.org/~guido)

From brett at python.org  Mon Jul 18 02:16:45 2011
From: brett at python.org (Brett Cannon)
Date: Sun, 17 Jul 2011 17:16:45 -0700
Subject: [Python-Dev] Latest draft of PEP 399 (Pure Python/C Accelerator
 Module Compatibility Requirements)
Message-ID: <CAP1=2W7wiE5t3dJxF-fFKOEXhykAdgtQphKvUYMjR2q5cKs9ZA@mail.gmail.com>

While at a mini-PyPy sprint w/ Alex Gaynor of PyPy and Phil Jenvey of
Jython, I decided to finally put the time in to update this PEP yet again.

The biggest changes is that the 100% branch coverage requirement has been
replaced with "comprehensive" coverage from the tests. I think we are all
enough grown-ups to not have to specify anything tighter than this. I also
added a paragraph in the Details section about using the abstract C APIs
(e.g., PyObject_GetItem) over type-specific ones (e.g., PyList_GetItem) in
order to be more supportive of duck typing from the Python code. I figure
the "be API compatible" assumes this, but mentioning it doesn't hurt (and
should help make Raymond less angry =).


PEP: 399
Title: Pure Python/C Accelerator Module Compatibility Requirements
Version: $Revision: 88219 $
Last-Modified: $Date: 2011-01-27 13:47:00 -0800 (Thu, 27 Jan 2011) $
Author: Brett Cannon <brett at python.org>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 04-Apr-2011
Python-Version: 3.3
Post-History: 04-Apr-2011, 12-Apr-2011, 17-Jul-2011

Abstract
========

The Python standard library under CPython contains various instances
of modules implemented in both pure Python and C (either entirely or
partially). This PEP requires that in these instances that the
C code *must* pass the test suite used for the pure Python code
so as to act as much as a drop-in replacement as reasonably possible
(C- and VM-specific tests are exempt). It is also required that new
C-based modules lacking a pure Python equivalent implementation get
special permission to be added to the standard library.


Rationale
=========

Python has grown beyond the CPython virtual machine (VM). IronPython_,
Jython_, and PyPy_ are all currently viable alternatives to the
CPython VM. The VM ecosystem that has sprung up around the Python
programming language has led to Python being used in many different
areas where CPython cannot be used, e.g., Jython allowing Python to be
used in Java applications.

A problem all of the VMs other than CPython face is handling modules
from the standard library that are implemented (to some extent) in C.
Since other VMs do not typically support the entire `C API of CPython`_
they are unable to use the code used to create the module. Often times
this leads these other VMs to either re-implement the modules in pure
Python or in the programming language used to implement the VM itself
(e.g., in C# for IronPython). This duplication of effort between
CPython, PyPy, Jython, and IronPython is extremely unfortunate as
implementing a module *at least* in pure Python would help mitigate
this duplicate effort.

The purpose of this PEP is to minimize this duplicate effort by
mandating that all new modules added to Python's standard library
*must* have a pure Python implementation _unless_ special dispensation
is given. This makes sure that a module in the stdlib is available to
all VMs and not just to CPython (pre-existing modules that do not meet
this requirement are exempt, although there is nothing preventing
someone from adding in a pure Python implementation retroactively).

Re-implementing parts (or all) of a module in C (in the case
of CPython) is still allowed for performance reasons, but any such
accelerated code must pass the same test suite (sans VM- or C-specific
tests) to verify semantics and prevent divergence. To accomplish this,
the test suite for the module must have comprehensive coverage of the
pure Python implementation before the acceleration code may be added.


Details
=======

Starting in Python 3.3, any modules added to the standard library must
have a pure Python implementation. This rule can only be ignored if
the Python development team grants a special exemption for the module.
Typically the exemption will be granted only when a module wraps a
specific C-based library (e.g., sqlite3_). In granting an exemption it
will be recognized that the module will be considered exclusive to
CPython and not part of Python's standard library that other VMs are
expected to support. Usage of ``ctypes`` to provide an
API for a C library will continue to be frowned upon as ``ctypes``
lacks compiler guarantees that C code typically relies upon to prevent
certain errors from occurring (e.g., API changes).

Even though a pure Python implementation is mandated by this PEP, it
does not preclude the use of a companion acceleration module. If an
acceleration module is provided it is to be named the same as the
module it is accelerating with an underscore attached as a prefix,
e.g., ``_warnings`` for ``warnings``. The common pattern to access
the accelerated code from the pure Python implementation is to import
it with an ``import *``, e.g., ``from _warnings import *``. This is
typically done at the end of the module to allow it to overwrite
specific Python objects with their accelerated equivalents. This kind
of import can also be done before the end of the module when needed,
e.g., an accelerated base class is provided but is then subclassed by
Python code. This PEP does not mandate that pre-existing modules in
the stdlib that lack a pure Python equivalent gain such a module. But
if people do volunteer to provide and maintain a pure Python
equivalent (e.g., the PyPy team volunteering their pure Python
implementation of the ``csv`` module and maintaining it) then such
code will be accepted. In those instances the C version is considered
the reference implementation in terms of expected semantics.

Any new accelerated code must act as a drop-in replacement as close
to the pure Python implementation as reasonable. Technical details of
the VM providing the accelerated code are allowed to differ as
necessary, e.g., a class being a ``type`` when implemented in C. To
verify that the Python and equivalent C code operate as similarly as
possible, both code bases must be tested using the same tests which
apply to the pure Python code (tests specific to the C code or any VM
do not follow under this requirement). The test suite is expected to
be extensive in order to verify expected semantics.

Acting as a drop-in replacement also dictates that no public API be
provided in accelerated code that does not exist in the pure Python
code.  Without this requirement people could accidentally come to rely
on a detail in the accelerated code which is not made available to
other VMs that use the pure Python implementation. To help verify
that the contract of semantic equivalence is being met, a module must
be tested both with and without its accelerated code as thoroughly as
possible.

As an example, to write tests which exercise both the pure Python and
C accelerated versions of a module, a basic idiom can be followed::

    import collections.abc
    from test.support import import_fresh_module, run_unittest
    import unittest

    c_heapq = import_fresh_module('heapq', fresh=['_heapq'])
    py_heapq = import_fresh_module('heapq', blocked=['_heapq'])


    class ExampleTest(unittest.TestCase):

        def test_heappop_exc_for_non_MutableSequence(self):
            # Raise TypeError when heap is not a
            # collections.abc.MutableSequence.
            class Spam:
                """Test class lacking many ABC-required methods
                (e.g., pop())."""
                def __len__(self):
                    return 0

            heap = Spam()
            self.assertFalse(isinstance(heap,
                                collections.abc.MutableSequence))
            with self.assertRaises(TypeError):
                self.heapq.heappop(heap)


    class AcceleratedExampleTest(ExampleTest):

        """Test using the accelerated code."""

        heapq = c_heapq


    class PyExampleTest(ExampleTest):

        """Test with just the pure Python code."""

        heapq = py_heapq


    def test_main():
        run_unittest(AcceleratedExampleTest, PyExampleTest)


    if __name__ == '__main__':
        test_main()


If this test were to provide extensive coverage for
``heapq.heappop()`` in the pure Python implementation then the
accelerated C code would be allowed to be added to CPython's standard
library. If it did not, then the test suite would need to be updated
until proper coverage was provided before the accelerated C code
could be added.

To also help with compatibility, C code should use abstract APIs on
objects to prevent accidental dependence on specific types. For
instance, if a function accepts a sequence then the C code should
default to using `PyObject_GetItem()` instead of something like
`PyList_GetItem()`. C code is allowed to have a fast path if the
proper `PyList_Check()` is used, but otherwise APIs should work with
any object that duck types to the proper interface instead of a
specific type.


Copyright
=========

This document has been placed in the public domain.


.. _IronPython: http://ironpython.net/
.. _Jython: http://www.jython.org/
.. _PyPy: http://pypy.org/
.. _C API of CPython: http://docs.python.org/py3k/c-api/index.html
.. _sqlite3: http://docs.python.org/py3k/library/sqlite3.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110717/47f4570b/attachment.html>

From brett at python.org  Mon Jul 18 04:19:59 2011
From: brett at python.org (Brett Cannon)
Date: Sun, 17 Jul 2011 19:19:59 -0700
Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any
 callable object - request commiter
In-Reply-To: <20110716103322.330f805c@vimes>
References: <20110716103322.330f805c@vimes>
Message-ID: <CAP1=2W7w9rc1E1=umrFwUoNea8rrjd8ZK_gaFEc1rHeyJs_tfg@mail.gmail.com>

Just so people know, I went ahead and fixed this for 3.3 (but not for 3.2
since it changes the API in a subtle way).

On Sat, Jul 16, 2011 at 01:33, lekmalek <lekmalek at gmail.com> wrote:

> Hello all,
>
> Can any of you core devs have a look at
> http://bugs.python.org/issue10271. It seems Brett is really busy right
> now and this uncontroversial (AFAICT) one liner only needs someone to
> review it and commit it. The pb is, it's holding me back a little bit,
> and I really would like to have it in the next 3.2 release if possible.
>
> Thanks for your help,
>
> lekma
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110717/c3470d43/attachment.html>

From mail at kerrickstaley.com  Mon Jul 18 08:43:46 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Mon, 18 Jul 2011 01:43:46 -0500
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
	/usr/bin/python2 symlink upstream)
Message-ID: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>

Hi,
These are two emails I sent a short while ago about finalizing PEP
394. There was no response, so in case the messages didn't go through,
I'm resending them.

Thanks,
Kerrick Staley



---------- Forwarded message ----------
From: Kerrick Staley <mail at kerrickstaley.com>
Date: Sat, Jul 9, 2011 at 7:45 PM
Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream
To: python-dev at python.org


Sorry that I dropped the ball on this. I'd still like to see this get
implemented, but I got distracted with school and forgot about it.
Updates I have made to the PEP will be sent as a patch immediately
after this email.

Here's a summary of what was happenening when we left off:
* The draft SVN version from March 4 was pretty much complete.

* Some were concerned about addressing Windows, but it was generally
agreed to leave the Windows issue to another PEP. PEP 397 was started
on March 15 to address the Windows side of the issue. PEP 397
recommends that the Windows Python launcher read the shebang and use
it to determine which Python version to use; this allows one syntax
for both operating systems that is compatible with the current PEP 394
recommendation.

* There were concerns from Ned Deily about the naming of other
binaries such as idle, pydoc, and python-config; these need to be
created as idle2, pydoc2, and python2-config, with links created at
the locations of the original binaries.

* There were concerns from Glenn Linderman that the shebang line
doesn't encode enough information to flexibly handle Windows launching
(or even launching in general).

====
Here are my thoughts:
* For Ned's comments, I agree. Although the issue isn't as large with
these programs, there's no reason we can't handle them in the same
way. I updated the PEP.

* For Glenn's comments, I think the method you propose adds too much
complexity. Regardless, if the #@ syntax is implemented, it can be
described in PEP 397; it won't have an impact on the contents of this
PEP. I think, though, that having multiple syntaxes may cause many
scripts to be incompatible with multiple platforms when they don't
have to be, since Unix coders will rarely add a #@ line, and Windows
coders will likely forget the #! line.

Also, #@ really ignores the purpose of a shebang: shebangs simply
indicate an interpreter that works with the script; the shebang allows
users to run arbitrary scripts without worrying about which
interpreter they should specify. There's no reason that a script
should use one interpreter on Unix, but be incompatible with that
interpreter on Windows yet compatible with another. The name of the
Unix binary is enough to determine the implementation and version of
the interpreter to be used on Unix, and a Windows launcher should
always invoke the same implementation/version on Windows (and this
won't require hard-coding anything into the launcher if done right).
If you want the script to run with a specific interpreter and version,
possibly contingent on which operating system you're running the
script under, then you can just invoke the interpreter by name with
the script as an argument (e.g. python3 myprogram.py).

TL;DR: shebangs encode a default implementation/version, and if you
need something special, you can just manually run python3 myprogram.py
or use a .bat file.

====
Also, I updated the PEP with the clarification that commands like
python3 should be hard links (because they'll be invoked from code and
are more efficient; also, hard links are just as flexible as symlinks
here), while commands like python should be soft links (because this
makes it clear to sysadmins that they can be "switched", and it's
needed for flexibility if python3 changes). This really doesn't
matter, but can we keep it this way unless there are serious
objections?

Regards,
Kerrick Staley



---------- Forwarded message ----------
From: Kerrick Staley <mail at kerrickstaley.com>
Date: Sat, Jul 9, 2011 at 7:46 PM
Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream
To: python-dev at python.org


$ svn diff
Index: pep-0394.txt
===================================================================
--- pep-0394.txt        (revision 88860)
+++ pep-0394.txt        (working copy)
@@ -1,5 +1,5 @@
 PEP: 394
-Title: The "python" command on Unix-Like Systems
+Title: The "python" Command on Unix-Like Systems
 Version: $Revision$
 Last-Modified: $Date$
 Author: Kerrick Staley <mail at kerrickstaley.com>,
@@ -53,10 +53,14 @@
 * When reinvoking the interpreter from a Python script, querying
  ``sys.executable`` to avoid hardcoded assumptions regarding the
  interpreter location remains the preferred approach.
+* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0
+should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``,
+with the original commands invoking these binaries by default, but possibly
+invoking the Python 3.0 versions instead.

 These recommendations are the outcome of the relevant python-dev discussion in
-March 2011 [1] (NOTE: More accurately, they will be such once that "Draft"
-status disappears from the PEP header, it has been moved into the "Other
+March to July 2011 [1] (NOTE: More accurately, they will be such once that
+"Draft" status disappears from the PEP header, it has been moved into
the "Other
 Informational PEP" section in PEP 0 and this note has been deleted)


@@ -144,11 +148,16 @@

 While technically a new feature, the ``make install`` command and the Mac OS
 X installer in the 2.7 version of CPython will be adjusted to create the
-new ``python2`` command in addition to the existing ``python`` and
-``python2.7`` commands. This feature will first appear in CPython 2.7.2.
+``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries,
+with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as hard links
+to the respective binaries, and ``python``, ``idle``, ``pydoc``, and
+``python-config`` as symbolic links to the respective hard links.  This feature
+will first appear in CPython 2.7.2.

-The ``make install`` command in the CPython 3.x series will continue to
-install only the ``python3`` symlink for the foreseeable future.
+The ``make install`` command in the CPython 3.x series will similarly install
+the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries
+(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and
+``python3-config`` as hard links.


 Impact on PYTHON* Environment Variables
@@ -166,27 +175,12 @@
 Exclusions of MS Windows
 ========================

-This PEP deliberately excludes any proposals relating to Microsoft Windows.
-The use of parallel installs on Windows suffers from numerous issues,
-including the "last installed wins" behaviour for handling of file
-associations, a lack of universal robust symlink support for easy aliasing of
-commands, the fact that the Python executable is not available on ``PATH`` by
-default, the fact that the ``python.exe`` and ``pythonw.exe`` names are
-used for both Python 2 and Python 3 binaries and the lack of distinction
-between the different Python versions when the Start menu shortcuts are
-divorced from their containing folders (e.g. when they appear in the
-"Recently Used" list.
+This PEP deliberately excludes any proposals relating to Microsoft Windows:
+devising an equivalent solution for Windows was deemed too complex to handle
+here. PEP 397 and the related discussion on the python-dev mailing list address
+this issue.

-While these questions are well worth addressing, they do not have easy
-answers. The authors of this particular PEP aren't inclined to even begin
-trying to answer them, but anyone that wants to tackle them should feel free
-to start working on their own PEP.

-Note that, while the topic has been excluded from this PEP, there is plenty of
-material in the linked python-dev discussion that may be useful in the design
-and implementation of a Windows-specific solution.
-
-
 References
 ==========

From nad at acm.org  Mon Jul 18 10:03:50 2011
From: nad at acm.org (Ned Deily)
Date: Mon, 18 Jul 2011 01:03:50 -0700
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
	/usr/bin/python2 symlink upstream)
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
Message-ID: <nad-62B81D.01034918072011@dough.gmane.org>

In article 
<CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA at mail.gmail.com>,
 Kerrick Staley <mail at kerrickstaley.com> wrote:
> Here are my thoughts:
> * For Ned's comments, I agree. Although the issue isn't as large with
> these programs, there's no reason we can't handle them in the same
> way. I updated the PEP.

Thanks.

> Also, I updated the PEP with the clarification that commands like
> python3 should be hard links (because they'll be invoked from code and
> are more efficient; also, hard links are just as flexible as symlinks
> here), while commands like python should be soft links (because this
> makes it clear to sysadmins that they can be "switched", and it's
> needed for flexibility if python3 changes). This really doesn't
> matter, but can we keep it this way unless there are serious
> objections?

I think adding the requirement to mandate hard link vs soft link usage 
is an unnecessary and unwarranted attempt at optimization.  For 
instance, IIRC, the OS X installers don't use any hard links: that may 
complicate the install, plus hard links on OS X HFS* file systems are a 
bit of a kludge and not necessarily more efficient than symlinks.   It's 
not a big deal but perhaps the wording should be changed to make a 
suggestion about hard links vs syminks rather than mandate which should 
be used.

-- 
 Ned Deily,
 nad at acm.org


From durga.disc at gmail.com  Mon Jul 18 17:32:38 2011
From: durga.disc at gmail.com (Durga D)
Date: Mon, 18 Jul 2011 21:02:38 +0530
Subject: [Python-Dev] Python web adv. ?
Message-ID: <CAANVYQYqzkDWcOsp067HapD_VbYzx1ENt38VeS1HgwiXO9-LXw@mail.gmail.com>

Hi All,

    My desktop application opens web page from tomcat/apache server in
button click event (MySQL database by using JSP/Servlets on internet from
user machine). Here, I found some delay in opening web pages.  Is there any
advantages with Python over JSP/Servlets?

    I am new to web programming. Please provide any inputs on Python over
jsp/servlets.

    Thanks in advance.

Regards,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/2ce1ff1c/attachment.html>

From _ at lvh.cc  Mon Jul 18 17:49:01 2011
From: _ at lvh.cc (Laurens Van Houtven)
Date: Mon, 18 Jul 2011 17:49:01 +0200
Subject: [Python-Dev] Python web adv. ?
In-Reply-To: <CAANVYQYqzkDWcOsp067HapD_VbYzx1ENt38VeS1HgwiXO9-LXw@mail.gmail.com>
References: <CAANVYQYqzkDWcOsp067HapD_VbYzx1ENt38VeS1HgwiXO9-LXw@mail.gmail.com>
Message-ID: <CAE_Hg6aGWbbfcz+sWU=mmt8BJmzUspBgL7XzibC-EYui7Qhzww@mail.gmail.com>

Hi!

This list is for developing Python itself, not about developing *in* Python.
For more support try the comp.lang.python newsgroup or the equivalent
http://mail.python.org/mailman/listinfo/python-list mailing list, or the
#python IRC channel on Freenode.

cheers
lvh

On Mon, Jul 18, 2011 at 5:32 PM, Durga D <durga.disc at gmail.com> wrote:

> Hi All,
>
>     My desktop application opens web page from tomcat/apache server in
> button click event (MySQL database by using JSP/Servlets on internet from
> user machine). Here, I found some delay in opening web pages.  Is there any
> advantages with Python over JSP/Servlets?
>
>     I am new to web programming. Please provide any inputs on Python over
> jsp/servlets.
>
>     Thanks in advance.
>
> Regards,
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/_%40lvh.cc
>
>


-- 
cheers
lvh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/d68b1cb6/attachment.html>

From v+python at g.nevcal.com  Mon Jul 18 21:10:46 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 18 Jul 2011 12:10:46 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
Message-ID: <4E248536.30006@g.nevcal.com>

Attached reduced test case works fine with Python 3.1, fails with Python3.2:

SyntaxError: Non-ASCII character '\xc3' in file D:\my\py\t32enc.py on 
line 1, but no encoding declared; see 
http://www.python.org/peps/pep-0263.html for details

I'm familiar with the PEP, but thought 3.x cured that.  Python 3.1 
produces no error; I'm not sure that 3.2 did or didn't report such 
problems, or if the programs I ran with 3.2 simply didn't contain 
non-ASCII characters.

The file is UTF-8 encoded with no pseudo-BOM.

Is this intentional, or a regression (in which case I will open a ticket)?

If a regression, does that mean we have no tests for it, possibly 
because most of the tests contain the PEP 263 encoding directive, 
because most people using 3.x are still also using 2.x (I am not)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/4dac2577/attachment.html>
-------------- next part --------------
"L ? W"

From tjreedy at udel.edu  Mon Jul 18 21:40:39 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 18 Jul 2011 15:40:39 -0400
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E248536.30006@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
Message-ID: <j0227l$lal$1@dough.gmane.org>

On 7/18/2011 3:10 PM, Glenn Linderman wrote:
> Attached reduced test case works fine with Python 3.1, fails with Python3.2:
>
> SyntaxError: Non-ASCII character '\xc3' in file D:\my\py\t32enc.py on
> line 1, but no encoding declared; see
> http://www.python.org/peps/pep-0263.html for details

It runs fine for me on winxp, 3.2.1, both command prompt and idle.
When cut/paste worked, I downloaded file and ran that.

> I'm familiar with the PEP, but thought 3.x cured that.  Python 3.1
> produces no error; I'm not sure that 3.2 did or didn't report such
> problems, or if the programs I ran with 3.2 simply didn't contain
> non-ASCII characters.
>
> The file is UTF-8 encoded with no pseudo-BOM.
>
> Is this intentional, or a regression (in which case I will open a ticket)?
>
> If a regression, does that mean we have no tests for it, possibly
> because most of the tests contain the PEP 263 encoding directive,
> because most people using 3.x are still also using 2.x (I am not)?

Or there could be a test (have not checked) which usually passes.


-- 
Terry Jan Reedy


From p.f.moore at gmail.com  Mon Jul 18 22:01:06 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 18 Jul 2011 21:01:06 +0100
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E248536.30006@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
Message-ID: <CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>

2011/7/18 Glenn Linderman <v+python at g.nevcal.com>:
> Attached reduced test case works fine with Python 3.1, fails with Python3.2:

PS D:\Data> py -3 .\t32enc.py
PS D:\Data> py -2 .\t32enc.py
  File ".\t32enc.py", line 1
SyntaxError: Non-ASCII character '\xc3' in file .\t32enc.py on line 1,
but no encoding declared; see http://www.python.o
rg/peps/pep-0263.html for details
PS D:\Data> py -3 --version
Python 3.2.1
PS D:\Data> py -2 --version
Python 2.7

Windows 7 32-bit, py is Vinay's implementation of the PEP 397 launcher
for Windows.

This looks like correct output to me.

Could it be your environment somehow?
Paul.

From anthony.hw.kong at gmail.com  Mon Jul 18 22:35:52 2011
From: anthony.hw.kong at gmail.com (Anthony Kong)
Date: Tue, 19 Jul 2011 06:35:52 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
Message-ID: <CAAyd8cnAzvGaDQ8XomN2etgzdJ1G3qwm4AUM1J0+EF3QWgWVrg@mail.gmail.com>

Similar outcome as Paul's.

$ python3 t32enc.py
$ python t32enc.py
  File "t32enc.py", line 1
SyntaxError: Non-ASCII character '\xc3' in file t32enc.py on line 1, but no
encoding declared; see http://www.python.org/peps/pep-0263.html for details
$ python3 -V
Python 3.2.1a0
$ python -V
Python 2.6.1

Running the script on OSX 10.6.8

(And I just realised I am running 3.2.1a0...)

Cheers

On Tue, Jul 19, 2011 at 6:01 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> 2011/7/18 Glenn Linderman <v+python at g.nevcal.com>:
> > Attached reduced test case works fine with Python 3.1, fails with
> Python3.2:
>
> PS D:\Data> py -3 .\t32enc.py
> PS D:\Data> py -2 .\t32enc.py
>  File ".\t32enc.py", line 1
> SyntaxError: Non-ASCII character '\xc3' in file .\t32enc.py on line 1,
> but no encoding declared; see http://www.python.o
> rg/peps/pep-0263.html for details
> PS D:\Data> py -3 --version
> Python 3.2.1
> PS D:\Data> py -2 --version
> Python 2.7
>
> Windows 7 32-bit, py is Vinay's implementation of the PEP 397 launcher
> for Windows.
>
> This looks like correct output to me.
>
> Could it be your environment somehow?
> Paul.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/anthony.hw.kong%40gmail.com
>



-- 

Tony Kong
*blog:* www.ahwkong.com

Don?t EVER make the mistake that you can design something better than what
you get from ruthless massively parallel trial-and-error with a feedback
cycle. That?s giving your intelligence *much* too much credit.


- Linus Torvalds
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110719/6519a441/attachment.html>

From v+python at g.nevcal.com  Mon Jul 18 22:36:48 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 18 Jul 2011 13:36:48 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
Message-ID: <4E249960.9000109@g.nevcal.com>

On 7/18/2011 1:01 PM, Paul Moore wrote:
> 2011/7/18 Glenn Linderman<v+python at g.nevcal.com>:
>> Attached reduced test case works fine with Python 3.1, fails with Python3.2:
> PS D:\Data>  py -3 .\t32enc.py
> PS D:\Data>  py -2 .\t32enc.py
>    File ".\t32enc.py", line 1
> SyntaxError: Non-ASCII character '\xc3' in file .\t32enc.py on line 1,
> but no encoding declared; see http://www.python.o
> rg/peps/pep-0263.html for details
> PS D:\Data>  py -3 --version
> Python 3.2.1
> PS D:\Data>  py -2 --version
> Python 2.7
>
> Windows 7 32-bit, py is Vinay's implementation of the PEP 397 launcher
> for Windows.
>
> This looks like correct output to me.
>
> Could it be your environment somehow?
> Paul.

Yes, a surprise.  But it is my environment.... I'm not sure exactly how 
or why yet.  I've continued investigating since posting that... but 
thanks for the confirmation that it isn't 3.2.1 !

Tweaked the program to show the version and it showed 2.6.4!  Which is 
the only version of Python 2.x installed here.

So my suspicion is that   launchsys.adm64.msi, installed just after 
3.2.1 (but which appeared to do nothing) has taken over ... something.  
Not sure what yet.

The reason I thought it did nothing, is that I checked assoc  ( 
=Python.File ) and ftype ( =c:\python32\python.exe "%1" %* ) both of 
which look familiar, and neither of which mention   py.exe  which is 
what I think the launcher is supposed to have been named; and running  
c:\python32\python.exe from the command line produces a Python that 
looks like 3.2.1 .

But somehow, running    t32enc.py  from the command line fails, and 
loads 2.6.4 !

 From what I've read about the launcher, I thought it was supposed to 
take over the assoc and ftype and point Python.File to  
c:\windows\system32\py.exe  -- but that hasn't happened (and yes, I've 
rebooted since doing the installs, so it is not just a leftover CMD that 
didn't pick up new values).

So my question now is: how does the launcher really get activated when 
invoking .py files from the command line, if the assoc and ftype 
indicate that it should run c:\python32\python.exe (which when I run by 
hand, seems to be what the name claims it to be)?

We don't need to create mysteries!

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/f4ee2476/attachment.html>

From vinay_sajip at yahoo.co.uk  Mon Jul 18 23:13:27 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 18 Jul 2011 21:13:27 +0000 (UTC)
Subject: [Python-Dev] 3.2.1 encoding surprise
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
Message-ID: <loom.20110718T230703-173@post.gmane.org>

>     The reason I thought it did nothing, is that I checked assoc? (
>     =Python.File ) and ftype ( =c:\python32\python.exe "%1" %* ) both of
>     which look familiar, and neither of which mention?? py.exe? which is
>     what I think the launcher is supposed to have been named; and
>     running? c:\python32\python.exe from the command line produces a
>     Python that looks like 3.2.1 .
>     But somehow, running??? t32enc.py? from the command line fails, and
>     loads 2.6.4 !
>     From what I've read about the launcher, I thought it was supposed to
>     take over the assoc and ftype and point Python.File to?
>     c:\windows\system32\py.exe? -- but that hasn't happened (and yes,
>     I've rebooted since doing the installs, so it is not just a leftover
>     CMD that didn't pick up new values).
>     So my question now is: how does the launcher really get activated
>     when invoking .py files from the command line, if the assoc and
>     ftype indicate that it should run c:\python32\python.exe (which when
>     I run by hand, seems to be what the name claims it to be)?
>     We don't need to create mysteries!
>     Glenn

The launcher's activation is done by the shell you're using, which does the
lookups in the registry.

Remember that there are two sets of locations - HKCU and HKLM - where the type
associations are potentially held. Please do a registry search (with
Administrator rights so you can search the whole registry) for "py.exe" or
"pyw.exe" and see if they show up anywhere at all. The launcher code tries to
add these keys in HKEY_CLASSES_ROOT, but I believe Windows can map this to HKCU
rather than HKLM if you don't have administrator access, at least on XP.

Regards,

Vinay Sajip


From v+python at g.nevcal.com  Tue Jul 19 00:05:14 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 18 Jul 2011 15:05:14 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <loom.20110718T230703-173@post.gmane.org>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
Message-ID: <4E24AE1A.3000308@g.nevcal.com>

On 7/18/2011 2:13 PM, Vinay Sajip wrote:
> Remember that there are two sets of locations - HKCU and HKLM - where the type
> associations are potentially held. Please do a registry search (with
> Administrator rights so you can search the whole registry) for "py.exe" or
> "pyw.exe" and see if they show up anywhere at all. The launcher code tries to
> add these keys in HKEY_CLASSES_ROOT, but I believe Windows can map this to HKCU
> rather than HKLM if you don't have administrator access, at least on XP.

Oh, and I'm running Windows 7 64-bit, but most of my registry knowledge 
was learned on Win2K and XP, and I have no knowledge of what they did in 
Vista or 7, and haven't yet attempted to research such.

I am running as an administrator, but a much more ignorant one on 7 than 
I was on XP.

I have no idea why CMD would show an ftype Python.File as one thing, but 
then the execution would use something from the registry that is different.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/2dfa20f0/attachment.html>

From v+python at g.nevcal.com  Tue Jul 19 00:06:30 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 18 Jul 2011 15:06:30 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <loom.20110718T230703-173@post.gmane.org>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
Message-ID: <4E24AE66.5030305@g.nevcal.com>

On 7/18/2011 2:13 PM, Vinay Sajip wrote:
>>      The reason I thought it did nothing, is that I checked assoc  (
>>      =Python.File ) and ftype ( =c:\python32\python.exe "%1" %* ) both of
>>      which look familiar, and neither of which mention   py.exe  which is
>>      what I think the launcher is supposed to have been named; and
>>      running  c:\python32\python.exe from the command line produces a
>>      Python that looks like 3.2.1 .
>>      But somehow, running    t32enc.py  from the command line fails, and
>>      loads 2.6.4 !
>>      From what I've read about the launcher, I thought it was supposed to
>>      take over the assoc and ftype and point Python.File to 
>>      c:\windows\system32\py.exe  -- but that hasn't happened (and yes,
>>      I've rebooted since doing the installs, so it is not just a leftover
>>      CMD that didn't pick up new values).
>>      So my question now is: how does the launcher really get activated
>>      when invoking .py files from the command line, if the assoc and
>>      ftype indicate that it should run c:\python32\python.exe (which when
>>      I run by hand, seems to be what the name claims it to be)?
>>      We don't need to create mysteries!
>>      Glenn
> The launcher's activation is done by the shell you're using, which does the
> lookups in the registry.
>
> Remember that there are two sets of locations - HKCU and HKLM - where the type
> associations are potentially held. Please do a registry search (with
> Administrator rights so you can search the whole registry) for "py.exe" or
> "pyw.exe" and see if they show up anywhere at all. The launcher code tries to
> add these keys in HKEY_CLASSES_ROOT, but I believe Windows can map this to HKCU
> rather than HKLM if you don't have administrator access, at least on XP.

Thanks for the response, Vinay... you are the one that would know, now 
that I've figured out it is a launcher issue rather than a Python issue.

Here is a list of py.exe references in my registry:

HCR\Python.CompiledFile
HCR\Python.File
HCR\Python.NoConFile
HCU\Software\Classes\Python.CompiledFile
HCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.py\OpenWithListk
HLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-21.....\Componer
HLM\SOFTWARE\Wow6432Note\Microsoft\Windows\CurrentVersion\SharedDLLs
HKU\S-1-5-21-.....\Software\Classes\Python.CompiledFile (and similar 
corresponding to the above HCR ones)

My (perhaps dated) understanding of the CMD commands attrib and ftype is 
that they reflect the contents of HCR.  But here HCR\Python.File is 
different than the value displayed by the CMD ftype Python.File.  Why?  
What am I missing?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/8c084fa4/attachment-0001.html>

From vinay_sajip at yahoo.co.uk  Tue Jul 19 01:55:11 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 18 Jul 2011 23:55:11 +0000 (UTC)
Subject: [Python-Dev] 3.2.1 encoding surprise
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
Message-ID: <loom.20110719T014549-927@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:


>     Here is a list of py.exe references in my registry:
>     HCR\Python.CompiledFile
>     HCR\Python.File
>     HCR\Python.NoConFile
      [snip]


There shouldn't be so many references, so I suggest you may want to try the
following (after backing up your registry first):

1. Uninstall all instances of the launcher.
2. Remove all registry keys for .py, .pyc, .pyo, .pyw, Python.File,
Python.NoConFile, and Python.CompiledFile.
3. Re-install the launcher (64-bit) using "msiexec /i launchsys.amd64.msi
ALLUSERS=1" (or launcher.amd64.msi if you prefer - the only difference is the
installation directory).
4. Check that the removed associations have appeared under HKEY_LOCAL_MACHINE,
and that they all point to the launcher installed in the previous step. These
may also appear in HKEY_CLASSES_ROOT if it is aliased to the
HKEY_LOCAL_MACHINE\Software\Classes tree.

Regards,

Vinay Sajip


From v+python at g.nevcal.com  Tue Jul 19 03:04:59 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 18 Jul 2011 18:04:59 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <loom.20110719T014549-927@post.gmane.org>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
Message-ID: <4E24D83B.8040100@g.nevcal.com>

On 7/18/2011 4:55 PM, Vinay Sajip wrote:
> Glenn Linderman<v+python<at>  g.nevcal.com>  writes:
>
>
>>      Here is a list of py.exe references in my registry:
>>      HCR\Python.CompiledFile
>>      HCR\Python.File
>>      HCR\Python.NoConFile
>        [snip]
>
>
> There shouldn't be so many references, so I suggest you may want to try the
> following (after backing up your registry first):

So, then, there must be 2 problems... one in the launcher installer, and 
the other being whatever happened to my machine.

Can you list the ones there should be?

> 1. Uninstall all instances of the launcher.

Well, there is only one.  But I can certainly remove it, and see what 
happens.

> 2. Remove all registry keys for .py, .pyc, .pyo, .pyw, Python.File,
> Python.NoConFile, and Python.CompiledFile.
Yes, I can do this.

> 3. Re-install the launcher (64-bit) using "msiexec /i launchsys.amd64.msi
> ALLUSERS=1" (or launcher.amd64.msi if you prefer - the only difference is the
> installation directory).
So what I had done was just double click launchsys.msi.  Why should 
there be command line parameters, rather than question/answer dialogs?  
This is Windows after all.  What does the parameter do, that would be 
different than what double click did?
> 4. Check that the removed associations have appeared under HKEY_LOCAL_MACHINE,
> and that they all point to the launcher installed in the previous step. These
> may also appear in HKEY_CLASSES_ROOT if it is aliased to the
> HKEY_LOCAL_MACHINE\Software\Classes tree.

I also still don't understand why CMD reports a different command line 
via its assoc/ftype commands for the Python.File than actually gets used 
by CMD to launch the .py file.

Found the following at http://zetconsultants.com/blog/?p=36:
> Second issue (not really an issue, however something that makes our 
> situation more complicated) is the fact that HKCR is in fact registry 
> symlink and this registry key doesn?t really exists on system. It is 
> merge of two different keys ? HKLM\Software\Classes and 
> HKCU\Software\Classes. You probably already guessed (correctly) that 
> this is the way how you can overwrite some settings on per-user basis 
> ? HKCU key always wins.

which is useful, but still doesn't explain the variant results from 
ftype and launching behavior.}

They further say:
>
> You need to create file type using FTYPE first and then associate this 
> TFYPE with executable using ASSOC command. Even though this process is 
> not very simple, you can use it. There is one HUGE disadvantage though 
> and I still don?t understand how is it possible ? both FTYPE and ASSOC 
> works only on per-machine level! Therefore you cannot configure 
> per-user associations using these tools!
>
>     Currently, there is no way through the UI to change or edit the
>     user-specific file type associations stored in the
>     HKEY_CURRENT_USER\SOFTWARE\Classes registry key. If you want to do
>     this, you have to directly edit the registry or develop your own
>     UI to gain access to this information.
>
> Above you can see official Microsoft statement 
> <http://support.microsoft.com/kb/257592/en-us> about this issue.
>

So maybe the problem is that ftype and assoc report the HLM values, but 
CMD actually uses the HCU values.  This actually would explain what I am 
seeing, and this is the piece of information I didn't understand that 
caused my confusion.

So as long as everything is installed for "all users", then there is no 
confusion!  I see I have less than 3 dozen extensions installed in HCU, 
and over 800 in HLM, so clearly I choose "all users" most of the time.  
It would be painstaking to see if the 3 dozen were all consistent or 
not, without writing a program to help.

Clearly the launcher installer should offer the same choices as Python 
itself for installing for one or all users, rather than defaulting 
silently to one user.  Better, the default should be inherited from the 
Python installation... if they are all consistent, and if permissions 
allow the modification of HLM settings.

> Regards,
>
> Vinay Sajip

So I can fix my machine, now that I understand what went wrong (delete 
py.exe entries from HCU, and put them in HLM instead).

Then the other problem I have, is why py.exe launched py 2.6.4 instead 
of py 3.2.1 when 3.2.1 is newer, and I don't have a #! line.  That is 
probably the defined behavior of the launcher, to prefer 2.x if 3 isn't 
specified.  I'll have to reread the launcher PEP.  In my environment, I 
would much prefer to use 3.x if 2.x isn't specified... so hopefully when 
I reread the launcher PEP, I'll discover a way to do that.  (I recall 
one can put -3 on the command line, so I could do that in the py.exe 
ftype commands, but hopefully there is a way to tweak the py.ini to do 
that.)

Thanks for your response, and help in creating the launcher.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/6deafa19/attachment.html>

From vinay_sajip at yahoo.co.uk  Tue Jul 19 03:41:16 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 19 Jul 2011 01:41:16 +0000 (UTC)
Subject: [Python-Dev] 3.2.1 encoding surprise
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
Message-ID: <loom.20110719T031535-263@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:

>     So, then, there must be 2 problems... one in the launcher installer,
>     and the other being whatever happened to my machine.

I think so. I know that Windows sometimes adds file type associations under
HKCU which shadow the ones set up in HKLM, but I'm not sure why or how it
happens.

>     Well, there is only one.  But I can certainly remove it, and see
>     what happens.

Okay, just making sure.

>     So what I had done was just double click launchsys.msi.  Why should
>     there be command line parameters, rather than question/answer
>     dialogs?  This is Windows after all.  What does the parameter do,
>     that would be different than what double click did?

Windows is unfortunately not consistent in its behaviour across different
versions - if you Google for ALLUSERS you'll probably find the information
that makes some people tear their hair out. So the ALLUSERS=1 forces Windows
to install under HKLM ("for all users" as opposed to "just for me") and to
fail if you don't have admin rights. Note that the ALLUSERS= is something
introduced by Microsoft, not something launcher-specific.

>     I also still don't understand why CMD reports a different command
>     line via its assoc/ftype commands for the Python.File than actually
>     gets used by CMD to launch the .py file.

I believe it's to do with multiple versions in the registry.

>       something that makes our situation more complicated) is the fact
>       that HKCR is in fact registry symlink and this registry key
>       doesn?t really exists on system. It is merge of two different keys
>       ? HKLM\Software\Classes and HKCU\Software\Classes. You probably
>       already guessed (correctly) that this is the way how you can
>       overwrite some settings on per-user basis ? HKCU key always wins.
>     
>     which is useful, but still doesn't explain the variant results from
>     ftype and launching behavior.}
>     They further say:
>       You need to create file type using FTYPE first and then
>         associate this TFYPE with executable using ASSOC command. Even
>         though this process is not very simple, you can use it. There is
>         one HUGE disadvantage though and I still don?t understand how is
>         it possible ? both FTYPE and ASSOC works only on per-machine
>         level! Therefore you cannot configure per-user associations
>         using these tools!

I have certainly seen file type associations for .py in HKCU, whose presence
there I can't explain, but it's probably wrongly created by some Windows
software common in the ecosystem (if not Windows itself), which manifests in
odd ways - search deadlybloodyserious.com for problems where command line
arguments aren't passed to scripts because of bad ftype/assoc registry
entries. I would advise not using ftype/assoc commands, as you certainly
shouldn't need them for Python files if you install the launcher.

>         Currently, there is no way through the UI to change or edit
>           the user-specific file type associations stored in the
>           HKEY_CURRENT_USER\SOFTWARE\Classes registry key. If you want
>           to do this, you have to directly edit the registry or develop
>           your own UI to gain access to this information. 
>       
>       Above you can see official Microsoft
>           statement about this issue.

Yes, but some software in Windows (perhaps "select program from a list" or
similar) definitely does set values in HKCU as a side-effect of doing
something else, which comes back to bite you later. I have personally
experienced the same problem as the blogger on deadlybloodyserious.com (which
is how I came across the solution on that site - by Googling to see if anyone
else had encountered it).


>     Clearly the launcher installer should offer the same choices as
>     Python itself for installing for one or all users, rather than
>     defaulting silently to one user.  Better, the default should be
>     inherited from the Python installation... if they are all
>     consistent, and if permissions allow the modification of HLM
>     settings.

Yes, but the launcher installer is beta software, and it's not yet clear
exactly how the launcher will be packaged with Python 3.3. Clearly if
packaged as an automatic installation with Python, it will use Python
defaults.

Note that ALLUSERS was introduced precisely because Microsoft didn't think
things through initially with sufficient care, and there's no way to get
consistent behaviour across XP and Vista/Windows 7. It's possible to force
ALLUSERS=1 in the .msi, but this means that non-admin users can't try the
launcher. Unfortunately AFAICT, there's no obvious solution which "just
works", though if someone were to point out a solution it could certainly be
tried.

>     So I can fix my machine, now that I understand what went wrong
>     (delete py.exe entries from HCU, and put them in HLM instead).
>     Then the other problem I have, is why py.exe launched py 2.6.4
>     instead of py 3.2.1 when 3.2.1 is newer, and I don't have a #!
>     line.  That is probably the defined behavior of the launcher, to
>     prefer 2.x if 3 isn't specified.  I'll have to reread the launcher
>     PEP.  In my environment, I would much prefer to use 3.x if 2.x isn't
>     specified... so hopefully when I reread the launcher PEP, I'll
>     discover a way to do that.  (I recall one can put -3 on the command
>     line, so I could do that in the py.exe ftype commands, but hopefully
>     there is a way to tweak the py.ini to do that.)

Yes, the PEP explains what you need to do to get Python 3 to be the default:
use the py.ini file or use an environment variable.

The use of py from the command line is merely a convenience for developers (as
the PEP says) - it's better to rely on shebang lines together with settings in
the .ini to get the behaviour you want.

Regards,

Vinay Sajip



From v+python at g.nevcal.com  Tue Jul 19 04:03:43 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 18 Jul 2011 19:03:43 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <loom.20110719T031535-263@post.gmane.org>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
Message-ID: <4E24E5FF.1010805@g.nevcal.com>

On 7/18/2011 6:41 PM, Vinay Sajip wrote:
> Yes, but the launcher installer is beta software, and it's not yet clear
> exactly how the launcher will be packaged with Python 3.3. Clearly if
> packaged as an automatic installation with Python, it will use Python
> defaults.

Sure, and that's why I'm trying it out now, to provide feedback.  But 
because I couldn't tell from assoc and ftype commands that it had done 
anything, I was a bit surprised by the behaviors I did encounter when I 
encountered them.  So now I've learned about the limits of ftype and 
assoc, sadly.  And they only work for setting things in HLM if you "Run 
as administrator", by the way.

Wonder if there is a third party command line tool which augments them 
for reading/setting HCU Classes?  Will search.  I know there is a 
command line registry tool somewhere, but specifying the paths correctly 
would be onerous.

Would be nice if the default of Py 2.x or 3.x were an install question, 
perhaps.  I know that only affects scripts without #! lines, but it 
would be nice to limit the required creation of those to the smaller 
class of preexisting scripts on a particular installation.

> Note that ALLUSERS was introduced precisely because Microsoft didn't think
> things through initially with sufficient care, and there's no way to get
> consistent behaviour across XP and Vista/Windows 7. It's possible to force
> ALLUSERS=1 in the .msi, but this means that non-admin users can't try the
> launcher. Unfortunately AFAICT, there's no obvious solution which "just
> works", though if someone were to point out a solution it could certainly be
> tried.

Could be, with .msi and/or .cab installers.  I have no knowledge of 
them, except that when I first needed to use an installer to package 
things, I read a bit about them, and determined that they are too 
complex for my brain.  I found NSIS, still complex, but it is much 
friendlier to my brain, but it seems to handle this case correctly, 
giving the installer creator the option of asking for or demanding 
elevation, and exposing the answer to the rest of the logic of the 
installer, so it can do different things for those cases if it desires.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/b87b2062/attachment.html>

From v+python at g.nevcal.com  Tue Jul 19 05:27:15 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 18 Jul 2011 20:27:15 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E24E5FF.1010805@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E24E5FF.1010805@g.nevcal.com>
Message-ID: <4E24F993.6090406@g.nevcal.com>

On 7/18/2011 7:03 PM, Glenn Linderman wrote:
> Wonder if there is a third party command line tool which augments them 
> for reading/setting HCU Classes?  Will search.  I know there is a 
> command line registry tool somewhere, but specifying the paths 
> correctly would be onerous.

XP+ has REG and REGINI commands which can manipulate the registry from 
the command line, without resorting to third party applications.  So a 
batch file could wrap them, to avoid mistyping keys, and one could make 
an FTYPE.CMD and ASSOC.CMD (I haven't, at least not yet).

Also <http://vim.wikia.com/wiki/Windows_file_associations> although 
specific to Vim, is addressing the general problem, and comments by 
readers on that page seem relevant:

> The following is a "for reader's information" comment; not a 
> suggestion that the tip be changed. In classic Microsoft style, using 
> HKEY_CLASSES_ROOT is ambiguous. I do not know the details, but on at 
> least some systems (after Windows 9x), I think the following is correct:
>
>     HKEY_LOCAL_MACHINE\Software\Classes : for all users 
>     HKEY_CURRENT_USER\Software\Classes : for interactive user 
>     HKEY_CLASSES_ROOT : merged view of above (and is used by Win9x
>     apps <http://vim.wikia.com/wiki/Windows_file_associations#>) 
>
> I don't know what happens if you write to HKEY_CLASSES_ROOT (which 
> does not exist). Perhaps (like installing some apps), if you are in 
> the Administrators group, you will write to HKLM, otherwise you will 
> write to HKCU. JohnBeckett 
> <http://vim.wikia.com/wiki/User:JohnBeckett> 08:24, 2 July 2009 (UTC)
>
> ------------------------------------------------------------------------
>
> This is correct as far as I know. I wanted to include some information 
> about how HKCR is a merged view of HKLM/Software/Classes and 
> HKCU/Software/Classes, but doing so would beg information such as 
> "what happens when I edit an entry that actually exists in HKCU?" etc. 
> From experimentation, it seems that creating /new/ keys in this area 
> will always create it in HKLM (system-wide), and ftype and assoc 
> certainly do that, but I don't really have any documentation of that 
> fact, and I don't know what it will do for limited-privilege accounts. 
> I also don't know what happens when you edit an existing key, but I 
> imagine it will "do the right thing" and keep the original where it 
> was. I don't know this for a fact though. For these reasons, I left 
> out that tidbit. But if we can answer some of these questions, it 
> would be a good thing to include.
>

So attempts by applications to use the HKEY_CLASSES_ROOT directly may 
result in variant behavior depending on the privilege of the 
application.  My overall opinion? It is sad that M$ even invented the 
registry.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110718/d19d3eeb/attachment.html>

From p.f.moore at gmail.com  Tue Jul 19 17:00:57 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 19 Jul 2011 16:00:57 +0100
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
	encoding surprise)
Message-ID: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>

On 19 July 2011 02:41, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> The use of py from the command line is merely a convenience for developers (as
> the PEP says) - it's better to rely on shebang lines together with settings in
> the .ini to get the behaviour you want.

But it's a *huge* convenience for running multiple Python versions,
particularly as no existing Python versions install executables with
the version in the name (python3.exe, python3.2.exe, etc).And BAT
files aren't a suitable option (I'll rant about the issues with BAT
files if you want, but I recommend you don't ask :-))

Being able to say py -3, py -2.7, etc, rather than having to hack
PATH, create renamed copies of exes, etc, is arguably more of a
benefit to me than shebang support.

This may explain why I'd like to see a command-line means of invoking
custom commands. Something like py.exe looking at an initial argument,
and if it's of the form "-cmd" for a command in py.ini, then run that
command, passing remaining arguments just as for py -3. (Maybe --cmd
to match standard long option usage would be better?)

Presumably, if this idea is to go anywhere, it would need adding to
the PEP. Mark, do you think it would be useful? It makes it possible
to invoke IronPython/Jython in the same manner as CPython. (And, of
course, to abuse things so that "py -perl" runs Perl, but no-one would
do that :-))

Paul.

From solipsis at pitrou.net  Tue Jul 19 17:16:47 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 19 Jul 2011 17:16:47 +0200
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
Message-ID: <20110719171647.03f9f9c9@pitrou.net>

On Tue, 19 Jul 2011 16:00:57 +0100
Paul Moore <p.f.moore at gmail.com> wrote:

> On 19 July 2011 02:41, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> > The use of py from the command line is merely a convenience for developers (as
> > the PEP says) - it's better to rely on shebang lines together with settings in
> > the .ini to get the behaviour you want.
> 
> But it's a *huge* convenience for running multiple Python versions,
> particularly as no existing Python versions install executables with
> the version in the name (python3.exe, python3.2.exe, etc).

Perhaps this could be changed? As far as I can see, python.exe is
a small executable around ~25KB (all the code being in the DLL), so
there doesn't seem to be any harm to make a copy of it named either
pythonXY.exe or pythonX.Y.exe.

Regards

Antoine.



From p.f.moore at gmail.com  Tue Jul 19 18:21:30 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 19 Jul 2011 17:21:30 +0100
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <20110719171647.03f9f9c9@pitrou.net>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
Message-ID: <CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>

On 19 July 2011 16:16, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 19 Jul 2011 16:00:57 +0100
> Paul Moore <p.f.moore at gmail.com> wrote:
>
>> On 19 July 2011 02:41, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> > The use of py from the command line is merely a convenience for developers (as
>> > the PEP says) - it's better to rely on shebang lines together with settings in
>> > the .ini to get the behaviour you want.
>>
>> But it's a *huge* convenience for running multiple Python versions,
>> particularly as no existing Python versions install executables with
>> the version in the name (python3.exe, python3.2.exe, etc).
>
> Perhaps this could be changed? As far as I can see, python.exe is
> a small executable around ~25KB (all the code being in the DLL), so
> there doesn't seem to be any harm to make a copy of it named either
> pythonXY.exe or pythonX.Y.exe.

I'm sure it could (and in fact, I thought that this had been discussed
some time back and it may even be already happening in 3.3) but it
doesn't help for existing versions, where the py.exe launcher does. So
as a longer-term solution, supplying pythonXY.exe binaries may be
useful (depending on how PEP 397 progresses), but the benefits won't
be for quite some time. (And there's still the question of what gets
put on PATH by default even if version-specific binaries exist).

It's a topic worthy of discussion, but I suspect that in actual fact,
PEP 397 may offer a more complete solution to the various Windows
installation niggles.

Two questions:
1. What level of support is there for PEP 397? If it's unlikely to get
accepted, there's little point in basing a solution on it.
2. Would it be worth extending the goals of the PEP to make
simplifying command line usage an explicit goal? Or is it better to
keep PEP 397 focused on one thing and have a separate PEP covering
such further extensions to the PEP 397 launcher?

Paul.

From 98k.master at gmail.com  Tue Jul 19 20:16:46 2011
From: 98k.master at gmail.com (Timothy D. Kadich)
Date: Tue, 19 Jul 2011 11:16:46 -0700
Subject: [Python-Dev] knee.py import hook in 2.6
Message-ID: <CALog0vx59KZaMT_t2YDz2-5g62vfMKdzpUzyRwKGk_2S8ntpfg@mail.gmail.com>

Hi,

I'm trying to use the import hook in Python2.6, but I'm having a problem. It
doesn't work for numpy. My error is such:

> >>> import knee
> >>> import numpy
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "knee.py", line 16, in import_hook
>     q, tail = find_head_package(parent, name)
>   File "knee.py", line 52, in find_head_package
>     q = import_module(head, qname, parent)
>   File "knee.py", line 101, in import_module
>     m = imp.load_module(fqname, fp, pathname, stuff)
>   File "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/__init__.py",
> line 130, in <module>
>     import add_newdocs
>   File "knee.py", line 16, in import_hook
>     q, tail = find_head_package(parent, name)
>   File "knee.py", line 52, in find_head_package
>     q = import_module(head, qname, parent)
>   File "knee.py", line 101, in import_module
>     m = imp.load_module(fqname, fp, pathname, stuff)
>   File
> "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/add_newdocs.py", line
> 9, in <module>
>     from lib import add_newdoc
>   File "knee.py", line 16, in import_hook
>     q, tail = find_head_package(parent, name)
>   File "knee.py", line 52, in find_head_package
>     q = import_module(head, qname, parent)
>   File "knee.py", line 101, in import_module
>     m = imp.load_module(fqname, fp, pathname, stuff)
>   File
> "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/lib/__init__.py",
> line 4, in <module>
>     from type_check import *
>   File "knee.py", line 16, in import_hook
>     q, tail = find_head_package(parent, name)
>   File "knee.py", line 52, in find_head_package
>     q = import_module(head, qname, parent)
>   File "knee.py", line 101, in import_module
>     m = imp.load_module(fqname, fp, pathname, stuff)
>   File
> "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/lib/type_check.py",
> line 8, in <module>
>     import numpy.core.numeric as _nx
>   File "knee.py", line 17, in import_hook
>     m = load_tail(q, tail)
>   File "knee.py", line 68, in load_tail
>     m = import_module(head, mname, m)
>   File "knee.py", line 101, in import_module
>     m = imp.load_module(fqname, fp, pathname, stuff)
>   File
> "/usr/apps/python2.6/lib/python2.6/site-packages/numpy/core/__init__.py",
> line 6, in <module>
>     import umath
>   File "knee.py", line 16, in import_hook
>     q, tail = find_head_package(parent, name)
>   File "knee.py", line 52, in find_head_package
>     q = import_module(head, qname, parent)
>   File "knee.py", line 101, in import_module
>     m = imp.load_module(fqname, fp, pathname, stuff)
> TypeError: import_hook() takes at most 4 arguments (5 given)



So I don't know what is going on, unless a "self" is being passed along the
way. (which seems like it could happen when looking at __import__ in the
source)
Can any of you identify my problem or let me know of a fixed import hook?

Thank you,
Timothy D. Kadich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110719/b0f06d8c/attachment.html>

From phd at phdru.name  Tue Jul 19 21:08:54 2011
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 19 Jul 2011 23:08:54 +0400
Subject: [Python-Dev] knee.py import hook in 2.6
In-Reply-To: <CALog0vx59KZaMT_t2YDz2-5g62vfMKdzpUzyRwKGk_2S8ntpfg@mail.gmail.com>
References: <CALog0vx59KZaMT_t2YDz2-5g62vfMKdzpUzyRwKGk_2S8ntpfg@mail.gmail.com>
Message-ID: <20110719190854.GA5691@iskra.aviel.ru>

Hello.

   We are sorry but we cannot help you. This mailing list is to work on
developing Python (adding new features to Python itself and fixing bugs);
if you're having problems learning, understanding or using Python, please
find another forum. Probably python-list/comp.lang.python mailing list/news
group is the best place; there are Python developers who participate in it;
you may get a faster, and probably more complete, answer there. See
http://www.python.org/community/ for other lists/news groups/fora. Thank
you for understanding.

On Tue, Jul 19, 2011 at 11:16:46AM -0700, Timothy D. Kadich wrote:
> I'm trying to use the import hook in Python2.6, but I'm having a problem. It
> doesn't work for numpy. My error is such:
[skip]
> > TypeError: import_hook() takes at most 4 arguments (5 given)

   Seems like import_hook is from an older version of Python.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From solipsis at pitrou.net  Tue Jul 19 22:59:54 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 19 Jul 2011 22:59:54 +0200
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
Message-ID: <20110719225954.0def2f75@pitrou.net>

On Tue, 19 Jul 2011 17:21:30 +0100
Paul Moore <p.f.moore at gmail.com> wrote:
> 
> Two questions:
> 1. What level of support is there for PEP 397? If it's unlikely to get
> accepted, there's little point in basing a solution on it.

It only needs support from our Windows users or developers.
It is doubtful than any Linux or OS X user would oppose it on purely
platonic grounds. I myself, as a casual Windows user, understand that
the current situation is not optimal and believe that any improvement is
welcome.

Practically, if Mark Hammond is satisfied with his own proposal, if
Martin doesn't oppose it, and if other users like you say it's a good
step forward, then I don't see any reason for it *not* to be accepted.

(if you want an explicit +1, here it is :-))

> 2. Would it be worth extending the goals of the PEP to make
> simplifying command line usage an explicit goal? Or is it better to
> keep PEP 397 focused on one thing and have a separate PEP covering
> such further extensions to the PEP 397 launcher?

I have no opinion about that.

Regards

Antoine.

From rdmurray at bitdance.com  Tue Jul 19 23:21:39 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 19 Jul 2011 17:21:39 -0400
Subject: [Python-Dev] email-6.0.0.a1
Message-ID: <20110719212139.D5D732500D5@webabinitio.net>

OK, so I've released the first iteration of the email6 package on pypi
as email-6.0.0a1.  After install you import it as email6.  This will
allow anyone curious and/or motivated to test it out under Python 3.2.
I'm especially interested in anyone with a working program that uses
email in 3.2: it should be completely backward compatible, and if it
isn't I want to know ASAP.[*]

I've also opened issue 12586 for review of the delta between default
and the code that is in the release.  I'd like to check the code in
to default and continue to work on it from there.  As I said in the
issue comments: "When we originally planned out email6 we thought we'd
be making a "compatibility break" with backward compatibility shims.
As things have turned out the work is more a matter of incremental
improvement of the API while maintaining the old API, and thus it seems
reasonable to me to work on it directly in default rather than continue
to work on it in a separate feature branch."  Assuming, that is, that
the general approach represented by *this* delta is accepted.

What this delta adds to email is a conversion to handling all headers as
full blown objects (as opposed to strings, tuples of strings, or Header
objects, depending on context).  The object type is a subclass of str,
so the headers act like strings if you don't use their additional API.
The basic additional API is that a 'source' attribute contains the
text the generator read from the input source, and a 'value' attribute
that contains the value with all the Content-Transfer-Encoding stuff
undone so that you have a real unicode string.  By changing a policy
setting, you can have that value as the string value of the header.
You can also assign a string with non-ASCII characters to a header, and
the right thing will happen.  (Well, eventually it will happen...right
now it only works correctly for unstructured headers).  Further, Date
headers have a datetime attribute (and accept being set to a datetime),
and address headers have attributes for accessing the individual addresses
in the header.  Other structured headers will eventually grow additional
attributes as well.

The general approach has been discussed with and approved by the email-sig,
but all comments are welcome.  I know there's room for bikeshedding
on some aspects of the API; in some cases I've dome some "placeholder"
stuff pending a more complete solution to certain design goals.

I have a big project in the offing over the next couple months.  QNX is
still fully behind the funding for email6 development, but I probably
won't be able to complete it until the fall.  So I'd like to get this
chunk (the biggest chunk of new code, considering the size of the parser)
reviewed and checked in if possible.  I'll keep working on the bits of
functionality that aren't quite complete and the bugs that I know are
there until my big project kicks off, but I wanted to release/post now
so that there might be a chance of some review happening while I still
have time to respond quickly to the feedback.

--
R. David Murray           http://www.bitdance.com

[*] I believe that if you try to use an email6 Message object with the 3.2
mailbox module you will run in to some trouble, but I think it ought to
be possible to make it work with the right magic :)

PS: I don't have much experience writing parsers, so I'm expecting some
critical comments about my parser design.  It had to be a custom parser
since otherwise I'd be blocked on waiting for some other software to
get accepted into the stdlib, but it certainly wound up being a bigger
chunk of code than I expected when I started writing it.

From ncoghlan at gmail.com  Wed Jul 20 01:08:20 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jul 2011 09:08:20 +1000
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <20110719225954.0def2f75@pitrou.net>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<20110719225954.0def2f75@pitrou.net>
Message-ID: <CADiSq7f=XZQEgu1a8+oBJ4sAJUkdQM=y-da858H4MG8tvQ6weA@mail.gmail.com>

On Wed, Jul 20, 2011 at 6:59 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> (if you want an explicit +1, here it is :-))

FWIW, +1 from me as well, but keep in mind that I actively avoid
programming on Windows (although I'm happy enough using it as a gaming
platform)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From skippy.hammond at gmail.com  Wed Jul 20 02:15:31 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 20 Jul 2011 10:15:31 +1000
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
Message-ID: <4E261E23.3020005@gmail.com>

On 20/07/2011 1:00 AM, Paul Moore wrote:
> On 19 July 2011 02:41, Vinay Sajip<vinay_sajip at yahoo.co.uk>  wrote:
>> The use of py from the command line is merely a convenience for developers (as
>> the PEP says) - it's better to rely on shebang lines together with settings in
>> the .ini to get the behaviour you want.
>
> But it's a *huge* convenience for running multiple Python versions,
> particularly as no existing Python versions install executables with
> the version in the name (python3.exe, python3.2.exe, etc).And BAT
> files aren't a suitable option (I'll rant about the issues with BAT
> files if you want, but I recommend you don't ask :-))
>
> Being able to say py -3, py -2.7, etc, rather than having to hack
> PATH, create renamed copies of exes, etc, is arguably more of a
> benefit to me than shebang support.

Ditto for me.

> This may explain why I'd like to see a command-line means of invoking
> custom commands. Something like py.exe looking at an initial argument,
> and if it's of the form "-cmd" for a command in py.ini, then run that
> command, passing remaining arguments just as for py -3. (Maybe --cmd
> to match standard long option usage would be better?)
>
> Presumably, if this idea is to go anywhere, it would need adding to
> the PEP. Mark, do you think it would be useful?

I doubt I will find it useful - but I'm on record as saying I wont find 
the custom command support itself useful :)  But similarly with that 
support, evidence that enough people *will* find it useful is enough for 
me to support the concept.

My current thinking re the PEP is to make it much smaller - just 
describe the concepts and try to avoid as much implementation detail as 
possible - I see no reason the PEP needs to take a stance on issues like 
that - this feature really could be treated just like any other feature 
request in Python - a loose consensus and acceptable patch is all that 
is needed.

Mark

From tjreedy at udel.edu  Wed Jul 20 04:21:31 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 19 Jul 2011 22:21:31 -0400
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
	encoding surprise)
In-Reply-To: <CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
Message-ID: <j05e3c$g9c$1@dough.gmane.org>

On 7/19/2011 12:21 PM, Paul Moore wrote:
> On 19 July 2011 16:16, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>> On Tue, 19 Jul 2011 16:00:57 +0100

>> Perhaps this could be changed? As far as I can see, python.exe is
>> a small executable around ~25KB (all the code being in the DLL), so
>> there doesn't seem to be any harm to make a copy of it named either
>> pythonXY.exe or pythonX.Y.exe.
>
> I'm sure it could (and in fact, I thought that this had been discussed
> some time back and it may even be already happening in 3.3) but it
> doesn't help for existing versions, where the py.exe launcher does. So
> as a longer-term solution, supplying pythonXY.exe binaries may be
> useful (depending on how PEP 397 progresses), but the benefits won't
> be for quite some time. (And there's still the question of what gets
> put on PATH by default even if version-specific binaries exist).

Suppose for Windows there were one '.../python' directory wherever the 
user first asks it to be put and that all pythons, not just cpython, are 
installed in directories below that and that the small startup file is 
copied into or linked from the python directory. Then the one python 
directory could be put on the path and left there and never removed by 
any python de-installer (unless perhaps it check that there are no 
subdirs and *asks* the user.

-- 
Terry Jan Reedy


From pje at telecommunity.com  Wed Jul 20 05:58:55 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 19 Jul 2011 23:58:55 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"
Message-ID: <20110720040505.400E23A4116@sparrow.telecommunity.com>

So, over on the Import-SIG, we were talking about the implementation 
and terminology for PEP 382, and it became increasingly obvious that 
things were, well, not entirely okay in the "implementation is easy 
to explain" department.

Anyway, to make a long story short, we came up with an alternative 
implementation plan that actually solves some other problems besides 
the one that PEP 382 sets out to solve, and whose implementation a 
bit is easier to explain.  (In fact, for users coming from various 
other languages, it hardly needs any explanation at all.)

However, for long-time users of Python, the approach may require a 
bit more justification, which is why roughly 2/3rds of the PEP 
consists of a detailed rationale, specification overview, rejected 
alternatives, and backwards-compatibility discussion...  which is 
still a lot less verbiage than reading through the lengthy Import-SIG 
threads that led up to the proposal.  ;-)  (The remaining 1/3rd of 
the PEP is the short, sweet, and easy-to-explain implementation detail.)

Anyway, the PEP has already been discussed on the Import-SIG, and is 
proposed as an alternative to PEP 382 ("Namespace packages").  We 
expect, however, that many people will be interested in it for 
reasons having little to do with the namespace packaging use case.

So, we would like to submit this for discussion, hole-finding, and 
eventual Pronouncement.  As Barry put it, "I think it's certainly 
worthy of posting to python-dev to see if anybody else can shoot 
holes in it, or come up with useful solutions to open 
questions.  I'll be very interested to see Guido's reaction to it. :)"

So, without further ado, here it is:

PEP: XXX
Title: Simplified Package Layout and Partitioning
Version: $Revision$
Last-Modified: $Date$
Author: P.J. Eby
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Jul-2011
Python-Version: 3.3
Post-History:
Replaces: 382

Abstract
========

This PEP proposes an enhancement to Python's package importing
to:

* Surprise users of other languages less,
* Make it easier to convert a module into a package, and
* Support dividing packages into separately installed components
   (ala "namespace packages", as described in PEP 382)

The proposed enhancements do not change the semantics of any
currently-importable directory layouts, but make it possible for
packages to use a simplified directory layout (that is not importable
currently).

However, the proposed changes do NOT add any performance overhead to
the importing of existing modules or packages, and performance for the
new directory layout should be about the same as that of previous
"namespace package" solutions (such as ``pkgutil.extend_path()``).


The Problem
===========

.. epigraph::

     "Most packages are like modules.  Their contents are highly
     interdependent and can't be pulled apart.  [However,] some
     packages exist to provide a separate namespace. ...  It should
     be possible to distribute sub-packages or submodules of these
     [namespace packages] independently."

     -- Jim Fulton, shortly before the release of Python 2.3 [1]_


When new users come to Python from other languages, they are often
confused by Python's packaging semantics.  At Google, for example,
Guido received complaints from "a large crowd with pitchforks" [2]_
that the requirement for packages to contain an ``__init__`` module
was a "misfeature", and should be dropped.

In addition, users coming from languages like Java or Perl are
sometimes confused by a difference in Python's import path searching.

In most other languages that have a similar path mechanism to Python's
``sys.path``, a package is merely a namespace that contains modules
or classes, and can thus be spread across multiple directories in
the language's path.  In Perl, for instance, a ``Foo::Bar`` module
will be searched for in ``Foo/`` subdirectories all along the module
include path, not just in the first such subdirectory found.

Worse, this is not just a problem for new users: it prevents *anyone*
from easily splitting a package into separately-installable
components.  In Perl terms, it would be as if every possible ``Net::``
module on CPAN had to be bundled up and shipped in a single tarball!

For that reason, various workarounds for this latter limitation exist,
circulated under the term "namespace packages".  The Python standard
library has provided one such workaround since Python 2.3 (via the
``pkgutil.extend_path()`` function), and the "setuptools" package
provides another (via ``pkg_resources.declare_namespace()``).

The workarounds themselves, however, fall prey to a *third* issue with
Python's way of laying out packages in the filesystem.

Because a package *must* contain an ``__init__`` module, any attempt
to distribute modules for that package must necessarily include that
``__init__`` module, if those modules are to be importable.

However, the very fact that each distribution of modules for a package
must contain this (duplicated) ``__init__`` module, means that OS
vendors who package up these module distributions must somehow handle
the conflict caused by several module distributions installing that
``__init__`` module to the same location in the filesystem.

This led to the proposing of PEP 382 ("Namespace Packages") - a way
to signal to Python's import machinery that a directory was
importable, using unique filenames per module distribution.

However, there was more than one downside to this approach.
Performance for all import operations would be affected, and the
process of designating a package became even more complex.  New
terminology had to be invented to explain the solution, and so on.

As terminology discussions continued on the Import-SIG, it soon became
apparent that the main reason it was so difficult to explain the
concepts related to "namespace packages" was because Python's
current way of handling packages is somewhat underpowered, when
compared to other languages.

That is, in other popular languages with package systems, no special
term is needed to describe "namespace packages", because *all*
packages generally behave in the desired fashion.

Rather than being an isolated single directory with a special marker
module (as in Python), packages in other languages are typically just
the *union* of appropriately-named directories across the *entire*
import or inclusion path.

In Perl, for example, the module ``Foo`` is always found in a
``Foo.pm`` file, and a module ``Foo::Bar`` is always found in a
``Foo/Bar.pm`` file.  (In other words, there is One Obvious Way to
find the location of a particular module.)

This is because Perl considers a module to be *different* from a
package: the package is purely a *namespace* in which other modules
may reside, and is only *coincidentally* the name of a module as well.

In current versions of Python, however, the module and the package are
more tightly bound together.  ``Foo`` is always a module -- whether it
is found in ``Foo.py`` or ``Foo/__init__.py`` -- and it is tightly
linked to its submodules (if any), which *must* reside in the exact
same directory where the ``__init__.py`` was found.

On the positive side, this design choice means that a package is quite
self-contained, and can be installed, copied, etc. as a unit just by
performing an operation on the package's root directory.

On the negative side, however, it is non-intuitive for beginners, and
requires a more complex step to turn a module into a package.  If
``Foo`` begins its life as ``Foo.py``, then it must be moved and
renamed to ``Foo/__init__.py``.

Conversely, if you intend to create a ``Foo.Bar`` module from the
start, but have no particular module contents to put in ``Foo``
itself, then you have to create an empty and seemingly-irrelevant
``Foo/__init__.py`` file, just so that ``Foo.Bar`` can be imported.

(And these issues don't just confuse newcomers to the language,
either: they annoy many experienced developers as well.)

So, after some discussion on the Import-SIG, this PEP was created
as an alternative to PEP \382, in an attempt to solve *all* of the
above problems, not just the "namespace package" use cases.

And, as a delightful side effect, the solution proposed in this PEP
does not affect the import performance of ordinary modules or
self-contained (i.e. ``__init__``-based) packages.


The Solution
============

In the past, various proposals have been made to allow more intuitive
approaches to package directory layout.  However, most of them failed
because of an apparent backward-compatibility problem.

That is, if the requirement for an ``__init__`` module were simply
dropped, it would open up the possibility for a directory named, say,
``string`` on ``sys.path``, to block importing of the standard library
``string`` module.

Paradoxically, however, the failure of this approach does *not* arise
from the elimination of the ``__init__`` requirement!

Rather, the failure arises because the underlying approach takes for
granted that a package is just ONE thing, instead of two.

In truth, a package comprises two separate, but related entities: a
module (with its own, optional contents), and a *namespace* where
*other* modules or packages can be found.

In current versions of Python, however, the module part (found in
``__init__``) and the namespace for submodule imports (represented
by the ``__path__`` attribute) are both initialized at the same time,
when the package is first imported.

And, if you assume this is the *only* way to initialize these two
things, then there is no way to drop the need for an ``__init__``
module, while still being backwards-compatible with existing directory
layouts.

After all, as soon as you encounter a directory on ``sys.path``
matching the desired name, that means you've "found" the package, and
must stop searching, right?

Well, not quite.


A Thought Experiment
--------------------

Let's hop into the time machine for a moment, and pretend we're back
in the early 1990s, shortly before Python packages and ``__init__.py``
have been invented.  But, imagine that we *are* familiar with
Perl-like package imports, and we want to implement a similar system
in Python.

We'd still have Python's *module* imports to build on, so we could
certainly conceive of having ``Foo.py`` as a parent ``Foo`` module
for a ``Foo`` package.  But how would we implement submodule and
subpackage imports?

Well, if we didn't have the idea of ``__path__`` attributes yet,
we'd probably just search ``sys.path`` looking for ``Foo/Bar.py``.

But we'd *only* do it when someone actually tried to *import*
``Foo.Bar``.

NOT when they imported ``Foo``.

And *that* lets us get rid of the backwards-compatibility problem
of dropping the ``__init__`` requirement, back here in 2011.

How?

Well, when we ``import Foo``, we're not even *looking* for ``Foo/``
directories on ``sys.path``, because we don't *care* yet.  The only
point at which we care, is the point when somebody tries to actually
import a submodule or subpackage of ``Foo``.

That means that if ``Foo`` is a standard library module (for example),
and I happen to have a ``Foo`` directory on ``sys.path`` (without
an ``__init__.py``, of course), then *nothing breaks*.  The ``Foo``
module is still just a module, and it's still imported normally.


Self-Contained vs. "Virtual" Packages
-------------------------------------

Of course, in today's Python, trying to ``import Foo.Bar`` will
fail if ``Foo`` is just a ``Foo.py`` module (and thus lacks a
``__path__`` attribute).

So, this PEP proposes to *dynamically* create a ``__path__``, in the
case where one is missing.

That is, if I try to ``import Foo.Bar`` the proposed change to the
import machinery will notice that the ``Foo`` module lacks a
``__path__``, and will therefore try to *build* one before proceeding.

And it will do this by making a list of all the existing ``Foo/``
subdirectories of the directories listed in ``sys.path``.

If the list is empty, the import will fail with ``ImportError``, just
like today.  But if the list is *not* empty, then it is saved in
a new ``Foo.__path__`` attribute, making the module a "virtual
package".

That is, because it now has a valid ``__path__``, we can proceed
to import submodules or subpackages in the normal way.

Now, notice that this change does not affect "classic", self-contained
packages that have an ``__init__`` module in them.  Such packages
already *have* a ``__path__`` attribute (initialized at import time)
so the import machinery won't try to create another one later.

This means that (for example) the standard library ``email`` package
will not be affected in any way by you having a bunch of unrelated
directories named ``email`` on ``sys.path``.  (Even if they contain
``*.py`` files.)

But it *does* mean that if you want to turn your ``Foo`` module into
a ``Foo`` package, all you have to do is add a ``Foo/`` directory
somewhere on ``sys.path``, and start adding modules to it.

But what if you only want a "namespace package"?  That is, a package
that is *only* a namespace for various separately-distributed
submodules and subpackages?

For example, if you're Zope Corporation, distributing dozens of
separate tools like ``zc.buildout``, each in packages under the ``zc``
namespace, you don't want to have to make and include an empty
``zc.py`` in every tool you ship.  (And, if you're a Linux or other
OS vendor, you don't want to deal with the package installation
conflicts created by trying to install ten copies of ``zc.py`` to the
same location!)

No problem.  All we have to do is make one more minor tweak to the
import process: if the "classic" import process fails to find a
self-contained module or package (e.g., if ``import zc`` fails to find
a ``zc.py`` or ``zc/__init__.py``), then we once more try to build a
``__path__`` by searching for all the ``zc/`` directories on
``sys.path``, and putting them in a list.

If this list is empty, we raise ``ImportError``.  But if it's
non-empty, we create an empty ``zc`` module, and put the list in
``zc.__path__``.  Congratulations: ``zc`` is now a namespace-only,
"pure virtual" package!  It has no module contents, but you can still
import submodules and subpackages from it, regardless of where they're
located on ``sys.path``.

(By the way, both of these additions to the import protocol (i.e. the
dynamically-added ``__path__``, and dynamically-created modules)
apply recursively to child packages, using the parent package's
``__path__`` in place of ``sys.path`` as a basis for generating a
child ``__path__``.  This means that self-contained and virtual
packages can contain each other without limitation, with the caveat
that if you put a virtual package inside a self-contained one, it's
gonna have a really short ``__path__``!)


Backwards Compatibility and Performance
---------------------------------------

Notice that these two changes *only* affect import operations that
today would result in ``ImportError``.  As a result, the performance
of imports that do not involve virtual packages is unaffected, and
potential backward compatibility issues are very restricted.

Today, if you try to import submodules or subpackages from a module
with no ``__path__``, it's an immediate error.  And of course, if you
don't have a ``zc.py`` or ``zc/__init__.py`` somewhere on ``sys.path``
today, ``import zc`` would likewise fail.

Thus, the only potential backwards-compatibility issues are:

1. Tools that expect package directories to have an ``__init__``
    module, that expect directories without an ``__init__`` module
    to be unimportable, or that expect ``__path__`` attributes to be
    static, will not recognize virtual packages as packages.

    (In practice, this just means that tools will need updating to
    support virtual packages, e.g. by using ``pkgutil.walk_modules()``
    instead of using hardcoded filesystem searches.)

2. Code that *expects* certain imports to fail may now do something
    unexpected.  This should be fairly rare in practice, as most sane,
    non-test code does not import things that are expected not to
    exist!

The biggest likely exception to the above would be when a piece of
code tries to check whether some package is installed by importing
it.  If this is done *only* by importing a top-level module (i.e., not
checking for a ``__version__`` or some other attribute), *and* there
is a directory of the same name as the sought-for package on
``sys.path`` somewhere, *and* the package is not actually installed,
then such code could *perhaps* be fooled into thinking a package is
installed that really isn't.

However, even in the rare case where all these conditions line up to
happen at once, the failure is more likely to be annoying than
damaging.  In most cases, after all, the code will simply fail a
little later on, when it actually tries to DO something with the
imported (but empty) module.  (And code that checks ``__version__``
attributes or for the presence of some desired function, class, or
module in the package will not see a false positive result in the
first place.)

Meanwhile, tools that expect to locate packages and modules by
walking a directory tree can be updated to use the existing
``pkgutil.walk_modules()`` API, and tools that need to inspect
packages in memory should use the other APIs described in the
`Standard Library Changes/Additions`_ section below.


Specification
=============

Two changes are made to the existing import process.

First, the built-in ``__import__`` function must not raise an
``ImportError`` when importing a submodule of a module with no
``__path__``.  Instead, it must attempt to *create* a ``__path__``
attribute for the parent module first, as described in `__path__
creation`_, below.

Second, if searching ``sys.meta_path`` and ``sys.path`` (or a parent
package ``__path__``) fails to find a module being imported, the
import process must attempt to create a ``__path__`` attribute for
the missing module.  If the attempt succeeds, an empty module is
created and its ``__path__`` is set.  Otherwise, importing fails.

In both of the above cases, if a non-empty ``__path__`` is created,
the name of the module whose ``__path__`` was created is added to
``sys.virtual_packages`` -- an initially-empty ``set()`` of package
names.

(This way, code that extends ``sys.path`` at runtime can find out
what virtual packages are currently imported, and thereby add any
new subdirectories to those packages' ``__path__`` attributes.  See
`Standard Library Changes/Additions`_ below for more details.)

Conversely, if an empty ``__path__`` results, an ``ImportError``
is immediately raised, and the module is not created or changed, nor
is its name added to ``sys.virtual_packages``.


``__path__`` Creation
---------------------

A virtual ``__path__`` is created by obtaining a PEP 302 "importer"
object for each of the path entries found in ``sys.path`` (for a
top-level module) or the parent ``__path__`` (for a submodule).

(Note: because ``sys.meta_path`` importers are not associated with
``sys.path`` or ``__path__`` entry strings, such importers do *not*
participate in this process.)

Each importer is checked for a ``get_subpath()`` method, and if
present, the method is called with the full name of the module/package
the ``__path__`` is being constructed for.  The return value is either
a string representing a subdirectory for the requested package, or
``None`` if no such subdirectory exists.

The strings returned by the importers are added to the ``__path__``
being built, in the same order as they are found.  (``None`` values
and missing ``get_subpath()`` methods are simply skipped.)

In Python code, the algorithm would look something like this::

     def get_virtual_path(modulename, parent_path=None):

         if parent_path is None:
             parent_path = sys.path

         path = []

         for entry in parent_path:
             # Obtain a PEP 302 importer object - see pkgutil module
             importer = pkgutil.get_importer(entry)

             if hasattr(importer, 'get_subpath'):
                 subpath = importer.get_subpath(modulename)
                 if subpath is not None:
                     path.append(subpath)

         return path

And a function like this one should be exposed in the standard
library as e.g. ``imp.get_virtual_path()``, so that people creating
``__import__`` replacements or ``sys.meta_path`` hooks can reuse it.


Standard Library Changes/Additions
----------------------------------

The ``pkgutil`` module should be updated to handle this
specification appropriately, including any necessary changes to
``extend_path()``, ``iter_modules()``, etc.

Specifically the proposed changes and additions to ``pkgutil`` are:

* A new ``extend_virtual_paths(path_entry)`` function, to extend
   existing, already-imported virtual packages' ``__path__`` attributes
   to include any portions found in a new ``sys.path`` entry.  This
   function should be called by applications extending ``sys.path``
   at runtime, e.g. when adding a plugin directory or an egg to the
   path.

   The implementation of this function does a simple top-down traversal
   of ``sys.virtual_packages``, and performs any necessary
   ``get_subpath()`` calls to identify what path entries need to
   be added to each package's ``__path__``, given that `path_entry`
   has been added to ``sys.path``.  (Or, in the case of sub-packages,
   adding a derived subpath entry, based on their parent namespace's
   ``__path__``.)

* A new ``iter_virtual_packages(parent='')`` function to allow
   top-down traversal of virtual packages in ``sys.virtual_packages``,
   by yielding the child virtual packages of `parent`.  For example,
   calling ``iter_virtual_packages("zope")`` might yield ``zope.app``
   and ``zope.products`` (if they are imported virtual packages listed
   in ``sys.virtual_packages``), but **not** ``zope.foo.bar``.
   (This function is needed to implement ``extend_virtual_paths()``,
   but is also potentially useful for other code that needs to inspect
   imported virtual packages.)

* ``ImpImporter.iter_modules()`` should be changed to also detect and
   yield the names of modules found in virtual packages.

In addition to the above changes, the ``zipimport`` importer should
have its ``iter_modules()`` implementation similarly changed.  (Note:
current versions of Python implement this via a shim in ``pkgutil``,
so technically this is also a change to ``pkgutil``.)

Last, but not least, the ``imp`` module (or ``importlib``, if
appropriate) should expose the algorithm described in the `__path__
creation`_ section above, as a
``get_virtual_path(modulename, parent_path=None)`` function, so that
creators of ``__import__`` replacements can use it.


Implementation Notes
--------------------

For users, developers, and distributors of virtual packages:

* While virtual packages are easy to set up and use, there is still
   a time and place for using self-contained packages.  While it's not
   strictly necessary, adding an ``__init__`` module to your
   self-contained packages lets users of the package (and Python
   itself) know that *all* of the package's code will be found in
   that single subdirectory.  In addition, it lets you define
   ``__all__``, expose a public API, provide a package-level docstring,
   and do other things that make more sense for a self-contained
   project than for a mere "namespace" package.

* ``sys.virtual_packages`` is allowed to contain non-existent or
   not-yet-imported package names; code that uses its contents should
   not assume that every name in this set is also present in
   ``sys.modules`` or that importing the name will necessarily succeed.

* If you are changing a currently self-contained package into a
   virtual one, it's important to note that you can no longer use its
   ``__file__`` attribute to locate data files stored in a package
   directory.  Instead, you must search ``__path__`` or use the
   ``__file__`` of a submodule adjacent to the desired files, or
   of a self-contained subpackage that contains the desired files.

   (Note: this caveat is already true for existing users of "namespace
   packages" today.  That is, it is an inherent result of being able
   to partition a package, that you must know *which* partition the
   desired data file lives in.  We mention it here simply so that
   *new* users converting from self-contained to virtual packages will
   also be aware of it.)

* XXX what is the __file__ of a "pure virtual" package?  ``None``?
   Some arbitrary string?  The path of the first directory with a
   trailing separator?  No matter what we put, *some* code is
   going to break, but the last choice might allow some code to
   accidentally work.  Is that good or bad?


For those implementing PEP \302 importer objects:

* Importers that support the ``iter_modules()`` method (used by
   ``pkgutil`` to locate importable modules and packages) and want to
   add virtual package support should modify their ``iter_modules()``
   method so that it discovers and lists virtual packages as well as
   standard modules and packages.  To do this, the importer should
   simply list all immediate subdirectory names in its jurisdiction
   that are valid Python identifiers.

   XXX This might list a lot of not-really-packages.  Should we
   require importable contents to exist?  If so, how deep do we
   search, and how do we prevent e.g. link loops, or traversing onto
   different filesystems, etc.?  Ick.

* "Meta" importers (i.e., importers placed on ``sys.meta_path``) do
   not need to implement ``get_subpath()``, because the method
   is only called on importers corresponding to ``sys.path`` entries
   and ``__path__`` entries.  If a meta importer wishes to support
   virtual packages, it must do so entirely within its own
   ``find_module()`` implementation.

   Unfortunately, it is unlikely that any such implementation will be
   able to merge its package subpaths with those of other meta
   importers or ``sys.path`` importers, so the meaning of "supporting
   virtual packages" for a meta importer is currently undefined!

   (However, since the intended use case for meta importers is to
   replace Python's normal import process entirely for some subset of
   modules, and the number of such importers currently implemented is
   quite small, this seems unlikely to be a big issue in practice.)


References
==========

.. [1] "namespace" vs "module" packages (mailing list thread)
    (http://mail.zope.org/pipermail/zope3-dev/2002-December/004251.html)

.. [2] "Dropping __init__.py requirement for subpackages"
    (http://mail.python.org/pipermail/python-dev/2006-April/064400.html)


Copyright
=========

This document has been placed in the public domain.


..
    Local Variables:
    mode: indented-text
    indent-tabs-mode: nil
    sentence-end-double-space: t
    fill-column: 70
    coding: utf-8
    End:


From mail at kerrickstaley.com  Wed Jul 20 08:53:09 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Wed, 20 Jul 2011 01:53:09 -0500
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <nad-62B81D.01034918072011@dough.gmane.org>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
Message-ID: <CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>

On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily <nad at acm.org> wrote:
> I think adding the requirement to mandate hard link vs soft link usage
> is an unnecessary and unwarranted attempt at optimization.  For
> instance, IIRC, the OS X installers don't use any hard links: that may
> complicate the install, plus hard links on OS X HFS* file systems are a
> bit of a kludge and not necessarily more efficient than symlinks.   It's
> not a big deal but perhaps the wording should be changed to make a
> suggestion about hard links vs syminks rather than mandate which should
> be used.

Ah, OK. The wording's been changed so that symbolic links will be
installed on Mac OS X and hard links elsewhere (although maybe
symbolic links are also better on certain other platforms; I'm not
sure).

I do think that specific instructions must be given (rather than just
a suggestion) because it's indicating what must be done to CPython.
The instructions *should* be as close as possible to what the
installer already does, but I'm not entirely sure what the installer
does by default, and the hard-link recommendation was based off a
cursory inspection of my own system, so further input from yourself
and the rest of python-dev would be appreciated.

Nick, can you please apply the patch (will be sent in the following
email) to the PEP SVN as soon as we get the hard-link issue is figured
out? Alternatively, could you provide me write access to just the
pep-0394.txt file so I can update it myself? Thanks.

-Kerrick Staley

From mail at kerrickstaley.com  Wed Jul 20 08:53:22 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Wed, 20 Jul 2011 01:53:22 -0500
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
Message-ID: <CANaWP3ws0M7JvaqhVaBvCKpEGwg9Z2StaSLbbPc=QhzJsJ4VfA@mail.gmail.com>

$ svn diff
Index: pep-0394.txt
===================================================================
--- pep-0394.txt        (revision 88866)
+++ pep-0394.txt        (working copy)
@@ -1,5 +1,5 @@
 PEP: 394
-Title: The "python" command on Unix-Like Systems
+Title: The "python" Command on Unix-Like Systems
 Version: $Revision$
 Last-Modified: $Date$
 Author: Kerrick Staley <mail at kerrickstaley.com>,
@@ -53,10 +53,14 @@
 * When reinvoking the interpreter from a Python script, querying
   ``sys.executable`` to avoid hardcoded assumptions regarding the
   interpreter location remains the preferred approach.
+* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0
+should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``,
+with the original commands invoking these binaries by default, but possibly
+invoking the Python 3.0 versions instead.

 These recommendations are the outcome of the relevant python-dev discussion in
-March 2011 [1] (NOTE: More accurately, they will be such once that "Draft"
-status disappears from the PEP header, it has been moved into the "Other
+March to July 2011 [1] (NOTE: More accurately, they will be such once that
+"Draft" status disappears from the PEP header, it has been moved into
the "Other
 Informational PEP" section in PEP 0 and this note has been deleted)


@@ -142,13 +146,21 @@
 Application to the CPython Reference Interpreter
 ================================================

-While technically a new feature, the ``make install`` command and the Mac OS
-X installer in the 2.7 version of CPython will be adjusted to create the
-new ``python2`` command in addition to the existing ``python`` and
-``python2.7`` commands. This feature will first appear in CPython 2.7.2.
+While technically a new feature, the ``make install`` command and the Mac
+OS X installer in the 2.7 version of CPython will be adjusted to create the
+``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries,
+with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as links to
+the respective binaries. On Mac OS X, these will be symbolic links; on other
+platforms, they will be hard links, because hard links improve performance
+on filesystems other than Mac OS X's HFS. ``python``, ``idle``, ``pydoc``,
+and ``python-config`` will be created as symbolic links to these links.
+This feature will first appear in CPython 2.7.2.

-The ``make install`` command in the CPython 3.x series will continue to
-install only the ``python3`` symlink for the foreseeable future.
+The ``make install`` command in the CPython 3.x series will similarly install
+the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries
+(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and
+``python3-config`` as symbolic links on Mac OS X and as hard links on other
+platforms.


 Impact on PYTHON* Environment Variables
@@ -166,27 +178,12 @@
 Exclusions of MS Windows
 ========================

-This PEP deliberately excludes any proposals relating to Microsoft Windows.
-The use of parallel installs on Windows suffers from numerous issues,
-including the "last installed wins" behaviour for handling of file
-associations, a lack of universal robust symlink support for easy aliasing of
-commands, the fact that the Python executable is not available on ``PATH`` by
-default, the fact that the ``python.exe`` and ``pythonw.exe`` names are
-used for both Python 2 and Python 3 binaries and the lack of distinction
-between the different Python versions when the Start menu shortcuts are
-divorced from their containing folders (e.g. when they appear in the
-"Recently Used" list.
+This PEP deliberately excludes any proposals relating to Microsoft Windows:
+devising an equivalent solution for Windows was deemed too complex to handle
+here. PEP 397 and the related discussion on the python-dev mailing list address
+this issue.

-While these questions are well worth addressing, they do not have easy
-answers. The authors of this particular PEP aren't inclined to even begin
-trying to answer them, but anyone that wants to tackle them should feel free
-to start working on their own PEP.

-Note that, while the topic has been excluded from this PEP, there is plenty of
-material in the linked python-dev discussion that may be useful in the design
-and implementation of a Windows-specific solution.
-
-
 References
 ==========

From p.f.moore at gmail.com  Wed Jul 20 09:22:02 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 20 Jul 2011 08:22:02 +0100
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <j05e3c$g9c$1@dough.gmane.org>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
Message-ID: <CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>

On 20 July 2011 03:21, Terry Reedy <tjreedy at udel.edu> wrote:
> Suppose for Windows there were one '.../python' directory wherever the user
> first asks it to be put and that all pythons, not just cpython, are
> installed in directories below that and that the small startup file is
> copied into or linked from the python directory. Then the one python
> directory could be put on the path and left there and never removed by any
> python de-installer (unless perhaps it check that there are no subdirs and
> *asks* the user.

Hmm. Suppose that directory was "C:\Program Files\Python Launcher" (or
"C:\Windows\system32" if you don't want to add an extra directory to
PATH). And suppose that instead of having a startup file per Python
installation you have a single file called py.exe. Then you have the
launcher!

Plus, the launcher has its own uninstaller, making it a "normal" part
of the Windows environment, rather than being a directory created by
something which doesn't get uninstalled.

Plus, the launcher has a means of dealing with the "generic" python,
python2 and python3 commands, which your proposal doesn't.

Plus, the launcher deals with existing versions of Python, which your
proposal doesn't (except by manual intervention).

But yes, the idea is sound, which is why it's so similar to what Vinay
did with the launcher IMO.

Paul.

From nad at acm.org  Wed Jul 20 09:56:29 2011
From: nad at acm.org (Ned Deily)
Date: Wed, 20 Jul 2011 00:56:29 -0700
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
	/usr/bin/python2 symlink upstream)
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
Message-ID: <nad-8582A3.00562920072011@news.gmane.org>

In article 
<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw at mail.gmail.com>,
 Kerrick Staley <mail at kerrickstaley.com> wrote:
> On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily <nad at acm.org> wrote:
> > I think adding the requirement to mandate hard link vs soft link usage
> > is an unnecessary and unwarranted attempt at optimization.  For
> > instance, IIRC, the OS X installers don't use any hard links: that may
> > complicate the install, plus hard links on OS X HFS* file systems are a
> > bit of a kludge and not necessarily more efficient than symlinks.   It's
> > not a big deal but perhaps the wording should be changed to make a
> > suggestion about hard links vs syminks rather than mandate which should
> > be used.
> 
> Ah, OK. The wording's been changed so that symbolic links will be
> installed on Mac OS X and hard links elsewhere (although maybe
> symbolic links are also better on certain other platforms; I'm not
> sure).
> 
> I do think that specific instructions must be given (rather than just
> a suggestion) because it's indicating what must be done to CPython.
> The instructions *should* be as close as possible to what the
> installer already does, but I'm not entirely sure what the installer
> does by default, and the hard-link recommendation was based off a
> cursory inspection of my own system, so further input from yourself
> and the rest of python-dev would be appreciated.

Thanks for addressing the comments.  Unfortunately, I was relying too 
much on memory and I should have checked more carefully.  I see now 
that, within the framework bin directory for Python 3, the python3 -> 
python3.x link is, in fact, a hard link because it is being produced by 
the standard Makefile target "bininstall".  I was thinking more of the 
optional /usr/local/bin links which are all symlinks.  So, if the Python 
3 installer does is, there's no reason why Python 2 can't do it, I 
suppose.

But if you look at the Python 3 "bininstall" target (Makefile.pre.in 
starting around line 870 or so), you'll see that, for Python 3, symlinks 
are made for all the versioned files except "python3".  I'm not sure 
that there is a particular reason why a distinction was made;  IIRC, the 
other versioned links were added later in the cycle.  The other Python 3 
versioned links could probably be changed to hard links as well.  In the 
end, I don't think it makes a lot of difference.  But it would be better 
if Python 2 and Python 3 were consistent here.

My apologies for the confusion and extra work.

-- 
 Ned Deily,
 nad at acm.org


From ncoghlan at gmail.com  Wed Jul 20 10:28:16 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jul 2011 18:28:16 +1000
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
Message-ID: <CADiSq7fC+iLSf8_39F82r08ytjnZEu0SpdBM+scW37j8ZQePjg@mail.gmail.com>

On Wed, Jul 20, 2011 at 4:53 PM, Kerrick Staley <mail at kerrickstaley.com> wrote:
> Nick, can you please apply the patch (will be sent in the following
> email) to the PEP SVN as soon as we get the hard-link issue is figured
> out? Alternatively, could you provide me write access to just the
> pep-0394.txt file so I can update it myself? Thanks.

Done: http://www.python.org/dev/peps/pep-0394/

Main thing needed now is a tracker issue to reference (ideally with a
first cut at a patch) and python-dev's collective blessing to take it
out of Draft status.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Wed Jul 20 10:46:27 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jul 2011 18:46:27 +1000
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <CADiSq7c7yCramnj4vbeywfvZDKme7CEedRvvLfwXczMFOx7D+w@mail.gmail.com>

On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby <pje at telecommunity.com> wrote:
> So, without further ado, here it is:

I pushed this version up to the PEPs repo, so it now has a number
(402) and can be read in prettier HTML format:
http://www.python.org/dev/peps/pep-0402/

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From piotr at debian.org  Wed Jul 20 10:58:33 2011
From: piotr at debian.org (Piotr =?utf-8?Q?O=C5=BCarowski?=)
Date: Wed, 20 Jul 2011 10:58:33 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <20110720085833.GE30625@piotro.eu>

+1 (and yay!)
-- 
Piotr O?arowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

From v+python at g.nevcal.com  Wed Jul 20 11:24:12 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 02:24:12 -0700
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <4E269EBC.90708@g.nevcal.com>

On 7/19/2011 8:58 PM, P.J. Eby wrote:
> Standard Library Changes/Additions
> ----------------------------------
>
> The ``pkgutil`` module should be updated to handle this
> specification appropriately, including any necessary changes to
> ``extend_path()``, ``iter_modules()``, etc.
>
> Specifically the proposed changes and additions to ``pkgutil`` are:
>
> * A new ``extend_virtual_paths(path_entry)`` function, to extend
>   existing, already-imported virtual packages' ``__path__`` attributes
>   to include any portions found in a new ``sys.path`` entry.  This
>   function should be called by applications extending ``sys.path``
>   at runtime, e.g. when adding a plugin directory or an egg to the
>   path.
>
>   The implementation of this function does a simple top-down traversal
>   of ``sys.virtual_packages``, and performs any necessary
>   ``get_subpath()`` calls to identify what path entries need to
>   be added to each package's ``__path__``, given that `path_entry`
>   has been added to ``sys.path``.  (Or, in the case of sub-packages,
>   adding a derived subpath entry, based on their parent namespace's
>   ``__path__``.) 

When I read about creating __path__ from sys.path, I immediately thought 
of the issue of programs that extend sys.path, and the above is the 
"workaround" for such programs.  but it requires such programs to do 
work, and there are a lot of such programs (I, a relative newbie, have 
had to write some).  As it turns out, I can't think of a situation where 
I have extended sys.path that would result in a problem for fancy 
namespace packages, because so far I've only written modules, not 
packages, and only modules are on the paths that I add to sys.path.  But 
that does not make for a general solution.

Is there some way to create a new __path__ that would reflect the fact 
that it has been dynamically created, rather than set from __init__.py, 
and then when it is referenced, calculate (and cache?) a new value of 
__path__ to actually search?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/710f4c6c/attachment.html>

From v+python at g.nevcal.com  Wed Jul 20 11:17:58 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 02:17:58 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <loom.20110719T031535-263@post.gmane.org>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
Message-ID: <4E269D46.1010806@g.nevcal.com>

On 7/18/2011 6:41 PM, Vinay Sajip wrote:
>> >      So I can fix my machine, now that I understand what went wrong
>> >       (delete py.exe entries from HCU, and put them in HLM instead).
>> >       Then the other problem I have, is why py.exe launched py 2.6.4
>> >       instead of py 3.2.1 when 3.2.1 is newer, and I don't have a #!
>> >       line.  That is probably the defined behavior of the launcher, to
>> >       prefer 2.x if 3 isn't specified.  I'll have to reread the launcher
>> >       PEP.  In my environment, I would much prefer to use 3.x if 2.x isn't
>> >       specified... so hopefully when I reread the launcher PEP, I'll
>> >       discover a way to do that.  (I recall one can put -3 on the command
>> >       line, so I could do that in the py.exe ftype commands, but hopefully
>> >       there is a way to tweak the py.ini to do that.)
> Yes, the PEP explains what you need to do to get Python 3 to be the default:
> use the py.ini file or use an environment variable.
>
> The use of py from the command line is merely a convenience for developers (as
> the PEP says) - it's better to rely on shebang lines together with settings in
> the .ini to get the behaviour you want.

OK, took a few minutes to play with the launcher again.  I have removed 
the HCU classes that point at the launcher, leaving me with all my 
Python.File stuff pointing at some version of Python or another.  I seem 
to have Python 2.6.4, Python 3.1, and Python 3.2.1 installed, all 64-bit.

py.exe is located in c:\Windows\system32.exe, I modified the empty 
py.ini that was installed to contain

Since I don't yet have associations set up that point at the launcher, I 
thought I'd play with saying "py" in front of the command.

So I have a command foo.py using Python 3 syntax in a directory on my 
PATH.  It works fine, since Python 3.2.1 is in Python.file.  foo.py does 
not have a #! line.

I can successfully execute:  foo.py

However, the following fails:  py foo.py
It fails, because foo.py is not found.  Instead, I have to specify: py 
d:\path\to\foo.py
This is annoying, py should walk the PATH for unqualified files (the 
Windows PATH implicitly includes the current directory, of course, but 
if it were in the current directory, it would just work).

So when I: type c:\windows\system32\py.ini
I get a set of h2 h3 v2 v3 commands in a section called [commands].  Not 
sure where my [defaults] went.  Is there some more Windows magic to be 
explained?  WOW6432-ness, perhaps?  Probably my editor is running as a 
32-bit process, so looked in the wrong C:\windows\system32 !!

OK, with that mystery solved, and using Notepad running as administrator 
to actually, successfully edit the file, it still runs the wrong version 
of Python.  Here is the content of the file, what is wrong?  And why is 
the spacing around the = in the [commands] section so inconsistent?

[defaults]
python=3

[commands]
h2=c:\Python26\python -h
h3 = c:\Python32\python -h
v2= c:\Python26\python -V
v3 =c:\Python32\python -V


OK, so when I specify: py d:\path\to\foo.py
It fails, but this time it is because it launches Python 2.6.4.  So the 
[defaults] section doesn't seem to have an effect.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/a9b88e9d/attachment.html>

From p.f.moore at gmail.com  Wed Jul 20 13:28:23 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 20 Jul 2011 12:28:23 +0100
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E269D46.1010806@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
Message-ID: <CACac1F9YjDT0ZsVE1qji6rx57fDESKeCAf-CUn6BcpDQrgNRUQ@mail.gmail.com>

On 20 July 2011 10:17, Glenn Linderman <v+python at g.nevcal.com> wrote:
> However, the following fails:? py foo.py
> It fails, because foo.py is not found.? Instead, I have to specify: py
> d:\path\to\foo.py
> This is annoying, py should walk the PATH for unqualified files (the Windows
> PATH implicitly includes the current directory, of course, but if it were in
> the current directory, it would just work).

Just as a reference point:

PS 12:26 D:\Data
>type .\tt.py
import sys
print sys.argv
PS 12:26 D:\Data
>py tt.py
['tt.py']

This is Windows XP 32-bit, using Powershell as my shell. Also works in cmd.exe.

Paul.

From pje at telecommunity.com  Wed Jul 20 14:57:55 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 08:57:55 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CADiSq7c7yCramnj4vbeywfvZDKme7CEedRvvLfwXczMFOx7D+w@mail.g
	mail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CADiSq7c7yCramnj4vbeywfvZDKme7CEedRvvLfwXczMFOx7D+w@mail.gmail.com>
Message-ID: <20110720131415.2644B3A409B@sparrow.telecommunity.com>

At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote:
>On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > So, without further ado, here it is:
>
>I pushed this version up to the PEPs repo, so it now has a number
>(402) and can be read in prettier HTML format:
>http://www.python.org/dev/peps/pep-0402/

Technically, shouldn't this be a 3XXX series PEP?  Or are we not 
doing those any more now that all PEPs would be 3XXX?


From pje at telecommunity.com  Wed Jul 20 15:05:47 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 09:05:47 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <4E269EBC.90708@g.nevcal.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
Message-ID: <20110720131415.A2DD43A4100@sparrow.telecommunity.com>

At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote:
>When I read about creating __path__ from sys.path, I immediately 
>thought of the issue of programs that extend sys.path, and the above 
>is the "workaround" for such programs.?  but it requires such 
>programs to do work, and there are a lot of such programs (I, a 
>relative newbie, have had to write some).?  As it turns out, I can't 
>think of a situation where I have extended sys.path that would 
>result in a problem for fancy namespace packages, because so far 
>I've only written modules, not packages, and only modules are on the 
>paths that I add to sys.path.?  But that does not make for a general solution.

Most programs extend sys.path in order to import things.  If those 
things aren't yet imported, they don't have a __path__ yet, and so 
don't need to be fixed.  Only programs that modify sys.path *after* 
importing something that has a dynamic __path__ would need to do 
anything about that.


>Is there some way to create a new __path__ that would reflect the 
>fact that it has been dynamically created, rather than set from 
>__init__.py, and then when it is referenced, calculate (and cache?) 
>a new value of __path__ to actually search?

That's what extend_virtual_paths() is for.  It updates the __path__ 
of all currently-imported virtual packages.  Where before you wrote:

      sys.path.append('foo')

You would now write:

      sys.path.append('foo')
      pkgutil.extend_virtual_paths('foo')

...assuming you have virtual packages you've already imported.  If 
you don't, there's no reason to call extend_virtual_paths().  But it 
doesn't hurt anything if you call it unnecessarily, because it uses 
sys.virtual_packages to find out what to update, and if you haven't 
imported any virtual packages, there's nothing to update and the call 
will be a quick no-op.


From eric at trueblade.com  Wed Jul 20 15:29:21 2011
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 20 Jul 2011 09:29:21 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720131415.2644B3A409B@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>	<CADiSq7c7yCramnj4vbeywfvZDKme7CEedRvvLfwXczMFOx7D+w@mail.gmail.com>
	<20110720131415.2644B3A409B@sparrow.telecommunity.com>
Message-ID: <4E26D831.8000307@trueblade.com>

On 07/20/2011 08:57 AM, P.J. Eby wrote:
> At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote:
>> On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> > So, without further ado, here it is:
>>
>> I pushed this version up to the PEPs repo, so it now has a number
>> (402) and can be read in prettier HTML format:
>> http://www.python.org/dev/peps/pep-0402/
> 
> Technically, shouldn't this be a 3XXX series PEP?  Or are we not doing
> those any more now that all PEPs would be 3XXX?

I think we're back to normal PEP numbering. PEP 382 was also 3.x only.

Eric.

From rdmurray at bitdance.com  Wed Jul 20 15:57:07 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 20 Jul 2011 09:57:07 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <20110720135708.657C92500D5@webabinitio.net>


On Tue, 19 Jul 2011 23:58:55 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
> Worse, this is not just a problem for new users: it prevents *anyone*
> from easily splitting a package into separately-installable
> components.  In Perl terms, it would be as if every possible ``Net::``
> module on CPAN had to be bundled up and shipped in a single tarball!

In general the simplicity of the proposed mechanism and implementation
is attractive.  However, this bit of discussion struck me as sending the
wrong message.  We don't *want* something like the CPAN module
hierarchy.  I prefer to keep things as flat as practical.  Namespace
packages clearly have utility, but please let's not descend into
java-esq package hierarchies.

--
R. David Murray           http://www.bitdance.com

From vinay_sajip at yahoo.co.uk  Wed Jul 20 16:19:08 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 20 Jul 2011 14:19:08 +0000 (UTC)
Subject: [Python-Dev] 3.2.1 encoding surprise
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
Message-ID: <loom.20110720T160430-209@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:


>     Since I don't yet have associations set up that point at the
>     launcher, I thought I'd play with saying "py" in front of the
>     command.

Why don't you have any associations pointing to the launcher? Did you delete
them? If you uninstall and install the launcher, are they present?

>     So I have a command foo.py using Python 3 syntax in a directory on
>     my PATH.? It works fine, since Python 3.2.1 is in Python.file.?
>     foo.py does not have a #! line.
>     I can successfully execute:? foo.py
>     However, the following fails:? py foo.py
>     It fails, because foo.py is not found.? Instead, I have to specify:
>     py d:\path\to\foo.py
>     This is annoying, py should walk the PATH for unqualified files (the
>     Windows PATH implicitly includes the current directory, of course,

It's not py's job to walk the path: the shell does that when you just type
"foo". It locates foo.py, and then invokes py because of file association - py
then checks the file for a shebang to decide which Python to dispatch it to.

>     OK, with that mystery solved, and using Notepad running as
>     administrator to actually, successfully edit the file, it still runs
>     the wrong version of Python.? Here is the content of the file, what

I'm afraid I can't reproduce this. When I invoke a script with the default
py.ini, py runs it with Python 2. When I add a [defaults]python=3 entry, py 
correctly runs it with Python 3.

>     is wrong?? And why is the spacing around the = in the [commands]
>     section so inconsistent?

That's just test data, not a real "production" py.ini. I was testing out
something, which is why the spaces around = are every which way, and I never got
around to changing it. More importantly, those customised commands, while
perhaps useful for testing, are useless in everyday Python usage: perhaps -O,
-Werror, -E, -S etc. might be more useful. I'll take suggestions as to what
might be useful customised commands to ship as a default.

Regards,

Vinay Sajip


From merwok at netwok.org  Wed Jul 20 16:21:28 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 20 Jul 2011 16:21:28 +0200
Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace
 characters lost via email transmission.
In-Reply-To: <E1QjXN2-000859-6x@dinsdale.python.org>
References: <E1QjXN2-000859-6x@dinsdale.python.org>
Message-ID: <4E26E468.20800@netwok.org>

Hi,

> changeset:   3903:728660b53208
> user:        pje
> date:        Wed Jul 20 09:56:48 2011 -0400
> summary:
>   Restore whitespace characters lost via email transmission.
> [...]
> diff --git a/pep-0402.txt b/pep-0402.txt
> --- a/pep-0402.txt
> +++ b/pep-0402.txt
> @@ -38,13 +38,13 @@
>  
>  .. epigraph::
>  
> -   "Most packages are like modules.  Their contents are highly
> [snip]
> +    "Most packages are like modules.  Their contents are highly

FYI, reST uses three-space indents, not four (so that blocks align
nicely under the leading two dots + one space), so I think the change
was intentional.  The ?Documenting Python? guide tells this (in the
standard docs), and I think it applies to PEPs too.

Regards

From ndbecker2 at gmail.com  Wed Jul 20 16:40:18 2011
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 20 Jul 2011 10:40:18 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <j06pcj$6sa$1@dough.gmane.org>

I wonder if this fixes the long-standing issue in OS vendor's distributions.  In 
Fedora, for example, there is both arch-specific and non-arch directories: 
/usr/lib/python2.7 + /usr/lib64/python2.7, for example.  Pure python goes into 
/usr/lib/python2.7, and code including binaries goes into /usr/lib64/python2.7.  
But if a package has both, it all has to go into /usr/lib64/python2.7, because 
the current loader can't find pieces in 2 different directories.

You can't have both /usr/lib/python2.7/site-packages/foo and 
/usr/lib64/python2.7/site-packages/foo.

So if this PEP will allow pieces of foo to be found in 2 different places, that 
would be helpful, IMO.



From pje at telecommunity.com  Wed Jul 20 16:55:04 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 10:55:04 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <j06pcj$6sa$1@dough.gmane.org>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<j06pcj$6sa$1@dough.gmane.org>
Message-ID: <20110720145543.E834D3A409B@sparrow.telecommunity.com>

At 10:40 AM 7/20/2011 -0400, Neal Becker wrote:
>I wonder if this fixes the long-standing issue in OS vendor's 
>distributions.  In
>Fedora, for example, there is both arch-specific and non-arch directories:
>/usr/lib/python2.7 + /usr/lib64/python2.7, for example.  Pure python 
>goes into
>/usr/lib/python2.7, and code including binaries goes into 
>/usr/lib64/python2.7.
>But if a package has both, it all has to go into 
>/usr/lib64/python2.7, because
>the current loader can't find pieces in 2 different directories.
>
>You can't have both /usr/lib/python2.7/site-packages/foo and
>/usr/lib64/python2.7/site-packages/foo.
>
>So if this PEP will allow pieces of foo to be found in 2 different 
>places, that
>would be helpful, IMO.

It's more of a long-term solution than a short-term one.  In order 
for it to work the way you want, 'foo' would need to have its main 
code in foo.py rather than foo/__init__.py.

You could of course make that change on the author's behalf for your 
distro, or remove it altogether if it doesn't contain any actual 
code.  However, if you're going to make changes, you could change its 
__init__.py right now to append extra directories to the module 
__path__...  and that's something you can do right now.


From pje at telecommunity.com  Wed Jul 20 16:55:39 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 10:55:39 -0400
Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace
 characters lost via email transmission.
Message-ID: <20110720145618.7A3F33A409B@sparrow.telecommunity.com>

At 04:21 PM 7/20/2011 +0200, ??ric Araujo wrote:
>FYI, reST uses three-space indents, not four (so that blocks align
>nicely under the leading two dots + one space), so I think the change
>was intentional.  The ???Documenting Python??? guide tells this (in the
>standard docs), and I think it applies to PEPs too.

PEP 12 prescribes four-space indents for PEPs, actually, wherever it 
mentions an specific indentation depth.  Also, a formfeed character 
was lost, not just the leading spaces.

Essentially, though, I was just merging my working copy, and those 
were the only differences that showed up (apart from the filled-in 
Post-History header), so I assumed it was just whitespace lost in transmission.

(I'm a bit surprised that three-space indents are mandated for 
anything involving documenting Python in reST, though, since that 
would mean you'd also have to indent your code samples by three 
spaces, or else have an editor that supports two different tab widths.)  


From merwok at netwok.org  Wed Jul 20 17:05:28 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 20 Jul 2011 17:05:28 +0200
Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace
 characters lost via email transmission.
In-Reply-To: <20110720144058.92DD33A409B@sparrow.telecommunity.com>
References: <E1QjXN2-000859-6x@dinsdale.python.org> <4E26E468.20800@netwok.org>
	<20110720144058.92DD33A409B@sparrow.telecommunity.com>
Message-ID: <4E26EEB8.4050200@netwok.org>

Le 20/07/2011 16:40, P.J. Eby a ?crit :
> PEP 12 prescribes four-space indents for PEPs, actually, wherever it
> mentions an specific indentation depth.

Ah, thanks.  I see in my docutils docs that David Goodger used three and
four-space indents (three for indenting the body of a directive, four
for other cases).

> (I'm a bit surprised that three-space indents are mandated for
> anything involving documenting Python in reST, though, since that
> would mean you'd also have to indent your code samples by three
> spaces, or else have an editor that supports two different tab widths.)

I don?t remember this being a pain.  Maybe it?s because the reST I tend
to edit has much more reST indents than Python code example indents, so
I don?t mind typing <Tab><Space>.

Cheers

From jdhardy at gmail.com  Wed Jul 20 17:56:33 2011
From: jdhardy at gmail.com (Jeff Hardy)
Date: Wed, 20 Jul 2011 08:56:33 -0700
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>

On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby <pje at telecommunity.com> wrote:
> The biggest likely exception to the above would be when a piece of
> code tries to check whether some package is installed by importing
> it. ?If this is done *only* by importing a top-level module (i.e., not
> checking for a ``__version__`` or some other attribute), *and* there
> is a directory of the same name as the sought-for package on
> ``sys.path`` somewhere, *and* the package is not actually installed,
> then such code could *perhaps* be fooled into thinking a package is
> installed that really isn't.

This part worries me slightly. Imagine a program as such:

datagen.py
json/foo.js
json/bar.js

datagen.py uses the files in json/ to generate sample data for a
database. In datagen.py is the following code:

try:
    import json
except ImportError:
    import simplejson as json

Currently, this works just fine, but if will break (as I understand
it) under the PEP because the json directory will become a virtual
package and no ImportError will be raised. Is there a mitigation for
this in the PEP that I've missed?

> However, even in the rare case where all these conditions line up to
> happen at once, the failure is more likely to be annoying than
> damaging.  In most cases, after all, the code will simply fail a
> little later on, when it actually tries to DO something with the
> imported (but empty) module.  (And code that checks ``__version__``
> attributes or for the presence of some desired function, class, or
> module in the package will not see a false positive result in the
> first place.)

It may only be annoying, but it's still a breaking change, and a
subtle one at that. Checking __version__ is of course possible, but
it's never been necessary before, so it's unlikely there's much code
that does it. It also makes the fallback code significantly less neat.

- Jeff

From merwok at netwok.org  Wed Jul 20 17:58:17 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 20 Jul 2011 17:58:17 +0200
Subject: [Python-Dev] New tests in stable versions
Message-ID: <4E26FB19.2070406@netwok.org>

Hello everyone,

I?ve seen recent commits in the default branch (3.3) that improve test
coverage (for example logging) or add new test files (cgitb, committed
by Brian).  Do we have a policy of not adding new test files to stable
branches?  For existing test files that get more tests, is the commit to
stable branches left to the decision of the committer, or should stable
versions get the new tests too?

Regards

From hyugaricdeau at gmail.com  Wed Jul 20 18:37:34 2011
From: hyugaricdeau at gmail.com (Erik)
Date: Wed, 20 Jul 2011 12:37:34 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
Message-ID: <CADgWxTVJbGexBe95vRytLzioyoxxiEypFfFhvFe1YrCN4T5teg@mail.gmail.com>

On Wed, Jul 20, 2011 at 11:56 AM, Jeff Hardy <jdhardy at gmail.com> wrote:
> On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> The biggest likely exception to the above would be when a piece of
>> code tries to check whether some package is installed by importing
>> it. ?If this is done *only* by importing a top-level module (i.e., not
>> checking for a ``__version__`` or some other attribute), *and* there
>> is a directory of the same name as the sought-for package on
>> ``sys.path`` somewhere, *and* the package is not actually installed,
>> then such code could *perhaps* be fooled into thinking a package is
>> installed that really isn't.
>
> This part worries me slightly. Imagine a program as such:
>
> datagen.py
> json/foo.js
> json/bar.js
>
> datagen.py uses the files in json/ to generate sample data for a
> database. In datagen.py is the following code:
>
> try:
> ? ?import json
> except ImportError:
> ? ?import simplejson as json
>
> Currently, this works just fine, but if will break (as I understand
> it) under the PEP because the json directory will become a virtual
> package and no ImportError will be raised. Is there a mitigation for
> this in the PEP that I've missed?

This problem was brought up a few times on import-sig, but I don't
think a solution was ever decided on.

The best solution I can think of would be to have a way for a module
to mark itself as "finalized" (I'm not sure if that's the best
term--just the first that popped into my head).  This would prevent
its __path__ from being created or extended in any way.  For example,
if the json module contains `__finalized__ = True` or something of the
like, any `import json.foo` would immediately fail.

Of course, this would put all the onus on the json module to solve
this problem, and other modules might actually wish to be extendable
into packages, in which case you'd still have this problem.  In that
case there would need to be a way to mark a directory as not
containing importable code.  Not sure what the best approach to that
would be, especially since one of the goals of this PEP seems to be to
avoid marker files.

Erik

From victor.stinner at haypocalc.com  Wed Jul 20 18:25:52 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 20 Jul 2011 18:25:52 +0200
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <4E26FB19.2070406@netwok.org>
References: <4E26FB19.2070406@netwok.org>
Message-ID: <4E270190.1000209@haypocalc.com>

Le 20/07/2011 17:58, ?ric Araujo a ?crit :
> Do we have a policy of not adding new test files to stable branches?
New logging tests failed during some weeks. If we add new tests, we may 
also break some stable buildbots. I don't think that we need to add 
these new tests to a stable version.

Victor

From pje at telecommunity.com  Wed Jul 20 19:04:05 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 13:04:05 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.g
	mail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
Message-ID: <20110720170448.D7D793A409B@sparrow.telecommunity.com>

At 08:56 AM 7/20/2011 -0700, Jeff Hardy wrote:
>On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > The biggest likely exception to the above would be when a piece of
> > code tries to check whether some package is installed by importing
> > it.  If this is done *only* by importing a top-level module (i.e., not
> > checking for a ``__version__`` or some other attribute), *and* there
> > is a directory of the same name as the sought-for package on
> > ``sys.path`` somewhere, *and* the package is not actually installed,
> > then such code could *perhaps* be fooled into thinking a package is
> > installed that really isn't.
>
>This part worries me slightly. Imagine a program as such:
>
>datagen.py
>json/foo.js
>json/bar.js
>
>datagen.py uses the files in json/ to generate sample data for a
>database. In datagen.py is the following code:
>
>try:
>     import json
>except ImportError:
>     import simplejson as json
>
>Currently, this works just fine, but if will break (as I understand
>it) under the PEP because the json directory will become a virtual
>package and no ImportError will be raised.

Well, it won't fail as long if there actually *is* a json module or 
package on the path.  ;-)  But I do see your point.


>Is there a mitigation for this in the PEP that I've missed?

A possible mitigation would be to require that get_subpath() only 
return a directory name if that directory in fact contains importable 
modules somewhere.  This is actually discussed a bit later as an open 
issue under "Implementation Notes", indicating that iter_modules() 
has this issue as well.

The main open questions in doing this kind of checking have to do 
with recursion: it's perfectly valid to have say, a 'zc/' directory 
whose only content is a 'buildout/' subdirectory.

Of course, it still wouldn't help if the 'json/' subdirectory in your 
example did contain .py files.

There is another possibility, though:

What if we change the logic for pure-virtual package creation so that 
the parent module is created *if and only if* a child module is found?

In that case, trying to import a pure virtual 'zc' package would 
fail, but importing 'zc.buildout' would succeed as long as there was 
a zc/buildout.py or a zc/buildout/__init__.py somewhere.

And in your example, 'import json' would fail -- which is to say, succeed.  ;-)

This is a minor change to the spec, though perhaps a bit hairier to 
implement in practice.

The current import.c loop over the module name parts (iterating over 
say, 'zc', then 'buildout', and importing them in turn) would need to 
be reworked so that it could either roll back the virtual package 
creation in the event of sub-import failure or conversely delay 
creation of the parent package(s) until a sub-import finds a module.

I certainly think it's *doable*, mind you, but I'd hate to have to do 
it in C.  ;-)

Hm.  Here's another variant that might be easier to implement (even 
in C), and could offer some other advantages as well.

Suppose we replace the sys.virtual_packages set() with a 
sys.virtual_paths dict(): a dictionary that maps from module names to 
__path__ lists, and that's populated by the __path__ creation 
algorithm described in the PEP.  (An empty list would mean that 
__path__ creation failed for that module/package name.)

Now, if a module doesn't have a __path__ (or doesn't exist), we look 
in sys.virtual_paths for the module name.  If the retrieved list is 
empty, we fail the import.  If it's not, we proceed...  but *don't* 
create a module or set the existing module's __path__.

Then, at the point where an import succeeds, and we're going to set 
an attribute on the parent module, we recursively construct parent 
modules and set their __path__ attributes from sys.virtual_paths, if 
a module doesn't exist in sys.path, or its __path__ isn't set.

Voila.  Now there are fewer introspection problems as well: trying to 
'import json.foo' when there's no 'foo.py' in any json/ directory 
will *not* create an empty 'json' package in sys.modules as a 
side-effect.  And it won't add a __path__ to the 'json' module if 
there were a json.py found, either.

What's more, since importing a pure virtual package now fails unless 
you've successfully imported something from it before, it makes more 
sense for it to not have a __file__, or a __file__ of None.

Actually, it's too bad that we have to have parent packages in 
sys.modules, or I'd suggest we just make pure virtual packages 
unimportable, period.

Technically, we *could* always create dummy parent modules for 
virtual packages and *not* put them in sys.modules, but I'm not sure 
if that's a good idea.  It would be more consistent in some ways with 
the idea that virtual packages are not directly importable, but an 
interesting side effect would be that if module A does:

   import foo.bar

and module B does:

   import foo.baz

Then module A's version of 'foo' has *only* a 'bar' attribute and B's 
version has *only* a 'baz' attribute.  This could be considered a 
good thing, a bad thing, or a weird thing, depending on how you look 
at it.  ;-)

Probably, we should stick with the current shared 'foo' instance, 
even for pure virtual packages.  It's just that 'foo' should not 
exist in sys.packages until one of the above imports succeeds.

Anyway, thanks for bringing this issue up, because now we can fix the 
hole *entirely*.  If pure virtual packages can never be imported 
directly, then they can *never* create false positive imports -- and 
the "Backward Compatibility" part of the PEP gets shorter.  ;-)

Hurray!  (I'm tempted to run off and tweak the PEP for this right 
now, but I want to see if any of the folks who'd be doing the actual 
3.x implementation of this want to weigh in on the details first.)


From pje at telecommunity.com  Wed Jul 20 19:22:36 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 13:22:36 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CADgWxTVJbGexBe95vRytLzioyoxxiEypFfFhvFe1YrCN4T5teg@mail.g
	mail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
	<CADgWxTVJbGexBe95vRytLzioyoxxiEypFfFhvFe1YrCN4T5teg@mail.gmail.com>
Message-ID: <20110720172317.21DAA3A409B@sparrow.telecommunity.com>

At 12:37 PM 7/20/2011 -0400, Erik wrote:
>The best solution I can think of would be to have a way for a module
>to mark itself as "finalized" (I'm not sure if that's the best
>term--just the first that popped into my head).  This would prevent
>its __path__ from being created or extended in any way.  For example,
>if the json module contains `__finalized__ = True` or something of the
>like, any `import json.foo` would immediately fail.

That wouldn't actually fix the problem Jeff brought up, which was the 
case where there *wasn't* a json.py.

In any case, we can fix this now by banning direct import of 
pure-virtual packages.


>In that case there would need to be a way to mark a directory as not
>containing importable code.  Not sure what the best approach to that
>would be, especially since one of the goals of this PEP seems to be to
>avoid marker files.

For this particular issue, we don't need it.  For tools that process 
Python code, or use pkgutil.walk_modules(), there may still be use 
cases, so we'll keep an eye open for relevant input.  Hopefully 
someone will say something that jars loose an idea or two, as 
happened with Jeff's issue above.

(Btw, as we speak, I am swiping Jeff's example and adding it into the 
PEP.  ;-)  It makes a great motivating example for banning 
pure-virtual package imports.)


From mail at kerrickstaley.com  Wed Jul 20 20:03:26 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Wed, 20 Jul 2011 13:03:26 -0500
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <CADiSq7fC+iLSf8_39F82r08ytjnZEu0SpdBM+scW37j8ZQePjg@mail.gmail.com>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<CADiSq7fC+iLSf8_39F82r08ytjnZEu0SpdBM+scW37j8ZQePjg@mail.gmail.com>
Message-ID: <CANaWP3wTsYmtBcKnH3+pyhrDw9GsNVgJUAcMTmOEQVss8VzjGA@mail.gmail.com>

On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Done: http://www.python.org/dev/peps/pep-0394/

Quick question: When I do "svn up", it doesn't show any changes. Has
it been switched over to Mercurial recently?

Thanks,
Kerrick Staley

From ericsnowcurrently at gmail.com  Wed Jul 20 20:14:17 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 20 Jul 2011 12:14:17 -0600
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <CANaWP3wTsYmtBcKnH3+pyhrDw9GsNVgJUAcMTmOEQVss8VzjGA@mail.gmail.com>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<CADiSq7fC+iLSf8_39F82r08ytjnZEu0SpdBM+scW37j8ZQePjg@mail.gmail.com>
	<CANaWP3wTsYmtBcKnH3+pyhrDw9GsNVgJUAcMTmOEQVss8VzjGA@mail.gmail.com>
Message-ID: <CALFfu7CLbQxf6tXcffRqq+ij-+R8oUDBeohOGm6BM4mm4fyxUQ@mail.gmail.com>

On Wed, Jul 20, 2011 at 12:03 PM, Kerrick Staley <mail at kerrickstaley.com> wrote:
> On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Done: http://www.python.org/dev/peps/pep-0394/
>
> Quick question: When I do "svn up", it doesn't show any changes. Has
> it been switched over to Mercurial recently?

http://hg.python.org/peps/

-eric

>
> Thanks,
> Kerrick Staley
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>

From tjreedy at udel.edu  Wed Jul 20 20:38:47 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Jul 2011 14:38:47 -0400
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
	encoding surprise)
In-Reply-To: <CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>	<20110719171647.03f9f9c9@pitrou.net>	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
Message-ID: <j077bn$855$1@dough.gmane.org>

On 7/20/2011 3:22 AM, Paul Moore wrote:
> On 20 July 2011 03:21, Terry Reedy<tjreedy at udel.edu>  wrote:
>> Suppose for Windows there were one '.../python' directory wherever the user
>> first asks it to be put and that all pythons, not just cpython, are
>> installed in directories below that and that the small startup file is
>> copied into or linked from the python directory. Then the one python
>> directory could be put on the path and left there and never removed by any
>> python de-installer (unless perhaps it check that there are no subdirs and
>> *asks* the user.
>
> Hmm. Suppose that directory was "C:\Program Files\Python Launcher" (or
> "C:\Windows\system32" if you don't want to add an extra directory to
> PATH). And suppose that instead of having a startup file per Python
> installation you have a single file called py.exe. Then you have the
> launcher!
>
> Plus, the launcher has its own uninstaller, making it a "normal" part
> of the Windows environment, rather than being a directory created by
> something which doesn't get uninstalled.
>
> Plus, the launcher has a means of dealing with the "generic" python,
> python2 and python3 commands, which your proposal doesn't.
>
> Plus, the launcher deals with existing versions of Python, which your
> proposal doesn't (except by manual intervention).
>
> But yes, the idea is sound, which is why it's so similar to what Vinay
> did with the launcher IMO.

Many installers first make an organization directory and then an app 
directory within that. This annoys me sometimes when they only have one 
app to ever install, but is useful when there might really be multiple 
directories, as in our case. (Ditto for start menu entries.) This is 
what python should have done a decade ago. Now is not too late.

The launcher has to be in a directory somewhere on the path. That 
directory could just as well be 'our' directory. The two proposals 
overlap but are not mutually exclusive. For future pythons, 'python33' 
is easier to remember and type than 'py -v 3.3' or whatever the proposed 
encantation is.

A python directory also gives a sensible (though optional) place to put 
other interpreters and even python-based apps. The launcher does not.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Wed Jul 20 20:48:18 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Jul 2011 14:48:18 -0400
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <4E270190.1000209@haypocalc.com>
References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com>
Message-ID: <j077ti$bql$1@dough.gmane.org>

On 7/20/2011 12:25 PM, Victor Stinner wrote:
> Le 20/07/2011 17:58, ?ric Araujo a ?crit :
>> Do we have a policy of not adding new test files to stable branches?
> New logging tests failed during some weeks. If we add new tests, we may
> also break some stable buildbots. I don't think that we need to add
> these new tests to a stable version.

When bugs are fixed in stable branches, they are usually accompanied by 
tests that fail without the bugfix. I have understood the policy to be 
that new tests go into stable branches. Failure indicates a bug in 
either the not-really-so-stable branch or the test. In the latter case, 
remove the test everywhere until fixed. In the former case, either fix 
the bug in the stable branch immediately or open an issue and attach the 
test code (skipping the test needed stage) or just disable it and note 
on the issue that a fix patch should re-enable. The logging tests may 
have been exceptional some way.

-- 
Terry Jan Reedy



From g.brandl at gmx.net  Wed Jul 20 21:10:01 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 20 Jul 2011 21:10:01 +0200
Subject: [Python-Dev] [Python-checkins] peps: Restore whitespace
 characters lost via email transmission.
In-Reply-To: <4E26EEB8.4050200@netwok.org>
References: <E1QjXN2-000859-6x@dinsdale.python.org> <4E26E468.20800@netwok.org>
	<20110720144058.92DD33A409B@sparrow.telecommunity.com>
	<4E26EEB8.4050200@netwok.org>
Message-ID: <j07962$iru$1@dough.gmane.org>

On 07/20/11 17:05, ?ric Araujo wrote:
> Le 20/07/2011 16:40, P.J. Eby a ?crit :
>> PEP 12 prescribes four-space indents for PEPs, actually, wherever it
>> mentions an specific indentation depth.
> 
> Ah, thanks.  I see in my docutils docs that David Goodger used three and
> four-space indents (three for indenting the body of a directive, four
> for other cases).
> 
>> (I'm a bit surprised that three-space indents are mandated for
>> anything involving documenting Python in reST, though, since that
>> would mean you'd also have to indent your code samples by three
>> spaces, or else have an editor that supports two different tab widths.)
> 
> I don?t remember this being a pain.  Maybe it?s because the reST I tend
> to edit has much more reST indents than Python code example indents, so
> I don?t mind typing <Tab><Space>.

Also, chances are that you've tried out your sample code (!) and thus copy
it from a Python file anyway.

Georg


From tjreedy at udel.edu  Wed Jul 20 20:12:42 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Jul 2011 14:12:42 -0400
Subject: [Python-Dev] [Python-checkins] cpython: #665194: support
 roundtripping RFC2822 date stamps in the email.utils module
In-Reply-To: <E1QjYu7-0001X3-Rc@dinsdale.python.org>
References: <E1QjYu7-0001X3-Rc@dinsdale.python.org>
Message-ID: <4E271A9A.1040703@udel.edu>

On 7/20/2011 11:41 AM, r.david.murray wrote:

> diff --git a/Lib/email/utils.py b/Lib/email/utils.py

>   # We need wormarounds for bugs in these methods in older Pythons (see below)

Is 'wormaround' (variation of workaround) an intentional play on the 
fact that some worms prey on other 'bugs' ;-? (rather then eating plant 
matter?)

Terry

From ericsnowcurrently at gmail.com  Wed Jul 20 21:35:46 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 20 Jul 2011 13:35:46 -0600
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720170448.D7D793A409B@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
	<20110720170448.D7D793A409B@sparrow.telecommunity.com>
Message-ID: <CALFfu7Dm6Qo37a=h8JL1ykaOWs7pT0JJtT+6brymRDwyzc=h2Q@mail.gmail.com>

On Wed, Jul 20, 2011 at 11:04 AM, P.J. Eby <pje at telecommunity.com> wrote:
> Hm. ?Here's another variant that might be easier to implement (even in C),
> and could offer some other advantages as well.
>
> Suppose we replace the sys.virtual_packages set() with a sys.virtual_paths
> dict(): a dictionary that maps from module names to __path__ lists, and
> that's populated by the __path__ creation algorithm described in the PEP.
> ?(An empty list would mean that __path__ creation failed for that
> module/package name.)
>
> Now, if a module doesn't have a __path__ (or doesn't exist), we look in
> sys.virtual_paths for the module name. ?If the retrieved list is empty, we
> fail the import. ?If it's not, we proceed... ?but *don't* create a module or
> set the existing module's __path__.
>
> Then, at the point where an import succeeds, and we're going to set an
> attribute on the parent module, we recursively construct parent modules and
> set their __path__ attributes from sys.virtual_paths, if a module doesn't
> exist in sys.path, or its __path__ isn't set.

(I'm guessing you meant sys.modules in that last sentence.)

This is a really nice solution.  So a virtual package is not imported
until a submodule of the virtual package is successfully imported
(except for direct import of pure virtual packages).  It seems like
sys.virtual_packages should be populated even during a failed
submodule import.  Is that right?

Also, it makes sense that the above applies to all virtual packages,
not just pure ones.

>
> Voila. ?Now there are fewer introspection problems as well: trying to
> 'import json.foo' when there's no 'foo.py' in any json/ directory will *not*
> create an empty 'json' package in sys.modules as a side-effect. ?And it
> won't add a __path__ to the 'json' module if there were a json.py found,
> either.
>
> What's more, since importing a pure virtual package now fails unless you've
> successfully imported something from it before, it makes more sense for it
> to not have a __file__, or a __file__ of None.
>
> Actually, it's too bad that we have to have parent packages in sys.modules,
> or I'd suggest we just make pure virtual packages unimportable, period.

It wouldn't be that hard to disallow their direct import entirely, but
still allow the indirect import when successfully importing a
submodule.  However, that would effectively imply that the import of
submodules of the virtual package will also fail.  In other words, it
may be a source of confusion if a package can't be imported but its
submodule can.

There is one remaining difference between the two types of virtual
packages that's derived from allowing direct import of pure virtual
packages.

When a pure virtual package is directly imported, a new [empty] module
is created and its __path__ is set to the matching value in
sys.virtual_packages.  However, an "impure" virtual package is not
created upon direct import, and its __path__ is not updated until a
submodule import is attempted.  Even the sys.virtual_packages entry is
not generated until the submodule attempt, since the virtual package
mechanism doesn't kick in until the point that an ImportError is
currently raised.

This isn't that big a deal, but it would be the one behavioral
difference between the two kinds of virtual packages.  So either leave
that one difference, disallow direct import of pure virtual packages,
or attempt to make virtual packages for all non-package imports.  That
last one would impose the virtual package overhead on many more
imports so it is probably too impractical.  I'm fine with leaving the
one difference.

>
> Technically, we *could* always create dummy parent modules for virtual
> packages and *not* put them in sys.modules, but I'm not sure if that's a
> good idea. ?It would be more consistent in some ways with the idea that
> virtual packages are not directly importable, but an interesting side effect
> would be that if module A does:
>
> ?import foo.bar
>
> and module B does:
>
> ?import foo.baz
>
> Then module A's version of 'foo' has *only* a 'bar' attribute and B's
> version has *only* a 'baz' attribute. ?This could be considered a good
> thing, a bad thing, or a weird thing, depending on how you look at it. ?;-)
>
> Probably, we should stick with the current shared 'foo' instance, even for
> pure virtual packages. ?It's just that 'foo' should not exist in
> sys.packages until one of the above imports succeeds.

(Guessing you meant sys.virtual_packages.)

Agreed.

FYI, last night I started on an importlib-based implementation for the
PEP and the above solution would be really easy to incorporate.

-eric

>
> Anyway, thanks for bringing this issue up, because now we can fix the hole
> *entirely*. ?If pure virtual packages can never be imported directly, then
> they can *never* create false positive imports -- and the "Backward
> Compatibility" part of the PEP gets shorter. ?;-)
>
> Hurray! ?(I'm tempted to run off and tweak the PEP for this right now, but I
> want to see if any of the folks who'd be doing the actual 3.x implementation
> of this want to weigh in on the details first.)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>

From tjreedy at udel.edu  Wed Jul 20 21:51:39 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Jul 2011 15:51:39 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720170448.D7D793A409B@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.g	mail.com>
	<20110720170448.D7D793A409B@sparrow.telecommunity.com>
Message-ID: <j07bkb$3q5$1@dough.gmane.org>

On 7/20/2011 1:04 PM, P.J. Eby wrote:

>> This part worries me slightly. Imagine a program as such:
>>
>> datagen.py
>> json/foo.js
>> json/bar.js
>>
>> datagen.py uses the files in json/ to generate sample data for a
>> database. In datagen.py is the following code:
>>
>> try:
>> import json
>> except ImportError:
>> import simplejson as json

While reading the PEP, I worried about this standard usage too but 
missed the scenario you imagined. Good catch.

> A possible mitigation would be to require that get_subpath() only return
> a directory name if that directory in fact contains importable modules
> somewhere. This is actually discussed a bit later as an open issue under
> "Implementation Notes", indicating that iter_modules() has this issue as
> well.

If one actually wants to create a bare-as-possible empty module, one can 
do that now either with a directory containing an empty __init__.py or, 
even cleaner, imp.new_module. So there is no need for the new mechanism 
to ever duplicate either ;-). So +1 on improving back-compatibility.

-- 
Terry Jan Reedy


From solipsis at pitrou.net  Wed Jul 20 22:40:17 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Jul 2011 22:40:17 +0200
Subject: [Python-Dev] [Python-checkins] cpython: #665194: support
 roundtripping RFC2822 date stamps in the email.utils module
References: <E1QjYu7-0001X3-Rc@dinsdale.python.org> <4E271A9A.1040703@udel.edu>
Message-ID: <20110720224017.4a2ddf83@pitrou.net>

On Wed, 20 Jul 2011 14:12:42 -0400
Terry Reedy <tjreedy at udel.edu> wrote:
> On 7/20/2011 11:41 AM, r.david.murray wrote:
> 
> > diff --git a/Lib/email/utils.py b/Lib/email/utils.py
> 
> >   # We need wormarounds for bugs in these methods in older Pythons (see below)
> 
> Is 'wormaround' (variation of workaround) an intentional play on the 
> fact that some worms prey on other 'bugs' ;-? (rather then eating plant 
> matter?)

Or perhaps worms dig their way carefully around known bugs?




From pje at telecommunity.com  Wed Jul 20 22:44:15 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 16:44:15 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CALFfu7Dm6Qo37a=h8JL1ykaOWs7pT0JJtT+6brymRDwyzc=h2Q@mail.g
	mail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
	<20110720170448.D7D793A409B@sparrow.telecommunity.com>
	<CALFfu7Dm6Qo37a=h8JL1ykaOWs7pT0JJtT+6brymRDwyzc=h2Q@mail.gmail.com>
Message-ID: <20110720204456.5D1AB3A409B@sparrow.telecommunity.com>

At 01:35 PM 7/20/2011 -0600, Eric Snow wrote:
>This is a really nice solution.  So a virtual package is not imported
>until a submodule of the virtual package is successfully imported

Correct...

>(except for direct import of pure virtual packages).

Not correct.  ;-)  What we do is avoid creating a parent module or 
altering its __path__ until a submodule/subpackage import is just 
about to be successfully completed.

See the change I just pushed to the PEP:

    http://hg.python.org/peps/rev/a6f02035c66c

Or read the revised Specification section here (which is a bit easier 
to read than the diff):

    http://www.python.org/dev/peps/pep-0402/#specification

The change is basically that we wait until a successful find_module() 
happens before creating or tweaking any parent modules.  This way, 
the load_module() part will still see an initialized parent package 
in sys.modules, and if it does any relative imports, they'll still work.

(It *does* mean that if an error happens during load_module(), then 
future imports of the virtual package will succeed, but I'm okay with 
that corner case.)


>It seems like
>sys.virtual_packages should be populated even during a failed
>submodule import.  Is that right?

Yes.  In the actual draft, btw, I dubbed it 
``sys.virtual_package_paths`` and made it a dictionary.  This 
actually makes the pkgutil.extend_path() code more general: it'll be 
able to fix the paths of things you haven't actually imported yet.  ;-)


>Also, it makes sense that the above applies to all virtual packages,
>not just pure ones.

Well, if the package isn't "pure" then what you've imported is really 
just an ordinary module, not a package at all.  ;-)



>When a pure virtual package is directly imported, a new [empty] module
>is created and its __path__ is set to the matching value in
>sys.virtual_packages.  However, an "impure" virtual package is not
>created upon direct import, and its __path__ is not updated until a
>submodule import is attempted.  Even the sys.virtual_packages entry is
>not generated until the submodule attempt, since the virtual package
>mechanism doesn't kick in until the point that an ImportError is
>currently raised.
>
>This isn't that big a deal, but it would be the one behavioral
>difference between the two kinds of virtual packages.  So either leave
>that one difference, disallow direct import of pure virtual packages,
>or attempt to make virtual packages for all non-package imports.  That
>last one would impose the virtual package overhead on many more
>imports so it is probably too impractical.  I'm fine with leaving the
>one difference.

At this point, I've updated the PEP to disallow direct imports of 
pure virtual packages.  AFAICT it's the only approach that ensures 
you can't get false positive imports by having 
unrelated-but-similarly-named directories floating around.

So, really, there's not a difference, except that you can't import a 
useless empty module that you have no real business importing in the 
first place...  and I'm fine with that.  ;-)


>FYI, last night I started on an importlib-based implementation for the
>PEP and the above solution would be really easy to incorporate.

Well, you might want to double-check that now that I've updated the 
spec.  ;-)  In the new approach, you cannot rely on parent modules 
existing before proceeding to the submodule import.

However, I've just glanced at the importlib trunk, and I think I see 
what you mean.  It's already using a recursive approach, rather than 
an iterative one, so the change should be a lot simpler there than in import.c.

There probably just needs to be a pair of functions like:

     def _get_parent_path(parent):
         pmod = sys.modules.get(parent)
         if pmod is None:
             try:
                 pmod = _gcd_import(parent)
             except ImportError:
                 # Can't import parent, is it a virtual package?
                 path = imp.get_virtual_path(parent)
                 if not path:
                     # no, allow the parent's import error to propagate
                     raise
                 return path
         if hasattr(pmod, '__path__'):
             return pmod.__path__
         else:
             return imp.get_virtual_path(parent)

     def _get_parent_module(parent):
         pmod = sys.modules.get(parent)
         if pmod is None:
             pmod = sys.modules[parent] = imp.new_module(parent)
             if '.' in parent:
                 head, _, tail = parent.rpartition('.')
                 setattr(_get_parent_module(head), tail, pmod)
         if not hasattr(pmod, '__path__'):
             pmod.__path__ = imp.get_virtual_path(parent)
         return pmod

And then instead of hanging on to parent_module during the import 
process, you'd just grab a path from _get_parent_path(), and 
initialize parent_module a little later, i.e.:

         if parent:
             path = _get_parent_path(parent)
             if not path:
                 msg = (_ERR_MSG + '; {} is not a 
package').format(name, parent)
                 raise ImportError(msg)

         meta_path = sys.meta_path + _IMPLICIT_META_PATH
         for finder in meta_path:
             loader = finder.find_module(name, path)
             if loader is not None:
                 # ensure parent module exists and is a package before loading
                 parent_module = _get_parent_module(parent)
                 loader.load_module(name)
                 break
         else:
             raise ImportError(_ERR_MSG.format(name))

So, yeah, actually, that's looking pretty sweet.  Basically, we just 
have to throw a virtual_package_paths dict into the sys module, and 
do the above along with the get_virtual_path() function and add 
get_subpath() to the importer objects, in order to get the PEP's core 
functionality working.


From ericsnowcurrently at gmail.com  Wed Jul 20 23:22:05 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 20 Jul 2011 15:22:05 -0600
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720204456.5D1AB3A409B@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
	<20110720170448.D7D793A409B@sparrow.telecommunity.com>
	<CALFfu7Dm6Qo37a=h8JL1ykaOWs7pT0JJtT+6brymRDwyzc=h2Q@mail.gmail.com>
	<20110720204456.5D1AB3A409B@sparrow.telecommunity.com>
Message-ID: <CALFfu7AgEvEt0Mf_3m_vSafQ6SV_MzdUs6_4RD-VquQg1J7wMQ@mail.gmail.com>

On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 01:35 PM 7/20/2011 -0600, Eric Snow wrote:
>>
>> This is a really nice solution. ?So a virtual package is not imported
>> until a submodule of the virtual package is successfully imported
>
> Correct...
>
>> (except for direct import of pure virtual packages).
>
> Not correct. ?;-) ?What we do is avoid creating a parent module or altering
> its __path__ until a submodule/subpackage import is just about to be
> successfully completed.

Good point, though I was talking about direct imports of pure virtual
packages (which you've indicated are disallowed by the current draft).

>> Also, it makes sense that the above applies to all virtual packages,
>> not just pure ones.
>
> Well, if the package isn't "pure" then what you've imported is really just
> an ordinary module, not a package at all. ?;-)

I meant that if the submodule import fails in the "impure" case, the
existing module does not end up with a __path__.

>> When a pure virtual package is directly imported, a new [empty] module
>> is created and its __path__ is set to the matching value in
>> sys.virtual_packages. ?However, an "impure" virtual package is not
>> created upon direct import, and its __path__ is not updated until a
>> submodule import is attempted. ?Even the sys.virtual_packages entry is
>> not generated until the submodule attempt, since the virtual package
>> mechanism doesn't kick in until the point that an ImportError is
>> currently raised.
>>
>> This isn't that big a deal, but it would be the one behavioral
>> difference between the two kinds of virtual packages. ?So either leave
>> that one difference, disallow direct import of pure virtual packages,
>> or attempt to make virtual packages for all non-package imports. ?That
>> last one would impose the virtual package overhead on many more
>> imports so it is probably too impractical. ?I'm fine with leaving the
>> one difference.
>
> At this point, I've updated the PEP to disallow direct imports of pure
> virtual packages. ?AFAICT it's the only approach that ensures you can't get
> false positive imports by having unrelated-but-similarly-named directories
> floating around.

I see what you mean.  That case is probably more important than the
case of having a package that fails to import but submodules of the
package that succeed.

>> FYI, last night I started on an importlib-based implementation for the
>> PEP and the above solution would be really easy to incorporate.
>
> Well, you might want to double-check that now that I've updated the spec.
> ?;-) ?In the new approach, you cannot rely on parent modules existing before
> proceeding to the submodule import.
>
> However, I've just glanced at the importlib trunk, and I think I see what
> you mean. ?It's already using a recursive approach, rather than an iterative
> one, so the change should be a lot simpler there than in import.c.
>
<snip>
>
> So, yeah, actually, that's looking pretty sweet. ?Basically, we just have to
> throw a virtual_package_paths dict into the sys module, and do the above
> along with the get_virtual_path() function and add get_subpath() to the
> importer objects, in order to get the PEP's core functionality working.


Exactly.  That's part of why the importlib approach is so appealing to
me.  Brett really did a nice job.

-eric

From v+python at g.nevcal.com  Wed Jul 20 23:43:43 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 14:43:43 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <loom.20110720T160430-209@post.gmane.org>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
Message-ID: <4E274C0F.40400@g.nevcal.com>

On 7/20/2011 7:19 AM, Vinay Sajip wrote:
> Glenn Linderman<v+python<at>  g.nevcal.com>  writes:
>
>
>>      Since I don't yet have associations set up that point at the
>>      launcher, I thought I'd play with saying "py" in front of the
>>      command.
> Why don't you have any associations pointing to the launcher? Did you delete
> them? If you uninstall and install the launcher, are they present?

This is a followon from the other day... the launcher installer had 
placed launcher associations in HCU, which are not visible from ftype 
and assoc commands... so I deleted those, which returned my system to 
not having launcher associations, but Python 3.2.1 associations instead 
(they were, and remained, in HLM throughout the install, but HCU 
temporarily overrode them in some circumstances).

So the launcher is installed in c:\windows\system32, but doesn't have 
associations.  Thought I'd play with it from the command line, before 
reinstalling with ALLUSERS=1 (or establishing the associations by hand).

It is still my opinion that the installer should ask whether it should 
be installed for all users or not, but from what you said the other day, 
this may be beyond the .msi installer.


>
>>      So I have a command foo.py using Python 3 syntax in a directory on
>>      my PATH.  It works fine, since Python 3.2.1 is in Python.file. 
>>      foo.py does not have a #! line.
>>      I can successfully execute:  foo.py
>>      However, the following fails:  py foo.py
>>      It fails, because foo.py is not found.  Instead, I have to specify:
>>      py d:\path\to\foo.py
>>      This is annoying, py should walk the PATH for unqualified files (the
>>      Windows PATH implicitly includes the current directory, of course,
> It's not py's job to walk the path: the shell does that when you just type
> "foo". It locates foo.py, and then invokes py because of file association - py
> then checks the file for a shebang to decide which Python to dispatch it to.

Certainly when the launcher is invoked via an association, this would be 
the case.  However, when the launcher is invoked via the command line, 
then the unqualified name is passed through.  To be useful from the 
command line, the launcher should walk the PATH to find the .py file.

>
>>      OK, with that mystery solved, and using Notepad running as
>>      administrator to actually, successfully edit the file, it still runs
>>      the wrong version of Python.  Here is the content of the file, what
> I'm afraid I can't reproduce this. When I invoke a script with the default
> py.ini, py runs it with Python 2. When I add a [defaults]python=3 entry, py
> correctly runs it with Python 3.

I still get the same behavior.  Is there any debugging output produced 
by py.exe that would tell what py.ini it looks in, and what the content 
is?  What diagnostic steps might I take to produce additional 
information that would help you (or me, along the way) analyze the 
issue?  I do not presently have a C compiler installed on this machine, 
however, unless it came along for a ride with something else.

>
>>      is wrong?  And why is the spacing around the = in the [commands]
>>      section so inconsistent?
> That's just test data, not a real "production" py.ini. I was testing out
> something, which is why the spaces around = are every which way, and I never got
> around to changing it. More importantly, those customised commands, while
> perhaps useful for testing, are useless in everyday Python usage: perhaps -O,
> -Werror, -E, -S etc. might be more useful. I'll take suggestions as to what
> might be useful customised commands to ship as a default.

Fine. I realize the launcher is still Beta.  This was just a curiosity 
question.

> Regards,
>
> Vinay Sajip

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/20678bc3/attachment.html>

From rdmurray at bitdance.com  Wed Jul 20 23:51:03 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 20 Jul 2011 17:51:03 -0400
Subject: [Python-Dev] [Python-checkins] cpython: #665194: support
	roundtripping RFC2822 date stamps in the email.utils module
In-Reply-To: <20110720224017.4a2ddf83@pitrou.net>
References: <E1QjYu7-0001X3-Rc@dinsdale.python.org> <4E271A9A.1040703@udel.edu>
	<20110720224017.4a2ddf83@pitrou.net>
Message-ID: <20110720215103.DD85B25008E@webabinitio.net>

On Wed, 20 Jul 2011 22:40:17 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 20 Jul 2011 14:12:42 -0400
> Terry Reedy <tjreedy at udel.edu> wrote:
> > On 7/20/2011 11:41 AM, r.david.murray wrote:
> > 
> > > diff --git a/Lib/email/utils.py b/Lib/email/utils.py
> > 
> > >   # We need wormarounds for bugs in these methods in older Pythons (see below)
> > 
> > Is 'wormaround' (variation of workaround) an intentional play on the 
> > fact that some worms prey on other 'bugs' ;-? (rather then eating plant 
> > matter?)
> 
> Or perhaps worms dig their way carefully around known bugs?

Hexlify, wormaround...our Barry is just full of interesting words :)

--
R. David Murray           http://www.bitdance.com

From vinay_sajip at yahoo.co.uk  Thu Jul 21 00:07:49 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 20 Jul 2011 22:07:49 +0000 (UTC)
Subject: [Python-Dev] New tests in stable versions
References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com>
Message-ID: <loom.20110721T000244-533@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:

> New logging tests failed during some weeks. If we add new tests, we may 
> also break some stable buildbots. I don't think that we need to add 
> these new tests to a stable version.

Just for my information, which logging test failures are you referring to?

Regards,

Vinay Sajip


From v+python at g.nevcal.com  Thu Jul 21 00:09:49 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 15:09:49 -0700
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720131415.A2DD43A4100@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
Message-ID: <4E27522D.8010300@g.nevcal.com>

On 7/20/2011 6:05 AM, P.J. Eby wrote:
> At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote:
>> When I read about creating __path__ from sys.path, I immediately 
>> thought of the issue of programs that extend sys.path, and the above 
>> is the "workaround" for such programs.?  but it requires such 
>> programs to do work, and there are a lot of such programs (I, a 
>> relative newbie, have had to write some).?  As it turns out, I can't 
>> think of a situation where I have extended sys.path that would result 
>> in a problem for fancy namespace packages, because so far I've only 
>> written modules, not packages, and only modules are on the paths that 
>> I add to sys.path.?  But that does not make for a general solution.
>
> Most programs extend sys.path in order to import things.  If those 
> things aren't yet imported, they don't have a __path__ yet, and so 
> don't need to be fixed.  Only programs that modify sys.path *after* 
> importing something that has a dynamic __path__ would need to do 
> anything about that.

Sure.  But there are a lot of things already imported by Python itself, 
and if this mechanism gets used in the stdlib, a program wouldn't know 
whether it is safe or not, to not bother with the 
pkgutil.extend_virtual_paths() call or not.

Plus, that requires importing pkgutil, which isn't necessarily done by 
every program that extends the sys.path ("import sys" is sufficient at 
present).

Plus, if some 3rd party packages are imported before sys.path is 
extended, the knowledge of how they are implement is required to make a 
choice about whether it is needed to import pkgutil and call 
extend_virtual_paths  or not.

So I am still left with my original question:

>
>> Is there some way to create a new __path__ that would reflect the 
>> fact that it has been dynamically created, rather than set from 
>> __init__.py, and then when it is referenced, calculate (and cache?) a 
>> new value of __path__ to actually search?
>
> That's what extend_virtual_paths() is for.  It updates the __path__ of 
> all currently-imported virtual packages.  Where before you wrote:
>
>      sys.path.append('foo')
>
> You would now write:
>
>      sys.path.append('foo')
>      pkgutil.extend_virtual_paths('foo')
>
> ...assuming you have virtual packages you've already imported.  If you 
> don't, there's no reason to call extend_virtual_paths().  But it 
> doesn't hurt anything if you call it unnecessarily, because it uses 
> sys.virtual_packages to find out what to update, and if you haven't 
> imported any virtual packages, there's nothing to update and the call 
> will be a quick no-op.

I think I would have to write

sys.path.append('foo')
import pkgutil
pkgutil.extend_virtual_paths('foo')

or I'd get an error.

And, in the absence of knowing (because I didn't write them) whether any 
of the packages I imported before extending sys.path are virtual 
packages or not, I would have to do this every time I extend sys.path.  
And so it becomes a burden on writing programs.

If the code is so boilerplate as you describe, should sys.path become an 
object that acts like a list, instead of a list, and have its append 
method automatically do the pkgutil.extend_virtual_paths for me?  Then I 
wouldn't have to worry about whether any of the packages I imported were 
virtual packages or not.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/c9adcddd/attachment.html>

From brett at python.org  Thu Jul 21 00:16:12 2011
From: brett at python.org (Brett Cannon)
Date: Wed, 20 Jul 2011 15:16:12 -0700
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <j077ti$bql$1@dough.gmane.org>
References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com>
	<j077ti$bql$1@dough.gmane.org>
Message-ID: <CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>

On Wed, Jul 20, 2011 at 11:48, Terry Reedy <tjreedy at udel.edu> wrote:

> On 7/20/2011 12:25 PM, Victor Stinner wrote:
>
>> Le 20/07/2011 17:58, ?ric Araujo a ?crit :
>>
>>> Do we have a policy of not adding new test files to stable branches?
>>>
>> New logging tests failed during some weeks. If we add new tests, we may
>> also break some stable buildbots. I don't think that we need to add
>> these new tests to a stable version.
>>
>
> When bugs are fixed in stable branches, they are usually accompanied by
> tests that fail without the bugfix. I have understood the policy to be that
> new tests go into stable branches. Failure indicates a bug in either the
> not-really-so-stable branch or the test. In the latter case, remove the test
> everywhere until fixed. In the former case, either fix the bug in the stable
> branch immediately or open an issue and attach the test code (skipping the
> test needed stage) or just disable it and note on the issue that a fix patch
> should re-enable. The logging tests may have been exceptional some way


Right, but Eric is asking about new tests that do nothing more than improve
test coverage, not exercise a fix for a bug.

I say don't add new tests for the sake of coverage or adding new tests to
stable branches. Tests for bugfixes are practically required.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/b2671452/attachment-0001.html>

From ethan at stoneleaf.us  Thu Jul 21 00:51:25 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 20 Jul 2011 15:51:25 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E274C0F.40400@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>	<4E249960.9000109@g.nevcal.com>	<loom.20110718T230703-173@post.gmane.org>	<4E24AE66.5030305@g.nevcal.com>	<loom.20110719T014549-927@post.gmane.org>	<4E24D83B.8040100@g.nevcal.com>	<loom.20110719T031535-263@post.gmane.org>	<4E269D46.1010806@g.nevcal.com>	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com>
Message-ID: <4E275BED.4050901@stoneleaf.us>

Glenn Linderman wrote:
>   On 7/20/2011 7:19 AM, Vinay Sajip wrote:
>> It's not py's job to walk the path: the shell does that when you just type
>> "foo". It locates foo.py, and then invokes py because of file association - py
>> then checks the file for a shebang to decide which Python to dispatch it to.
> 
> Certainly when the launcher is invoked via an association, this would be 
> the case.  However, when the launcher is invoked via the command line, 
> then the unqualified name is passed through.  To be useful from the 
> command line, the launcher should walk the PATH to find the .py file.

I would say that would be a cool enhancement, as it could save a bit of 
typing, but I think the launcher is quite useful even without path 
traversal.

~Ethan~

From pje at telecommunity.com  Thu Jul 21 00:39:14 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 18:39:14 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CALFfu7AgEvEt0Mf_3m_vSafQ6SV_MzdUs6_4RD-VquQg1J7wMQ@mail.g
	mail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAF7AXFFZBLPXh_G=fArGLa=fDOvETx49OfEFxqYPzQm3pvfuOg@mail.gmail.com>
	<20110720170448.D7D793A409B@sparrow.telecommunity.com>
	<CALFfu7Dm6Qo37a=h8JL1ykaOWs7pT0JJtT+6brymRDwyzc=h2Q@mail.gmail.com>
	<20110720204456.5D1AB3A409B@sparrow.telecommunity.com>
	<CALFfu7AgEvEt0Mf_3m_vSafQ6SV_MzdUs6_4RD-VquQg1J7wMQ@mail.gmail.com>
Message-ID: <20110720223954.124953A409B@sparrow.telecommunity.com>

At 03:22 PM 7/20/2011 -0600, Eric Snow wrote:
>On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby <pje at telecommunity.com> wrote:
> >
> > So, yeah, actually, that's looking pretty sweet.  Basically, we 
> just have to
> > throw a virtual_package_paths dict into the sys module, and do the above
> > along with the get_virtual_path() function and add get_subpath() to the
> > importer objects, in order to get the PEP's core functionality working.
>
>
>Exactly.  That's part of why the importlib approach is so appealing to
>me.

Actually, it turns out I was a little too optimistic -- the sketch I 
gave doesn't work right for anything but top-level virtual packages, 
because I didn't take into account the part where get_virtual_path() 
needs a parent path.

Fixing *that* error then leads to a really nasty bit of mutual 
recursion in which the parent module imports are attempted over and 
over again in something like O(N**2), I think.  In order to get rid 
of that, _gcd_import would have to grow some internal memoization so 
it doesn't retry the same imports repeatedly.

Ironically enough, this is because _gcd_import() is recursive, and 
thus attempts the imports in the opposite order (sort of) than 
import.c does, which means that you can't get hold of the parent's 
__path__ without recursing (again).  :-(

And trying to work around that with memoization, led me to the 
realization that you actually can't implement PEP 402 using that type 
of recursion.  That is, to implement the spec correctly, _gcd_import 
is going to have to be refactored to iterate left-to-right over 
module name parts, rather than recursing right-to-left.

That's because PEP 402 only allows for processing a virtual path if a 
module is not found, *not* if a module is found but can't be loaded.

But, with importlib currently being recursive, it only knows that a 
parent import failed via ImportError, not whether that error arose 
from failing to find the module, or failing to load the module!

So, the core part of the _gcd_import() function will need to be 
rewritten to iterate instead of recursing.

(Still, it's probably not going to be *terribly* difficult.  I'll 
take a look at doing a sketch of that next, but if I do one I'll send 
it to Import-SIG instead of here; it's not a detail that matters to 
the general PEP discussion.)


From pje at telecommunity.com  Thu Jul 21 01:03:58 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Jul 2011 19:03:58 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <4E27522D.8010300@g.nevcal.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
	<4E27522D.8010300@g.nevcal.com>
Message-ID: <20110720230439.6ADE53A409B@sparrow.telecommunity.com>

At 03:09 PM 7/20/2011 -0700, Glenn Linderman wrote:
>On 7/20/2011 6:05 AM, P.J. Eby wrote:
>>At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote:
>>>When I read about creating __path__ from sys.path, I immediately 
>>>thought of the issue of programs that extend sys.path, and the 
>>>above is the "workaround" for such programs.???  but it requires 
>>>such programs to do work, and there are a lot of such programs (I, 
>>>a relative newbie, have had to write some).???  As it turns out, I 
>>>can't think of a situation where I have extended sys.path that 
>>>would result in a problem for fancy namespace packages, because so 
>>>far I've only written modules, not packages, and only modules are 
>>>on the paths that I add to sys.path.???  But that does not make 
>>>for a general solution.
>>
>>Most programs extend sys.path in order to import things.?  If those 
>>things aren't yet imported, they don't have a __path__ yet, and so 
>>don't need to be fixed.?  Only programs that modify sys.path 
>>*after* importing something that has a dynamic __path__ would need 
>>to do anything about that.
>
>Sure.  But there are a lot of things already imported by Python 
>itself, and if this mechanism gets used in the stdlib, a program 
>wouldn't know whether it is safe or not, to not bother with the 
>pkgutil.extend_virtual_paths() call or not.

I'm not sure I see how the mechanism could meaningfully be used in 
the stdlib, since IIUC we're not going for Perl-style package 
naming.  So, all stdlib packages would be self-contained.


>Plus, that requires importing pkgutil, which isn't necessarily done 
>by every program that extends the sys.path ("import sys" is 
>sufficient at present).
>
>Plus, if some 3rd party packages are imported before sys.path is 
>extended, the knowledge of how they are implement is required to 
>make a choice about whether it is needed to import pkgutil and call 
>extend_virtual_paths or not.

I'd recommend *always* using it, outside of simple startup code.


>So I am still left with my original question:
>
>>>Is there some way to create a new __path__ that would reflect the 
>>>fact that it has been dynamically created, rather than set from 
>>>__init__.py, and then when it is referenced, calculate (and 
>>>cache?) a new value of __path__ to actually search?

Hm.  Yes, there is a way to do something like that, but it would 
complicate things a bit.  We'd need to:

1. Leave __path__ off of the modules, and always pull them from 
sys.virtual_package_paths, and

2. Before using a value in sys.virtual_package_paths, we'd need to 
check whether sys.path had changed since we last cached anything, and 
if so, clear sys.virtual_package_paths first, to force a refresh.

This doesn't sound particularly forbidding, but there are various 
unpleasant consequences, like being unable to tell whether a module 
is a package or not, and whether it's a virtual package or not.  We'd 
have to invent new ways to denote these things.

On the bright side, though, it *would* allow transparent live updates 
to virtual package paths, so it might be worth considering.

By the way, the reason we have to get rid of __path__ is that if we 
kept it, then code could change it, and then we wouldn't know if it 
was actually safe to change it automatically...  even if no code had 
actually changed it.

In principle, we could keep __path__ attributes around, and 
automatically update them in the case where sys.path has changed, so 
long as user code hasn't directly altered or replaced the 
__path__.  But it seems to me to be a dangerous corner case; I'd 
rather that code which touches __path__ be taking responsibility for 
that path's correctness from then on, rather than having it get 
updated (possibly incorrectly) behind its back.

So, I'd say that for this approach, we'd have to actually leave 
__path__ off of virtual packages' parent modules.

Anyway, it seems worth considering.  We just need to sort out what 
the downsides are for any current tools thinking that such modules 
aren't packages.  (But hey, at least it'll be consistent with what 
such tools would think of the on-disk representation!  That is, a 
tool that thinks foo.py alongside a foo/ subdirectory is just a 
module with no package, will also think that 'foo', once imported, is 
a module with no package.)


>And, in the absence of knowing (because I didn't write them) whether 
>any of the packages I imported before extending sys.path are virtual 
>packages or not, I would have to do this every time I extend 
>sys.path.?  And so it becomes a burden on writing programs.
>
>If the code is so boilerplate as you describe, should sys.path 
>become an object that acts like a list, instead of a list, and have 
>its append method automatically do the pkgutil.extend_virtual_paths 
>for me??  Then I wouldn't have to worry about whether any of the 
>packages I imported were virtual packages or not.

Well, then we'd have to worry about other mutation methods, and 
things like 'sys.path = [blah, blah]', as well.  So if we're going to 
ditch the need for extend_virtual_paths(), we should probably do it 
via the absence of __path__ attributes.


From v+python at g.nevcal.com  Thu Jul 21 01:39:35 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 16:39:35 -0700
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720230439.6ADE53A409B@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
	<4E27522D.8010300@g.nevcal.com>
	<20110720230439.6ADE53A409B@sparrow.telecommunity.com>
Message-ID: <4E276737.8080104@g.nevcal.com>

On 7/20/2011 4:03 PM, P.J. Eby wrote:
> I'd recommend *always* using it, outside of simple startup code.

So that is a burden on every program.  Documentation would help, but it 
certainly makes updating sys.path much more complex -- 3 lines (counting 
import of pkgutil) instead of one, and the complexity of understanding 
why there is a need for it, when in simple cases the single line works 
fine, but it would be bug prone to have both ways.


>
>> So I am still left with my original question:
>>
>>>> Is there some way to create a new __path__ that would reflect the 
>>>> fact that it has been dynamically created, rather than set from 
>>>> __init__.py, and then when it is referenced, calculate (and cache?) 
>>>> a new value of __path__ to actually search?
>
> Hm.  Yes, there is a way to do something like that, but it would 
> complicate things a bit

 From what you said, it would complicate the solution for complex 
packaging tasks, but would return simple extensions of sys.path to being 
simple again.  Sounds like a good tradeoff, but I'll leave that to you 
and other more knowledgeable people to figure out the details and 
implementation... I snipped the explanation, because it is beyond my 
present knowledge base.

> Anyway, it seems worth considering.  We just need to sort out what the 
> downsides are for any current tools thinking that such modules aren't 
> packages.  (But hey, at least it'll be consistent with what such tools 
> would think of the on-disk representation!  That is, a tool that 
> thinks foo.py alongside a foo/ subdirectory is just a module with no 
> package, will also think that 'foo', once imported, is a module with 
> no package.)

Please consider it.  I think your initial proposal solves some problems, 
but a version that doesn't complicate the normal, simple, extension of 
sys.path would be a much better solution, so I am happy to hear that you 
have ideas in that regard.  Hopefully, they don't complicate things too 
much more.  So far, I haven't gotten my head around packages as they 
presently exist (this __init__.py stuff seems much more complex than the 
simplicity of Perl imports that I was used to, although I certainly like 
many things about Python better than Perl, and have switched 
whole-heartedly, although I  still have a fair bit of Perl code to port 
in the fullness of time).  I think your proposal here, although 
maintaining some amount of backward-compatibility may require complexity 
of implementation, can simplify the requirements for creating new 
packages, to the extent I understand it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/d28ac334/attachment.html>

From skippy.hammond at gmail.com  Thu Jul 21 01:41:09 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 09:41:09 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E274C0F.40400@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com>
Message-ID: <4E276795.9050404@gmail.com>

On 21/07/2011 7:43 AM, Glenn Linderman wrote:
...
>
> I still get the same behavior.  Is there any debugging output produced
> by py.exe that would tell what py.ini it looks in, and what the content
> is?  What diagnostic steps might I take to produce additional
> information that would help you (or me, along the way) analyze the
> issue?  I do not presently have a C compiler installed on this machine,
> however, unless it came along for a ride with something else.

It doesn't do exactly what you ask, but setting an environment variable 
PYLAUNCH_DEBUG to any value will cause some debug output to be generated 
to stdout.

Mark

From ncoghlan at gmail.com  Thu Jul 21 01:54:21 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Jul 2011 09:54:21 +1000
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>
References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com>
	<j077ti$bql$1@dough.gmane.org>
	<CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>
Message-ID: <CADiSq7d9RTjgM8=BH5WOpxQUT034SmL-AxjpEWsGwgdFUae+Tg@mail.gmail.com>

On Thu, Jul 21, 2011 at 8:16 AM, Brett Cannon <brett at python.org> wrote:
> I say don't add new tests for the sake of coverage or adding new tests to
> stable branches. Tests for bugfixes are practically required.

I don't *object* to enhanced tests going into maintenance branches,
but the workflow of committing directly to trunk is so much simpler
that I personally would only apply such changes if the new tests
actually uncovered implementation bugs. Then backporting the tests
would useful either as part of fixing the same bugs or else to
demonstrate that the maintenance branch did not have the problem.

So slightly more relaxed than Brett's view:
- definitely apply bug fixes and their tests to affected maintenance branches
- *optionally* apply coverage enhancements to maintenance branches,
but don't feel obliged to do so

I see it as a productivity trade-off - time spent improving coverage
on multiple branches is time not spent on other things. I'm more than
willing to sacrifice some test coverage on the maintenance branches to
get even better test coverage and appropriate new features on trunk
and more bug fixes on maintenance branches.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From skippy.hammond at gmail.com  Thu Jul 21 01:55:28 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 09:55:28 +1000
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <j077bn$855$1@dough.gmane.org>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>	<20110719171647.03f9f9c9@pitrou.net>	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org>
Message-ID: <4E276AF0.5090409@gmail.com>

On 21/07/2011 4:38 AM, Terry Reedy wrote:

> Many installers first make an organization directory and then an app
> directory within that. This annoys me sometimes when they only have one
> app to ever install, but is useful when there might really be multiple
> directories, as in our case. (Ditto for start menu entries.) This is
> what python should have done a decade ago.

I disagree.  If we followed that advice we would also be in "\Program 
Files".  I have no problem with multiple Python versions installed 
directly off the root, especially given most users probably have a very 
small number of such installations.  I think Python being a developer 
tool rather than a user app is a reasonable justification for that (and 
the justification used when the existing scheme was decided)

 > The two proposals
> overlap but are not mutually exclusive. For future pythons, 'python33'
> is easier to remember and type than 'py -v 3.3' or whatever the proposed
> encantation is.

'py -3.3' - less chars to type than 'python33' and with no need to have 
every Python directory on your PATH.  IMO it is also simple enough that 
people will remember it fairly easily.

Also, the launcher supports the ability to select either the 32 or 64bit 
implementation - so maybe 'python33.exe' isn't really good enough and 
should reflect the bittedness?

> A python directory also gives a sensible (though optional) place to put
> other interpreters and even python-based apps. The launcher does not.

What other interpreters?  IMO it doesn't make sense to have IronPython, 
jython etc be installed there.  Ditto for apps - especially given most 
apps tend to be tied to a subset of all possible Python versions.

Mark

From v+python at g.nevcal.com  Thu Jul 21 02:02:04 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 17:02:04 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E276795.9050404@gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
Message-ID: <4E276C7C.1020605@g.nevcal.com>

On 7/20/2011 4:41 PM, Mark Hammond wrote:
> On 21/07/2011 7:43 AM, Glenn Linderman wrote:
> ...
>>
>> I still get the same behavior.  Is there any debugging output produced
>> by py.exe that would tell what py.ini it looks in, and what the content
>> is?  What diagnostic steps might I take to produce additional
>> information that would help you (or me, along the way) analyze the
>> issue?  I do not presently have a C compiler installed on this machine,
>> however, unless it came along for a ride with something else.
>
> It doesn't do exactly what you ask, but setting an environment 
> variable PYLAUNCH_DEBUG to any value will cause some debug output to 
> be generated to stdout.
>
> Mark
>

It produces:

d:\>py d:\path\to\foo.py
launcher build: 64bit
launcher executable: Console
File 'C:\Users\Glenn\AppData\Roaming\py.ini' non-existent
maybe_handle_shebang: read 256 bytes
maybe_handle_shebang: BOM not found, using UTF-8
locate_pythons_for_key: unable to open PythonCore key
locate_pythons_for_key: unable to open PythonCore key
run_child: about to run 'C:\Python26\python.exe d:\path\to\foo.py'
   File "d:\path\to\foo.py", line 469
child process exit code: 1

So this tells me that it didn't find a local py.ini (no surprise, I 
don't have one) but doesn't tell me that it did find or read 
c:\Windows\system32\py.ini much less whether I have the syntax correct 
for my [defaults] section.  It doesn't tell me that it didn't find a #! 
line (but there isn't one).

Perhaps the problem is that there isn't one?  If it finds no virtual 
command, maybe it doesn't obey the [defaults] python=3 directive?  The 
PEP says it should act like '!#python' (I think the PEP meant 
'#!python', though????).  There is no " " after '!#python' in the PEP, 
so that disqualifies it from being a customized command, there is no 
'/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means 
it is not a virtual command; and it is not a fully-qualified name, so it 
doesn't mean that case either.... looks like the PEP needs a bit of 
clarification here.

I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and 
it runs 2.6.4 as the output shows.  And I would expect it to run 3.2.1 
with the [defaults] python=3 directive, since that is newer than 3.1.2, 
which is on my PATH.

So, still a mystery.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/3705e75e/attachment.html>

From ncoghlan at gmail.com  Thu Jul 21 02:07:31 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Jul 2011 10:07:31 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E275BED.4050901@stoneleaf.us>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
Message-ID: <CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>

On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> I would say that would be a cool enhancement, as it could save a bit of
> typing, but I think the launcher is quite useful even without path
> traversal.

Two related points:

1. Walking PATH isn't necessary, but the cwd of the py process should
be inherited from the shell correctly. If it is, then 'py foo.py'
shouldn't need path traversal, it should just look in the current
directory. Using PATHEXT to turn 'foo.py' directly into an executable
command on PATH from any directory is different and out of scope for
the launcher.

2. The defined launched command line handling means that "py -m foo"
should also work, so long as the current directory is inherited
correctly.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From solipsis at pitrou.net  Thu Jul 21 02:08:56 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 21 Jul 2011 02:08:56 +0200
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org> <4E276AF0.5090409@gmail.com>
Message-ID: <20110721020856.2b38528b@pitrou.net>

On Thu, 21 Jul 2011 09:55:28 +1000
Mark Hammond <skippy.hammond at gmail.com> wrote:
> 
>  > The two proposals
> > overlap but are not mutually exclusive. For future pythons, 'python33'
> > is easier to remember and type than 'py -v 3.3' or whatever the proposed
> > encantation is.
> 
> 'py -3.3' - less chars to type than 'python33' and with no need to have 
> every Python directory on your PATH.  IMO it is also simple enough that 
> people will remember it fairly easily.

Given that Python 2.x has a "-3" option, isn't "py -3.3" kind of
confusing, at least to the eye?

Regards

Antoine.



From skippy.hammond at gmail.com  Thu Jul 21 02:11:43 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 10:11:43 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E276C7C.1020605@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
	<4E276C7C.1020605@g.nevcal.com>
Message-ID: <4E276EBF.6000509@gmail.com>

On 21/07/2011 10:02 AM, Glenn Linderman wrote:
> So this tells me that it didn't find a local py.ini (no surprise, I
> don't have one) but doesn't tell me that it did find or read
> c:\Windows\system32\py.ini much less whether I have the syntax correct
> for my [defaults] section.  It doesn't tell me that it didn't find a #!
> line (but there isn't one).

I'll have a go at enhancing the debug output for most of the above 
(although note that if it *did* find a shebang line extra output would 
have been generated.)

> Perhaps the problem is that there isn't one?  If it finds no virtual
> command, maybe it doesn't obey the [defaults] python=3 directive?  The
> PEP says it should act like '!#python' (I think the PEP meant
> '#!python', though????).  There is no " " after '!#python' in the PEP,
> so that disqualifies it from being a customized command, there is no
> '/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means
> it is not a virtual command; and it is not a fully-qualified name, so it
> doesn't mean that case either.... looks like the PEP needs a bit of
> clarification here.

I'm not sure the PEP needs clarification - possibly just the 
implementation ;)  But let me know if you think otherwise.

> I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and
> it runs 2.6.4 as the output shows.

FYI, what is on the path isn't relevant to the launcher.

> And I would expect it to run 3.2.1
> with the [defaults] python=3 directive, since that is newer than 3.1.2,
> which is on my PATH.

It may be that your copy of the launcher is a little old - some changes 
were pushed just yesterday (but I'm not sure if Vinay made a new 
installer yet).  It has slightly better debug output (although generally 
not what you are asking for yet) and better "cross-bittedness" support.

Cheers,

Mark

From v+python at g.nevcal.com  Thu Jul 21 02:13:57 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 17:13:57 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E274C0F.40400@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com>
Message-ID: <4E276F45.7070105@g.nevcal.com>

On 7/20/2011 2:43 PM, Glenn Linderman wrote:
>> It's not py's job to walk the path: the shell does that when you just type
>> "foo". It locates foo.py, and then invokes py because of file association - py
>> then checks the file for a shebang to decide which Python to dispatch it to.
>
> Certainly when the launcher is invoked via an association, this would 
> be the case.  However, when the launcher is invoked via the command 
> line, then the unqualified name is passed through.  To be useful from 
> the command line, the launcher should walk the PATH to find the .py file.

The Windows SearchPath API 
<http://msdn.microsoft.com/en-us/library/aa365527%28v=VS.85%29.aspx> 
makes it a pretty easy job to walk the PATH.  Would even allow low cost 
additional feature of searching for both   foo and foo.py  at the same time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/67faf951/attachment.html>

From skippy.hammond at gmail.com  Thu Jul 21 02:17:44 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 10:17:44 +1000
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <20110721020856.2b38528b@pitrou.net>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org> <4E276AF0.5090409@gmail.com>
	<20110721020856.2b38528b@pitrou.net>
Message-ID: <4E277028.4040202@gmail.com>

On 21/07/2011 10:08 AM, Antoine Pitrou wrote:
> On Thu, 21 Jul 2011 09:55:28 +1000
> Mark Hammond<skippy.hammond at gmail.com>  wrote:
>>
>>   >  The two proposals
>>> overlap but are not mutually exclusive. For future pythons, 'python33'
>>> is easier to remember and type than 'py -v 3.3' or whatever the proposed
>>> encantation is.
>>
>> 'py -3.3' - less chars to type than 'python33' and with no need to have
>> every Python directory on your PATH.  IMO it is also simple enough that
>> people will remember it fairly easily.
>
> Given that Python 2.x has a "-3" option, isn't "py -3.3" kind of
> confusing, at least to the eye?

A little, yeah, but IMO practicality beats purity here.  I'd probably 
feel different if I felt 'python -3' was regularly used and would 
continue to be so in the future.  Also, I think most people who would 
potentially use 'python -3' will be aware that running 'py' is a totally 
different command and will adjust accordingly (either by continuing to 
use 'python -3' or adjusting to running 'py -2 -3'.)

The PEP does make explicit mention of this...

Cheers,

Mark

From v+python at g.nevcal.com  Thu Jul 21 02:18:33 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 17:18:33 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
	<CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
Message-ID: <4E277059.3080301@g.nevcal.com>

On 7/20/2011 5:07 PM, Nick Coghlan wrote:
> On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman<ethan at stoneleaf.us>  wrote:
>> I would say that would be a cool enhancement, as it could save a bit of
>> typing, but I think the launcher is quite useful even without path
>> traversal.
> Two related points:
>
> 1. Walking PATH isn't necessary, but the cwd of the py process should
> be inherited from the shell correctly. If it is, then 'py foo.py'
> shouldn't need path traversal, it should just look in the current
> directory. Using PATHEXT to turn 'foo.py' directly into an executable
> command on PATH from any directory is different and out of scope for
> the launcher.

Sorry, I disagree that it is out of scope.  Looking in the current 
directory is fine, when the script is there, but my scripts are seldom 
in my data directories, and I want to run scripts (from the PATH) on 
data that is in the CWD.  I consider this a _very common_ use case for 
using scripts/programs, but then if you want to use py from the command 
line to tweak which version of Python gets used to execute the script 
(if the default one didn't work, for example, and you want to try a 
different one), then suddenly, you have to find the path to the script, 
and specify it explicitly.

> 2. The defined launched command line handling means that "py -m foo"
> should also work, so long as the current directory is inherited
> correctly.

Even if CWD is a data directory, and foo.py is somewhere on the path?

> Cheers,
> Nick.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/5ff610b8/attachment.html>

From v+python at g.nevcal.com  Thu Jul 21 02:22:03 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Jul 2011 17:22:03 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E276EBF.6000509@gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
	<4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com>
Message-ID: <4E27712B.2030704@g.nevcal.com>

On 7/20/2011 5:11 PM, Mark Hammond wrote:
> On 21/07/2011 10:02 AM, Glenn Linderman wrote:
>> So this tells me that it didn't find a local py.ini (no surprise, I
>> don't have one) but doesn't tell me that it did find or read
>> c:\Windows\system32\py.ini much less whether I have the syntax correct
>> for my [defaults] section.  It doesn't tell me that it didn't find a #!
>> line (but there isn't one).
>
> I'll have a go at enhancing the debug output for most of the above 
> (although note that if it *did* find a shebang line extra output would 
> have been generated.)
>
>> Perhaps the problem is that there isn't one?  If it finds no virtual
>> command, maybe it doesn't obey the [defaults] python=3 directive?  The
>> PEP says it should act like '!#python' (I think the PEP meant
>> '#!python', though????).  There is no " " after '!#python' in the PEP,
>> so that disqualifies it from being a customized command, there is no
>> '/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means
>> it is not a virtual command; and it is not a fully-qualified name, so it
>> doesn't mean that case either.... looks like the PEP needs a bit of
>> clarification here.
>
> I'm not sure the PEP needs clarification - possibly just the 
> implementation ;)  But let me know if you think otherwise.

Well, at least !# should be changed to #! :)  And then the example of 
"customized command" shows "#! vpython" with the space before the 
vpython, but the text describes the necessary space as being after the 
customized command, at least the way I read it.  So I really don't know 
whether the example is showing an extraneous space after the ! (and not 
one after vpython) or exactly what is meant.

>
>> I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and
>> it runs 2.6.4 as the output shows.
>
> FYI, what is on the path isn't relevant to the launcher.

I didn't expect it to be, I just mentioned it in passing, because the 
launcher doesn't seem to be doing what I expect it to do, but neither 
does it seem to be using the PATH to find a Python either.

>
>> And I would expect it to run 3.2.1
>> with the [defaults] python=3 directive, since that is newer than 3.1.2,
>> which is on my PATH.
>
> It may be that your copy of the launcher is a little old - some 
> changes were pushed just yesterday (but I'm not sure if Vinay made a 
> new installer yet).  It has slightly better debug output (although 
> generally not what you are asking for yet) and better 
> "cross-bittedness" support.
>
> Cheers,
>
> Mark
>

Mine's a week old, yes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/f8c34773/attachment.html>

From skippy.hammond at gmail.com  Thu Jul 21 02:34:34 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 10:34:34 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E276F45.7070105@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com>
Message-ID: <4E27741A.1040202@gmail.com>

On 21/07/2011 10:13 AM, Glenn Linderman wrote:
> On 7/20/2011 2:43 PM, Glenn Linderman wrote:
>>> It's not py's job to walk the path: the shell does that when you just type
>>> "foo". It locates foo.py, and then invokes py because of file association - py
>>> then checks the file for a shebang to decide which Python to dispatch it to.
>>
>> Certainly when the launcher is invoked via an association, this would
>> be the case.  However, when the launcher is invoked via the command
>> line, then the unqualified name is passed through.  To be useful from
>> the command line, the launcher should walk the PATH to find the .py file.
>
> The Windows SearchPath API
> <http://msdn.microsoft.com/en-us/library/aa365527%28v=VS.85%29.aspx>
> makes it a pretty easy job to walk the PATH.  Would even allow low cost
> additional feature of searching for both   foo and foo.py  at the same time.

But that would also require us to parse the command-line, understand 
which options require 2 args and which just require 1 (making it fragile 
in the face of later versions introducing new options), then 
re-stitching a modified command-line back together for the child 
process.  IMO that is all out of scope.

Are you maybe forgetting about the PATHEXT functionality?  If you 
include .py in your PATHEXT and file associations are set so .py files 
are handled by the launcher, you should be able to directly execute just 
'foo' and have the command processor search the PATH for a foo.py and 
invoke it via the launcher.

Mark

From skippy.hammond at gmail.com  Thu Jul 21 02:27:31 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 10:27:31 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E277059.3080301@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
	<CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
	<4E277059.3080301@g.nevcal.com>
Message-ID: <4E277273.5060508@gmail.com>

On 21/07/2011 10:18 AM, Glenn Linderman wrote:
> On 7/20/2011 5:07 PM, Nick Coghlan wrote:
>> On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman<ethan at stoneleaf.us>  wrote:
>>> I would say that would be a cool enhancement, as it could save a bit of
>>> typing, but I think the launcher is quite useful even without path
>>> traversal.
>> Two related points:
>>
>> 1. Walking PATH isn't necessary, but the cwd of the py process should
>> be inherited from the shell correctly. If it is, then 'py foo.py'
>> shouldn't need path traversal, it should just look in the current
>> directory. Using PATHEXT to turn 'foo.py' directly into an executable
>> command on PATH from any directory is different and out of scope for
>> the launcher.
>
> Sorry, I disagree that it is out of scope.  Looking in the current
> directory is fine, when the script is there, but my scripts are seldom
> in my data directories, and I want to run scripts (from the PATH) on
> data that is in the CWD.  I consider this a _very common_ use case for
> using scripts/programs, but then if you want to use py from the command
> line to tweak which version of Python gets used to execute the script
> (if the default one didn't work, for example, and you want to try a
> different one), then suddenly, you have to find the path to the script,
> and specify it explicitly.

The arguments above apply equally to python.exe.  The launcher's job is 
to find an appropriate python.exe and launch it, not to locate the 
scripts and all the command-line parsing that would entail.  If you want 
this feature you should advocate for it to be added to Python itself and 
it will then automatically work in the launcher too.

Mark

From ben+python at benfinney.id.au  Thu Jul 21 03:31:21 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 21 Jul 2011 11:31:21 +1000
Subject: [Python-Dev] Indentation in reStructuredText documents (was:
	[Python-checkins] peps: Restore whitespace characters lost
	via email transmission.)
References: <E1QjXN2-000859-6x@dinsdale.python.org> <4E26E468.20800@netwok.org>
Message-ID: <87livss9li.fsf_-_@benfinney.id.au>

?ric Araujo <merwok at netwok.org> writes:

> FYI, reST uses three-space indents, not four (so that blocks align
> nicely under the leading two dots + one space), so I think the change
> was intentional.

No, reST doesn't specify any particular level of indentation. Like most
Python programmers I prefer four-space indentation, so I do the same in
my reST documents::

    ..  epigraph::
        ?Foo bar baz? ?Fnorble

I strongly recommend the same for Python documentation.

-- 
 \     ?I was in the first submarine. Instead of a periscope, they had |
  `\               a kaleidoscope. ?We're surrounded.?? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From ncoghlan at gmail.com  Thu Jul 21 03:52:23 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Jul 2011 11:52:23 +1000
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720230439.6ADE53A409B@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
	<4E27522D.8010300@g.nevcal.com>
	<20110720230439.6ADE53A409B@sparrow.telecommunity.com>
Message-ID: <CADiSq7cNZ=ypGK--YfVa2xnkCMq5yM3VWdMwY5FUyZtOFYBsAA@mail.gmail.com>

On Thu, Jul 21, 2011 at 9:03 AM, P.J. Eby <pje at telecommunity.com> wrote:
> Hm. ?Yes, there is a way to do something like that, but it would complicate
> things a bit. ?We'd need to:
>
> 1. Leave __path__ off of the modules, and always pull them from
> sys.virtual_package_paths, and

Setting __path__ to a sentinel value (imp.VirtualPath?) would break
less code, as hasattr(mod, '__path__') checks would still work.

Even better would be for these (and sys.path) to be list subclasses
that did the right thing under the hood as Glenn suggested. Code that
*replaces* rather than modifies these attributes would still
potentially break virtual packages, but code that modifies them in
place would do the right thing automatically. (Note that all code that
manipulates sys.path and __path__ attributes requires explicit calls
to correctly support current namespace package mechanisms, so this
would actually be an improvement on the status quo rather than making
anything worse).

I'll note that this kind of thing is one of the key reasons the import
state should some day move to a real class - state coherency is one of
the major use cases for the descriptor protocol, which is unavailable
when interdependent state is stored as module attributes. (Don't
worry, that day is a very long way away, if it ever happens at all)

> 2. Before using a value in sys.virtual_package_paths, we'd need to check
> whether sys.path had changed since we last cached anything, and if so, clear
> sys.virtual_package_paths first, to force a refresh.
>
> This doesn't sound particularly forbidding, but there are various unpleasant
> consequences, like being unable to tell whether a module is a package or
> not, and whether it's a virtual package or not. ?We'd have to invent new
> ways to denote these things.

Trying to change how packages are identified at the Python level makes
PEP 382 sound positively appealing. __path__ needs to stay :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From brian.curtin at gmail.com  Thu Jul 21 04:24:38 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 20 Jul 2011 21:24:38 -0500
Subject: [Python-Dev] Indentation in reStructuredText documents (was:
 [Python-checkins] peps: Restore whitespace characters lost via email
 transmission.)
In-Reply-To: <87livss9li.fsf_-_@benfinney.id.au>
References: <E1QjXN2-000859-6x@dinsdale.python.org> <4E26E468.20800@netwok.org>
	<87livss9li.fsf_-_@benfinney.id.au>
Message-ID: <CAD+XWwrhPmZ=c+T41dqCzTHZXuAcWRiVURmo_5DWd6SWbn-QCQ@mail.gmail.com>

On Wed, Jul 20, 2011 at 20:31, Ben Finney <ben+python at benfinney.id.au>wrote:

> ?ric Araujo <merwok at netwok.org> writes:
>
> > FYI, reST uses three-space indents, not four (so that blocks align
> > nicely under the leading two dots + one space), so I think the change
> > was intentional.
>
> No, reST doesn't specify any particular level of indentation. Like most
> Python programmers I prefer four-space indentation, so I do the same in
> my reST documents::
>
>    ..  epigraph::
>        ?Foo bar baz? ?Fnorble
>
> I strongly recommend the same for Python documentation.
>

We already use three and it seems to look and function properly.

-1 to a mass re-spacing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/f421298d/attachment.html>

From ericsnowcurrently at gmail.com  Thu Jul 21 04:39:19 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 20 Jul 2011 20:39:19 -0600
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <CADiSq7cNZ=ypGK--YfVa2xnkCMq5yM3VWdMwY5FUyZtOFYBsAA@mail.gmail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
	<4E27522D.8010300@g.nevcal.com>
	<20110720230439.6ADE53A409B@sparrow.telecommunity.com>
	<CADiSq7cNZ=ypGK--YfVa2xnkCMq5yM3VWdMwY5FUyZtOFYBsAA@mail.gmail.com>
Message-ID: <CALFfu7AYXza5Wz4xc+9xDY9cooym+jUC8H137-fre8HwURZrqg@mail.gmail.com>

On Wed, Jul 20, 2011 at 7:52 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Even better would be for these (and sys.path) to be list subclasses
> that did the right thing under the hood as Glenn suggested. Code that
> *replaces* rather than modifies these attributes would still
> potentially break virtual packages, but code that modifies them in
> place would do the right thing automatically. (Note that all code that
> manipulates sys.path and __path__ attributes requires explicit calls
> to correctly support current namespace package mechanisms, so this
> would actually be an improvement on the status quo rather than making
> anything worse).

+1 as a solution to the problem Glenn brought up.  However, I'm still
not clear on how much code out there changes sys.path in the offending
way, forcing the need to provide a more implicit solution in this PEP
than extend_virtual_paths().  And in cases where sys.path *is*
changed, and it impacts some virtual package, how many places is that
going to happen in one project?  My guess is not many (and so not many
"boilerplate" calls).  Is it worth adding implicit __path__ updates
for that use case, rather than just the extend_virtual_paths()
function?

As an aside, my first reaction to Glenn's suggestion was "that would
be cool".  Would it be a pursuable option?  We can take this over to
import-sig if it is.

-eric

From stephen at xemacs.org  Thu Jul 21 04:45:10 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 21 Jul 2011 11:45:10 +0900
Subject: [Python-Dev] [Python-checkins] cpython: #665194:
	support	roundtripping RFC2822 date stamps in the email.utils module
In-Reply-To: <20110720215103.DD85B25008E@webabinitio.net>
References: <E1QjYu7-0001X3-Rc@dinsdale.python.org> <4E271A9A.1040703@udel.edu>
	<20110720224017.4a2ddf83@pitrou.net>
	<20110720215103.DD85B25008E@webabinitio.net>
Message-ID: <87aac8s66h.fsf@uwakimon.sk.tsukuba.ac.jp>

R. David Murray writes:

 > Hexlify, wormaround...our Barry is just full of interesting words :)

Not "full" at all---there's no there there to put them in.

He's a generator!

The FLUFL-always-uses-efficient-idioms-ly y'rs,

From rdmurray at bitdance.com  Thu Jul 21 04:52:15 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 20 Jul 2011 22:52:15 -0400
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E277273.5060508@gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
	<CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
	<4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com>
Message-ID: <20110721025216.63DCF2500D6@webabinitio.net>

On Thu, 21 Jul 2011 10:27:31 +1000, Mark Hammond <skippy.hammond at gmail.com> wrote:
> On 21/07/2011 10:18 AM, Glenn Linderman wrote:
> > On 7/20/2011 5:07 PM, Nick Coghlan wrote:
> >> On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman<ethan at stoneleaf.us>  wrote:
> >>> I would say that would be a cool enhancement, as it could save a bit of
> >>> typing, but I think the launcher is quite useful even without path
> >>> traversal.
> >> Two related points:
> >>
> >> 1. Walking PATH isn't necessary, but the cwd of the py process should
> >> be inherited from the shell correctly. If it is, then 'py foo.py'
> >> shouldn't need path traversal, it should just look in the current
> >> directory. Using PATHEXT to turn 'foo.py' directly into an executable
> >> command on PATH from any directory is different and out of scope for
> >> the launcher.
> >
> > Sorry, I disagree that it is out of scope.  Looking in the current
> > directory is fine, when the script is there, but my scripts are seldom
> > in my data directories, and I want to run scripts (from the PATH) on
> > data that is in the CWD.  I consider this a _very common_ use case for
> > using scripts/programs, but then if you want to use py from the command
> > line to tweak which version of Python gets used to execute the script
> > (if the default one didn't work, for example, and you want to try a
> > different one), then suddenly, you have to find the path to the script,
> > and specify it explicitly.
> 
> The arguments above apply equally to python.exe.  The launcher's job is 
> to find an appropriate python.exe and launch it, not to locate the 
> scripts and all the command-line parsing that would entail.  If you want 
> this feature you should advocate for it to be added to Python itself and 
> it will then automatically work in the launcher too.

Indeed.  If I want to run a script with a different python version
on a unix-like system, I need to know the path to said script.
We're trying to make python as easy to use on Windows as it is on Unix.
If find-script-on-path is considered a worthwhile feature, then as Mark
said it should be added to base Python (on all platforms), not special-cased
in the Windows launcher.

--
R. David Murray           http://www.bitdance.com

From ncoghlan at gmail.com  Thu Jul 21 05:22:11 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Jul 2011 13:22:11 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <20110721025216.63DCF2500D6@webabinitio.net>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
	<CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
	<4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com>
	<20110721025216.63DCF2500D6@webabinitio.net>
Message-ID: <CADiSq7fCQO-Geg1pqbHcZhDoSXDOTvFkvMQ6VjHJ-faueicCig@mail.gmail.com>

On Thu, Jul 21, 2011 at 12:52 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> Indeed. ?If I want to run a script with a different python version
> on a unix-like system, I need to know the path to said script.
> We're trying to make python as easy to use on Windows as it is on Unix.
> If find-script-on-path is considered a worthwhile feature, then as Mark
> said it should be added to base Python (on all platforms), not special-cased
> in the Windows launcher.

And given the diverse range of what Python considers to be an
executable script these days, -1000 to that particular feature. Use
`which scriptname`, that's what it's for. The lack of such
functionality in the underpowered cmd shell on Windows isn't Python's
problem to solve - ask MS for a better shell and command line
utilities (assuming Powershell doesn't already offer something
comparable to 'which').

There are reasons I only code specifically for Windows if someone pays
me to do so :P

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From lekmalek at gmail.com  Thu Jul 21 08:05:52 2011
From: lekmalek at gmail.com (lekmalek)
Date: Thu, 21 Jul 2011 08:05:52 +0200
Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any
 callable object - request commiter
In-Reply-To: <CAP1=2W7w9rc1E1=umrFwUoNea8rrjd8ZK_gaFEc1rHeyJs_tfg@mail.gmail.com>
References: <20110716103322.330f805c@vimes>
	<CAP1=2W7w9rc1E1=umrFwUoNea8rrjd8ZK_gaFEc1rHeyJs_tfg@mail.gmail.com>
Message-ID: <20110721080552.40e767a1@vimes>

On Sun, 17 Jul 2011 19:19:59 -0700
Brett Cannon <brett at python.org> wrote:

> Just so people know, I went ahead and fixed this for 3.3 (but not for
> 3.2 since it changes the API in a subtle way).
Yeah, but that shouldn't break anything.

Anyway, thanks a lot for your help, I'm happy it's in 3.3.

lekma

> 
> On Sat, Jul 16, 2011 at 01:33, lekmalek <lekmalek at gmail.com> wrote:
> 
> > Hello all,
> >
> > Can any of you core devs have a look at
> > http://bugs.python.org/issue10271. It seems Brett is really busy
> > right now and this uncontroversial (AFAICT) one liner only needs
> > someone to review it and commit it. The pb is, it's holding me back
> > a little bit, and I really would like to have it in the next 3.2
> > release if possible.
> >
> > Thanks for your help,
> >
> > lekma
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/brett%40python.org
> >


From skippy.hammond at gmail.com  Thu Jul 21 08:35:38 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 16:35:38 +1000
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
Message-ID: <4E27C8BA.8040000@gmail.com>

I've updated PEP 397 - "Python launcher for Windows" based on recent 
discussions and Vinay's implementation.

http://www.python.org/dev/peps/pep-0397/ and a copy is attached for your 
convenience.

The main changes are:

* All mentions of the Python reference implementation have been removed 
now that a C implementation exists and is a more accurate implementation 
of the PEP than the Python version.

* Some "implementation details" have been removed and links added to the 
launcher docs.  This was done mainly so the implementation is free to 
make changes based on user feedback and still stay true to the PEP. 
Note that the launcher doc link doesn't exist right now, but should 
magically appear over the next couple of days as Vinay pushes some 
changes I just sent him.

* The recommendation to install into System32 has been changed to a 
recommendation to install directly into \Windows, as the System32 
directory is not on the default path for 32bit processes on a 64bit 
Windows.  Vinay even has a version of an MSI installer which installs 
into this directory.  The PEP also gives installers more leeway on where 
to install the launcher if this directory is not writable.

I think this PEP is getting close to being finalized - please suggest 
whatever changes you feel are necessary ASAP as soon I'll be asking for 
pronouncement.

Thanks - especially to Vinay for taking on the C implementation!

Mark
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pep-0397.txt
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/16279753/attachment.txt>

From raymond.hettinger at gmail.com  Thu Jul 21 08:58:14 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Wed, 20 Jul 2011 23:58:14 -0700
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>
References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com>
	<j077ti$bql$1@dough.gmane.org>
	<CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>
Message-ID: <AEA2ACFB-E9A7-455B-8686-53B64EF80CB5@gmail.com>


On Jul 20, 2011, at 3:16 PM, Brett Cannon wrote:

> 
> 
> On Wed, Jul 20, 2011 at 11:48, Terry Reedy <tjreedy at udel.edu> wrote:
> On 7/20/2011 12:25 PM, Victor Stinner wrote:
> Le 20/07/2011 17:58, ?ric Araujo a ?crit :
> Do we have a policy of not adding new test files to stable branches?
> New logging tests failed during some weeks. If we add new tests, we may
> also break some stable buildbots. I don't think that we need to add
> these new tests to a stable version.
> 
> When bugs are fixed in stable branches, they are usually accompanied by tests that fail without the bugfix. I have understood the policy to be that new tests go into stable branches. Failure indicates a bug in either the not-really-so-stable branch or the test. In the latter case, remove the test everywhere until fixed. In the former case, either fix the bug in the stable branch immediately or open an issue and attach the test code (skipping the test needed stage) or just disable it and note on the issue that a fix patch should re-enable. The logging tests may have been exceptional some way
> 
> Right, but Eric is asking about new tests that do nothing more than improve test coverage, not exercise a fix for a bug.
> 
> I say don't add new tests for the sake of coverage or adding new tests to stable branches. Tests for bugfixes are practically required.

I concur with Brett.   Nothing good will come from backporting tests that aren't aimed at a specific bugfix.


Raymond

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/31b6f588/attachment-0001.html>

From v+python at g.nevcal.com  Thu Jul 21 10:06:57 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 01:06:57 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <CADiSq7fCQO-Geg1pqbHcZhDoSXDOTvFkvMQ6VjHJ-faueicCig@mail.gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
	<CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
	<4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com>
	<20110721025216.63DCF2500D6@webabinitio.net>
	<CADiSq7fCQO-Geg1pqbHcZhDoSXDOTvFkvMQ6VjHJ-faueicCig@mail.gmail.com>
Message-ID: <4E27DE21.4050304@g.nevcal.com>

On 7/20/2011 8:22 PM, Nick Coghlan wrote:
> On Thu, Jul 21, 2011 at 12:52 PM, R. David Murray<rdmurray at bitdance.com>  wrote:
>> Indeed.  If I want to run a script with a different python version
>> on a unix-like system, I need to know the path to said script.
>> We're trying to make python as easy to use on Windows as it is on Unix.
>> If find-script-on-path is considered a worthwhile feature, then as Mark
>> said it should be added to base Python (on all platforms), not special-cased
>> in the Windows launcher.
> And given the diverse range of what Python considers to be an
> executable script these days, -1000 to that particular feature. Use
> `which scriptname`, that's what it's for. The lack of such
> functionality in the underpowered cmd shell on Windows isn't Python's
> problem to solve - ask MS for a better shell and command line
> utilities (assuming Powershell doesn't already offer something
> comparable to 'which').
>
> There are reasons I only code specifically for Windows if someone pays
> me to do so :P

Interesting feedback.  Well, I have a "which" program on my machine, but 
as a 32-bit executable, it won't find py in the 64-bit 
c:\windows\system32 directory!  Another good reason to demand pay for 
Windows programming.  There are some interesting gotchas to the way 32- 
vs 64-bit "compatibility" is achieved in Windows (groan).  I'll find or 
write a better one, in due time.  Meantime, the launcher testing has 
been a good learning exercise for me.

Interesting, David, that you feel it that Python usability on Windows 
should be limited to its usability on Unix, rather than to exceed it. I 
don't see that as a necessary or appropriate limit.  Windows and Unix 
are different.  Unix people are accustomed to using tools like which, 
and using command lines, and path manipulations; Windows people are 
not.  So the use of the command line is already somewhat foreign to 
them, and limiting the launcher so that they have to use other command 
line tools to get the work done, would only serve to frustrate them.  
Now the argument can possibly be made that people that want to use 
launcher from the command line would be those that are already command 
line experts might be realistic, but I will note that Perl has a -S 
option to find its script on the PATH, not that that is a sufficient 
reason to add such to Python, or even to the launcher, but just to note 
that there are at least some people besides myself that might think that 
is a friendly idea.

My goal in writing software is to make it easy to use, regardless of 
whether some other system should be the responsible party or not -- 
especially when I don't control the other system.  But then, I haven't 
found time to write a competing launcher, either, with friendlier 
features.  So, I'll just reiterate that I would find it friendly if the 
launcher searched the path to find the script, and agree that if Python 
had a feature to do so, that would also be friendly to the Windows 
platform, but less necessary on Unix where you can  `which script.py` 
(although that would still require more typing than if the path 
searching were automatic.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/dbfa504b/attachment.html>

From v+python at g.nevcal.com  Thu Jul 21 10:13:56 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 01:13:56 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E27741A.1040202@gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com>
	<4E27741A.1040202@gmail.com>
Message-ID: <4E27DFC4.1000801@g.nevcal.com>

On 7/20/2011 5:34 PM, Mark Hammond wrote:
> On 21/07/2011 10:13 AM, Glenn Linderman wrote:
>> On 7/20/2011 2:43 PM, Glenn Linderman wrote:
>>>> It's not py's job to walk the path: the shell does that when you 
>>>> just type
>>>> "foo". It locates foo.py, and then invokes py because of file 
>>>> association - py
>>>> then checks the file for a shebang to decide which Python to 
>>>> dispatch it to.
>>>
>>> Certainly when the launcher is invoked via an association, this would
>>> be the case.  However, when the launcher is invoked via the command
>>> line, then the unqualified name is passed through.  To be useful from
>>> the command line, the launcher should walk the PATH to find the .py 
>>> file.
>>
>> The Windows SearchPath API
>> <http://msdn.microsoft.com/en-us/library/aa365527%28v=VS.85%29.aspx>
>> makes it a pretty easy job to walk the PATH.  Would even allow low cost
>> additional feature of searching for both   foo and foo.py  at the 
>> same time.
>
> But that would also require us to parse the command-line, understand 
> which options require 2 args and which just require 1 (making it 
> fragile in the face of later versions introducing new options), then 
> re-stitching a modified command-line back together for the child 
> process.  IMO that is all out of scope.

Yes it would be more work.

>
> Are you maybe forgetting about the PATHEXT functionality?  If you 
> include .py in your PATHEXT and file associations are set so .py files 
> are handled by the launcher, you should be able to directly execute 
> just 'foo' and have the command processor search the PATH for a foo.py 
> and invoke it via the launcher.

Not at all.  I was just testing the use of the launcher from the command 
line, and seeing the cumbersomeness of using it as a prefix to a command 
that would work on its own, but fails when launched by the launcher from 
the command line.  And the SearchPath API has a limited form of PATHEXT 
support built in, which is what I was trying to point out above.  Yes, 
the use of the launcher via file associations can be friendly because it 
leverages Windows features, but its use from the command line presently 
seems rather unfriendly, because it leverages Windows misfeatures.
>
>
> Mark 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/ab01036c/attachment.html>

From v+python at g.nevcal.com  Thu Jul 21 10:32:59 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 01:32:59 -0700
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <4E27C8BA.8040000@gmail.com>
References: <4E27C8BA.8040000@gmail.com>
Message-ID: <4E27E43B.2070906@g.nevcal.com>

On 7/20/2011 11:35 PM, Mark Hammond wrote:
>      * If the command starts with the definition of a customized command
>        followed by a space character, the customized command will be used.
>        See below for a description of customized commands.

>      Then a shebang line of '#! vpython' in a script named 'doit.py' will
>      result in the launcher using the command-line 'c:\bin\vpython.exe -foo
>      doit.py'

Shouldn't the second paragraph include a space before the 2nd ' ?  In 
other words, the command as quoted does not show a "customized command 
followed by a space character" as the definition requires.  I don't know 
why the space character is required, or maybe "white space" was meant, 
so that the line terminating newline character qualifies also to delimit 
the customized command?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/31945ef6/attachment.html>

From ncoghlan at gmail.com  Thu Jul 21 10:47:57 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Jul 2011 18:47:57 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E27DE21.4050304@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
	<CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
	<4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com>
	<20110721025216.63DCF2500D6@webabinitio.net>
	<CADiSq7fCQO-Geg1pqbHcZhDoSXDOTvFkvMQ6VjHJ-faueicCig@mail.gmail.com>
	<4E27DE21.4050304@g.nevcal.com>
Message-ID: <CADiSq7eC2KmbjDsZMkE9=dvLZFCLiA3JKJ8JOuqAkN2oRNiUOA@mail.gmail.com>

On Thu, Jul 21, 2011 at 6:06 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> Interesting, David, that you feel it that Python usability on Windows should
> be limited to its usability on Unix, rather than to exceed it. I don't see
> that as a necessary or appropriate limit.? Windows and Unix are different.
> Unix people are accustomed to using tools like which, and using command
> lines, and path manipulations; Windows people are not.? So the use of the
> command line is already somewhat foreign to them, and limiting the launcher
> so that they have to use other command line tools to get the work done,
> would only serve to frustrate them.? Now the argument can possibly be made
> that people that want to use launcher from the command line would be those
> that are already command line experts might be realistic, but I will note
> that Perl has a -S option to find its script on the PATH, not that that is a
> sufficient reason to add such to Python, or even to the launcher, but just
> to note that there are at least some people besides myself that might think
> that is a friendly idea.

If a PEP is put forward proposing such a feature, with a reference
implementation that supports at least Windows, *nix and OS X and works
for at least the 4 script types understood by the CPython executable
without a command line option (source and bytecode files along with
directories and zipfiles providing top level __main__ modules), then
I'd be prepared to moderate my response all the way to a +0 (reserving
the extreme negative reaction for proposals that are either platform
specific or only handle some script types).

I believe that is significantly easier said than done, though.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From skippy.hammond at gmail.com  Thu Jul 21 10:51:15 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 21 Jul 2011 18:51:15 +1000
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <4E27E43B.2070906@g.nevcal.com>
References: <4E27C8BA.8040000@gmail.com> <4E27E43B.2070906@g.nevcal.com>
Message-ID: <4E27E883.2060404@gmail.com>

On 21/07/2011 6:32 PM, Glenn Linderman wrote:
> On 7/20/2011 11:35 PM, Mark Hammond wrote:
>>      * If the command starts with the definition of a customized command
>>        followed by a space character, the customized command will be used.
>>        See below for a description of customized commands.
>
>>      Then a shebang line of '#! vpython' in a script named 'doit.py' will
>>      result in the launcher using the command-line 'c:\bin\vpython.exe -foo
>>      doit.py'
>
> Shouldn't the second paragraph include a space before the 2nd ' ?  In
> other words, the command as quoted does not show a "customized command
> followed by a space character" as the definition requires.  I don't know
> why the space character is required, or maybe "white space" was meant,
> so that the line terminating newline character qualifies also to delimit
> the customized command?

Yes, thanks for pointing that out - I've pushed a new version of the PEP 
with that paragraph reading as:

"""
If the command starts with the definition of a customized command 
followed by a whitespace character (including a newline), the customized 
command will be used.  See below for a description of customized commands.
"""


Thanks,

Mark
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/skippy.hammond%40gmail.com


From victor.stinner at haypocalc.com  Thu Jul 21 12:44:11 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 21 Jul 2011 12:44:11 +0200
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <loom.20110721T000244-533@post.gmane.org>
References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com>
	<loom.20110721T000244-533@post.gmane.org>
Message-ID: <4E2802FB.9080009@haypocalc.com>

On 21/07/2011 00:07, Vinay Sajip wrote:
> Victor Stinner<victor.stinner<at>  haypocalc.com>  writes:
>
>> New logging tests failed during some weeks. If we add new tests, we may
>> also break some stable buildbots. I don't think that we need to add
>> these new tests to a stable version.
> Just for my information, which logging test failures are you referring to?
Sorry, I don't remember the details, I just remember that some new tests 
added to test_logging were failing during some days/weeks.

Victor

From rdmurray at bitdance.com  Thu Jul 21 14:00:10 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 21 Jul 2011 08:00:10 -0400
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E27DE21.4050304@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E275BED.4050901@stoneleaf.us>
	<CADiSq7daJUkyVLo=2SeYQnpOqkOG4HHwC+nPHhsBVZgOB7Mp0A@mail.gmail.com>
	<4E277059.3080301@g.nevcal.com> <4E277273.5060508@gmail.com>
	<20110721025216.63DCF2500D6@webabinitio.net>
	<CADiSq7fCQO-Geg1pqbHcZhDoSXDOTvFkvMQ6VjHJ-faueicCig@mail.gmail.com>
	<4E27DE21.4050304@g.nevcal.com>
Message-ID: <20110721120011.2E460B14005@webabinitio.net>

On Thu, 21 Jul 2011 01:06:57 -0700, Glenn Linderman <v+python at g.nevcal.com> wrote:
> Interesting, David, that you feel it that Python usability on Windows 
> should be limited to its usability on Unix, rather than to exceed it. I 
> don't see that as a necessary or appropriate limit.  Windows and Unix 

That wasn't how I intended my comment.  My point was that the goal of
the *PEP* is to make it "as usable" (and actually just in the specific
case of #! lines), and that if the additional feature is desirable
why not make it available on all platforms?  I can see how you read what
you did in what I wrote, though.

--
R. David Murray           http://www.bitdance.com

From techtonik at gmail.com  Thu Jul 21 14:13:16 2011
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 21 Jul 2011 15:13:16 +0300
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <4E27C8BA.8040000@gmail.com>
References: <4E27C8BA.8040000@gmail.com>
Message-ID: <CAPkN8xLSQzemorNFxry-FjCmKNrNdDtDROqicJyUi7sN0S_JsQ@mail.gmail.com>

If you're going to include this into standard Python distribution, it
needs more attention from _users_. As a user, I can not find any
references to any user stories in this PEP article. Abstract chapter
is totally useless

"This PEP (named 'Python launcher for Windows') describes a Python
launcher for the Windows platform." - it is a waste of time to read
such stuff.

"A Python launcher is a single executable which uses a number of
heuristics to locate a Python executable and launch it with a
specified command line." - this doesn't answer main questions - Why
Launcher is needed, i.e. is there a problem? Can it be solved? How
this PEP helps to solve the problem?


I see that this launcher is an .msi file. Shouldn't it be portable and
don't require administrative privileges? How about KISS?
--
anatoly t.



On Thu, Jul 21, 2011 at 9:35 AM, Mark Hammond <skippy.hammond at gmail.com> wrote:
> I've updated PEP 397 - "Python launcher for Windows" based on recent
> discussions and Vinay's implementation.
>
> http://www.python.org/dev/peps/pep-0397/ and a copy is attached for your
> convenience.
>
> The main changes are:
>
> * All mentions of the Python reference implementation have been removed now
> that a C implementation exists and is a more accurate implementation of the
> PEP than the Python version.
>
> * Some "implementation details" have been removed and links added to the
> launcher docs. ?This was done mainly so the implementation is free to make
> changes based on user feedback and still stay true to the PEP. Note that the
> launcher doc link doesn't exist right now, but should magically appear over
> the next couple of days as Vinay pushes some changes I just sent him.
>
> * The recommendation to install into System32 has been changed to a
> recommendation to install directly into \Windows, as the System32 directory
> is not on the default path for 32bit processes on a 64bit Windows. ?Vinay
> even has a version of an MSI installer which installs into this directory.
> ?The PEP also gives installers more leeway on where to install the launcher
> if this directory is not writable.
>
> I think this PEP is getting close to being finalized - please suggest
> whatever changes you feel are necessary ASAP as soon I'll be asking for
> pronouncement.
>
> Thanks - especially to Vinay for taking on the C implementation!
>
> Mark
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com
>
>

From brian.curtin at gmail.com  Thu Jul 21 14:54:26 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Thu, 21 Jul 2011 07:54:26 -0500
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <CAPkN8xLSQzemorNFxry-FjCmKNrNdDtDROqicJyUi7sN0S_JsQ@mail.gmail.com>
References: <4E27C8BA.8040000@gmail.com>
	<CAPkN8xLSQzemorNFxry-FjCmKNrNdDtDROqicJyUi7sN0S_JsQ@mail.gmail.com>
Message-ID: <CAD+XWwre4gUeJ=bYjJTs-TfaJWYupBDP54sNTchsgdAww5fbaQ@mail.gmail.com>

On Jul 21, 2011 7:15 AM, "anatoly techtonik" <techtonik at gmail.com> wrote:
>
> If you're going to include this into standard Python distribution, it
> needs more attention from _users_. As a user, I can not find any
> references to any user stories in this PEP article. Abstract chapter
> is totally useless
>
> "This PEP (named 'Python launcher for Windows') describes a Python
> launcher for the Windows platform." - it is a waste of time to read
> such stuff.
>
> "A Python launcher is a single executable which uses a number of
> heuristics to locate a Python executable and launch it with a
> specified command line." - this doesn't answer main questions - Why
> Launcher is needed, i.e. is there a problem? Can it be solved? How
> this PEP helps to solve the problem?
>
>
> I see that this launcher is an .msi file. Shouldn't it be portable and
> don't require administrative privileges? How about KISS?

No. As stated in the title, this is for Windows. It also installs to system
directories which is likely the need for admin access.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/1f204098/attachment.html>

From mhammond at skippinet.com.au  Thu Jul 21 15:07:12 2011
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 21 Jul 2011 23:07:12 +1000
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <CAPkN8xLSQzemorNFxry-FjCmKNrNdDtDROqicJyUi7sN0S_JsQ@mail.gmail.com>
References: <4E27C8BA.8040000@gmail.com>
	<CAPkN8xLSQzemorNFxry-FjCmKNrNdDtDROqicJyUi7sN0S_JsQ@mail.gmail.com>
Message-ID: <4E282480.7000700@skippinet.com.au>

On 21/07/2011 10:13 PM, anatoly techtonik wrote:
> If you're going to include this into standard Python distribution, it
> needs more attention from _users_. As a user, I can not find any
> references to any user stories in this PEP article. Abstract chapter
> is totally useless

Could you please be a little more constructive and offer concrete 
improvements?

Mark

From pje at telecommunity.com  Thu Jul 21 15:20:53 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 21 Jul 2011 09:20:53 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CADiSq7cNZ=ypGK--YfVa2xnkCMq5yM3VWdMwY5FUyZtOFYBsAA@mail.g
	mail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
	<4E27522D.8010300@g.nevcal.com>
	<20110720230439.6ADE53A409B@sparrow.telecommunity.com>
	<CADiSq7cNZ=ypGK--YfVa2xnkCMq5yM3VWdMwY5FUyZtOFYBsAA@mail.gmail.com>
Message-ID: <20110721132153.D18B03A411E@sparrow.telecommunity.com>

At 11:52 AM 7/21/2011 +1000, Nick Coghlan wrote:
>Trying to change how packages are identified at the Python level makes
>PEP 382 sound positively appealing. __path__ needs to stay :)

In which case, it should be a list, not a sentinel.  ;-)


>Even better would be for these (and sys.path) to be list subclasses
>that did the right thing under the hood as Glenn suggested. Code that
>*replaces* rather than modifies these attributes would still
>potentially break virtual packages, but code that modifies them in
>place would do the right thing automatically. (Note that all code that
>manipulates sys.path and __path__ attributes requires explicit calls
>to correctly support current namespace package mechanisms, so this
>would actually be an improvement on the status quo rather than making
>anything worse).

I think the simplest thing, if we're keeping __path__ (and on 
reflection, I think we should), would be to simply call 
extend_virtual_paths() automatically on new path entries found in 
sys.path when an import is performed, relative to the previous value 
of sys.path.

That is, we save an "old" copy of sys.path somewhere, and whenever 
__import__() is called (well, once it gets past checking if the 
target is already in sys.modules, anyway), it checks the current 
sys.path against it, and calls extend_virtual_paths() on any sys.path 
entries that weren't in the "old" sys.path.

This is not the most efficient thing in the world, as it will cause a 
bunch of stat calls to happen against the new directories, in the 
middle of a possibly-entirely-unrelated import operation, but it 
would certainly address the issue in the Simplest Way That Could Possibly Work.

A stricter (safer) version of the same thing would be one where we 
only update __path__ values that are unchanged since we created them, 
and rather than only appending new entries, we replace the __path__ 
with a newly-computed one.

This version is safer because it avoids corner cases like "I imported 
foo.bar while foo.baz 1.1 was on my path, then I prepended a 
directory to sys.path that has foo.baz 1.2, but I still get foo.baz 
1.1 when I import."  But it loses in cases where people do direct 
__path__ manipulation.

On the other hand, it's a lot easier to say "you break it, you bought 
it" where __path__ manipulation is concerned, so I'm actually pretty 
inclined towards using the strict version.

Hey...  here's a crazy idea.  Suppose that a virtual package __path__ 
is a *tuple* instead of a list?  Now, in order to change it, you 
*have* to replace it.  And we can cache the tuple we initially set it 
to in sys.virtual_package_paths, so we can do an 'is' check before 
replacing it.

Voila: __path__ still exists and is still a sequence for a virtual 
path, but you have to explicitly replace it if you want to do 
anything funky -- at which point you're responsible for maintaining it.

I'm tempted to say, "well, why not use a list-subclass proxy, then?", 
but that means more work for no real difference.  I just went through 
dozens of examples of __path__ usage (found via Google), and I found 
exactly two examples of code that modifies a __path__ that is not:

1. In the __init__.py whose __path__ it is (i.e., code that'll still 
have a list), or
2. Modifying the __path__ of an explicitly-named self-contained 
package that's part of the same distribution.

The two examples are from Twisted, and Google AppEngine.  In the 
Twisted case, it's some sort of namespace package-like plugin 
chicanery, and in the AppEngine case, well, I'm not sure what the 
heck it's doing, but it seems to be making sure that you can still 
import stuff that has the same name as stdlib stuff, or something.

The Twisted case (and an apparent copy of the same code in a project 
called "flumotion") uses ihooks, though, so I'm not sure it'll even 
get executed for virtual packages.  The Google case loops over 
everything in sys.modules, in a function by the name of 
appengine.dist.fix_paths()...  but I wasn't able to find out who 
calls this function, when and why.

So, pretty much, except for these bits of "nosy" code, the vast 
majority of code out there seems to only mess with its own 
self-contained paths, making the use of tuples seem like a pretty safe choice.

(Oh, and all the code I found that reads paths without modifying them 
only use tuple-safe operations.)

So, if we implement automatic __path__ updates for virtual packages, 
I'm currently leaning towards the strict approach using tuples, but 
could possibly be persuaded towards read-only list-proxies instead.

Side note: it looks like a *lot* of code out there abuses __path__[0] 
to find data files, so I probably need to add a note to the PEP about 
not doing that when you convert a self-contained package to a virtual 
one.  Of course, I suppose using a sentinel could address *that* 
problem, or an iteration-only proxy.

The main concern here is that using __path__[0] will *seem* to work 
when you first use it with a virtual package, because it'll be the 
right directory.  But it'll be wrong long-term.

This seems to lean in favor of making a simple reiterable wrapper 
type for the __path__, that only allows you to take the length and 
iterate over it.  With an appropriate design, it could actually 
update itself automatically, given a subname and a parent 
__path__/sys.path.  That is, it could keep a tuple copy of the 
last-seen parent path, and before iteration, compare 
tuple(self.parent_path) to self.last_seen_path.  If they're 
different, it rebuilds the value to be iterated over.

Voila: transparent updating of all virtual __path__ values from 
sys.path changes (or modifications to self-contained __path__ 
parents, btw), and trying to change it (or read an item from it 
positionally) will not create any silent failures.

Alright...  *if* we support automatic updates to virtual __paths__, 
this is probably how we should do it.  (It will require, though, that 
imp.find_module be changed to use a different iteration method than 
PyList_GetItem, as it's quite possible a virtual __path__ will get 
passed into it.)

Also, we *long* ago passed the point where any of this can be sanely 
backported to Python 2.x with a simple shim, alas.  For my purposes 
at least, needing a full importlib for the implementation is a 
no-go.  :-(  Still, for the future of Python, this all makes good 
sense.  I just wish we'd thought of all this in 2006 when the 
discussion came up before: we maybe could've had this in Python 
2.6.  Where's that damn time machine when you *really* need it?  ;-)


From p.f.moore at gmail.com  Thu Jul 21 16:43:04 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 21 Jul 2011 15:43:04 +0100
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E27DFC4.1000801@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com>
	<4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com>
Message-ID: <CACac1F-BwEbKaoW-oc16eccif6fKf0tzC7=CfkVFrOZsk_Jz3Q@mail.gmail.com>

On 21 July 2011 09:13, Glenn Linderman <v+python at g.nevcal.com> wrote:
> Certainly when the launcher is invoked via an association, this would
> be the case.? However, when the launcher is invoked via the command
> line, then the unqualified name is passed through.? To be useful from
> the command line, the launcher should walk the PATH to find the .py file.

It's equally as arguable (and would match my expectations much more
closely) that "py a_file.py" should do whatever "python a_file.py"
would do. So path search in that context would only be reasonable if
it were a Python feature rather than a feature of the launcher.

This is what the launcher currently does (so I guess it's not
surprising that I'm happy with the current behaviour).

I can see the benefits of path search, but I'd want it to be a Python
feature (and hence inherited "for free" by the launcher) and not a
launcher-only one.

Paul.

From fuzzyman at voidspace.org.uk  Thu Jul 21 17:20:07 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 21 Jul 2011 16:20:07 +0100
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <CACac1F-BwEbKaoW-oc16eccif6fKf0tzC7=CfkVFrOZsk_Jz3Q@mail.gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com>
	<4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com>
	<CACac1F-BwEbKaoW-oc16eccif6fKf0tzC7=CfkVFrOZsk_Jz3Q@mail.gmail.com>
Message-ID: <4E2843A7.4090502@voidspace.org.uk>

On 21/07/2011 15:43, Paul Moore wrote:
> On 21 July 2011 09:13, Glenn Linderman<v+python at g.nevcal.com>  wrote:
>> Certainly when the launcher is invoked via an association, this would
>> be the case.  However, when the launcher is invoked via the command
>> line, then the unqualified name is passed through.  To be useful from
>> the command line, the launcher should walk the PATH to find the .py file.
> It's equally as arguable (and would match my expectations much more
> closely) that "py a_file.py" should do whatever "python a_file.py"
> would do. So path search in that context would only be reasonable if
> it were a Python feature rather than a feature of the launcher.
>
> This is what the launcher currently does (so I guess it's not
> surprising that I'm happy with the current behaviour).
>
> I can see the benefits of path search, but I'd want it to be a Python
> feature (and hence inherited "for free" by the launcher) and not a
> launcher-only one.

What he said ^^. (+1)

py launcher and python binaries behaving differently in this regard 
would be a recipe for confusion and hard to debug problems.

Michael

>
> Paul.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From brett at python.org  Thu Jul 21 20:15:49 2011
From: brett at python.org (Brett Cannon)
Date: Thu, 21 Jul 2011 11:15:49 -0700
Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any
 callable object - request commiter
In-Reply-To: <20110721080552.40e767a1@vimes>
References: <20110716103322.330f805c@vimes>
	<CAP1=2W7w9rc1E1=umrFwUoNea8rrjd8ZK_gaFEc1rHeyJs_tfg@mail.gmail.com>
	<20110721080552.40e767a1@vimes>
Message-ID: <CAP1=2W6x22cPrcMP+3sdYvRDg5gN+H2XDhxc2WZUaTPs+1wT7w@mail.gmail.com>

On Wed, Jul 20, 2011 at 23:05, lekmalek <lekmalek at gmail.com> wrote:

> On Sun, 17 Jul 2011 19:19:59 -0700
> Brett Cannon <brett at python.org> wrote:
>
> > Just so people know, I went ahead and fixed this for 3.3 (but not for
> > 3.2 since it changes the API in a subtle way).
> Yeah, but that shouldn't break anything.
>

It won't break any _existing_ code, but it could cause compatibility for
_future_ code. Imagine I wrote some code for 3.2.2 where this change was
backported and worked *only* with this fix. That would mean my code would
fail in any Python 3.2.1 or older interpreter. That's simply not acceptable
as that means that code would be magically busted for some versions of
Python 3.2 but not all of them.


>
> Anyway, thanks a lot for your help, I'm happy it's in 3.3.
>

I'm just sorry it took so long to resolve. Life has been crazy for me in
2011.

-Brett


>
> lekma
>
> >
> > On Sat, Jul 16, 2011 at 01:33, lekmalek <lekmalek at gmail.com> wrote:
> >
> > > Hello all,
> > >
> > > Can any of you core devs have a look at
> > > http://bugs.python.org/issue10271. It seems Brett is really busy
> > > right now and this uncontroversial (AFAICT) one liner only needs
> > > someone to review it and commit it. The pb is, it's holding me back
> > > a little bit, and I really would like to have it in the next 3.2
> > > release if possible.
> > >
> > > Thanks for your help,
> > >
> > > lekma
> > > _______________________________________________
> > > Python-Dev mailing list
> > > Python-Dev at python.org
> > > http://mail.python.org/mailman/listinfo/python-dev
> > > Unsubscribe:
> > > http://mail.python.org/mailman/options/python-dev/brett%40python.org
> > >
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/162cdb3b/attachment.html>

From tjreedy at udel.edu  Thu Jul 21 20:49:47 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Jul 2011 14:49:47 -0400
Subject: [Python-Dev] Issue10271 - warnings.showwarning should allow any
 callable object - request commiter
In-Reply-To: <CAP1=2W6x22cPrcMP+3sdYvRDg5gN+H2XDhxc2WZUaTPs+1wT7w@mail.gmail.com>
References: <20110716103322.330f805c@vimes>	<CAP1=2W7w9rc1E1=umrFwUoNea8rrjd8ZK_gaFEc1rHeyJs_tfg@mail.gmail.com>	<20110721080552.40e767a1@vimes>
	<CAP1=2W6x22cPrcMP+3sdYvRDg5gN+H2XDhxc2WZUaTPs+1wT7w@mail.gmail.com>
Message-ID: <j09sca$c4n$1@dough.gmane.org>

On 7/21/2011 2:15 PM, Brett Cannon wrote:

>
> It won't break any _existing_ code, but it could cause compatibility for
> _future_ code. Imagine I wrote some code for 3.2.2 where this change was
> backported and worked *only* with this fix. That would mean my code
> would fail in any Python 3.2.1 or older interpreter. That's simply not
> acceptable as that means that code would be magically busted for some
> versions of Python 3.2 but not all of them.

In other words, Python 3.2 is a fixed language, cpython3.2.z's are 
intended to be increasingly better implementations of that language, 
although regressions can and do happen (as with issue 12540).

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Thu Jul 21 21:20:59 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Jul 2011 15:20:59 -0400
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <AEA2ACFB-E9A7-455B-8686-53B64EF80CB5@gmail.com>
References: <4E26FB19.2070406@netwok.org>
	<4E270190.1000209@haypocalc.com>	<j077ti$bql$1@dough.gmane.org>	<CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>
	<AEA2ACFB-E9A7-455B-8686-53B64EF80CB5@gmail.com>
Message-ID: <j09u6r$pls$1@dough.gmane.org>

On 7/21/2011 2:58 AM, Raymond Hettinger wrote:

> I concur with Brett. Nothing good will come from backporting tests that
> aren't aimed at a specific bugfix.

They could catch reversions that otherwise would not be caught. This 
would mainly apply to 2.7. It would not be an issue for 3.2 if all fixes 
are forward ported to 3.3 and tested there (before pushing) where there 
are tests not in 3.2. If people fix in 3.2, test, commit, and push, and 
just assume OK in 3.3, the new test will not do any good until someone 
else runs them with the fix.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Thu Jul 21 21:42:44 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Jul 2011 15:42:44 -0400
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
	encoding surprise)
In-Reply-To: <4E276AF0.5090409@gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>	<20110719171647.03f9f9c9@pitrou.net>	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>	<j05e3c$g9c$1@dough.gmane.org>	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>	<j077bn$855$1@dough.gmane.org>
	<4E276AF0.5090409@gmail.com>
Message-ID: <j09vfk$1v5$1@dough.gmane.org>

On 7/20/2011 7:55 PM, Mark Hammond wrote:
> On 21/07/2011 4:38 AM, Terry Reedy wrote:
>
>> Many installers first make an organization directory and then an app
>> directory within that. This annoys me sometimes when they only have one
>> app to ever install, but is useful when there might really be multiple
>> directories, as in our case. (Ditto for start menu entries.) This is
>> what python should have done a decade ago.

> I disagree. If we followed that advice we would also be in "\Program
> Files".

That is not what I suggested. I said let the use pick.

> I have no problem with multiple Python versions installed
> directly off the root, especially given most users probably have a very
> small number of such installations. I think Python being a developer
> tool rather than a user app is a reasonable justification for that (and
> the justification used when the existing scheme was decided)

I put the multiple installations and several other directories into 
/programs. On my next machine, on order, I will use /devel.

>  > The two proposals
>> overlap but are not mutually exclusive. For future pythons, 'python33'
>> is easier to remember and type than 'py -v 3.3' or whatever the proposed
>> encantation is.
>
> 'py -3.3' - less chars to type than 'python33' and with no need to have
> every Python directory on your PATH.

My proposal, as I clearly said, was for EXACTLY ONE directory to be 
added to PATH. In spite of Microsoft making is damned difficult for 
users to edit PATH, (and deleted programs not deleting their entries) I 
added 'C:/programs;'. I copied python32/python as py32 and 
python27/python as py27. Those are even fewer characters to type (4 
versus 7). Now I can click a 'Command Prompt' icon and enter 'py32 -m 
test.regrtest' and it works without cd-ing to /programs/python32. Of 
course, I will have to re-copy with every install, which is why I would 
like something like this as part of installs.


> IMO it is also simple enough that
> people will remember it fairly easily.

py32 is even easier to remember.

> Also, the launcher supports the ability to select either the 32 or 64bit
> implementation - so maybe 'python33.exe' isn't really good enough and
> should reflect the bittedness?

Like py32-6? If I install both Pythons on my new 64 bit machine, I will 
think about it, though I have no need for both now.

>> A python directory also gives a sensible (though optional) place to put
>> other interpreters and even python-based apps. The launcher does not.
>
> What other interpreters? IMO it doesn't make sense to have IronPython,
> jython etc be installed there. Ditto for apps - especially given most
> apps tend to be tied to a subset of all possible Python versions.

If I install pypy, /programs is exactly where I would put it until I 
somehow discovered that to be a problem. Its startup could be copied as 
pp26 or something.


My idea may be not so good for general use, even though is now solves my 
problems, but please criticize what I said, allowing for obvious 
modifications like py32 instead of python32, and not a strawman that is 
wildly different.

-- 
Terry Jan Reedy


From v+python at g.nevcal.com  Thu Jul 21 22:05:45 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 13:05:45 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E2843A7.4090502@voidspace.org.uk>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com>
	<4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com>
	<CACac1F-BwEbKaoW-oc16eccif6fKf0tzC7=CfkVFrOZsk_Jz3Q@mail.gmail.com>
	<4E2843A7.4090502@voidspace.org.uk>
Message-ID: <4E288699.3060609@g.nevcal.com>

On 7/21/2011 8:20 AM, Michael Foord wrote:
> On 21/07/2011 15:43, Paul Moore wrote:
>> On 21 July 2011 09:13, Glenn Linderman<v+python at g.nevcal.com>  wrote:
>>> Certainly when the launcher is invoked via an association, this would
>>> be the case.  However, when the launcher is invoked via the command
>>> line, then the unqualified name is passed through.  To be useful from
>>> the command line, the launcher should walk the PATH to find the .py 
>>> file.
>> It's equally as arguable (and would match my expectations much more
>> closely) that "py a_file.py" should do whatever "python a_file.py"
>> would do. So path search in that context would only be reasonable if
>> it were a Python feature rather than a feature of the launcher.
>>
>> This is what the launcher currently does (so I guess it's not
>> surprising that I'm happy with the current behaviour).
>>
>> I can see the benefits of path search, but I'd want it to be a Python
>> feature (and hence inherited "for free" by the launcher) and not a
>> launcher-only one.
>
> What he said ^^. (+1)
>
> py launcher and python binaries behaving differently in this regard 
> would be a recipe for confusion and hard to debug problems. 

I see the point.  Although the incremental benefit is higher to Windows 
users, and although we are creating a Windows-only piece of code that 
could be the vehicle for adding the functionality, it would be 
beneficial for all platforms, and a common implementation would serve 
that need better.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/7490ea4b/attachment.html>

From pje at telecommunity.com  Thu Jul 21 23:12:41 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 21 Jul 2011 17:12:41 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <4E288527.8060202@gmail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
	<4E27522D.8010300@g.nevcal.com>
	<20110720230439.6ADE53A409B@sparrow.telecommunity.com>
	<CADiSq7cNZ=ypGK--YfVa2xnkCMq5yM3VWdMwY5FUyZtOFYBsAA@mail.gmail.com>
	<20110721132153.D18B03A411E@sparrow.telecommunity.com>
	<4E288527.8060202@gmail.com>
Message-ID: <20110721211336.7C1393A40AA@sparrow.telecommunity.com>

At 12:59 PM 7/21/2011 -0700, Reliable Domains wrote:
>I assume that the implicit extend_virtual_paths() would be smart 
>enough to only do real work if there are virtual packages to do it 
>in, so much of the performance costs (bunch of stats) are bounded by 
>the existence of and number of virtual packages that have actually 
>been imported, correct?

Yes - this is true even for an explicit call.  It only does this for 
imported virtual packages, and child virtual packages are only 
checked for if the parent package exists.  So, in the case of a 
directory being added that has no parent packages, then the cost in 
stats is equal to the number of top-level, *imported* virtual packages.

The __path__ wrapper scheme can do this even better, and defer doing 
any of the stat calls until/unless another import occurs for one of 
those packages.  So if you munge sys.path and then don't import 
anything from a virtual package, no extra stat calls would happen at all.


From ncoghlan at gmail.com  Fri Jul 22 00:06:30 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 22 Jul 2011 08:06:30 +1000
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <j09u6r$pls$1@dough.gmane.org>
References: <4E26FB19.2070406@netwok.org> <4E270190.1000209@haypocalc.com>
	<j077ti$bql$1@dough.gmane.org>
	<CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>
	<AEA2ACFB-E9A7-455B-8686-53B64EF80CB5@gmail.com>
	<j09u6r$pls$1@dough.gmane.org>
Message-ID: <CADiSq7fs6ojcqPHq9VFSwECaJhMqO+DxrLkwzaiPnX6XaOfsyw@mail.gmail.com>

On Fri, Jul 22, 2011 at 5:20 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 7/21/2011 2:58 AM, Raymond Hettinger wrote:
>
>> I concur with Brett. Nothing good will come from backporting tests that
>> aren't aimed at a specific bugfix.
>
> They could catch reversions that otherwise would not be caught. This would
> mainly apply to 2.7. It would not be an issue for 3.2 if all fixes are
> forward ported to 3.3 and tested there (before pushing) where there are
> tests not in 3.2. If people fix in 3.2, test, commit, and push, and just
> assume OK in 3.3, the new test will not do any good until someone else runs
> them with the fix.

None of that contradicts what Raymond and Brett said. Backporting test
improvements that aren't targeting specific known bugs does not make
efficient use of limited development resources. Forward porting of any
changes made to maintenance branches (or explicitly blocking same as
being irrelevant), OTOH, is mandatory.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Fri Jul 22 00:17:24 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 22 Jul 2011 08:17:24 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E288699.3060609@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276F45.7070105@g.nevcal.com>
	<4E27741A.1040202@gmail.com> <4E27DFC4.1000801@g.nevcal.com>
	<CACac1F-BwEbKaoW-oc16eccif6fKf0tzC7=CfkVFrOZsk_Jz3Q@mail.gmail.com>
	<4E2843A7.4090502@voidspace.org.uk> <4E288699.3060609@g.nevcal.com>
Message-ID: <CADiSq7ez702qshzCdmgSM4zptcApLV=yifzgwV8ji1sEOPH1=w@mail.gmail.com>

On Fri, Jul 22, 2011 at 6:05 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 7/21/2011 8:20 AM, Michael Foord wrote:
>> py launcher and python binaries behaving differently in this regard would be
>> a recipe for confusion and hard to debug problems.
>
> I see the point.? Although the incremental benefit is higher to Windows
> users, and although we are creating a Windows-only piece of code that could
> be the vehicle for adding the functionality, it would be beneficial for all
> platforms, and a common implementation would serve that need better.

Well that, and the desire to have the Windows launcher *just* find an
interpreter to run, so that "py -3.2 <args>" and "c:\python32\python
<args>" handle the arguments the same way.

While further discussion of the PATH walking concept should be done in
a new thread on python-ideas, I'll note that I'm no longer actively
opposed to the idea. However, I still think it needs its own PEP to
work out the details (and whether or not it happens at all).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Fri Jul 22 00:35:15 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 22 Jul 2011 08:35:15 +1000
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110721132153.D18B03A411E@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<4E269EBC.90708@g.nevcal.com>
	<20110720131415.A2DD43A4100@sparrow.telecommunity.com>
	<4E27522D.8010300@g.nevcal.com>
	<20110720230439.6ADE53A409B@sparrow.telecommunity.com>
	<CADiSq7cNZ=ypGK--YfVa2xnkCMq5yM3VWdMwY5FUyZtOFYBsAA@mail.gmail.com>
	<20110721132153.D18B03A411E@sparrow.telecommunity.com>
Message-ID: <CADiSq7d2TO=gt3NORTUFpeE_f19Cg0Gdsdw3nv-xZJ4w=cF1EA@mail.gmail.com>

On Thu, Jul 21, 2011 at 11:20 PM, P.J. Eby <pje at telecommunity.com> wrote:
> This seems to lean in favor of making a simple reiterable wrapper type for
> the __path__, that only allows you to take the length and iterate over it.
> ?With an appropriate design, it could actually update itself automatically,
> given a subname and a parent __path__/sys.path. ?That is, it could keep a
> tuple copy of the last-seen parent path, and before iteration, compare
> tuple(self.parent_path) to self.last_seen_path. ?If they're different, it
> rebuilds the value to be iterated over.
>
> Voila: transparent updating of all virtual __path__ values from sys.path
> changes (or modifications to self-contained __path__ parents, btw), and
> trying to change it (or read an item from it positionally) will not create
> any silent failures.
>
> Alright... ?*if* we support automatic updates to virtual __paths__, this is
> probably how we should do it. ?(It will require, though, that
> imp.find_module be changed to use a different iteration method than
> PyList_GetItem, as it's quite possible a virtual __path__ will get passed
> into it.)

A no-indexing tuple wrapper for virtual package __path__ values that
automatically updates itself in response to parent path modifications
sounds good to me (errors shall not pass silently, etc). This also
allows virtual packages to be indicated clearly just through the type
of their __path__ attribute rather than having to look them up in the
import state.

I still like the idea of keeping sys.virtual_packages as a dict
mapping to the path values, though - it makes it easier to debug
erroneous __path__ replacement in virtual packages by checking
"pkg.__path__ is sys.virtual_package_paths[pkg.__name__]"

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From raymond.hettinger at gmail.com  Fri Jul 22 00:37:01 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Thu, 21 Jul 2011 15:37:01 -0700
Subject: [Python-Dev] [Python-checkins] devguide: Add a communications
	section to the devguide FAQ (closes #11690)
In-Reply-To: <CADiSq7cFFhKfskThNu9ZryQT+M-XHBoA63pZmc-2AhENRsBQyA@mail.gmail.com>
References: <E1QYJLt-00027Q-Ia@dinsdale.python.org>
	<4E27EE68.4000801@gmail.com>
	<CADiSq7cFFhKfskThNu9ZryQT+M-XHBoA63pZmc-2AhENRsBQyA@mail.gmail.com>
Message-ID: <44F06A8A-8153-443C-A4FB-0E57ABCBFDA6@gmail.com>

Please don't add the IRC link to the devguide.

Based on conversations with Guido, he is against it being part of the core development process.


Raymond



On Jul 21, 2011, at 4:08 AM, Nick Coghlan wrote:

> On Thu, Jul 21, 2011 at 7:16 PM, Ezio Melotti <ezio.melotti at gmail.com> wrote:
>> 
>> FWIW you can make #python a link using either irc://chat.freenode.net/python
>> (this will open the default IRC app, and I think Firefox even ask if you
>> want to use a webchat) or http://webchat.freenode.net/?channels=python (for
>> the freenode webchat).  If you are going to do it, it might be worth
>> mentioning that the channel requires registration.
>> 
>> I agree with Victor that the network (Freenode or irc.freenode.net) should
>> be specified.
>> Also the #python-dev channel should be mentioned (it doesn't require
>> registration, so links are fine here).
> 
> If someone more knowledgeable on IRC matters than me could either
> commit a fix directly to the devguide repo or else put a patch on the
> tracker, that would be great.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins


From solipsis at pitrou.net  Fri Jul 22 00:45:20 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Jul 2011 00:45:20 +0200
Subject: [Python-Dev] devguide: Add a communications section to the
 devguide FAQ (closes #11690)
References: <E1QYJLt-00027Q-Ia@dinsdale.python.org>
	<4E27EE68.4000801@gmail.com>
	<CADiSq7cFFhKfskThNu9ZryQT+M-XHBoA63pZmc-2AhENRsBQyA@mail.gmail.com>
	<44F06A8A-8153-443C-A4FB-0E57ABCBFDA6@gmail.com>
Message-ID: <20110722004520.1f430e6d@pitrou.net>

On Thu, 21 Jul 2011 15:37:01 -0700
Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> Please don't add the IRC link to the devguide.
> 
> Based on conversations with Guido, he is against it being part of the core
> development process.

IRC is very much outside of the core development process, but it's
still a reasonable place to ask help in.

Regards

Antoine.



From riscutiavlad at gmail.com  Fri Jul 22 00:54:30 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Thu, 21 Jul 2011 15:54:30 -0700
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <CAJ-9HZ3+Cpr50gYy7aqG0DcSuawpBLEnRMfE-eJauynT=ZZxFg@mail.gmail.com>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
	<4E1AAA75.7010903@v.loewis.de>
	<CAJ-9HZ3+Cpr50gYy7aqG0DcSuawpBLEnRMfE-eJauynT=ZZxFg@mail.gmail.com>
Message-ID: <CAJ-9HZ1cKz0AySJoj+bamiKEpmER6dCvuFWxoyS1DX9R7U-LZA@mail.gmail.com>

Anyone care to take a look at this?

Thank you,
Vlad

On Mon, Jul 11, 2011 at 8:59 AM, Vlad Riscutia <riscutiavlad at gmail.com>wrote:

> Actually I already attached patch implementing everything to the issue (not
> too much time spent :)). I'm hoping someone can review.
>
> Thank you,
> Vlad
>
>
> On Mon, Jul 11, 2011 at 12:47 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:
>
>> Am 25.06.2011 18:33, schrieb Vlad Riscutia:
>> > I would like to hear some opinions on this.
>>
>> Sounds fine to me in principle. Warnings can be added to the
>> documentation at any time; please submit a patch to the tracker.
>> As for the API change - make sure you post your proposed API
>> to the list before spending time to implement it.
>>
>> Regards,
>> Martin
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/e8379edc/attachment.html>

From v+python at g.nevcal.com  Fri Jul 22 01:02:46 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 16:02:46 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E276EBF.6000509@gmail.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
	<4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com>
Message-ID: <4E28B016.5050500@g.nevcal.com>

On 7/20/2011 5:11 PM, Mark Hammond wrote:
> It may be that your copy of the launcher is a little old - some 
> changes were pushed just yesterday (but I'm not sure if Vinay made a 
> new installer yet).  It has slightly better debug output (although 
> generally not what you are asking for yet) and better 
> "cross-bittedness" support. 

Installed new version:

msiexec /i launchwin.amd64.msi ALLUSERS=1

Expected behaviors of registry changes occurred.  Still launches python 
2, though, whereas c:\windows\py.ini contains:

[defaults]
python=3

[commands]
/usr/bin/perl=C:\perl\bin\perl.exe


Here is the debug output.  Seems like it isn't recognizing the python=3, 
even the new version.

d:\path\to\data>>set PYLAUNCH_DEBUG=1
set PYLAUNCH_DEBUG=1

d:\path\to\data>foo.py --pyver --clean
foo.py --pyver --clean
launcher build: 64bit
launcher executable: Console
File 'C:\Users\Glenn\AppData\Roaming\py.ini' non-existent
Using global configuration file 'C:\Windows\py.ini'
maybe_handle_shebang: read 256 bytes
maybe_handle_shebang: BOM not found, using UTF-8
locating Pythons in 32bit registry
locate_pythons_for_key: unable to open PythonCore key in HKCU
locate_pythons_for_key: C:\Python26\python.exe is a 32bit executable
locate_pythons_for_key: C:\Python26\PCBuild\python.exe: The system 
cannot find the path specified.
locate_pythons_for_key: C:\Python26\PCBuild\amd64\python.exe: The system 
cannot find the path specified.
locate_pythons_for_key: C:\Python31\python.exe is a 32bit executable
locate_pythons_for_key: C:\Python31\PCBuild\python.exe: The system 
cannot find the path specified.
locate_pythons_for_key: C:\Python31\PCBuild\amd64\python.exe: The system 
cannot find the path specified.
locating Pythons in native registry
locate_pythons_for_key: unable to open PythonCore key in HKCU
locate_pythons_for_key: C:\Python32\python.exe is a 64bit executable
locate_pythons_for_key: C:\Python32\PCBuild\python.exe: The system 
cannot find the path specified.
locate_pythons_for_key: C:\Python32\PCBuild\amd64\python.exe: The system 
cannot find the path specified.
found no configured value for 'python'
search for default Python found version 2.6 at 'C:\Python26\python.exe'
run_child: about to run 'C:\Python26\python.exe "D:\my\py\foo.py"  
--pyver --clean'
   File "D:\my\py\foo.py", line 469
SyntaxError: Non-ASCII character '\xc3' in file D:\my\py\foo.py on line 
470, but no encoding declared; see 
http://www.python.org/peps/pep-0263.html for details
child process exit code: 1

d:\path\to\data>

So, looking at the code, get_configured_value produces that message, but 
there are 3 places to look for "python".  Setting the environment 
variable makes it work.  Eliminating the environment variable, I then 
copied c:\Windows\py.ini to c:\users\glenn\appdata\roaming\py.ini.  That 
worked.  So I guess the syntax of my py.ini file is correct.  But 
apparently it isn't properly using c:\windows\py.ini !!  Yet curiously, 
it reports the correct name for the global configuration file.

Aha!

Bad logic is get_configured_value!  get_configured_value only looks in 
the global configuration file if there is a local configuration file 
that doesn't have the setting.  It should look in the global 
configuration file if there is no local configuration file _OR_ the is a 
local configuration file without the setting.

I'll await a new launcher.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/2453cf6c/attachment-0001.html>

From v+python at g.nevcal.com  Fri Jul 22 01:19:17 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 16:19:17 -0700
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <4E27C8BA.8040000@gmail.com>
References: <4E27C8BA.8040000@gmail.com>
Message-ID: <4E28B3F5.8080802@g.nevcal.com>

On 7/20/2011 11:35 PM, Mark Hammond wrote:
> Customized Commands:
>
>      The launcher will support the ability to define "Customized Commands" in a
>      Windows .ini file (ie, a file which can be parsed by the Windows function
>      GetPrivateProfileString).  A section called '[commands]' can be created
>      with key names defining the virtual command and the value specifying the
>      actual command-line to be used for this virtual command.
>
>      For example, if an INI file has the contents:
>
>      [commands]
>      vpython=c:\bin\vpython.exe -foo
>
>      Then a shebang line of '#! vpython' in a script named 'doit.py' will
>      result in the launcher using the command-line 'c:\bin\vpython.exe -foo
>      doit.py'

I experimented, and empirically determined, that a customized command 
can be of the form

[commands]
/usr/bin/perl=C:\perl\bin\perl.exe

and that this works to launch Unix perl scripts on Windows 
(successfully, if the perl script is actually portable).

This does not contradict the above description, but may be surprising to 
some.  I think it is a good thing.

Of course, the extra handling of versions, and locating of corresponding 
installed versions of things applies only to Python, but some may find 
uses for launching non-Python programs.  This also would work for python 
programs using non-CPython implementations that may not set the registry 
in the same way.  However, because virtual commands take precedence over 
Customized Commands, there is no way to override even a specific virtual 
command to point at a Python other than a CPython.  (perhaps some 
serious registry hacking could make a non-CPython masquerade in the 
registry as a version of CPython.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/b40ed116/attachment.html>

From v+python at g.nevcal.com  Fri Jul 22 01:12:45 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 16:12:45 -0700
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <4E27C8BA.8040000@gmail.com>
References: <4E27C8BA.8040000@gmail.com>
Message-ID: <4E28B26D.7070902@g.nevcal.com>

On 7/20/2011 11:35 PM, Mark Hammond wrote:
> Virtual commands in shebang lines:
>
>      Virtual Commands are shebang lines which start with strings which would
>      be expected to work on Unix platforms - examples include
>      '/usr/bin/python', '/usr/bin/env python' and 'python'.  Optionally, the
>      virtual command may be suffixed with a version qualifier (see below),
>      such as '/usr/bin/python2' or '/usr/bin/python3.2'.  The command executed
>      is based on the rules described in Python Version Qualifiers below.

I note in passing that '/usr/bin/env python' is hard-coded in the 
launcher, which conforms to the above documentation.  But there is no 
hard requirement in Unix, if I correctly understand it, that 
'/usr/bin/env' be separated from 'python' (or whatever) by exactly one 
space.  While I doubt it is frequently used with other than a single 
space, I think it would be legal to have 2 or 3 or 10 spaces, and maybe 
even tabs or a mixture, and it would work on Unix... but not in the 
launcher.

It would somewhat complicate the launcher code to have an additional 
case to check for /usr/bin/env, skip following white space, and then 
compare to python, but it would be more robust.

If it is thought that hard-coding a single space covers most of the 
uses, it should at least be emphasized in the documentation that only 
commands of than nature containing a single space will work with the 
launcher.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/9ae3d4f4/attachment.html>

From solipsis at pitrou.net  Fri Jul 22 01:35:31 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Jul 2011 01:35:31 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <20110722013531.4fa9ac30@pitrou.net>

On Tue, 19 Jul 2011 23:58:55 -0400
"P.J. Eby" <pje at telecommunity.com> wrote:
> 
> Anyway, to make a long story short, we came up with an alternative 
> implementation plan that actually solves some other problems besides 
> the one that PEP 382 sets out to solve, and whose implementation a 
> bit is easier to explain.  (In fact, for users coming from various 
> other languages, it hardly needs any explanation at all.)

I have a question.

If I have (on sys.path) a module "x.py" containing, say:

    y = 5

and (also on sys.path), a directory "x" containing a "y.py" module.

What is "from x import y" supposed to do?

(currently, it would bind "y" to its value in x.py)

Regards

Antoine.



From merwok at netwok.org  Fri Jul 22 01:37:05 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 22 Jul 2011 01:37:05 +0200
Subject: [Python-Dev] New tests in stable versions
In-Reply-To: <CADiSq7d9RTjgM8=BH5WOpxQUT034SmL-AxjpEWsGwgdFUae+Tg@mail.gmail.com>
References: <4E26FB19.2070406@netwok.org>
	<4E270190.1000209@haypocalc.com>	<j077ti$bql$1@dough.gmane.org>	<CAP1=2W7MMnBK+O=KFzj=jofFy4WbSx8mLhmOQHx59tdNDU3hvQ@mail.gmail.com>
	<CADiSq7d9RTjgM8=BH5WOpxQUT034SmL-AxjpEWsGwgdFUae+Tg@mail.gmail.com>
Message-ID: <4E28B821.3000800@netwok.org>

Le 21/07/2011 01:54, Nick Coghlan a ?crit :
> [...]
> So slightly more relaxed than Brett's view:
> - definitely apply bug fixes and their tests to affected maintenance branches
> - *optionally* apply coverage enhancements to maintenance branches,
> but don't feel obliged to do so
> 
> I see it as a productivity trade-off - time spent improving coverage
> on multiple branches is time not spent on other things. [...]

Thanks all!  Nick?s viewpoint and other people?s more-or-less according
replies are clear and make sense.

Cheers

From merwok at netwok.org  Fri Jul 22 01:42:55 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 22 Jul 2011 01:42:55 +0200
Subject: [Python-Dev] Indentation in reStructuredText documents
In-Reply-To: <87livss9li.fsf_-_@benfinney.id.au>
References: <E1QjXN2-000859-6x@dinsdale.python.org>
	<4E26E468.20800@netwok.org> <87livss9li.fsf_-_@benfinney.id.au>
Message-ID: <4E28B97F.7050106@netwok.org>

Le 21/07/2011 03:31, Ben Finney a ?crit :
> ?ric Araujo <merwok at netwok.org> writes:
>> FYI, reST uses three-space indents, not four (so that blocks align
>> nicely under the leading two dots + one space), so I think the change
>> was intentional.
> 
> No, reST doesn't specify any particular level of indentation.

My phrasing was a shortcut for ?the reST files in our documentation?, as
was made clear by followups.

Regards

From ncoghlan at gmail.com  Fri Jul 22 01:53:57 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 22 Jul 2011 09:53:57 +1000
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110722013531.4fa9ac30@pitrou.net>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
Message-ID: <CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>

On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 19 Jul 2011 23:58:55 -0400
> "P.J. Eby" <pje at telecommunity.com> wrote:
>>
>> Anyway, to make a long story short, we came up with an alternative
>> implementation plan that actually solves some other problems besides
>> the one that PEP 382 sets out to solve, and whose implementation a
>> bit is easier to explain. ?(In fact, for users coming from various
>> other languages, it hardly needs any explanation at all.)
>
> I have a question.
>
> If I have (on sys.path) a module "x.py" containing, say:
>
> ? ?y = 5
>
> and (also on sys.path), a directory "x" containing a "y.py" module.
>
> What is "from x import y" supposed to do?
>
> (currently, it would bind "y" to its value in x.py)

It would behave the same as it does today: the imported value of 'y' would be 5.

Virtual packages only kick in if an import would otherwise fail.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From solipsis at pitrou.net  Fri Jul 22 02:00:07 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Jul 2011 02:00:07 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
Message-ID: <1311292807.3039.4.camel@localhost.localdomain>

Le vendredi 22 juillet 2011 ? 09:53 +1000, Nick Coghlan a ?crit :
> On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Tue, 19 Jul 2011 23:58:55 -0400
> > "P.J. Eby" <pje at telecommunity.com> wrote:
> >>
> >> Anyway, to make a long story short, we came up with an alternative
> >> implementation plan that actually solves some other problems besides
> >> the one that PEP 382 sets out to solve, and whose implementation a
> >> bit is easier to explain.  (In fact, for users coming from various
> >> other languages, it hardly needs any explanation at all.)
> >
> > I have a question.
> >
> > If I have (on sys.path) a module "x.py" containing, say:
> >
> >    y = 5
> >
> > and (also on sys.path), a directory "x" containing a "y.py" module.
> >
> > What is "from x import y" supposed to do?
> >
> > (currently, it would bind "y" to its value in x.py)
> 
> It would behave the same as it does today: the imported value of 'y' would be 5.
> 
> Virtual packages only kick in if an import would otherwise fail.

Wouldn't it produce confusing situations like the above example?

Regards

Antoine.



From v+python at g.nevcal.com  Fri Jul 22 02:31:04 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 17:31:04 -0700
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <1311292807.3039.4.camel@localhost.localdomain>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
Message-ID: <4E28C4C8.5060104@g.nevcal.com>

On 7/21/2011 5:00 PM, Antoine Pitrou wrote:
> Le vendredi 22 juillet 2011 ? 09:53 +1000, Nick Coghlan a ?crit :
>> On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>>> On Tue, 19 Jul 2011 23:58:55 -0400
>>> "P.J. Eby"<pje at telecommunity.com>  wrote:
>>>> Anyway, to make a long story short, we came up with an alternative
>>>> implementation plan that actually solves some other problems besides
>>>> the one that PEP 382 sets out to solve, and whose implementation a
>>>> bit is easier to explain.  (In fact, for users coming from various
>>>> other languages, it hardly needs any explanation at all.)
>>> I have a question.
>>>
>>> If I have (on sys.path) a module "x.py" containing, say:
>>>
>>>     y = 5
>>>
>>> and (also on sys.path), a directory "x" containing a "y.py" module.
>>>
>>> What is "from x import y" supposed to do?
>>>
>>> (currently, it would bind "y" to its value in x.py)
>> It would behave the same as it does today: the imported value of 'y' would be 5.
>>
>> Virtual packages only kick in if an import would otherwise fail.
> Wouldn't it produce confusing situations like the above example?
>
> Regards
>
> Antoine.

If I have (on sys.path), a directory "x" containing a "y.py" module, and 
later (on sys.path), another directory "x" containing a "y.py" module, 
what is "from x import y" supposed to do?

OR

If I have (on sys.path), a module "x.py" containing, say:

    y = 5

and later (on sys.path), another module "x.py" containing, say:

    y = 6

what is "from x import y" supposed to do?


I guess I don't see how this new proposal makes anything more confusing 
than it already is?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/1c4d6a31/attachment.html>

From solipsis at pitrou.net  Fri Jul 22 02:38:05 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Jul 2011 02:38:05 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<4E28C4C8.5060104@g.nevcal.com>
Message-ID: <20110722023805.623f4a7f@pitrou.net>

On Thu, 21 Jul 2011 17:31:04 -0700
Glenn Linderman <v+python at g.nevcal.com> wrote:
> 
> If I have (on sys.path), a directory "x" containing a "y.py" module, and 
> later (on sys.path), another directory "x" containing a "y.py" module, 
> what is "from x import y" supposed to do?
> 
> OR
> 
> If I have (on sys.path), a module "x.py" containing, say:
> 
>     y = 5
> 
> and later (on sys.path), another module "x.py" containing, say:
> 
>     y = 6
> 
> what is "from x import y" supposed to do?
> 
> 
> I guess I don't see how this new proposal makes anything more confusing 
> than it already is?

It does. In your two examples, the "x.py" files (or the "x" directories)
live in two different base directories; imports are then resolved in
sys.path order, which is expected and intuitive.

However, you can have a "x.py" file and a "x" directory *in the same
base directory which is present in sys.path*, meaning sys.path can't
help disambiguate in this case.

Regards

Antoine.



From mhammond at skippinet.com.au  Fri Jul 22 02:44:01 2011
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Fri, 22 Jul 2011 10:44:01 +1000
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E28B016.5050500@g.nevcal.com>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
	<4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com>
	<4E28B016.5050500@g.nevcal.com>
Message-ID: <4E28C7D1.3010002@skippinet.com.au>

On 22/07/2011 9:02 AM, Glenn Linderman wrote:
> Bad logic is get_configured_value!  get_configured_value only looks in
> the global configuration file if there is a local configuration file
> that doesn't have the setting.  It should look in the global
> configuration file if there is no local configuration file _OR_ the is a
> local configuration file without the setting.
>
> I'll await a new launcher.

I just pushed a fix and hopefully Vinay will push a new MSI soon.

Thanks,

Mark

From v+python at g.nevcal.com  Fri Jul 22 02:53:09 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 21 Jul 2011 17:53:09 -0700
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110722023805.623f4a7f@pitrou.net>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<4E28C4C8.5060104@g.nevcal.com>
	<20110722023805.623f4a7f@pitrou.net>
Message-ID: <4E28C9F5.3040904@g.nevcal.com>

On 7/21/2011 5:38 PM, Antoine Pitrou wrote:
> However, you can have a "x.py" file and a "x" directory *in the same
> base directory which is present in sys.path*, meaning sys.path can't
> help disambiguate in this case.

Ah yes.  It means there has to be one more rule for disambiguation, 
which Nick supplied.  Your case wasn't clear to me from your first 
description, however.  As long as there is an ordering, and it is 
documented, it is not particularly confusing, however.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/9b289a7d/attachment.html>

From ncoghlan at gmail.com  Fri Jul 22 02:58:20 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 22 Jul 2011 10:58:20 +1000
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <1311292807.3039.4.camel@localhost.localdomain>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
Message-ID: <CADiSq7dKv43mcgkLcPkRjcba2AY6-zY2C1sVME7uc8O+zcqqOQ@mail.gmail.com>

On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Wouldn't it produce confusing situations like the above example?

I don't see how it is any more confusing than any other form of module
shadowing.

For backwards compatibility reasons, the precedence model will be:

1. Modules and self-contained packages that can satisfy the import
request are checked for first (along the whole length of sys.path).
2. If that fails, the virtual package mechanism is checked

PEP 402 eliminates some cases of package shadowing by making
__init__.py files optional, so your scenario will actually *work*, so
long as the submodule name doesn't conflict with a module attribute.

*Today* if you have:

x.py
x.pyd
x.so
x/__init__.py

in the same sys.path directory, x.py wins (search order is controlled
by the internal order of checks within the import system - and source
files are first on that list).

With PEP 302, x.py still wins, but the submodules within the x
directory become accessible so long as they don't conflict with
*actual* attributes set in the x module.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From riscutiavlad at gmail.com  Fri Jul 22 03:03:22 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Thu, 21 Jul 2011 18:03:22 -0700
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <j09vfk$1v5$1@dough.gmane.org>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org> <4E276AF0.5090409@gmail.com>
	<j09vfk$1v5$1@dough.gmane.org>
Message-ID: <CAJ-9HZ3CKc2+8Ko-KiXF2fsZbxAjbujBax-L8Fcw5Y=Lmtz4gg@mail.gmail.com>

I'm kind of -1 on changing Python executable name. It would make sense for
different major versions, where there are known incompatibilities, so
python2-python3 would make sense but python31 python32 not that much...

If my team is using Python and it gets pre-installed with other dev-tools,
do I need to let everyone know they must call python*31*? And if we upgrade,
make sure everyone knows they should now call python*32*? What if we have
scripts that call python? Make sure we update all of them whenever minor
version is changed?

The way I look at it, most people have only one version of Python installed
at one time and it's just extra burden to make them remember major+minor
version number they have. If you actually install multiple versions, you do
that for a reason and, since you know what you're doing, you would rather
remember to pass correct -v argument to py than users who *just want to use
Python*.

Thank you,
Vlad

On Thu, Jul 21, 2011 at 12:42 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 7/20/2011 7:55 PM, Mark Hammond wrote:
>
>> On 21/07/2011 4:38 AM, Terry Reedy wrote:
>>
>>  Many installers first make an organization directory and then an app
>>> directory within that. This annoys me sometimes when they only have one
>>> app to ever install, but is useful when there might really be multiple
>>> directories, as in our case. (Ditto for start menu entries.) This is
>>> what python should have done a decade ago.
>>>
>>
>  I disagree. If we followed that advice we would also be in "\Program
>> Files".
>>
>
> That is not what I suggested. I said let the use pick.
>
>
>  I have no problem with multiple Python versions installed
>> directly off the root, especially given most users probably have a very
>> small number of such installations. I think Python being a developer
>> tool rather than a user app is a reasonable justification for that (and
>> the justification used when the existing scheme was decided)
>>
>
> I put the multiple installations and several other directories into
> /programs. On my next machine, on order, I will use /devel.
>
>
>   > The two proposals
>>
>>> overlap but are not mutually exclusive. For future pythons, 'python33'
>>> is easier to remember and type than 'py -v 3.3' or whatever the proposed
>>> encantation is.
>>>
>>
>> 'py -3.3' - less chars to type than 'python33' and with no need to have
>> every Python directory on your PATH.
>>
>
> My proposal, as I clearly said, was for EXACTLY ONE directory to be added
> to PATH. In spite of Microsoft making is damned difficult for users to edit
> PATH, (and deleted programs not deleting their entries) I added
> 'C:/programs;'. I copied python32/python as py32 and python27/python as
> py27. Those are even fewer characters to type (4 versus 7). Now I can click
> a 'Command Prompt' icon and enter 'py32 -m test.regrtest' and it works
> without cd-ing to /programs/python32. Of course, I will have to re-copy with
> every install, which is why I would like something like this as part of
> installs.
>
>
>
>  IMO it is also simple enough that
>> people will remember it fairly easily.
>>
>
> py32 is even easier to remember.
>
>
>  Also, the launcher supports the ability to select either the 32 or 64bit
>> implementation - so maybe 'python33.exe' isn't really good enough and
>> should reflect the bittedness?
>>
>
> Like py32-6? If I install both Pythons on my new 64 bit machine, I will
> think about it, though I have no need for both now.
>
>
>  A python directory also gives a sensible (though optional) place to put
>>> other interpreters and even python-based apps. The launcher does not.
>>>
>>
>> What other interpreters? IMO it doesn't make sense to have IronPython,
>> jython etc be installed there. Ditto for apps - especially given most
>> apps tend to be tied to a subset of all possible Python versions.
>>
>
> If I install pypy, /programs is exactly where I would put it until I
> somehow discovered that to be a problem. Its startup could be copied as pp26
> or something.
>
>
> My idea may be not so good for general use, even though is now solves my
> problems, but please criticize what I said, allowing for obvious
> modifications like py32 instead of python32, and not a strawman that is
> wildly different.
>
> --
> Terry Jan Reedy
>
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/f86914c7/attachment.html>

From ncoghlan at gmail.com  Fri Jul 22 03:03:26 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 22 Jul 2011 11:03:26 +1000
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <4E28C9F5.3040904@g.nevcal.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<4E28C4C8.5060104@g.nevcal.com>
	<20110722023805.623f4a7f@pitrou.net>
	<4E28C9F5.3040904@g.nevcal.com>
Message-ID: <CADiSq7fhd1OFeeH6smCQJ8n_WvKgFeTtTOVc-EZxQ9u6D928OA@mail.gmail.com>

On Fri, Jul 22, 2011 at 10:53 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> Ah yes.? It means there has to be one more rule for disambiguation, which
> Nick supplied.? Your case wasn't clear to me from your first description,
> however.? As long as there is an ordering, and it is documented, it is not
> particularly confusing, however.

The genuinely confusing part is that x.py still takes precedence, even
if it appears on sys.path *after* x/y.py.

However, we're forced into that behaviour by backwards compatibility
requirements. The alternative of allowing x/y.py to take precedence
has been rejected on those grounds more than once.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From solipsis at pitrou.net  Fri Jul 22 03:04:49 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Jul 2011 03:04:49 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <CADiSq7dKv43mcgkLcPkRjcba2AY6-zY2C1sVME7uc8O+zcqqOQ@mail.gmail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<CADiSq7dKv43mcgkLcPkRjcba2AY6-zY2C1sVME7uc8O+zcqqOQ@mail.gmail.com>
Message-ID: <1311296689.3039.7.camel@localhost.localdomain>


Le vendredi 22 juillet 2011 ? 10:58 +1000, Nick Coghlan a ?crit :
> On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > Wouldn't it produce confusing situations like the above example?
> 
> I don't see how it is any more confusing than any other form of module
> shadowing.

The additional confusion lies in the fact that a module can be shadowed
by something which is not a module (a mere global variable). I find it
rather baffling.

Regards

Antoine.



From merwok at netwok.org  Fri Jul 22 03:07:31 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 22 Jul 2011 03:07:31 +0200
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <CAJ-9HZ3CKc2+8Ko-KiXF2fsZbxAjbujBax-L8Fcw5Y=Lmtz4gg@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>	<20110719171647.03f9f9c9@pitrou.net>	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>	<j05e3c$g9c$1@dough.gmane.org>	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>	<j077bn$855$1@dough.gmane.org>
	<4E276AF0.5090409@gmail.com>	<j09vfk$1v5$1@dough.gmane.org>
	<CAJ-9HZ3CKc2+8Ko-KiXF2fsZbxAjbujBax-L8Fcw5Y=Lmtz4gg@mail.gmail.com>
Message-ID: <4E28CD53.1090707@netwok.org>

Hi,

Le 22/07/2011 03:03, Vlad Riscutia a ?crit :
> I'm kind of -1 on changing Python executable name. It would make sense for
> different major versions, where there are known incompatibilities, so
> python2-python3 would make sense but python31 python32 not that much...
> 
> If my team is using Python and it gets pre-installed with other dev-tools,
> do I need to let everyone know they must call python*31*? And if we upgrade,
> make sure everyone knows they should now call python*32*? What if we have
> scripts that call python? Make sure we update all of them whenever minor
> version is changed?

If I understand correctly, adding versioned filenames like python3.3.exe
would happen in addition of python.exe, not in replacement.

Regards

From riscutiavlad at gmail.com  Fri Jul 22 03:30:21 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Thu, 21 Jul 2011 18:30:21 -0700
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <4E28CD53.1090707@netwok.org>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org> <4E276AF0.5090409@gmail.com>
	<j09vfk$1v5$1@dough.gmane.org>
	<CAJ-9HZ3CKc2+8Ko-KiXF2fsZbxAjbujBax-L8Fcw5Y=Lmtz4gg@mail.gmail.com>
	<4E28CD53.1090707@netwok.org>
Message-ID: <CAJ-9HZ0YCW4OXgwgFo60TmK_fY3vmCjXLkqjZwXZ7adzntq1SA@mail.gmail.com>

If versioned filenames are added in addition to python.exe, it still might
look confusing for most users: Why do I have python and python3.2
executables? What's the difference? I'd rather go with -v argument either
way, for people that *know* they want to call Python 3.2 instead of Python
3.1...

Thank you,
Vlad

On Thu, Jul 21, 2011 at 6:07 PM, ?ric Araujo <merwok at netwok.org> wrote:

> Hi,
>
> Le 22/07/2011 03:03, Vlad Riscutia a ?crit :
> > I'm kind of -1 on changing Python executable name. It would make sense
> for
> > different major versions, where there are known incompatibilities, so
> > python2-python3 would make sense but python31 python32 not that much...
> >
> > If my team is using Python and it gets pre-installed with other
> dev-tools,
> > do I need to let everyone know they must call python*31*? And if we
> upgrade,
> > make sure everyone knows they should now call python*32*? What if we have
> > scripts that call python? Make sure we update all of them whenever minor
> > version is changed?
>
> If I understand correctly, adding versioned filenames like python3.3.exe
> would happen in addition of python.exe, not in replacement.
>
> Regards
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110721/d5a8a0fe/attachment.html>

From pje at telecommunity.com  Fri Jul 22 04:05:22 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 21 Jul 2011 22:05:22 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <1311296689.3039.7.camel@localhost.localdomain>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<CADiSq7dKv43mcgkLcPkRjcba2AY6-zY2C1sVME7uc8O+zcqqOQ@mail.gmail.com>
	<1311296689.3039.7.camel@localhost.localdomain>
Message-ID: <20110722020615.71C913A40AA@sparrow.telecommunity.com>

At 03:04 AM 7/22/2011 +0200, Antoine Pitrou wrote:
>The additional confusion lies in the fact that a module can be 
>shadowed by something which is not a module (a mere global 
>variable). I find it rather baffling.

If you move x.py to x/__init__.py, it does *exactly the same thing* 
in current versions of Python:

Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> from x import y
 >>> import x.y
 >>> x.y
<module 'x.y' from 'x\y.py'>
 >>> y
5

The PEP does nothing new or different here.  If something is baffling 
you, it's the behavior of "from ... import", not the actual importing process.

"from x import y" means "import x; y = x.y".  The PEP does not 
propose we change this.  ;-)


From ncoghlan at gmail.com  Fri Jul 22 04:21:51 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 22 Jul 2011 12:21:51 +1000
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <1311296689.3039.7.camel@localhost.localdomain>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<CADiSq7dKv43mcgkLcPkRjcba2AY6-zY2C1sVME7uc8O+zcqqOQ@mail.gmail.com>
	<1311296689.3039.7.camel@localhost.localdomain>
Message-ID: <CADiSq7c1EeiOd7HBPQ5dTweBAdBiy1+HGrjE-yoR+RcwMenP+Q@mail.gmail.com>

On Fri, Jul 22, 2011 at 11:04 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Le vendredi 22 juillet 2011 ? 10:58 +1000, Nick Coghlan a ?crit :
>> On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > Wouldn't it produce confusing situations like the above example?
>>
>> I don't see how it is any more confusing than any other form of module
>> shadowing.
>
> The additional confusion lies in the fact that a module can be shadowed
> by something which is not a module (a mere global variable). I find it
> rather baffling.

It's still an improvement on current Python. There a submodule can be
shadowed uselessly by something that doesn't even exist. For example:

x.py <-- No 'y' attribute
x/__init__.py <-- not needed in PEP 402
x/y.py

from x import y  <-- ImportError now, but would work in PEP 402

However, this does highlight an interesting corner case not yet
covered by the PEP: when building a virtual path to add to an existing
module, what do we do with directories that contain __init__.py[co]
files?

1. Ignore the entire directory (i.e leave it out of the created path)?
(always emit ImportWarning)
2. Ignore the file and add the directory to the created path anyway?
(never emit ImportWarning)
3. Ignore the file and add the directory to the created path anyway?
(emit ImportWarning if __init__.py is not empty)
4. Ignore the file only if it is empty, otherwise ignore the whole
directory? (emit ImportWarning if __init__.py is not empty)
5. Execute the file in the namespace of the existing module?

I suspect option 1 will lead to the fewest quirks, since it preserves
current shadowing behaviour for modules and self-contained packages.

Cheers,
Nick.


-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From stephen at xemacs.org  Fri Jul 22 07:22:52 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 22 Jul 2011 14:22:52 +0900
Subject: [Python-Dev] New update to PEP397 - Python launcher for Windows
In-Reply-To: <4E28B26D.7070902@g.nevcal.com>
References: <4E27C8BA.8040000@gmail.com>
	<4E28B26D.7070902@g.nevcal.com>
Message-ID: <877h7asxcj.fsf@uwakimon.sk.tsukuba.ac.jp>

Glenn Linderman writes:

 > I note in passing that '/usr/bin/env python' is hard-coded in the 
 > launcher, which conforms to the above documentation.

A single character (space or tab) is also hard-coded in Emacs's
python-mode.

 > But there is no hard requirement in Unix, if I correctly understand
 > it, that '/usr/bin/env' be separated from 'python' (or whatever) by
 > exactly one space.

No, there is no hard requirement; the shebang still is not
standardized AFAIK.  A single space is probably the most portable
choice, however.

It would not be hard to allow arbitrary whitespace, but I think
documenting the restriction is sufficient.


From greg.ewing at canterbury.ac.nz  Fri Jul 22 09:37:40 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 22 Jul 2011 19:37:40 +1200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <1311296689.3039.7.camel@localhost.localdomain>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<CADiSq7dKv43mcgkLcPkRjcba2AY6-zY2C1sVME7uc8O+zcqqOQ@mail.gmail.com>
	<1311296689.3039.7.camel@localhost.localdomain>
Message-ID: <4E2928C4.70506@canterbury.ac.nz>

Antoine Pitrou wrote:

> The additional confusion lies in the fact that a module can be shadowed
> by something which is not a module (a mere global variable). I find it
> rather baffling.

I think we're stuck with that as long as we use the same
syntax for importing a submodule and importing a non-module
name from a module, i.e. 'from x import y'.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Jul 22 09:37:45 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 22 Jul 2011 19:37:45 +1200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <20110722020615.71C913A40AA@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<20110722013531.4fa9ac30@pitrou.net>
	<CADiSq7egN=c82CnyMw-Hntw9-YGvkAyq5BVo2=wNHuGW1XESWA@mail.gmail.com>
	<1311292807.3039.4.camel@localhost.localdomain>
	<CADiSq7dKv43mcgkLcPkRjcba2AY6-zY2C1sVME7uc8O+zcqqOQ@mail.gmail.com>
	<1311296689.3039.7.camel@localhost.localdomain>
	<20110722020615.71C913A40AA@sparrow.telecommunity.com>
Message-ID: <4E2928C9.2070903@canterbury.ac.nz>

P.J. Eby wrote:

> "from x import y" means "import x; y = x.y".

It actually means slightly more that that if y is a submodule,
in which case it means "import x.y; y = x.y".

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Jul 22 11:29:23 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 22 Jul 2011 21:29:23 +1200
Subject: [Python-Dev] Import lock considered mysterious
Message-ID: <4E2942F3.3090704@canterbury.ac.nz>

I recently encountered a very mysterious bug in which
a background thread would inexplicably hang while attempting
to make a connection using httplib.

Debugging as far as I could at the Python level led to
the surprising conclusion that it was hanging while
using the ascii codec to encode the host name.

I couldn't imagine why, until I resorted to breaking into
it with gdb and found that it was trying to import
"codecs.ascii", and blocked on acquiring the import lock.

The reason for *that* was that my main module was a stub
that imported the real main module, which did all its
work directly from the module code. So the whole program
was effectively running inside an import statement and
holding onto the import lock.

Once I realised what was happening, it was easy to fix,
but I'm a bit disturbed about how difficult it was to
track down the cause.

This whole episode seems to have resulted from a collision
between two rather obscure things: that import statements
involve locking things, and that some fairly innocuous
looking calls, such as s.encode('ascii'), can trigger an
import.

I'm wondering whether anything can be done to make problems
like this less likely to occur, or at least to make the
cause more readily apparent. One shouldn't really have to
use gdb to track down bugs in Python code.

Is it really necessary to hold the import lock for so long?
Presumably the import lock is there to protect access to
things like sys.modules. Is that all? Could it be released
after the module code is loaded and sys.modules has been
updated, but before executing the module body?

-- 
Greg

From fuzzyman at voidspace.org.uk  Fri Jul 22 11:49:04 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 22 Jul 2011 10:49:04 +0100
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <CAJ-9HZ0YCW4OXgwgFo60TmK_fY3vmCjXLkqjZwXZ7adzntq1SA@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org> <4E276AF0.5090409@gmail.com>
	<j09vfk$1v5$1@dough.gmane.org>
	<CAJ-9HZ3CKc2+8Ko-KiXF2fsZbxAjbujBax-L8Fcw5Y=Lmtz4gg@mail.gmail.com>
	<4E28CD53.1090707@netwok.org>
	<CAJ-9HZ0YCW4OXgwgFo60TmK_fY3vmCjXLkqjZwXZ7adzntq1SA@mail.gmail.com>
Message-ID: <4E294790.9020409@voidspace.org.uk>

On 22/07/2011 02:30, Vlad Riscutia wrote:
> If versioned filenames are added in addition to python.exe, it still 
> might look confusing for most users: Why do I have python and 
> python3.2 executables? What's the difference? I'd rather go with -v 
> argument either way, for people that /know/ they want to call Python 
> 3.2 instead of Python 3.1...
>

It doesn't seem to be too confusing for Linux / Mac OS X users where you 
have both. What's more it's very useful. I still like being able to 
specify version from the launcher, it's probably what I will use it most 
for (on the rare occasions I still use Windows).

Michael

> Thank you,
> Vlad
>
> On Thu, Jul 21, 2011 at 6:07 PM, ?ric Araujo <merwok at netwok.org 
> <mailto:merwok at netwok.org>> wrote:
>
>     Hi,
>
>     Le 22/07/2011 03:03, Vlad Riscutia a ?crit :
>     > I'm kind of -1 on changing Python executable name. It would make
>     sense for
>     > different major versions, where there are known
>     incompatibilities, so
>     > python2-python3 would make sense but python31 python32 not that
>     much...
>     >
>     > If my team is using Python and it gets pre-installed with other
>     dev-tools,
>     > do I need to let everyone know they must call python*31*? And if
>     we upgrade,
>     > make sure everyone knows they should now call python*32*? What
>     if we have
>     > scripts that call python? Make sure we update all of them
>     whenever minor
>     > version is changed?
>
>     If I understand correctly, adding versioned filenames like
>     python3.3.exe
>     would happen in addition of python.exe, not in replacement.
>
>     Regards
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110722/0d2a0dd7/attachment.html>

From cs at zip.com.au  Fri Jul 22 11:58:46 2011
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 22 Jul 2011 19:58:46 +1000
Subject: [Python-Dev] Import lock considered mysterious
In-Reply-To: <4E2942F3.3090704@canterbury.ac.nz>
References: <4E2942F3.3090704@canterbury.ac.nz>
Message-ID: <20110722095846.GA32631@cskk.homeip.net>

On 22Jul2011 21:29, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
[...]
| This whole episode seems to have resulted from a collision
| between two rather obscure things: that import statements
| involve locking things,

Necessary to avoid performing the module definitons twice when a module
is imported twice, surely?

| and that some fairly innocuous
| looking calls, such as s.encode('ascii'), can trigger an
| import.

There are plenty of occasions where an import might be deferred until a
function call - it is a common workaround for otherwise circular
imports. Personally I also sometimes defer an import if I'm importing
something "large" that the module would only depend upon on in unusual
(or anyway, often avoidable) circumstances; for example, a facility not
in the stdlib, which a caller might avoid requiring by choosing a
different aspect of the importing module such as choosing a CSV storage
backend over and SQLalchemy backend.

| Is it really necessary to hold the import lock for so long?
| Presumably the import lock is there to protect access to
| things like sys.modules. Is that all? Could it be released
| after the module code is loaded and sys.modules has been
| updated, but before executing the module body?

Sometimes names are made by executing the module body.
Since one expects to access the module's names after the import, how is
this avoidable?

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

What's the point of having Nebraska if we can't put it to its highest and
best use?       - Patrick Bedard arguing for a 100 mph speed limit

From p.f.moore at gmail.com  Fri Jul 22 12:30:40 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 22 Jul 2011 11:30:40 +0100
Subject: [Python-Dev] Import lock considered mysterious
In-Reply-To: <4E2942F3.3090704@canterbury.ac.nz>
References: <4E2942F3.3090704@canterbury.ac.nz>
Message-ID: <CACac1F8+1yyNjxYQdPP5aWiKaGxQPhEkg1OUqmFQ5VwPuxurhA@mail.gmail.com>

On 22 July 2011 10:29, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> The reason for *that* was that my main module was a stub
> that imported the real main module, which did all its
> work directly from the module code. So the whole program
> was effectively running inside an import statement and
> holding onto the import lock.

I realise you probably know this, but I think this is covered by the
standard advice to avoid having substantial code running at module
level (i.e., on import). It may be useful to clarify or further
emphasize that advice in the Python documentation.

Paul.

PS This is not to say that I oppose trying to reduce the chance of
this type of thing happening...

From solipsis at pitrou.net  Fri Jul 22 14:48:58 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Jul 2011 14:48:58 +0200
Subject: [Python-Dev] Import lock considered mysterious
References: <4E2942F3.3090704@canterbury.ac.nz>
Message-ID: <20110722144858.49cb7f2f@pitrou.net>

On Fri, 22 Jul 2011 21:29:23 +1200
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
> This whole episode seems to have resulted from a collision
> between two rather obscure things: that import statements
> involve locking things, and that some fairly innocuous
> looking calls, such as s.encode('ascii'), can trigger an
> import.

Indeed.

> I'm wondering whether anything can be done to make problems
> like this less likely to occur, or at least to make the
> cause more readily apparent. One shouldn't really have to
> use gdb to track down bugs in Python code.

See http://bugs.python.org/issue9260

There's a patch there but it needs additional sophistication to remove
deadlocks when doing concurrent circular imports.

Regards

Antoine.



From brian.curtin at gmail.com  Fri Jul 22 15:41:26 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Fri, 22 Jul 2011 08:41:26 -0500
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <CAJ-9HZ0YCW4OXgwgFo60TmK_fY3vmCjXLkqjZwXZ7adzntq1SA@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org> <4E276AF0.5090409@gmail.com>
	<j09vfk$1v5$1@dough.gmane.org>
	<CAJ-9HZ3CKc2+8Ko-KiXF2fsZbxAjbujBax-L8Fcw5Y=Lmtz4gg@mail.gmail.com>
	<4E28CD53.1090707@netwok.org>
	<CAJ-9HZ0YCW4OXgwgFo60TmK_fY3vmCjXLkqjZwXZ7adzntq1SA@mail.gmail.com>
Message-ID: <CAD+XWwoRvQUaBz1aTeOdUDerhixSRDXREodOjCeeR9ZHWFcGKw@mail.gmail.com>

On Thu, Jul 21, 2011 at 20:30, Vlad Riscutia <riscutiavlad at gmail.com> wrote:

> If versioned filenames are added in addition to python.exe, it still might
> look confusing for most users: Why do I have python and python3.2
> executables? What's the difference? I'd rather go with -v argument either
> way, for people that *know* they want to call Python 3.2 instead of Python
> 3.1...
>
> Thank you,
> Vlad
>

Honestly, would it really be that confusing? Seeing python32.exe inside
C:\Python32 shouldn't be a huge surprise, and ActiveState has been doing
something like this for years (forever?).

Versioned executables in addition to the standard python.exe is something
I've wanted for a while, but that's outside of this PEP. This way you could
have C:\Python27 and C:\Python32 on your path and explicitly open up the
right one.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110722/ac0c8d33/attachment.html>

From ncoghlan at gmail.com  Fri Jul 22 16:24:20 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 23 Jul 2011 00:24:20 +1000
Subject: [Python-Dev] Import lock considered mysterious
In-Reply-To: <4E2942F3.3090704@canterbury.ac.nz>
References: <4E2942F3.3090704@canterbury.ac.nz>
Message-ID: <CADiSq7ekhO=SMrcCskVO+NS92aLT3e+Eibs+9PJcYvRUB0usVA@mail.gmail.com>

On Fri, Jul 22, 2011 at 7:29 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Is it really necessary to hold the import lock for so long?
> Presumably the import lock is there to protect access to
> things like sys.modules. Is that all? Could it be released
> after the module code is loaded and sys.modules has been
> updated, but before executing the module body?

No, because we don't know if we're keeping it until the module body is
executed successfully and we don't want other threads to see the
partially initialised module.

There's a reason the docs say never to spawn and then wait for threads
as a side effect of import
(http://docs.python.org/library/threading#importing-in-threaded-code)

If you want to delegate main module execution to another file, then
runpy.run_module is available.

Antoine's fine grained import locking patch (or something along those
lines) would improve matters, but currently causes its own deadlock
problems.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From pje at telecommunity.com  Fri Jul 22 17:01:57 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 22 Jul 2011 11:01:57 -0400
Subject: [Python-Dev] Import lock considered mysterious
In-Reply-To: <20110722144858.49cb7f2f@pitrou.net>
References: <4E2942F3.3090704@canterbury.ac.nz>
	<20110722144858.49cb7f2f@pitrou.net>
Message-ID: <20110722150247.6C7083A409B@sparrow.telecommunity.com>

At 02:48 PM 7/22/2011 +0200, Antoine Pitrou wrote:

>See http://bugs.python.org/issue9260
>
>There's a patch there but it needs additional sophistication to remove
>deadlocks when doing concurrent circular imports.

I don't think that approach can work, as PEP 302 loaders can 
currently assume the global import lock is being held when they 
run...  and in general, there are too many global data structures in 
sys that need to be protected by code that uses such things.

A simpler solution to Greg's problem would be to have a timeout on 
attempts to acquire the import lock, and have it fail with a 
RuntimeError describing the problem.  (*Not* an ImportError, mind 
you, which might get ignored and trigger a different code path.)

The timeout would need to be on the order of seconds to prevent false 
positives, and there'd need to be a way to change or remove the 
timeout in the event somebody really needs to.  But it would 
eliminate the mysteriousness.  A unique and Google-able error message 
would let someone find a clear explanation of what's going on, as well.

A second thing that *could* be helpful would be to issue a warning 
when a new thread is started (or waited on) while the import lock is 
held.  This is already known to be a bad thing to do.

The tricky part is issuing the warning for the right caller level, 
but I suppose you could walk back up the call stack until you found 
some module-level code, and then fingered that line of code as the culprit.

Yes, that might do it: the code for starting or waiting on a thread, 
could check to see if the import lock is held by the current thread, 
and if so, walk up the stack to find a module frame (one where 
f_globals is f_locals and __name__ in f_locals and 
sys.modules[__name__].__dict__ is f_locals), and if one is found, 
issue a warning about not starting or waiting on threads in module-level code.

Between that and the timeout, the mysteriousness could be completely 
done away with, without throwing a monkey wrench into the current 
import mechanisms.


From solipsis at pitrou.net  Fri Jul 22 17:07:07 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Jul 2011 17:07:07 +0200
Subject: [Python-Dev] Import lock considered mysterious
In-Reply-To: <20110722150247.6C7083A409B@sparrow.telecommunity.com>
References: <4E2942F3.3090704@canterbury.ac.nz>
	<20110722144858.49cb7f2f@pitrou.net>
	<20110722150247.6C7083A409B@sparrow.telecommunity.com>
Message-ID: <1311347227.3605.4.camel@localhost.localdomain>


> >See http://bugs.python.org/issue9260
> >
> >There's a patch there but it needs additional sophistication to remove
> >deadlocks when doing concurrent circular imports.
> 
> I don't think that approach can work, as PEP 302 loaders can 
> currently assume the global import lock is being held when they 
> run...  and in general, there are too many global data structures in 
> sys that need to be protected by code that uses such things.

Some of the comments in the issue are about that. The global import lock
could *still* protect those structures when needed. It just doesn't have
to be held when executing a module's body.

> Between that and the timeout, the mysteriousness could be completely 
> done away with, without throwing a monkey wrench into the current 
> import mechanisms.

Isn't the current import mechanism already a monkey wrench?

Regards

Antoine.



From riscutiavlad at gmail.com  Fri Jul 22 18:02:35 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Fri, 22 Jul 2011 09:02:35 -0700
Subject: [Python-Dev] Python launcher command line usage (Was: 3.2.1
 encoding surprise)
In-Reply-To: <CAD+XWwoRvQUaBz1aTeOdUDerhixSRDXREodOjCeeR9ZHWFcGKw@mail.gmail.com>
References: <CACac1F_VJ04KJ+oyOmm1sTwEKSay7z3R0hvQVNRLUcV1eVMFjQ@mail.gmail.com>
	<20110719171647.03f9f9c9@pitrou.net>
	<CACac1F9CAJDsa0jCdw6uXVMkzmW19rk7a1+vUWkcJ3gTEisyyA@mail.gmail.com>
	<j05e3c$g9c$1@dough.gmane.org>
	<CACac1F-VUHcMt5atV3wWDgbkHE_xqboPrnHkpw6rzoaikEWg3g@mail.gmail.com>
	<j077bn$855$1@dough.gmane.org> <4E276AF0.5090409@gmail.com>
	<j09vfk$1v5$1@dough.gmane.org>
	<CAJ-9HZ3CKc2+8Ko-KiXF2fsZbxAjbujBax-L8Fcw5Y=Lmtz4gg@mail.gmail.com>
	<4E28CD53.1090707@netwok.org>
	<CAJ-9HZ0YCW4OXgwgFo60TmK_fY3vmCjXLkqjZwXZ7adzntq1SA@mail.gmail.com>
	<CAD+XWwoRvQUaBz1aTeOdUDerhixSRDXREodOjCeeR9ZHWFcGKw@mail.gmail.com>
Message-ID: <CAJ-9HZ0GzHw42i=ZxLwy6ijXxJuXV3YxZtLspqzUweNsQh8TOw@mail.gmail.com>

OK then. I don't have a *strong* opinion against it, just thought that most
people have one version of Python, maybe 2 versions as in 2.x and 3.x, so I
would understand python2.exe, python3.exe but yeah, it's not that big of a
deal either way.

Thank you,
Vlad

On Fri, Jul 22, 2011 at 6:41 AM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Thu, Jul 21, 2011 at 20:30, Vlad Riscutia <riscutiavlad at gmail.com>wrote:
>
>> If versioned filenames are added in addition to python.exe, it still might
>> look confusing for most users: Why do I have python and python3.2
>> executables? What's the difference? I'd rather go with -v argument either
>> way, for people that *know* they want to call Python 3.2 instead of
>> Python 3.1...
>>
>> Thank you,
>> Vlad
>>
>
> Honestly, would it really be that confusing? Seeing python32.exe inside
> C:\Python32 shouldn't be a huge surprise, and ActiveState has been doing
> something like this for years (forever?).
>
> Versioned executables in addition to the standard python.exe is something
> I've wanted for a while, but that's outside of this PEP. This way you could
> have C:\Python27 and C:\Python32 on your path and explicitly open up the
> right one.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110722/7451ce49/attachment.html>

From status at bugs.python.org  Fri Jul 22 18:07:26 2011
From: status at bugs.python.org (Python tracker)
Date: Fri, 22 Jul 2011 18:07:26 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110722160726.B02D01DE28@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-07-15 - 2011-07-22)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2886 ( +6)
  closed 21507 (+33)
  total  24393 (+39)

Open issues with patches: 1253 


Issues opened (29)
==================

#10881: test_site and macframework builds fails
http://bugs.python.org/issue10881  reopened by vinay.sajip

#12266: str.capitalize contradicts oneself
http://bugs.python.org/issue12266  reopened by ezio.melotti

#12575: add a AST validator
http://bugs.python.org/issue12575  opened by benjamin.peterson

#12576: urlib.request fails to open some sites
http://bugs.python.org/issue12576  opened by daniel.ugra

#12578: Erratic socket.gaierror: [Errno 11004] when using smtplib
http://bugs.python.org/issue12578  opened by David.Ward

#12581: Increased test coverage of test_urlparse
http://bugs.python.org/issue12581  opened by haggholm

#12583: More detailed ImportError messages
http://bugs.python.org/issue12583  opened by cool-RR

#12585: distutils dereferences symlinks for zip but not for bztar/gzta
http://bugs.python.org/issue12585  opened by fberger

#12586: Enhanced email API: header objects
http://bugs.python.org/issue12586  opened by r.david.murray

#12588: test_capi.test_subinterps() failed on OpenBSD (powerpc)
http://bugs.python.org/issue12588  opened by rpointel

#12589: test_long.test_nan_inf() failed on OpenBSD (powerpc)
http://bugs.python.org/issue12589  opened by rpointel

#12590: First line and cursor not visible when opening files
http://bugs.python.org/issue12590  opened by sforman

#12591: text files returned by subprocess.Popen with universal_newline
http://bugs.python.org/issue12591  opened by anacrolix

#12592: modify configure.in to detect OpenBSD 5.x
http://bugs.python.org/issue12592  opened by rpointel

#12594: Docs for py3k still refer to "MacPython 2.5 folder"
http://bugs.python.org/issue12594  opened by hop

#12595: python.h redundant redeclarations
http://bugs.python.org/issue12595  opened by verticalduck

#12596: cPickle - stored data differ for same dictionary
http://bugs.python.org/issue12596  opened by Philipp.M??lders

#12598: Move sys variable initialization from import.c to sysmodule.c
http://bugs.python.org/issue12598  opened by ericsnow

#12599: Use 'is not None' where appropriate in importlib
http://bugs.python.org/issue12599  opened by ncoghlan

#12600: Support parameterized TestCases in unittest
http://bugs.python.org/issue12600  opened by abingham

#12602: Missing using docs cross-references
http://bugs.python.org/issue12602  opened by ncoghlan

#12603: pydoc.synopsis breaks if filesystem returns mtime of 0 (common
http://bugs.python.org/issue12603  opened by joshtriplett

#12604: VTRACE macro in _sre.c should use do {} while (0)
http://bugs.python.org/issue12604  opened by joshtriplett

#12605: Enhancements to gdb 7 debugging hooks
http://bugs.python.org/issue12605  opened by dmalcolm

#12606: Mutable Sequence Type works different for lists and bytearrays
http://bugs.python.org/issue12606  opened by py.user

#12607: subprocess(stdout=..., stderr=sys.stdout) breaks stderr for ch
http://bugs.python.org/issue12607  opened by chn

#12608: crash in PyAST_Compile when running Python code
http://bugs.python.org/issue12608  opened by Albert.Zeyer

#12610: Fatal Python error: non-string found in code slot
http://bugs.python.org/issue12610  opened by Albert.Zeyer

#12611: 2to3 crashes when converting doctest using reduce()
http://bugs.python.org/issue12611  opened by VPeric



Most recent 15 issues with no replies (15)
==========================================

#12611: 2to3 crashes when converting doctest using reduce()
http://bugs.python.org/issue12611

#12610: Fatal Python error: non-string found in code slot
http://bugs.python.org/issue12610

#12604: VTRACE macro in _sre.c should use do {} while (0)
http://bugs.python.org/issue12604

#12603: pydoc.synopsis breaks if filesystem returns mtime of 0 (common
http://bugs.python.org/issue12603

#12599: Use 'is not None' where appropriate in importlib
http://bugs.python.org/issue12599

#12598: Move sys variable initialization from import.c to sysmodule.c
http://bugs.python.org/issue12598

#12595: python.h redundant redeclarations
http://bugs.python.org/issue12595

#12592: modify configure.in to detect OpenBSD 5.x
http://bugs.python.org/issue12592

#12590: First line and cursor not visible when opening files
http://bugs.python.org/issue12590

#12586: Enhanced email API: header objects
http://bugs.python.org/issue12586

#12562: calling mmap twice fails on Windows
http://bugs.python.org/issue12562

#12553: email should default to 8bit CTE unless policy.must_be_7bit is
http://bugs.python.org/issue12553

#12542: Remove duplicates of cmp_to_key used in tests
http://bugs.python.org/issue12542

#12541: Accepting Badly formed headers in urllib HTTPBasicAuth
http://bugs.python.org/issue12541

#12532: PyPI server index names with '/' in them
http://bugs.python.org/issue12532



Most recent 15 issues waiting for review (15)
=============================================

#12607: subprocess(stdout=..., stderr=sys.stdout) breaks stderr for ch
http://bugs.python.org/issue12607

#12605: Enhancements to gdb 7 debugging hooks
http://bugs.python.org/issue12605

#12598: Move sys variable initialization from import.c to sysmodule.c
http://bugs.python.org/issue12598

#12586: Enhanced email API: header objects
http://bugs.python.org/issue12586

#12581: Increased test coverage of test_urlparse
http://bugs.python.org/issue12581

#12575: add a AST validator
http://bugs.python.org/issue12575

#12572: HP/UX compiler workarounds
http://bugs.python.org/issue12572

#12567: curses implementation of Unicode is wrong in Python 3
http://bugs.python.org/issue12567

#12561: Compiler workaround for wide string constants in Modules/getpa
http://bugs.python.org/issue12561

#12560: libpython.so not built on OpenBSD
http://bugs.python.org/issue12560

#12559: gzip.open() needs an optional encoding argument
http://bugs.python.org/issue12559

#12555: PEP 3151 implementation
http://bugs.python.org/issue12555

#12546: builtin __format__ methods cannot fill with \x00 char
http://bugs.python.org/issue12546

#12542: Remove duplicates of cmp_to_key used in tests
http://bugs.python.org/issue12542

#12531: documentation index entries for * and **
http://bugs.python.org/issue12531



Top 10 most discussed issues (10)
=================================

#7897: Support parametrized tests in unittest
http://bugs.python.org/issue7897  24 msgs

#11343: Make errors due to full parser stack identifiable
http://bugs.python.org/issue11343  18 msgs

#12589: test_long.test_nan_inf() failed on OpenBSD (powerpc)
http://bugs.python.org/issue12589  15 msgs

#12572: HP/UX compiler workarounds
http://bugs.python.org/issue12572  13 msgs

#12583: More detailed ImportError messages
http://bugs.python.org/issue12583  13 msgs

#1626300: 'Installing Python Modules' does not work for Windows
http://bugs.python.org/issue1626300   9 msgs

#12326: Linux 3: tests should avoid using sys.platform == 'linux2'
http://bugs.python.org/issue12326   8 msgs

#12540: "Restart Shell" command leaves pythonw.exe processes running
http://bugs.python.org/issue12540   8 msgs

#1170: shlex have problems with parsing unicode
http://bugs.python.org/issue1170   7 msgs

#6721: Locks in python standard library should be sanitized on fork
http://bugs.python.org/issue6721   7 msgs



Issues closed (34)
==================

#7484: smtplib: verify breaks with Postfix servers
http://bugs.python.org/issue7484  closed by r.david.murray

#10271: warnings.showwarning should allow any callable object
http://bugs.python.org/issue10271  closed by brett.cannon

#10309: dlmalloc.c needs _GNU_SOURCE for mremap()
http://bugs.python.org/issue10309  closed by barry

#11055: OS X IDLE 3.2 Save As menu accelerator opens two Save windows
http://bugs.python.org/issue11055  closed by ned.deily

#11321: 9th import of module _pickle always crashes
http://bugs.python.org/issue11321  closed by pitrou

#11436: Clarify struct doc for format 's', when it is mentioned withou
http://bugs.python.org/issue11436  closed by python-dev

#11603: Python crashes or hangs when rebinding __repr__ as __str__
http://bugs.python.org/issue11603  closed by pitrou

#11627: segfault raising an arbitrary object as an exception
http://bugs.python.org/issue11627  closed by python-dev

#11629: Reference implementation for PEP 397
http://bugs.python.org/issue11629  closed by mhammond

#12273: Change ast.__version__ calculation to provide consistent order
http://bugs.python.org/issue12273  closed by python-dev

#12279: Add build_distinfo command to packaging
http://bugs.python.org/issue12279  closed by eric.araujo

#12372: semaphore errors on AIX 7.1
http://bugs.python.org/issue12372  closed by neologix

#12434: Strengthen 2.7 io types warning
http://bugs.python.org/issue12434  closed by eli.bendersky

#12478: Possible error in HTTPErrorProcessor documentation
http://bugs.python.org/issue12478  closed by python-dev

#12479: Add HTTPErrorProcessor class definition
http://bugs.python.org/issue12479  closed by python-dev

#12521: IDLE 3.2 crashes on Mac OS 10.6 with ActiveState Tcl/Tk 8.5
http://bugs.python.org/issue12521  closed by ned.deily

#12524: change httplib docs POST example
http://bugs.python.org/issue12524  closed by python-dev

#12551: Provide data for TLS channel binding
http://bugs.python.org/issue12551  closed by pitrou

#12556: Disable size checks in mmap.mmap()
http://bugs.python.org/issue12556  closed by superbobry

#12570: BaseHTTPServer.shutdown() locks if the last request 404'd
http://bugs.python.org/issue12570  closed by orsenthil

#12571: Python built on Linux 3.0 doesn't include DLFCN
http://bugs.python.org/issue12571  closed by pitrou

#12573: add resource checks for dangling threads and processes
http://bugs.python.org/issue12573  closed by pitrou

#12574: Documentation for Queue in 2.x has an incorrect title
http://bugs.python.org/issue12574  closed by eli.bendersky

#12577: Misleading shutil.move docs regarding when os.rename is used
http://bugs.python.org/issue12577  closed by python-dev

#12579: str.format_map raises a SystemError for format strings with po
http://bugs.python.org/issue12579  closed by python-dev

#12580: Documentation error in Decimal module
http://bugs.python.org/issue12580  closed by holdenweb

#12582: lib-dynload missing in python install
http://bugs.python.org/issue12582  closed by ned.deily

#12584: win.protocol('WM_DELETE_WINDOW'...) still deletes window on OS
http://bugs.python.org/issue12584  closed by ned.deily

#12587: tokenize_tests-utf8-coding-cookie-and-no-utf8-bom-sig.txt has 
http://bugs.python.org/issue12587  closed by ned.deily

#12593: define OpenBSD in configure.in if Py_ENABLE_SHARED == 1
http://bugs.python.org/issue12593  closed by skrah

#12597: List created by multiplication behaves different
http://bugs.python.org/issue12597  closed by sandro.tosi

#12601: Spelling error in the comments in the webbrowser module.
http://bugs.python.org/issue12601  closed by ezio.melotti

#12609: SystemError: Objects/codeobject.c:64: bad argument to internal
http://bugs.python.org/issue12609  closed by python-dev

#665194: datetime-RFC2822 roundtripping
http://bugs.python.org/issue665194  closed by r.david.murray

From v+python at g.nevcal.com  Sat Jul 23 05:01:23 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 22 Jul 2011 20:01:23 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <4E28C7D1.3010002@skippinet.com.au>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
	<4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com>
	<4E28B016.5050500@g.nevcal.com> <4E28C7D1.3010002@skippinet.com.au>
Message-ID: <4E2A3983.2060901@g.nevcal.com>

On 7/21/2011 5:44 PM, Mark Hammond wrote:
> On 22/07/2011 9:02 AM, Glenn Linderman wrote:
>> Bad logic is get_configured_value!  get_configured_value only looks in
>> the global configuration file if there is a local configuration file
>> that doesn't have the setting.  It should look in the global
>> configuration file if there is no local configuration file _OR_ the is a
>> local configuration file without the setting.
>>
>> I'll await a new launcher.
>
> I just pushed a fix and hopefully Vinay will push a new MSI soon.
>
> Thanks,
>
> Mark
>

What (free?) toolset is needed for building the launcher?  I don't even 
have a C compiler installed on this computer yet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110722/f7a696e9/attachment.html>

From ncoghlan at gmail.com  Sat Jul 23 08:55:36 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 23 Jul 2011 16:55:36 +1000
Subject: [Python-Dev] [Python-checkins] r88867 - in
 tracker/instances/python-dev: extensions/pydevutils.py html/file.item.html
 html/issue.item.html html/msg.item.html
In-Reply-To: <3RLRcY6MHZzMml@mail.python.org>
References: <3RLRcY6MHZzMml@mail.python.org>
Message-ID: <CADiSq7eQVcVH5MPjy69nspi0D7YntoK3uyGo_FWKJJAMvDaLwA@mail.gmail.com>

On Sat, Jul 23, 2011 at 8:34 AM, ezio.melotti
<python-checkins at python.org> wrote:
> Author: ezio.melotti
> Date: Sat Jul 23 00:34:53 2011
> New Revision: 88867
>
> Log:
> #267: remove the remove button from the issue page, move it to the msg/file page, and add a button to add back removed messages/files.

Thank you! (I accidentally deleted one of RDM's comments just the other day)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From g.brandl at gmx.net  Sat Jul 23 09:44:50 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 23 Jul 2011 09:44:50 +0200
Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build
 on OpenBSD 5 (and future major
In-Reply-To: <E1QkNdO-0001zv-Pd@dinsdale.python.org>
References: <E1QkNdO-0001zv-Pd@dinsdale.python.org>
Message-ID: <j0du42$7g7$1@dough.gmane.org>

Am 22.07.2011 23:51, schrieb charles-francois.natali:
> http://hg.python.org/cpython/rev/63de97ae832e
> changeset:   71462:63de97ae832e
> parent:      71459:d3f0f72c31f8
> user:        Charles-Fran?ois Natali <neologix at free.fr>
> date:        Fri Jul 22 23:52:02 2011 +0200
> summary:
>   Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major
> releases).
> 
> files:
>   Misc/NEWS    |    2 +
>   configure    |  597 ++++++++++++++++++--------------------
>   configure.in |    4 +-
>   3 files changed, 291 insertions(+), 312 deletions(-)

Note that this commit wasn't actually a merge -- you'll have to use the
"hg merge" command for that.

Georg


From ezio.melotti at gmail.com  Sat Jul 23 10:45:25 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sat, 23 Jul 2011 11:45:25 +0300
Subject: [Python-Dev] [Python-checkins] r88867 - in
 tracker/instances/python-dev: extensions/pydevutils.py html/file.item.html
 html/issue.item.html html/msg.item.html
In-Reply-To: <CADiSq7eQVcVH5MPjy69nspi0D7YntoK3uyGo_FWKJJAMvDaLwA@mail.gmail.com>
References: <3RLRcY6MHZzMml@mail.python.org>
	<CADiSq7eQVcVH5MPjy69nspi0D7YntoK3uyGo_FWKJJAMvDaLwA@mail.gmail.com>
Message-ID: <4E2A8A25.2010006@gmail.com>

On 23/07/2011 9.55, Nick Coghlan wrote:
> On Sat, Jul 23, 2011 at 8:34 AM, ezio.melotti
> <python-checkins at python.org>  wrote:
>> Author: ezio.melotti
>> Date: Sat Jul 23 00:34:53 2011
>> New Revision: 88867
>>
>> Log:
>> #267: remove the remove button from the issue page, move it to the msg/file page, and add a button to add back removed messages/files.
> Thank you! (I accidentally deleted one of RDM's comments just the other day)

You are welcome! (That was actually one of the reasons that made me look 
at that issue again)

Now restoring messages and files that got deleted should be trivial, and 
deleting them accidentally (almost) impossible.
Messages and files can be deleted/restored from their pages.  Links to 
these pages can be found in the history of the issue.  For messages and 
files that are still linked to the issue, the "(view)" and "edit" links 
can also be used.

Best Regards,
Ezio Melotti

>
> Cheers,
> Nick.
>


From cf.natali at gmail.com  Sat Jul 23 11:46:05 2011
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Sat, 23 Jul 2011 11:46:05 +0200
Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build
 on OpenBSD 5 (and future major
In-Reply-To: <j0du42$7g7$1@dough.gmane.org>
References: <E1QkNdO-0001zv-Pd@dinsdale.python.org>
	<j0du42$7g7$1@dough.gmane.org>
Message-ID: <CAH_1eM3KioavqKUAjmMtSPzVvDW4aK2EPSyWMzN+OF9egN6oRw@mail.gmail.com>

> Note that this commit wasn't actually a merge -- you'll have to use the
> "hg merge" command for that.

You're right.
I guess that's what happens when I try to work past my usual bedtime ;-)

By the way, I'm still getting errors upon push, and it looks like when
I push a patch, this doesn't trigger any build on the buildbots. It
used to work, any idea what's going on?

From vinay_sajip at yahoo.co.uk  Sat Jul 23 14:34:28 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 23 Jul 2011 12:34:28 +0000 (UTC)
Subject: [Python-Dev] 3.2.1 encoding surprise
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
	<4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com>
	<4E28B016.5050500@g.nevcal.com> <4E28C7D1.3010002@skippinet.com.au>
	<4E2A3983.2060901@g.nevcal.com>
Message-ID: <loom.20110723T143146-382@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:

I aim to update the launcher downloads Real Soon Now.

>     What (free?) toolset is needed for building the launcher?? I don't
>     even have a C compiler installed on this computer yet.
> 

The tools I use for building the launcher are:

Windows SDK (for the 64-bit compilers), Visual Studio Express C++ (2008
Edition), WiX for the installers, and Python.

All are free as in beer, and some are also free as in speech.

Regards,

Vinay Sajip


From solipsis at pitrou.net  Sat Jul 23 15:44:07 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 23 Jul 2011 15:44:07 +0200
Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build
 on OpenBSD 5 (and future major
References: <E1QkNdO-0001zv-Pd@dinsdale.python.org>
	<j0du42$7g7$1@dough.gmane.org>
	<CAH_1eM3KioavqKUAjmMtSPzVvDW4aK2EPSyWMzN+OF9egN6oRw@mail.gmail.com>
Message-ID: <20110723154407.4b10ba34@pitrou.net>

On Sat, 23 Jul 2011 11:46:05 +0200
Charles-Fran?ois Natali <cf.natali at gmail.com> wrote:

> > Note that this commit wasn't actually a merge -- you'll have to use the
> > "hg merge" command for that.
> 
> You're right.
> I guess that's what happens when I try to work past my usual bedtime ;-)
> 
> By the way, I'm still getting errors upon push, and it looks like when
> I push a patch, this doesn't trigger any build on the buildbots. It
> used to work, any idea what's going on?

What is the error? Is a traceback printed?

Regards

Antoine.



From cf.natali at gmail.com  Sat Jul 23 16:10:38 2011
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Sat, 23 Jul 2011 16:10:38 +0200
Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build
 on OpenBSD 5 (and future major
In-Reply-To: <20110723154407.4b10ba34@pitrou.net>
References: <E1QkNdO-0001zv-Pd@dinsdale.python.org>
	<j0du42$7g7$1@dough.gmane.org>
	<CAH_1eM3KioavqKUAjmMtSPzVvDW4aK2EPSyWMzN+OF9egN6oRw@mail.gmail.com>
	<20110723154407.4b10ba34@pitrou.net>
Message-ID: <CAH_1eM2hX0HUs4NCnMOTwhvCbaHP50s3CRsod0k0hYm5Ftk0Xw@mail.gmail.com>

(See http://mail.python.org/pipermail/python-dev/2011-July/112309.html
for the original mail):

"""
$ hg push
pushing to ssh://hg at hg.python.org/cpython
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 2 changes to 2 files
remote: change(s) NOT sent, something went wrong:
remote: [Failure instance: Traceback from remote host -- Traceback unavailable
remote: ]
remote: sent email to roundup at report at bugs.python.org
remote: notified python-checkins at python.org of incoming changeset
ca077f2672e3
"""

No traceback.
Last time I did a push (yesterday), there was a slight difference, the
lines reporting the failure were prefixed by "buildbot:" (IIRC). And
indeed, none of my pushes triggers a build on the buildbots (and I'm
sure it worked a couple weeks ago).

Thanks,

cf

From solipsis at pitrou.net  Sat Jul 23 16:34:20 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 23 Jul 2011 16:34:20 +0200
Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build
 on OpenBSD 5 (and future major
In-Reply-To: <CAH_1eM2hX0HUs4NCnMOTwhvCbaHP50s3CRsod0k0hYm5Ftk0Xw@mail.gmail.com>
References: <E1QkNdO-0001zv-Pd@dinsdale.python.org>
	<j0du42$7g7$1@dough.gmane.org>
	<CAH_1eM3KioavqKUAjmMtSPzVvDW4aK2EPSyWMzN+OF9egN6oRw@mail.gmail.com>
	<20110723154407.4b10ba34@pitrou.net>
	<CAH_1eM2hX0HUs4NCnMOTwhvCbaHP50s3CRsod0k0hYm5Ftk0Xw@mail.gmail.com>
Message-ID: <20110723163420.7047a6b9@pitrou.net>

On Sat, 23 Jul 2011 16:10:38 +0200
Charles-Fran?ois Natali <cf.natali at gmail.com> wrote:
> 
> No traceback.
> Last time I did a push (yesterday), there was a slight difference, the
> lines reporting the failure were prefixed by "buildbot:" (IIRC). And
> indeed, none of my pushes triggers a build on the buildbots (and I'm
> sure it worked a couple weeks ago).

Ok, it should be fixed now.

Regards

Antoine.

From cf.natali at gmail.com  Sat Jul 23 19:22:34 2011
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Sat, 23 Jul 2011 19:22:34 +0200
Subject: [Python-Dev] cpython: Merge - Issue #12592: Make Python build
 on OpenBSD 5 (and future major
In-Reply-To: <20110723163420.7047a6b9@pitrou.net>
References: <E1QkNdO-0001zv-Pd@dinsdale.python.org>
	<j0du42$7g7$1@dough.gmane.org>
	<CAH_1eM3KioavqKUAjmMtSPzVvDW4aK2EPSyWMzN+OF9egN6oRw@mail.gmail.com>
	<20110723154407.4b10ba34@pitrou.net>
	<CAH_1eM2hX0HUs4NCnMOTwhvCbaHP50s3CRsod0k0hYm5Ftk0Xw@mail.gmail.com>
	<20110723163420.7047a6b9@pitrou.net>
Message-ID: <CAH_1eM2Or7+DMMjhk77ErP=dd34g2iQyxX+NDH9Mu40umtqr-A@mail.gmail.com>

> Ok, it should be fixed now.
>

Indeed.
Thanks!

cf

From andrew.svetlov at gmail.com  Sun Jul 24 00:20:08 2011
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Sun, 24 Jul 2011 01:20:08 +0300
Subject: [Python-Dev] tuple[int] optimization
Message-ID: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>

tuple[int] is 30% slower than list[int].
Please let me explain the problem.

(1, 2, 3)[1] has compiled as BINARY_SUBSCR operation.
ceval.c has optimization for list:

        case BINARY_SUBSCR:
            w = POP();
            v = TOP();
            if (PyList_CheckExact(v) && PyInt_CheckExact(w)) {
                /* INLINE: list[int] */
                Py_ssize_t i = PyInt_AsSsize_t(w);
                if (i < 0)
                    i += PyList_GET_SIZE(v);
                if (i >= 0 && i < PyList_GET_SIZE(v)) {
                    x = PyList_GET_ITEM(v, i);
                    Py_INCREF(x);
                }
                else
                    goto slow_get;
            }
            else
              slow_get:
                x = PyObject_GetItem(v, w);
            Py_DECREF(v);
            Py_DECREF(w);
            SET_TOP(x);
            if (x != NULL) continue;
            break;

For tuple code follows slow way via tp_as_mapping slot and calls
tuplesubscript from Objects/tupleobject.c

Looks like tuple is goot applicant to be optimized as list does.
tuples is used in python programs very often and getting element from
tuple by index as common operation as list[int].

Is there reasons to prevent special case for tuples in BINARY_SUBSCR?

From ryan at rfk.id.au  Sun Jul 24 01:13:07 2011
From: ryan at rfk.id.au (Ryan Kelly)
Date: Sun, 24 Jul 2011 09:13:07 +1000
Subject: [Python-Dev] tuple[int] optimization
In-Reply-To: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>
References: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>
Message-ID: <1311462787.2225.5.camel@durian>

On Sun, 2011-07-24 at 01:20 +0300, Andrew Svetlov wrote:
> tuple[int] is 30% slower than list[int].
> Please let me explain the problem.
> 
> (1, 2, 3)[1] has compiled as BINARY_SUBSCR operation.
> ceval.c has optimization for list:
> 
>         case BINARY_SUBSCR:
>             w = POP();
>             v = TOP();
>             if (PyList_CheckExact(v) && PyInt_CheckExact(w)) {
>                 /* INLINE: list[int] */
>                 Py_ssize_t i = PyInt_AsSsize_t(w);
>                 if (i < 0)
>                     i += PyList_GET_SIZE(v);
>                 if (i >= 0 && i < PyList_GET_SIZE(v)) {
>                     x = PyList_GET_ITEM(v, i);
>                     Py_INCREF(x);
>                 }
>                 else
>                     goto slow_get;
>             }
>             else
>               slow_get:
>                 x = PyObject_GetItem(v, w);
>             Py_DECREF(v);
>             Py_DECREF(w);
>             SET_TOP(x);
>             if (x != NULL) continue;
>             break;
> 
> For tuple code follows slow way via tp_as_mapping slot and calls
> tuplesubscript from Objects/tupleobject.c
> 
> Looks like tuple is goot applicant to be optimized as list does.
> tuples is used in python programs very often and getting element from
> tuple by index as common operation as list[int].
> 
> Is there reasons to prevent special case for tuples in BINARY_SUBSCR?

In latest trunk this optimisation seems to have gone away, the code is
now:

        TARGET(BINARY_SUBSCR)
            w = POP();
            v = TOP();
            x = PyObject_GetItem(v, w);
            Py_DECREF(v);
            Py_DECREF(w);
            SET_TOP(x);
            if (x != NULL) DISPATCH();
            break;

The implementation of PyObject_GetItem doesn't appear to have changed
though.  Maybe this optimisation was no longer worth it in practice? 


  Ryan



-- 
Ryan Kelly
http://www.rfk.id.au  |  This message is digitally signed. Please visit
ryan at rfk.id.au        |  http://www.rfk.id.au/ramblings/gpg/ for details

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110724/6815fe62/attachment.pgp>

From solipsis at pitrou.net  Sun Jul 24 01:27:55 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 24 Jul 2011 01:27:55 +0200
Subject: [Python-Dev] tuple[int] optimization
References: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>
	<1311462787.2225.5.camel@durian>
Message-ID: <20110724012755.32bf6a8f@pitrou.net>

On Sun, 24 Jul 2011 09:13:07 +1000
Ryan Kelly <ryan at rfk.id.au> wrote:
> 
> In latest trunk this optimisation seems to have gone away, the code is
> now:
> 
>         TARGET(BINARY_SUBSCR)
>             w = POP();
>             v = TOP();
>             x = PyObject_GetItem(v, w);
>             Py_DECREF(v);
>             Py_DECREF(w);
>             SET_TOP(x);
>             if (x != NULL) DISPATCH();
>             break;
> 
> The implementation of PyObject_GetItem doesn't appear to have changed
> though.  Maybe this optimisation was no longer worth it in practice? 

The optimization was probably removed because PyInt objects don't exist
anymore. There's a related but more ambitious patch at
http://bugs.python.org/issue10044.

In practice however, such micro-optimizations usually have little or no
effect.

Regards

Antoine.



From andrew.svetlov at gmail.com  Sun Jul 24 02:15:52 2011
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Sun, 24 Jul 2011 03:15:52 +0300
Subject: [Python-Dev] tuple[int] optimization
In-Reply-To: <20110724012755.32bf6a8f@pitrou.net>
References: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>
	<1311462787.2225.5.camel@durian>
	<20110724012755.32bf6a8f@pitrou.net>
Message-ID: <CAL3CFcWXR-kDJ4CvGX5X0K_KBkcYAhQZdkY+SjYCzzrZJdzYHQ@mail.gmail.com>

You right. Sorry, I missed changes in ceval.c for py3k.
Please note, simple test like:

from timeit import timeit

print('list', timeit("l[0]", "l = [1]"))
print('tuple', timeit("l[0]", "l = (1,)"))

Has results:

andrew at ocean ~/p/cpython> python2.7 z.py
('list', 0.03479599952697754)
('tuple', 0.046610116958618164)

andrew at ocean ~/p/cpython> python3.2 z.py
list 0.04870104789733887
tuple 0.04825997352600098

For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and
trunk) very close to "unoptimized" 2.7 version.

On Sun, Jul 24, 2011 at 2:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 24 Jul 2011 09:13:07 +1000
> Ryan Kelly <ryan at rfk.id.au> wrote:
>>
>> In latest trunk this optimisation seems to have gone away, the code is
>> now:
>>
>> ? ? ? ? TARGET(BINARY_SUBSCR)
>> ? ? ? ? ? ? w = POP();
>> ? ? ? ? ? ? v = TOP();
>> ? ? ? ? ? ? x = PyObject_GetItem(v, w);
>> ? ? ? ? ? ? Py_DECREF(v);
>> ? ? ? ? ? ? Py_DECREF(w);
>> ? ? ? ? ? ? SET_TOP(x);
>> ? ? ? ? ? ? if (x != NULL) DISPATCH();
>> ? ? ? ? ? ? break;
>>
>> The implementation of PyObject_GetItem doesn't appear to have changed
>> though. ?Maybe this optimisation was no longer worth it in practice?
>
> The optimization was probably removed because PyInt objects don't exist
> anymore. There's a related but more ambitious patch at
> http://bugs.python.org/issue10044.
>
> In practice however, such micro-optimizations usually have little or no
> effect.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>

From solipsis at pitrou.net  Sun Jul 24 02:18:23 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 24 Jul 2011 02:18:23 +0200
Subject: [Python-Dev] tuple[int] optimization
In-Reply-To: <CAL3CFcWXR-kDJ4CvGX5X0K_KBkcYAhQZdkY+SjYCzzrZJdzYHQ@mail.gmail.com>
References: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>
	<1311462787.2225.5.camel@durian> <20110724012755.32bf6a8f@pitrou.net>
	<CAL3CFcWXR-kDJ4CvGX5X0K_KBkcYAhQZdkY+SjYCzzrZJdzYHQ@mail.gmail.com>
Message-ID: <1311466703.3001.12.camel@localhost.localdomain>

Le dimanche 24 juillet 2011 ? 03:15 +0300, Andrew Svetlov a ?crit :
> You right. Sorry, I missed changes in ceval.c for py3k.
> Please note, simple test like:
> 
> from timeit import timeit
> 
> print('list', timeit("l[0]", "l = [1]"))
> print('tuple', timeit("l[0]", "l = (1,)"))
> 
> Has results:
> 
> andrew at ocean ~/p/cpython> python2.7 z.py
> ('list', 0.03479599952697754)
> ('tuple', 0.046610116958618164)
> 
> andrew at ocean ~/p/cpython> python3.2 z.py
> list 0.04870104789733887
> tuple 0.04825997352600098
> 
> For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and
> trunk) very close to "unoptimized" 2.7 version.

My point is that on non-trivial benchmarks, the savings are almost zero.
If you look at the (much more complex) patch I linked to, the savings
are at most 10% on a couple of select benchmarks, other benchmarks
showing almost no difference.

Regards

Antoine.



From andrew.svetlov at gmail.com  Sun Jul 24 02:26:26 2011
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Sun, 24 Jul 2011 03:26:26 +0300
Subject: [Python-Dev] tuple[int] optimization
In-Reply-To: <1311466703.3001.12.camel@localhost.localdomain>
References: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>
	<1311462787.2225.5.camel@durian>
	<20110724012755.32bf6a8f@pitrou.net>
	<CAL3CFcWXR-kDJ4CvGX5X0K_KBkcYAhQZdkY+SjYCzzrZJdzYHQ@mail.gmail.com>
	<1311466703.3001.12.camel@localhost.localdomain>
Message-ID: <CAL3CFcVb82ApxsE9XrzhFsFcBdOpGxWZ+rUPa0zZZkvw_ox7FQ@mail.gmail.com>

I undrestand your point. Thank you for explanation.

On Sun, Jul 24, 2011 at 3:18 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le dimanche 24 juillet 2011 ? 03:15 +0300, Andrew Svetlov a ?crit :
>> You right. Sorry, I missed changes in ceval.c for py3k.
>> Please note, simple test like:
>>
>> from timeit import timeit
>>
>> print('list', timeit("l[0]", "l = [1]"))
>> print('tuple', timeit("l[0]", "l = (1,)"))
>>
>> Has results:
>>
>> andrew at ocean ~/p/cpython> python2.7 z.py
>> ('list', 0.03479599952697754)
>> ('tuple', 0.046610116958618164)
>>
>> andrew at ocean ~/p/cpython> python3.2 z.py
>> list 0.04870104789733887
>> tuple 0.04825997352600098
>>
>> For python2.7 list[int] microoptimization saves 25-30%, while 3.2 (and
>> trunk) very close to "unoptimized" 2.7 version.
>
> My point is that on non-trivial benchmarks, the savings are almost zero.
> If you look at the (much more complex) patch I linked to, the savings
> are at most 10% on a couple of select benchmarks, other benchmarks
> showing almost no difference.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>

From raymond.hettinger at gmail.com  Sun Jul 24 03:34:16 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sat, 23 Jul 2011 18:34:16 -0700
Subject: [Python-Dev] tuple[int] optimization
In-Reply-To: <1311466703.3001.12.camel@localhost.localdomain>
References: <CAL3CFcX_v5GsQ4hHG89-YjhGU9e+cqm82G6Rw=C+DSYc1tC1ag@mail.gmail.com>
	<1311462787.2225.5.camel@durian>
	<20110724012755.32bf6a8f@pitrou.net>
	<CAL3CFcWXR-kDJ4CvGX5X0K_KBkcYAhQZdkY+SjYCzzrZJdzYHQ@mail.gmail.com>
	<1311466703.3001.12.camel@localhost.localdomain>
Message-ID: <645E4C22-4532-42A0-A87C-B65D8EB49B7B@gmail.com>


On Jul 23, 2011, at 5:18 PM, Antoine Pitrou wrote:
> 
> My point is that on non-trivial benchmarks, the savings are almost zero.

That could be said of any optimization in Python.
Typical Python scripts exercise many features at time,
so any one optimization by itself if almost useless.
Collectively however, we've gotten nice speed-ups on real programs.

Raymond

From mail at kerrickstaley.com  Sun Jul 24 04:48:43 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Sat, 23 Jul 2011 21:48:43 -0500
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <CADiSq7fC+iLSf8_39F82r08ytjnZEu0SpdBM+scW37j8ZQePjg@mail.gmail.com>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<CADiSq7fC+iLSf8_39F82r08ytjnZEu0SpdBM+scW37j8ZQePjg@mail.gmail.com>
Message-ID: <CANaWP3zhoG68gNvnh90BoU5rZfQmuob4rzstbj=4fwcsi7ucfg@mail.gmail.com>

I put up a tracker issue at http://bugs.python.org/issue12627

There are patches for 2.7 as well as tip, but they only fix the
Makefiles; no changes are done to documentation.

Also, Ned, it appears that Python 2.7 doesn't install the Idle or
PyDoc binaries (grep the 2.7 Makefile to see what I mean), so I didn't
make any changes related to idle2 or pydoc2.

-Kerrick Staley

On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> On Wed, Jul 20, 2011 at 4:53 PM, Kerrick Staley <mail at kerrickstaley.com> wrote:
> > Nick, can you please apply the patch (will be sent in the following
> > email) to the PEP SVN as soon as we get the hard-link issue is figured
> > out? Alternatively, could you provide me write access to just the
> > pep-0394.txt file so I can update it myself? Thanks.
>
> Done: http://www.python.org/dev/peps/pep-0394/
>
> Main thing needed now is a tracker issue to reference (ideally with a
> first cut at a patch) and python-dev's collective blessing to take it
> out of Draft status.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From eliben at gmail.com  Sun Jul 24 05:35:56 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Sun, 24 Jul 2011 06:35:56 +0300
Subject: [Python-Dev] Convention on functions that shadow existing stdlib
	functions
Message-ID: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>

Some background: I'm working (on and off) on issue 11015 - documenting
the public functions in test.support

Some of the functions in test.support (for example unlink, rmtree)
simply shadow existing & popular stdlib functions, with the aim of
swallowing the exceptions these may throw. This is confusing, IMHO.
For example, grepping 'unlink' on Lib/test/test_*.py files doesn't say
much about which unlink is being used.

A couple of options to handle this are:

1. Remove these functions altogether, trying to use existing
constructs (such as the ignore_errors parameter in rmtree).
2. Adapt a naming convention for such functions, for instance
rmtree_silent and unlink_silent (or a similar convention, if one
exists)

Opinions?

Eli

From ericsnowcurrently at gmail.com  Sun Jul 24 06:53:38 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 23 Jul 2011 22:53:38 -0600
Subject: [Python-Dev] is sys.modules not meant to be replaced?
Message-ID: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>

The documentation[1] doesn't say, but the implementation of the imp
module makes me wonder if sys.modules was not meant to be replaceable.
 No doubt this has to do with its tie to the interpreter's modules
dict.  I ran into this doing "sys.modules = sys.modules.copy()" to
protect the actual sys.modules dict during some import related test
cases.  If the modules I imported were extension modules it broke.

So, is sys.modules not meant to be open to re-binding?

-eric

p.s. I tried opening a tracker ticket on this, but it wouldn't go
through.  I'll try again later.

[1] http://docs.python.org/library/sys.html#sys.modules

From benjamin at python.org  Sun Jul 24 06:55:38 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 23 Jul 2011 23:55:38 -0500
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
Message-ID: <CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>

2011/7/23 Eric Snow <ericsnowcurrently at gmail.com>:
> The documentation[1] doesn't say, but the implementation of the imp
> module makes me wonder if sys.modules was not meant to be replaceable.
> ?No doubt this has to do with its tie to the interpreter's modules
> dict. ?I ran into this doing "sys.modules = sys.modules.copy()" to
> protect the actual sys.modules dict during some import related test
> cases. ?If the modules I imported were extension modules it broke.
>
> So, is sys.modules not meant to be open to re-binding?

Not any more or less than other global mutable objects. You can expect
other code to be holding on to old references.


-- 
Regards,
Benjamin

From ericsnowcurrently at gmail.com  Sun Jul 24 07:05:24 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 23 Jul 2011 23:05:24 -0600
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
	<CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>
Message-ID: <CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>

On Sat, Jul 23, 2011 at 10:55 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2011/7/23 Eric Snow <ericsnowcurrently at gmail.com>:
>> The documentation[1] doesn't say, but the implementation of the imp
>> module makes me wonder if sys.modules was not meant to be replaceable.
>> ?No doubt this has to do with its tie to the interpreter's modules
>> dict. ?I ran into this doing "sys.modules = sys.modules.copy()" to
>> protect the actual sys.modules dict during some import related test
>> cases. ?If the modules I imported were extension modules it broke.
>>
>> So, is sys.modules not meant to be open to re-binding?
>
> Not any more or less than other global mutable objects. You can expect
> other code to be holding on to old references.

But, isn't sys.modules a little different because other modules (at
least the imp module) don't use it.  From what I understand the
interpreter object's modules dict (to which sys.modules is initially
bound) is used directly instead.  So rebinding sys.modules causes a
disconnect.  That's why I am wondering if sys.modules is not meant to
be rebound.

Are there other objects in the interpreter state that are exposed in
sys that would have the same problem?

-eric

>
>
> --
> Regards,
> Benjamin
>

From benjamin at python.org  Sun Jul 24 07:09:13 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 24 Jul 2011 00:09:13 -0500
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
	<CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>
	<CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>
Message-ID: <CAPZV6o-XW6WiV9DhSb1-pxCUmPz+asjxkLnAN+2x8dUKaaz-8A@mail.gmail.com>

2011/7/24 Eric Snow <ericsnowcurrently at gmail.com>:
> On Sat, Jul 23, 2011 at 10:55 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2011/7/23 Eric Snow <ericsnowcurrently at gmail.com>:
>>> The documentation[1] doesn't say, but the implementation of the imp
>>> module makes me wonder if sys.modules was not meant to be replaceable.
>>> ?No doubt this has to do with its tie to the interpreter's modules
>>> dict. ?I ran into this doing "sys.modules = sys.modules.copy()" to
>>> protect the actual sys.modules dict during some import related test
>>> cases. ?If the modules I imported were extension modules it broke.
>>>
>>> So, is sys.modules not meant to be open to re-binding?
>>
>> Not any more or less than other global mutable objects. You can expect
>> other code to be holding on to old references.
>
> But, isn't sys.modules a little different because other modules (at
> least the imp module) don't use it. ?From what I understand the
> interpreter object's modules dict (to which sys.modules is initially
> bound) is used directly instead. ?So rebinding sys.modules causes a
> disconnect. ?That's why I am wondering if sys.modules is not meant to
> be rebound.

Sure. I'm not sure what point you're trying to make, though.

>
> Are there other objects in the interpreter state that are exposed in
> sys that would have the same problem?

Is it problematic?

-- 
Regards,
Benjamin

From v+python at g.nevcal.com  Sun Jul 24 08:29:55 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 23 Jul 2011 23:29:55 -0700
Subject: [Python-Dev] 3.2.1 encoding surprise
In-Reply-To: <loom.20110723T143146-382@post.gmane.org>
References: <4E248536.30006@g.nevcal.com>
	<CACac1F9As3dycopPH2RfLTeVdNbLKbaKtcQDPxk_R7ZEqPJrmw@mail.gmail.com>
	<4E249960.9000109@g.nevcal.com>
	<loom.20110718T230703-173@post.gmane.org>
	<4E24AE66.5030305@g.nevcal.com>
	<loom.20110719T014549-927@post.gmane.org>
	<4E24D83B.8040100@g.nevcal.com>
	<loom.20110719T031535-263@post.gmane.org>
	<4E269D46.1010806@g.nevcal.com>
	<loom.20110720T160430-209@post.gmane.org>
	<4E274C0F.40400@g.nevcal.com> <4E276795.9050404@gmail.com>
	<4E276C7C.1020605@g.nevcal.com> <4E276EBF.6000509@gmail.com>
	<4E28B016.5050500@g.nevcal.com> <4E28C7D1.3010002@skippinet.com.au>
	<4E2A3983.2060901@g.nevcal.com>
	<loom.20110723T143146-382@post.gmane.org>
Message-ID: <4E2BBBE3.2050304@g.nevcal.com>

On 7/23/2011 5:34 AM, Vinay Sajip wrote:
> Glenn Linderman<v+python<at>  g.nevcal.com>  writes:
>
> I aim to update the launcher downloads Real Soon Now.

Has fixed my problem with not having a local py.ini file, and now is 
picking up python=3 from the py.ini coresident to the py.exe.  Thanks, 
Mark & Vinay.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110723/938b00ba/attachment.html>

From ncoghlan at gmail.com  Sun Jul 24 08:54:33 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 24 Jul 2011 16:54:33 +1000
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
	<CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>
	<CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>
Message-ID: <CADiSq7dQjEDtr232tbdQkntzEHBPGnG-HvK=N1+jUJ8f3WSWtg@mail.gmail.com>

On Sun, Jul 24, 2011 at 3:05 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> Are there other objects in the interpreter state that are exposed in
> sys that would have the same problem?

Rebinding (rather than mutating) any global state in any module is
always dubious due to the potential for direct references to the
previous value. (This is a large part of the reason why imp.reload()
often doesn't work in practice)

sys.modules, sys.path, sys.argv, sys.stdout, et al are all subject to
that caveat. For mutable state, the onus is typically on the developer
performing the substitution to decide whether or not those
consequences are acceptable (usually, they're not, hence you get
idioms like "sys.path[:] = saved_copy" to revert sys.path to a saved
value). For immutable state (such as sys.stdout), where modification
in place isn't possible, the obligation often goes the other way, with
developers deliberately avoiding cached references in order to
correctly react to runtime changes.

sys.modules is by far the worst case though, since dropping modules
out of that cache is only safe for modules that correctly support
imp.reload(). This is not the case for any module that mutates other
global state (or is otherwise referenced from other modules), nor does
it work properly for extension modules.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From petri at digip.org  Sun Jul 24 19:51:37 2011
From: petri at digip.org (Petri Lehtinen)
Date: Sun, 24 Jul 2011 20:51:37 +0300
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
Message-ID: <20110724175137.GA1624@ihaa>

Eric Snow wrote:
> p.s. I tried opening a tracker ticket on this, but it wouldn't go
> through.  I'll try again later.

Adding the following line to /etc/hosts makes the bug tracker fast
when python.org is down:

127.0.0.1  python.org

This is because bugs.python.org works fine, but it links to CSS files
and images that are on python.org. Forcing DNS lookups for python.org
to return localhost makes the requests for these resources fail fast.

A more long-term solution would be to duplicate all the resources to
bugs.python.org...

Petri

From solipsis at pitrou.net  Sun Jul 24 23:16:55 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 24 Jul 2011 23:16:55 +0200
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
Message-ID: <20110724231655.0b69d36a@pitrou.net>

On Wed, 20 Jul 2011 01:53:09 -0500
Kerrick Staley <mail at kerrickstaley.com> wrote:
> On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily <nad at acm.org> wrote:
> > I think adding the requirement to mandate hard link vs soft link usage
> > is an unnecessary and unwarranted attempt at optimization.  For
> > instance, IIRC, the OS X installers don't use any hard links: that may
> > complicate the install, plus hard links on OS X HFS* file systems are a
> > bit of a kludge and not necessarily more efficient than symlinks.   It's
> > not a big deal but perhaps the wording should be changed to make a
> > suggestion about hard links vs syminks rather than mandate which should
> > be used.
> 
> Ah, OK. The wording's been changed so that symbolic links will be
> installed on Mac OS X and hard links elsewhere (although maybe
> symbolic links are also better on certain other platforms; I'm not
> sure).
> 
> I do think that specific instructions must be given (rather than just
> a suggestion) because it's indicating what must be done to CPython.
> The instructions *should* be as close as possible to what the
> installer already does, but I'm not entirely sure what the installer
> does by default, and the hard-link recommendation was based off a
> cursory inspection of my own system, so further input from yourself
> and the rest of python-dev would be appreciated.

I think the recommendation should be symbolic links for all systems.
Hard links are generally harder to discover, while it is trivial to
find out that a given file is a symbolink link, and to which other file.
The optimization is probably not useful in the real world (our
executables are relatively small, and people worried about a couple of
megabytes can always go for the shared library option).

Regards

Antoine.



From solipsis at pitrou.net  Sun Jul 24 23:21:06 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 24 Jul 2011 23:21:06 +0200
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
Message-ID: <20110724232106.53161528@pitrou.net>

On Sun, 24 Jul 2011 23:16:55 +0200
Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> I think the recommendation should be symbolic links for all systems.
> Hard links are generally harder to discover, while it is trivial to
> find out that a given file is a symbolink link, and to which other file.
> The optimization is probably not useful in the real world (our
> executables are relatively small, and people worried about a couple of
> megabytes can always go for the shared library option).

Oh, I don't even know why I thought hard links would be a size
optimization anyway. I guess I thought "file copy" instead of "symbolic
link".

Regards

Antoine.



From victor.stinner at haypocalc.com  Mon Jul 25 01:56:32 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 25 Jul 2011 01:56:32 +0200
Subject: [Python-Dev] Comments of the PEP 3151
Message-ID: <4E2CB130.7060701@haypocalc.com>

Arguments in favor of specific errors
-------------------------------------

Using specific errors avoids the need of "import errno". The import is
sometimes done in the except block, which may raise a new error (and may
be slow).

I like specific exceptions because it avoids the need of re-raise
unwanted exception. It makes the code more readable: we ignore
explicitly specific errors instead of re-raised unwanted exceptions and
implicitly ignore some errors. For example:

     try:
         os.mkdir(dir)
     except OSError as err:
         if err.errno != errno.EEXIST:
             raise

becomes

     try:
         os.mkdir(dir)
     except FileExistsError:
         pass

By the way, is it faster to not handle and than re-raise unwanted
exceptions?

(I don't like tests like "err.errno != errno.EEXIST", it's confusing to
me. It uses errno identifier twice: once as an attribute, once as a
module name.)


Make specific errors builtins
-----------------------------

I agree that it's better to make specific errors builtins.

If we move exceptions to more specific modules, like io and socket, you
have to load these modules to catch an exception and remember in which
module the exception is. An advantage of builtin specific exceptions is
to avoid an import (import errno).

Making them builtin may also avoid a bootstrap issue (catch a specific
error at startup).


Choice of the specific errors
-----------------------------

I don't understand how Antoine decided which errno should have an
exception or not.

For example, EINTR and EPIPE are handled in many modules of the standard
library but don't have their exception, whereas EISDIR
(IsADirectoryError) and ENOTDIR (NotADirectoryError) are very rare
(EISDIR is never handled in the standard library) but have their
exception.

If we add EINTR, I don't know if it's better to add it to
BlockingIOError or to create a new exception (InterruptError?).

Another good candidate is EINVAL.

Would it be possible to have an (exhaustive?) list of errno with their
popularity? At least, their popularity in the Python standard library?


Raise a specific error
----------------------

Does raising a specific error (e.g. raise FileNotFound()) set errno and
message attributes (to something different than None)?

If we provide an error message error: should it be localized? The
description of FileDescriptorError tells about the "default error
message". It we use a localized message, it's not possible to
preallocate or cache instances.

I don't see how we could choose a default errno value, because some
exceptions handle multiple errno. For example, PermissionError.errno can
be EACCES or EPERM.

Do specific errors slows down raising exceptions? Do raising an IOError
(or a "legacy" error) use a O(1) lookup to choose the exception class?


PermissionError
---------------

EACCES and EPERM have a different meaning. Even that they are very
similar, we might provide two different exceptions. EPERM is only used
once in the Python stdlib, so we might only provide AccesError.

On Linux, EPERM usually indicates an operation requiring root
priviledge.  EACCES can be related to filesystem permissions (read-only,
user is not allowed to write, etc.) or can be an mmap error.


Deprecation
-----------

Because IOError, OSError, select.error, etc. are well known exceptions,
I'm not in favor of removing them from Python 3. It would make porting
from Python 2 worse. If we don't remove them, they should not be
deprecated.

I'm in favor of adding a note in the documentation of all legacy
exceptions to advice to use IOError or specific exceptions instead. I
suppose that these notes will have to indicate a Python version.


Test the implementation
-----------------------

Did someone test Blender, Django or another major applications on the
implementation of the PEP 3151?


-1 on FileSystemError
---------------------

I'm not sure that we need FileSystemError or ConnectionError. Careless
code will use IOError, whereas careful code will use an explicit list
like (ConnectionAbortedError, ConnectionRefusedError,
ConnectionResetError).

If we remove IsADirectoryError and NotADirectoryError, FileSystemError
only contains FileExistsError and FileNotFoundError. I don't think that
these errors can occur on a same function. For example, rmdir() only
raises ENOTDIR.

I only see one advantage of FileSystemError: it doesn't handle
FileDescriptorError. Advice usage of FileSystemError would avoid to hide
real bugs like FileDescriptorError.

I don't really care of ConnectionError. Anyway, FileSystemError and
ConnectionError can be added later if needed.

WindowsError and winerror attribute
-----------------------------------

I don't like the idea of having a winerror attribute platforms other
than Windows. Why not using hasattr(err, "winerror") instead?
WindowsError is already specific to Windows. I don't see any usage of
the winerror attribute in the Python stdlib.

shutil module uses:

     try:
         WindowsError
     except NameError:
         WindowsError = None

     (...)

     try:
         copystat(src, dst)
     except OSError as why:
         if WindowsError is not None and isinstance(why, WindowsError):
             # Copying file access times may fail on Windows
             pass
         else:
             errors.extend((src, dst, str(why)))


Misc
----

Lock.acquire() doesn't raise an exception on timeout: just remove "(for
example in Lock.acquire())".

Should FileNotFound handle ENODEV? (see test_ossaudiodev)

Victor


From solipsis at pitrou.net  Mon Jul 25 02:24:30 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 25 Jul 2011 02:24:30 +0200
Subject: [Python-Dev] Comments of the PEP 3151
References: <4E2CB130.7060701@haypocalc.com>
Message-ID: <20110725022430.0ee6f5b4@pitrou.net>


Hello,

> By the way, is it faster to not handle and than re-raise unwanted
> exceptions?

You mean "is it faster to not handle than re-raise unwanted
exceptions?". It probably is, yes.

> I don't understand how Antoine decided which errno should have an
> exception or not.

Mostly experience with the stdlib combined with intuition. The list is
not cast in stone.

> If we add EINTR, I don't know if it's better to add it to
> BlockingIOError or to create a new exception (InterruptError?).

EINTR is not related to non-blocking I/O, so it shouldn't map to
BlockingIOError.

> Would it be possible to have an (exhaustive?) list of errno with their
> popularity? At least, their popularity in the Python standard library?

How do you suggest to do that?

> Does raising a specific error (e.g. raise FileNotFound()) set errno and
> message attributes (to something different than None)?

No, it doesn't. "errno" should IMO only be set if it's truely returned
by a system routine. As for the message, like with every other exception
it should be supplied by the developer.

> Do specific errors slows down raising exceptions? Do raising an IOError
> (or a "legacy" error) use a O(1) lookup to choose the exception class?

It uses a dict. You could run some benchmarks if you want, but I doubt
it makes much sense to worry about that.

> EACCES and EPERM have a different meaning. Even that they are very
> similar, we might provide two different exceptions. EPERM is only used
> once in the Python stdlib, so we might only provide AccesError.
> 
> On Linux, EPERM usually indicates an operation requiring root
> priviledge.  EACCES can be related to filesystem permissions (read-only,
> user is not allowed to write, etc.) or can be an mmap error.

I'd have to research that a bit more. Perhaps EACCES can be mapped to
AccessError and EPERM to PermissionError, but I think the proximity of
the two concepts will make things confusing, for little benefit.

> Because IOError, OSError, select.error, etc. are well known exceptions,
> I'm not in favor of removing them from Python 3. It would make porting
> from Python 2 worse. If we don't remove them, they should not be
> deprecated.

Ok. Does anyone else have an opinion on deprecations?
(not deprecating them means less work for me, anyway)

> Test the implementation
> -----------------------
> 
> Did someone test Blender, Django or another major applications on the
> implementation of the PEP 3151?

Does Django have a working Python 3 port yet?

> WindowsError and winerror attribute
> -----------------------------------
> 
> I don't like the idea of having a winerror attribute platforms other
> than Windows.

Well, there isn't, as you can see in the tests
(test_windows_error in Lib/test/test_pep3151.py).

> shutil module uses:
> 
>      (...)
> 
>      try:
>          copystat(src, dst)
>      except OSError as why:
>          if WindowsError is not None and isinstance(why, WindowsError):
>              # Copying file access times may fail on Windows
>              pass

That's the kind of code the PEP hopes to discourage :)

> Lock.acquire() doesn't raise an exception on timeout: just remove "(for
> example in Lock.acquire())".

Oops, will fix.

> Should FileNotFound handle ENODEV? (see test_ossaudiodev)

That's not really the same thing. For example, mmap says:

[ENODEV]
    The fildes argument refers to a file whose type is not supported by
mmap().

(http://www.opengroup.org/sud/sud1/xsh/mmap.htm)

Regards

Antoine.



From stephen at xemacs.org  Mon Jul 25 05:25:21 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 25 Jul 2011 12:25:21 +0900
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <20110725022430.0ee6f5b4@pitrou.net>
References: <4E2CB130.7060701@haypocalc.com>
	<20110725022430.0ee6f5b4@pitrou.net>
Message-ID: <87wrf7qbxa.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > > Did someone test Blender, Django or another major applications on the
 > > implementation of the PEP 3151?
 > 
 > Does Django have a working Python 3 port yet?

Define "working".  Martin ported Django to Python 3 years ago as proof
of concept, but never claimed it was ready for production use or as
the main line of development.

From ericsnowcurrently at gmail.com  Mon Jul 25 06:50:47 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sun, 24 Jul 2011 22:50:47 -0600
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CADiSq7dQjEDtr232tbdQkntzEHBPGnG-HvK=N1+jUJ8f3WSWtg@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
	<CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>
	<CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>
	<CADiSq7dQjEDtr232tbdQkntzEHBPGnG-HvK=N1+jUJ8f3WSWtg@mail.gmail.com>
Message-ID: <CALFfu7AXcift=eZftwmBRno7j6NCuAfEMuLa3akkunVFCSwH_g@mail.gmail.com>

On Sun, Jul 24, 2011 at 12:54 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Jul 24, 2011 at 3:05 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
>> Are there other objects in the interpreter state that are exposed in
>> sys that would have the same problem?
>
> Rebinding (rather than mutating) any global state in any module is
> always dubious due to the potential for direct references to the
> previous value. (This is a large part of the reason why imp.reload()
> often doesn't work in practice)
>
> sys.modules, sys.path, sys.argv, sys.stdout, et al are all subject to
> that caveat. For mutable state, the onus is typically on the developer
> performing the substitution to decide whether or not those
> consequences are acceptable (usually, they're not, hence you get
> idioms like "sys.path[:] = saved_copy" to revert sys.path to a saved
> value). For immutable state (such as sys.stdout), where modification
> in place isn't possible, the obligation often goes the other way, with
> developers deliberately avoiding cached references in order to
> correctly react to runtime changes.

I agree with what you are saying wholeheartedly, but still think there
is something fishy with the way that sys.modules works.  I'll take it
from here to a tracker issue though. :)

-eric

> sys.modules is by far the worst case though, since dropping modules
> out of that cache is only safe for modules that correctly support
> imp.reload(). This is not the case for any module that mutates other
> global state (or is otherwise referenced from other modules), nor does
> it work properly for extension modules.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia
>

From ncoghlan at gmail.com  Mon Jul 25 07:28:47 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 25 Jul 2011 15:28:47 +1000
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <4E2CB130.7060701@haypocalc.com>
References: <4E2CB130.7060701@haypocalc.com>
Message-ID: <CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>

On Mon, Jul 25, 2011 at 9:56 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> By the way, is it faster to not handle and than re-raise unwanted
> exceptions?

Yes, but probably not that much faster given the overhead of
instantiating the exception and unwinding the stack in the first
place.

> Choice of the specific errors
> -----------------------------
>
> I don't understand how Antoine decided which errno should have an
> exception or not.
>
> For example, EINTR and EPIPE are handled in many modules of the standard
> library but don't have their exception, whereas EISDIR
> (IsADirectoryError) and ENOTDIR (NotADirectoryError) are very rare
> (EISDIR is never handled in the standard library) but have their
> exception.

We do tend to call isdir() fairly often, though. Part of the appeal of
isdir() is the ugliness of using EAFP when you have to do explicit
errno checks.

> If we add EINTR, I don't know if it's better to add it to
> BlockingIOError or to create a new exception (InterruptError?).

InterruptedError seems like a reasonable candidate for addition to me
- catch and retry in that case is something developers are likely to
want to do.

Perhaps EPIPE should map to FileDescriptorError along with EBADF, with
different messages based on the exact error code? Potentially renamed
to DescriptorStateError? I'm not sure of the reason for singling out
EBADF for special treatment in this case - there are several other
errno values that indicate a descriptor is no longer in a usable state
(ECONSHUTDOWN would be another one, in the same vein as EPIPE).

To be honest though, what's the use case for *catching*
FileDescriptorError without catching IOError in general? Perhaps this
one should be dropped entirely, to be handled by broadly catching
IOError?

> Another good candidate is EINVAL.

As with EBADF, I'm having trouble visualising a clear use case for
handling things like EINVAL and EOPNOTSUP differently from other kinds
of IO errors.

> Would it be possible to have an (exhaustive?) list of errno with their
> popularity? At least, their popularity in the Python standard library?

Probably, but the stdlib is more a *generator* of low level
exceptions, so our code likely isn't representative enough to get a
decent set of numbers.

> If we provide an error message error: should it be localized? The
> description of FileDescriptorError tells about the "default error
> message". It we use a localized message, it's not possible to
> preallocate or cache instances.

We don't localize anything else, so there's no reason to start here.

> PermissionError
> ---------------
>
> EACCES and EPERM have a different meaning. Even that they are very
> similar, we might provide two different exceptions. EPERM is only used
> once in the Python stdlib, so we might only provide AccesError.
>
> On Linux, EPERM usually indicates an operation requiring root
> priviledge. ?EACCES can be related to filesystem permissions (read-only,
> user is not allowed to write, etc.) or can be an mmap error.

Code that cares can still fall back to exc.errno == EPERM. I don't
think we'd be doing anyone any favours by exposing subtle distinctions
like this at the Python level.

> Deprecation
> -----------
>
> Because IOError, OSError, select.error, etc. are well known exceptions,
> I'm not in favor of removing them from Python 3. It would make porting
> from Python 2 worse. If we don't remove them, they should not be
> deprecated.
>
> I'm in favor of adding a note in the documentation of all legacy
> exceptions to advice to use IOError or specific exceptions instead. I
> suppose that these notes will have to indicate a Python version.

+1 for grandfathering in the old exception names, but documenting the
recommended alternatives as of 3.3.

> -1 on FileSystemError
> ---------------------
>
> I'm not sure that we need FileSystemError or ConnectionError. Careless
> code will use IOError, whereas careful code will use an explicit list
> like (ConnectionAbortedError, ConnectionRefusedError,
> ConnectionResetError).
>
> If we remove IsADirectoryError and NotADirectoryError, FileSystemError
> only contains FileExistsError and FileNotFoundError. I don't think that
> these errors can occur on a same function. For example, rmdir() only
> raises ENOTDIR.
>
> I only see one advantage of FileSystemError: it doesn't handle
> FileDescriptorError. Advice usage of FileSystemError would avoid to hide
> real bugs like FileDescriptorError.

And that's precisely why FileSystemError is worthwhile - to separate
out when the FS is objecting, rather than there being something wrong
with the internal application state or when a previously valid
descriptor has become unusable for some reason.

It may also be reasonable to return a new DeviceNotAvailableError for
ENODEV and EBUSY (as a new FileSystemError subclass).

> I don't really care of ConnectionError. Anyway, FileSystemError and
> ConnectionError can be added later if needed.

But the use case for grouping them is quite obvious - there's is
plenty of application code that will want to handle them in a
particular way, while allowing other kinds of IOError to propagate
further up the stack. Baking this into the exception heirarchy is far
more future proof than people making their own explicit lists.

There may be some error codes that we choose to map to these generic
errors, even if we don't give them their own exception types at this
point (e.g. ECONSHUTDOWN could map directly to ConnectionError).

> Should FileNotFound handle ENODEV? (see test_ossaudiodev)

See above for my suggestion of a specific DeviceNotAvailable exception.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Jul 25 07:33:09 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 25 Jul 2011 15:33:09 +1000
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CALFfu7AXcift=eZftwmBRno7j6NCuAfEMuLa3akkunVFCSwH_g@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>
	<CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>
	<CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>
	<CADiSq7dQjEDtr232tbdQkntzEHBPGnG-HvK=N1+jUJ8f3WSWtg@mail.gmail.com>
	<CALFfu7AXcift=eZftwmBRno7j6NCuAfEMuLa3akkunVFCSwH_g@mail.gmail.com>
Message-ID: <CADiSq7chYbi=Mfrt0xyDGkNxT+6-rk8tcJSJy3VbYgPtqd=D_w@mail.gmail.com>

On Mon, Jul 25, 2011 at 2:50 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> I agree with what you are saying wholeheartedly, but still think there
> is something fishy with the way that sys.modules works. ?I'll take it
> from here to a tracker issue though. :)

Yup - the import system has a whole mess of interrelated global state
that really needs to be on a class that can use descriptors to
correctly invalidate caches when attributes are mutated or rebound.

The ImportEngine GSoC project is the first step along the long,
winding path towards doing something about that :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From martin at v.loewis.de  Mon Jul 25 08:26:09 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 25 Jul 2011 08:26:09 +0200
Subject: [Python-Dev] is sys.modules not meant to be replaced?
In-Reply-To: <CALFfu7AXcift=eZftwmBRno7j6NCuAfEMuLa3akkunVFCSwH_g@mail.gmail.com>
References: <CALFfu7AUpWY3F7fO0ta+4+tmhbWmQ7M0VLhfjLykZU8g2JybDw@mail.gmail.com>	<CAPZV6o-tvSFPJe-EwVBf8YDh+R-VEWL437ztMjQB3nXp7BR-Cw@mail.gmail.com>	<CALFfu7Btpwzmp28wm+_7aQjdfp24GVQA=+m8vKsLG4LwVoAWmQ@mail.gmail.com>	<CADiSq7dQjEDtr232tbdQkntzEHBPGnG-HvK=N1+jUJ8f3WSWtg@mail.gmail.com>
	<CALFfu7AXcift=eZftwmBRno7j6NCuAfEMuLa3akkunVFCSwH_g@mail.gmail.com>
Message-ID: <4E2D0C81.7070509@v.loewis.de>

> I agree with what you are saying wholeheartedly, but still think there
> is something fishy with the way that sys.modules works.  I'll take it
> from here to a tracker issue though. :)

Well, there is a short answer to you question: sys.modules  is *not*
meant to be replaced. Doing so will result in undefined behavior.

So there is nothing fishy about it.

Regards,
Martin

From solipsis at pitrou.net  Mon Jul 25 12:43:51 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 25 Jul 2011 12:43:51 +0200
Subject: [Python-Dev] Comments of the PEP 3151
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
Message-ID: <20110725124351.15529bc7@pitrou.net>

On Mon, 25 Jul 2011 15:28:47 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> > If we add EINTR, I don't know if it's better to add it to
> > BlockingIOError or to create a new exception (InterruptError?).
> 
> InterruptedError seems like a reasonable candidate for addition to me
> - catch and retry in that case is something developers are likely to
> want to do.

Ok, let's call it InterruptError then. InterruptedError sounds like the
error was interrupted ;)

> Perhaps EPIPE should map to FileDescriptorError along with EBADF, with
> different messages based on the exact error code? Potentially renamed
> to DescriptorStateError?

I'd rather have a separate BrokenPipeError for EPIPE.
EBADF and EPIPE are really quite different; EPIPE indicates that the
other end of the pipe closed it, while EBADF generally points to a
programming error (you are giving an invalid file descriptor to a
system routine).

> To be honest though, what's the use case for *catching*
> FileDescriptorError without catching IOError in general? Perhaps this
> one should be dropped entirely, to be handled by broadly catching
> IOError?

Good point indeed.

> > I'm in favor of adding a note in the documentation of all legacy
> > exceptions to advice to use IOError or specific exceptions instead. I
> > suppose that these notes will have to indicate a Python version.
> 
> +1 for grandfathering in the old exception names, but documenting the
> recommended alternatives as of 3.3.

Ok.

> It may also be reasonable to return a new DeviceNotAvailableError for
> ENODEV and EBUSY (as a new FileSystemError subclass).

EBUSY is really quite different from these other errors. It is
triggered by runtime protections in the OS (can't destroy some object
that is in use, see for example in pthread_cond_destroy:
http://pubs.opengroup.org/onlinepubs/009604499/functions/pthread_cond_destroy.html),
rather than indication some misassumption about the filesystem layout.

As for ENODEV, I'll have to take a look.

> There may be some error codes that we choose to map to these generic
> errors, even if we don't give them their own exception types at this
> point (e.g. ECONSHUTDOWN could map directly to ConnectionError).

Good point as well.

Regards

Antoine.



From v+python at g.nevcal.com  Mon Jul 25 18:17:36 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 25 Jul 2011 09:17:36 -0700
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <20110725124351.15529bc7@pitrou.net>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110725124351.15529bc7@pitrou.net>
Message-ID: <4E2D9720.8040901@g.nevcal.com>

On 7/25/2011 3:43 AM, Antoine Pitrou wrote:
> On Mon, 25 Jul 2011 15:28:47 +1000
> Nick Coghlan<ncoghlan at gmail.com>  wrote:
>> >  
>>> >  >  If we add EINTR, I don't know if it's better to add it to
>>> >  >  BlockingIOError or to create a new exception (InterruptError?).
>> >  
>> >  InterruptedError seems like a reasonable candidate for addition to me
>> >  - catch and retry in that case is something developers are likely to
>> >  want to do.
> Ok, let's call it InterruptError then. InterruptedError sounds like the
> error was interrupted;)
>

Sorry, no.  "InterruptError" sounds too much like a CPU interrupt 
signal, which the error is not.  "InterruptedError" is OK by me, I don't 
see the confusion you do.  But maybe "InterruptedOperationError" would 
be the most clear.  Way too long, of course, so maybe 
"InterruptedAPIError" or "InterruptedOpError" or "EINTRError" in my 
order of preference.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110725/1cb0946f/attachment.html>

From victor.stinner at haypocalc.com  Mon Jul 25 18:33:49 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 25 Jul 2011 18:33:49 +0200
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <20110725022430.0ee6f5b4@pitrou.net>
References: <4E2CB130.7060701@haypocalc.com>
	<20110725022430.0ee6f5b4@pitrou.net>
Message-ID: <4E2D9AED.8090404@haypocalc.com>

On 25/07/2011 02:24, Antoine Pitrou wrote:
>
> Hello,
>
>> By the way, is it faster to not handle and than re-raise unwanted
>> exceptions?
>
> You mean "is it faster to not handle than re-raise unwanted
> exceptions?". It probably is, yes.

I asked if:

try:
  ...
except FileNotFound:
    pass

is faster than:

try:
  ...
except IOError as err:
    if err.errno != errno.ENOENT:
       raise
    else:
       pass

f an IOError with errno != ENOENT is raised.

>> I don't understand how Antoine decided which errno should have an
>> exception or not.
>
> Mostly experience with the stdlib combined with intuition. The list is
> not cast in stone.

"The list is not cast in stone." : ok :-)

>> If we add EINTR, I don't know if it's better to add it to
>> BlockingIOError or to create a new exception (InterruptError?).
>
> EINTR is not related to non-blocking I/O, so it shouldn't map to
> BlockingIOError.

Ok, so let add InterruptError.

>> Would it be possible to have an (exhaustive?) list of errno with their
>> popularity? At least, their popularity in the Python standard library?
>
> How do you suggest to do that?

Use the list of all errno from the Appendix A, and then count the number 
of occurence in the Python source code (excluding the test suite). You 
can for example using a shell for that:

$ grep -h -c EINVAL $(find -name "*.py"|grep -v Lib/test)|grep -v '^0$'
1
1
2

These numbers give an approximation of the popularity of the different 
errno.

>> Does raising a specific error (e.g. raise FileNotFound()) set errno and
>> message attributes (to something different than None)?
>
> No, ... As for the message, like with every other exception
> it should be supplied by the developer.

Ok, I agree.

>> Do specific errors slows down raising exceptions? Do raising an IOError
>> (or a "legacy" error) use a O(1) lookup to choose the exception class?
>
> It uses a dict. You could run some benchmarks if you want, but I doubt
> it makes much sense to worry about that.

Ok, a dict should be fast.

>> EACCES and EPERM have a different meaning. Even that they are very
>> similar, we might provide two different exceptions. EPERM is only used
>> once in the Python stdlib, so we might only provide AccesError.
>>
>> On Linux, EPERM usually indicates an operation requiring root
>> priviledge.  EACCES can be related to filesystem permissions (read-only,
>> user is not allowed to write, etc.) or can be an mmap error.
>
> I'd have to research that a bit more. Perhaps EACCES can be mapped to
> AccessError and EPERM to PermissionError, but I think the proximity of
> the two concepts will make things confusing, for little benefit.

You are probable right.

>> WindowsError and winerror attribute
>> -----------------------------------
>>
>> I don't like the idea of having a winerror attribute platforms other
>> than Windows.
>
> Well, there isn't, as you can see in the tests

Oh, it's not clear in the PEP.

>> Should FileNotFound handle ENODEV? (see test_ossaudiodev)
>
> That's not really the same thing.

Ok.

Victor

From ethan at stoneleaf.us  Mon Jul 25 20:44:29 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 25 Jul 2011 11:44:29 -0700
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <4E2D9720.8040901@g.nevcal.com>
References: <4E2CB130.7060701@haypocalc.com>	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>	<20110725124351.15529bc7@pitrou.net>
	<4E2D9720.8040901@g.nevcal.com>
Message-ID: <4E2DB98D.20106@stoneleaf.us>

Glenn Linderman wrote:
>   On 7/25/2011 3:43 AM, Antoine Pitrou wrote:
>> On Mon, 25 Jul 2011 15:28:47 +1000
>> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> > 
>>>> > > If we add EINTR, I don't know if it's better to add it to
>>>> > > BlockingIOError or to create a new exception (InterruptError?).
>>> > 
>>> > InterruptedError seems like a reasonable candidate for addition to me
>>> > - catch and retry in that case is something developers are likely to
>>> > want to do.
>> Ok, let's call it InterruptError then. InterruptedError sounds like the
>> error was interrupted ;)
>>
> 
> Sorry, no.  "InterruptError" sounds too much like a CPU interrupt 
> signal, which the error is not.

It does, a bit -- but is that something we need to worry about at the 
Python level?  Seems to me we should have the names make sense for 
Python, and not worry about what assembly, C, Pascal, Perl, or <insert 
language X here> names might mean for them.

>  "InterruptedError" is OK by me, I don't 
> see the confusion you do.  But maybe "InterruptedOperationError" would 
> be the most clear.  Way too long, of course, so maybe 
> "InterruptedAPIError" or "InterruptedOpError" or "EINTRError" in my 
> order of preference.

Please not that last one!  ;)

I prefer InterruptedError or InterruptError.

~Ethan~

From brett at python.org  Tue Jul 26 02:34:33 2011
From: brett at python.org (Brett Cannon)
Date: Mon, 25 Jul 2011 17:34:33 -0700
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
Message-ID: <CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>

On Sat, Jul 23, 2011 at 20:35, Eli Bendersky <eliben at gmail.com> wrote:

> Some background: I'm working (on and off) on issue 11015 - documenting
> the public functions in test.support
>
> Some of the functions in test.support (for example unlink, rmtree)
> simply shadow existing & popular stdlib functions, with the aim of
> swallowing the exceptions these may throw. This is confusing, IMHO.
> For example, grepping 'unlink' on Lib/test/test_*.py files doesn't say
> much about which unlink is being used.
>
> A couple of options to handle this are:
>
> 1. Remove these functions altogether, trying to use existing
> constructs (such as the ignore_errors parameter in rmtree).
> 2. Adapt a naming convention for such functions, for instance
> rmtree_silent and unlink_silent (or a similar convention, if one
> exists)
>
> Opinions?
>

The mere fact that these functions exist in a different module suggests
different semantics from those found in other places in the stdlib. I don't
think they should be renamed simply because some code imports the functions
directly instead of the module itself (suggesting they should be doing the
latter over the former). Now if the functions are redundant that's something
else entirely and removal should be fine.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110725/de04e5b4/attachment.html>

From ncoghlan at gmail.com  Tue Jul 26 02:36:11 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 26 Jul 2011 10:36:11 +1000
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <4E2DB98D.20106@stoneleaf.us>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110725124351.15529bc7@pitrou.net>
	<4E2D9720.8040901@g.nevcal.com> <4E2DB98D.20106@stoneleaf.us>
Message-ID: <CADiSq7eMqcDdoTLHPjdeZDGXy5H-4S7fup1yHQH9_0DBORJJHw@mail.gmail.com>

On Tue, Jul 26, 2011 at 4:44 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> Glenn Linderman wrote:
>>
>> ?On 7/25/2011 3:43 AM, Antoine Pitrou wrote:
>>> Ok, let's call it InterruptError then. InterruptedError sounds like the
>>> error was interrupted ;)
>>>
>>
>> Sorry, no. ?"InterruptError" sounds too much like a CPU interrupt signal,
>> which the error is not.
>
> It does, a bit -- but is that something we need to worry about at the Python
> level? ?Seems to me we should have the names make sense for Python, and not
> worry about what assembly, C, Pascal, Perl, or <insert language X here>
> names might mean for them.

Like Glenn, I believe "InterruptError" sounds wrong - the event being
reported is that a system call was interrupted for some unknown
reason, not necessarily that the process received an interrupt.
'Interrupt' in computer science requires context to distinguish
between the verb and noun forms, and an exception name doesn't provide
that context. 'Interrupted' forces interpretation as the past tense of
the verb form, which is the meaning we want. If the subject of the
verb feels too ambiguous then I'd prefer to switch to the explicit
adjective form 'InterruptedCallError' rather than allowing the
verb/noun ambiguity.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From andrew at bemusement.org  Tue Jul 26 02:26:44 2011
From: andrew at bemusement.org (Andrew Bennetts)
Date: Tue, 26 Jul 2011 10:26:44 +1000
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <4E2DB98D.20106@stoneleaf.us>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110725124351.15529bc7@pitrou.net>
	<4E2D9720.8040901@g.nevcal.com> <4E2DB98D.20106@stoneleaf.us>
Message-ID: <20110726002644.GJ25354@aihal.home.puzzling.org>

Ethan Furman wrote:
> > [?] or "EINTRError" in my order of preference.
> 
> Please not that last one!  ;)

Why not, exactly?

When EINTR happens it's frequently a surprise, but programmers new to
the concept can always search the web for advice on what causes it and
how to deal with it (and after several attempts at dealing with it they
may even get it right).  Searching Google for ?InterruptedError? isn't
going to find anything helpful today, and eventually what I expect it
would find is a bunch of pages saying ?Look up EINTR.?  How about we
just cut out that middle step and call it what the rest of the world
calls it?

If ?InterruptedError? were going to be used for anything other than
EINTR then I could see an argument for abstracting the concept behind a
platform-independent name.  But I think it would be a mistake to treat
anything else as being the same as EINTR.

-Andrew.


From ncoghlan at gmail.com  Tue Jul 26 03:45:33 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 26 Jul 2011 11:45:33 +1000
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <20110726002644.GJ25354@aihal.home.puzzling.org>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110725124351.15529bc7@pitrou.net>
	<4E2D9720.8040901@g.nevcal.com> <4E2DB98D.20106@stoneleaf.us>
	<20110726002644.GJ25354@aihal.home.puzzling.org>
Message-ID: <CADiSq7fdj7cWfY7E3BdKFQvmpDiC9DirpZ36arRm_YcCedugow@mail.gmail.com>

On Tue, Jul 26, 2011 at 10:26 AM, Andrew Bennetts <andrew at bemusement.org> wrote:
> When EINTR happens it's frequently a surprise, but programmers new to
> the concept can always search the web for advice on what causes it and
> how to deal with it (and after several attempts at dealing with it they
> may even get it right). ?Searching Google for ?InterruptedError? isn't
> going to find anything helpful today, and eventually what I expect it
> would find is a bunch of pages saying ?Look up EINTR.? ?How about we
> just cut out that middle step and call it what the rest of the world
> calls it?
>
> If ?InterruptedError? were going to be used for anything other than
> EINTR then I could see an argument for abstracting the concept behind a
> platform-independent name. ?But I think it would be a mistake to treat
> anything else as being the same as EINTR.

The whole point of PEP 3151 is so that Python programmers can perform
key tasks without needing to worry about the existence of error
numbers under the hood. Including the cryptic errno abbreviations in
the interrupt names would completely miss the point.

However, the docs will point to appropriate information in the errno
module (and the exception details and docstrings may also mention the
errno codes). Abstractions do leak, after all, but that doesn't mean
we need to go punching holes in them with an icepick.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ethan at stoneleaf.us  Tue Jul 26 04:31:48 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 25 Jul 2011 19:31:48 -0700
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <20110726002644.GJ25354@aihal.home.puzzling.org>
References: <4E2CB130.7060701@haypocalc.com>	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>	<20110725124351.15529bc7@pitrou.net>	<4E2D9720.8040901@g.nevcal.com>
	<4E2DB98D.20106@stoneleaf.us>
	<20110726002644.GJ25354@aihal.home.puzzling.org>
Message-ID: <4E2E2714.3040809@stoneleaf.us>

Andrew Bennetts wrote:
> Ethan Furman wrote:
>>> [?] or "EINTRError" in my order of preference.
 >>
>> Please not that last one!  ;)
> 
> Why not, exactly?

Because this is Python, and readability counts.  Yes, it does take some 
getting used to (I finally stopped typing 'enum' a couple months ago ;) 
, but it is a worthwhile goal -- a goal we take a step back from with 
names like EINTRError instead of InterruptedError.

~Ethan~

From stephen at xemacs.org  Tue Jul 26 05:47:01 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 26 Jul 2011 12:47:01 +0900
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <4E2D9720.8040901@g.nevcal.com>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110725124351.15529bc7@pitrou.net>
	<4E2D9720.8040901@g.nevcal.com>
Message-ID: <87oc0hr9e2.fsf@uwakimon.sk.tsukuba.ac.jp>

Glenn Linderman writes:

 > Sorry, no.  "InterruptError" sounds too much like a CPU interrupt 
 > signal, which the error is not.  "InterruptedError" is OK by me, I don't 
 > see the confusion you do.  But maybe "InterruptedOperationError" would 
 > be the most clear.  Way too long, of course, so maybe 
 > "InterruptedAPIError" or "InterruptedOpError" or "EINTRError" in my 
 > order of preference.

Eh, doesn't it bother anybody that it's not an error, but a user
action?

If it needs a separate name, something like InterruptException seems
most accurate to me.

From ncoghlan at gmail.com  Tue Jul 26 06:18:58 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 26 Jul 2011 14:18:58 +1000
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <87oc0hr9e2.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110725124351.15529bc7@pitrou.net>
	<4E2D9720.8040901@g.nevcal.com>
	<87oc0hr9e2.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7e1kyZzavUs6+EjTKiR3KHFPKF4Qp2-SZ1yW7W6WUKJtQ@mail.gmail.com>

On Tue, Jul 26, 2011 at 1:47 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Eh, doesn't it bother anybody that it's not an error, but a user
> action?

Nope, doesn't bother me in the slightest. It's an error number code,
just like all the others. Several other error numbers may or may not
be errors too, depending on context. Rather than quibbling about that
part of the naming scheme it's easier to say that the call failing to
complete successfully is an error by default, but an application may
choose to interpret some cases as not really being errors, since
there's a defined response (most obvious case: a file or directory
already existing or not existing often isn't an error from the
application's point of view, it's just the application ensuring that a
particular configuration exists on the file system in an idempotent
fashion).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From merwok at netwok.org  Tue Jul 26 15:17:53 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 26 Jul 2011 15:17:53 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Issue #12102: Merge with 3.2.
In-Reply-To: <E1QlDWD-0002za-Am@dinsdale.python.org>
References: <E1QlDWD-0002za-Am@dinsdale.python.org>
Message-ID: <4E2EBE81.8010701@netwok.org>

Hi,

> changeset:   71497:4898b14dcd69
> user:        Ross Lagerwall <rosslagerwall at gmail.com>
> summary:
>   Issue #12102: Merge with 3.2.
> 
> files:
>   Doc/ACKS.txt         |  1 +
>   Doc/library/mmap.rst |  6 ++++++
>   Misc/NEWS            |  3 +++

> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -237,6 +237,9 @@
>  Library
>  -------
>  
> +- Issue #12102: Document that buffered files must be flushed before being used
> +  with mmap. Patch by Steffen Daode Nurpmeso.

We don?t add NEWS entries for each and every doc fix, otherwise it would
be very huge :)  We rather document large changes to the documentation,
like adding links to the source files, using ?python3? instead of
?python? in all examples, etc.  In addition, Library is the wrong
section, it should be Documentation.

I?m doing commits this afternoon, so I?ll take the occasion to remove
this entry.

Cheers

From merwok at netwok.org  Tue Jul 26 15:20:55 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 26 Jul 2011 15:20:55 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <E1QkV7i-0002pL-Av@dinsdale.python.org>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
Message-ID: <4E2EBF37.7020702@netwok.org>

Hi,

> changeset:   71465:be558ad15789
> user:        Eli Bendersky <eliben at gmail.com>
> summary:
>   Issue #11049: adding some tests to test.support
> Based on original patch by Giampaolo Rodola with contributions from R. David Murray
> 
> files:
>   Lib/test/support.py      |   21 +-
>   Lib/test/test_support.py |  178 +++++++++++++++++++++++++++
>   2 files changed, 189 insertions(+), 10 deletions(-)
> 
> diff --git a/Lib/test/support.py b/Lib/test/support.py
> --- a/Lib/test/support.py
> +++ b/Lib/test/support.py
> @@ -170,7 +170,7 @@
>          attribute = getattr(obj, name)
>      except AttributeError:
>          raise unittest.SkipTest("module %s has no attribute %s" % (
> -            obj.__name__, name))
> +            repr(obj), name))

I would use %r instead of %s for both fields here.  Non-ASCII characters
and unseen whitespace are at least two reasons to overuse %r in
debug/error messages instead of %s.

Regards

From solipsis at pitrou.net  Tue Jul 26 15:30:25 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 26 Jul 2011 15:30:25 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org>
Message-ID: <20110726153025.66b0de8b@pitrou.net>

On Tue, 26 Jul 2011 15:20:55 +0200
?ric Araujo <merwok at netwok.org> wrote:
> > 
> > diff --git a/Lib/test/support.py b/Lib/test/support.py
> > --- a/Lib/test/support.py
> > +++ b/Lib/test/support.py
> > @@ -170,7 +170,7 @@
> >          attribute = getattr(obj, name)
> >      except AttributeError:
> >          raise unittest.SkipTest("module %s has no attribute %s" % (
> > -            obj.__name__, name))
> > +            repr(obj), name))
> 
> I would use %r instead of %s for both fields here.  Non-ASCII characters
> and unseen whitespace are at least two reasons to overuse %r in
> debug/error messages instead of %s.

Actually, you want %a for non-ASCII messages to be escaped.
(however, there's hardly any reason to worry about it when it comes to
stdlib module names)

Regards

Antoine.



From merwok at netwok.org  Tue Jul 26 16:10:47 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 26 Jul 2011 16:10:47 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11784: Improve
 multiprocessing.Process.join() documentation. Patch by
In-Reply-To: <E1QlO8d-00034r-1x@dinsdale.python.org>
References: <E1QlO8d-00034r-1x@dinsdale.python.org>
Message-ID: <4E2ECAE7.503@netwok.org>

> changeset:   71499:8d67fd820627
> parent:      71497:4898b14dcd69
> user:        Charles-Fran?ois Natali <neologix at free.fr>
> date:        Mon Jul 25 18:35:49 2011 +0200
> summary:
>   Issue #11784: Improve multiprocessing.Process.join() documentation. Patch by
> Patrick Sabin.
> 
> files:
>   Doc/library/multiprocessing.rst |  7 +++----
>   Misc/ACKS                       |  1 +

There?s a dedicated file to thank doc contributors: Doc/ACKS.rst

Regards

From merwok at netwok.org  Tue Jul 26 16:10:50 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 26 Jul 2011 16:10:50 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <20110726153025.66b0de8b@pitrou.net>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>	<4E2EBF37.7020702@netwok.org>
	<20110726153025.66b0de8b@pitrou.net>
Message-ID: <4E2ECAEA.4090308@netwok.org>

Le 26/07/2011 15:30, Antoine Pitrou a ?crit :
> Actually, you want %a for non-ASCII messages to be escaped.

Thanks for the reminder, I should use more %a instead of %r.  In the
packaging code however, we can?t, given that we want to backport.

> (however, there's hardly any reason to worry about it when it comes to
> stdlib module names)

I lacked context to see that.  If the code in question only ever handles
stdlib modules, then okay.

Cheers

From merwok at netwok.org  Tue Jul 26 16:39:31 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 26 Jul 2011 16:39:31 +0200
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #8746:
 Correct faulty configure checks so that os.chflags() and
In-Reply-To: <E1QbSUY-00083q-8g@dinsdale.python.org>
References: <E1QbSUY-00083q-8g@dinsdale.python.org>
Message-ID: <4E2ED1A3.3050208@netwok.org>

Hi,

> changeset:   71030:abfe28e7e5cd
> branch:      2.7
> user:        Ned Deily <nad at acm.org>

> diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py
> --- a/Lib/test/test_posix.py
> +++ b/Lib/test/test_posix.py
> @@ -11,10 +11,12 @@
>  import os
>  import pwd
>  import shutil
> +import stat
>  import sys
>  import unittest
>  import warnings
>  
> +_DUMMY_SYMLINK = '%s/dummy-symlink' % os.getenv('TMPDIR', '/tmp')

Why not let the tempfile module do this for you?

Regards

From sdaoden at googlemail.com  Tue Jul 26 16:42:18 2011
From: sdaoden at googlemail.com (Steffen Daode Nurpmeso)
Date: Tue, 26 Jul 2011 16:42:18 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Issue #12102: Merge with 3.2.
In-Reply-To: <4E2EBE81.8010701@netwok.org>
References: <E1QlDWD-0002za-Am@dinsdale.python.org>
	<4E2EBE81.8010701@netwok.org>
Message-ID: <20110726144218.GA15388@sherwood.local>

@ ?ric Araujo <merwok at netwok.org> wrote (2011-07-26 15:17+0200):
> > [.]
> >   Doc/library/mmap.rst |  6 ++++++
> > [.]
> > +  with mmap. Patch by Steffen Daode Nurpmeso.
> 
> I?m doing commits this afternoon, so I?ll take the occasion to remove
> this entry.

Murmur... but it hits me little: where is the difference in
between Doc/ACKS.txt and Misc/ACKS?
I'm mentioned in the latter (on multiple branches), but not at all
in the former.

Sob.

But this i tell you - if i would be an italian.. then i.. would..

SOOOOOOOOOB!!

Viva la mamma - per que!!!

--Steffen
Ciao, sdaoden(*)(gmail.com)
ASCII ribbon campaign           ( ) More nuclear fission plants
  against HTML e-mail            X    can serve more coloured
    and proprietary attachments / \     and sounding animations

From merwok at netwok.org  Tue Jul 26 16:48:22 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 26 Jul 2011 16:48:22 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Issue #12102: Merge with 3.2.
In-Reply-To: <20110726144218.GA15388@sherwood.local>
References: <E1QlDWD-0002za-Am@dinsdale.python.org>	<4E2EBE81.8010701@netwok.org>
	<20110726144218.GA15388@sherwood.local>
Message-ID: <4E2ED3B6.9090907@netwok.org>

Hi,

> Murmur... but it hits me little: where is the difference in
> between Doc/ACKS.txt and Misc/ACKS?

Doc/ACKS.txt is used for doc contributions, Misc/ACKS for other
contributions (but sometimes doc too!).  Doc/A is also displayed on
docs.python.org whereas Misc/A is only readable in tarballs.

We talked about merging them a little time ago; I have to check my
archives, I don?t remember what issues there were.

Regards

From mail at kerrickstaley.com  Tue Jul 26 17:56:56 2011
From: mail at kerrickstaley.com (Kerrick Staley)
Date: Tue, 26 Jul 2011 10:56:56 -0500
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <20110724232106.53161528@pitrou.net>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
Message-ID: <CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>

I'm indifferent either way. python3 is a hard link to python3.2, so I
thought we'd make everything that way for consistency. Higher-level
links (python/idle/pydoc/python-config) have to be soft links so that
if, e.g., python points to python3, and python3 is then pointed to
another location, python also gets changed.

-Kerrick Staley

From rosslagerwall at gmail.com  Tue Jul 26 18:03:21 2011
From: rosslagerwall at gmail.com (Ross Lagerwall)
Date: Tue, 26 Jul 2011 18:03:21 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Issue #12102: Merge with 3.2.
In-Reply-To: <4E2EBE81.8010701@netwok.org>
References: <E1QlDWD-0002za-Am@dinsdale.python.org>
	<4E2EBE81.8010701@netwok.org>
Message-ID: <1311696201.1508.0.camel@hobo>

> We don?t add NEWS entries for each and every doc fix, otherwise it would
> be very huge :)  We rather document large changes to the documentation,
> like adding links to the source files, using ?python3? instead of
> ?python? in all examples, etc.  In addition, Library is the wrong
> section, it should be Documentation.
> 
> I?m doing commits this afternoon, so I?ll take the occasion to remove
> this entry.
> 

Thanks for the heads up. I'll keep it in mind for next time.

Cheers
Ross


From solipsis at pitrou.net  Tue Jul 26 18:05:28 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 26 Jul 2011 18:05:28 +0200
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
Message-ID: <1311696328.3648.0.camel@localhost.localdomain>

Le mardi 26 juillet 2011 ? 10:56 -0500, Kerrick Staley a ?crit :
> I'm indifferent either way. python3 is a hard link to python3.2, so I
> thought we'd make everything that way for consistency.

Is it? Yikes, I didn't know about that.

Regards

Antoine.



From merwok at netwok.org  Tue Jul 26 18:15:15 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 26 Jul 2011 18:15:15 +0200
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
 /usr/bin/python2 symlink upstream)
In-Reply-To: <1311696328.3648.0.camel@localhost.localdomain>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>	<nad-62B81D.01034918072011@dough.gmane.org>	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>	<20110724231655.0b69d36a@pitrou.net>	<20110724232106.53161528@pitrou.net>	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
Message-ID: <4E2EE813.1080407@netwok.org>

Le 26/07/2011 18:05, Antoine Pitrou a ?crit :
> Le mardi 26 juillet 2011 ? 10:56 -0500, Kerrick Staley a ?crit :
>> I'm indifferent either way. python3 is a hard link to python3.2, so I
>> thought we'd make everything that way for consistency.
>
> Is it? Yikes, I didn't know about that.

Yikes for me too.  I?ve had a quick look at the Makefile (look for
$(LN)) and found that all scripts use symbolic links, but the python3 to
python3.y link is hard.  I wonder why this is.

FTR, downstream packagers may change this.  Example on Debian:

/usr/bin/python3:     symbolic link to `python3.2'
/usr/bin/python3.2:   symbolic link to `python3.2mu'

Cheers

From nad at acm.org  Tue Jul 26 20:50:47 2011
From: nad at acm.org (Ned Deily)
Date: Tue, 26 Jul 2011 11:50:47 -0700
Subject: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the
	/usr/bin/python2 symlink upstream)
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org>
Message-ID: <nad-971FD6.11504626072011@news.gmane.org>

In article <4E2EE813.1080407 at netwok.org>,
 ?ric Araujo <merwok at netwok.org> wrote:

> Le 26/07/2011 18:05, Antoine Pitrou a ?crit :
> > Le mardi 26 juillet 2011 ? 10:56 -0500, Kerrick Staley a ?crit :
> >> I'm indifferent either way. python3 is a hard link to python3.2, so I
> >> thought we'd make everything that way for consistency.
> > Is it? Yikes, I didn't know about that. 
> Yikes for me too.  I?ve had a quick look at the Makefile (look for
> $(LN)) and found that all scripts use symbolic links, but the python3 to
> python3.y link is hard.  I wonder why this is.

I pointed that out earlier in the thread:

"But if you look at the Python 3 "bininstall" target (Makefile.pre.in 
starting around line 870 or so), you'll see that, for Python 3, symlinks 
are made for all the versioned files except "python3".  I'm not sure 
that there is a particular reason why a distinction was made;  IIRC, the 
other versioned links were added later in the cycle.  The other Python 3 
versioned links could probably be changed to hard links as well.  In the 
end, I don't think it makes a lot of difference.  But it would be better 
if Python 2 and Python 3 were consistent here."

I don't think it makes all that much difference one way or the other.  
But it would be better for them all to be one kind or the other.

-- 
 Ned Deily,
 nad at acm.org


From cf.natali at gmail.com  Tue Jul 26 22:43:00 2011
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Tue, 26 Jul 2011 22:43:00 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11784: Improve
 multiprocessing.Process.join() documentation. Patch by
In-Reply-To: <4E2ECAE7.503@netwok.org>
References: <E1QlO8d-00034r-1x@dinsdale.python.org> <4E2ECAE7.503@netwok.org>
Message-ID: <CAH_1eM30pD7ZAs4Puco9RiYkNKdMvCiwZVFtPmDd_sECdgiDOA@mail.gmail.com>

> There?s a dedicated file to thank doc contributors: Doc/ACKS.rst

I didn't know about this file, thanks.
In my "defense", there's this comment at the top of Misc/ACKS:
"""
This list is not complete and not in any useful order, but I would
like to thank everybody who contributed in any way, with code, hints,
bug reports, ideas, moral support, endorsement, or even complaints....
Without you, I would've stopped working on Python long ago!

        --Guido
"""

What's the rationale for having a dedicated file?

From nad at acm.org  Tue Jul 26 23:17:24 2011
From: nad at acm.org (Ned Deily)
Date: Tue, 26 Jul 2011 14:17:24 -0700
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #8746:
	Correct faulty configure checks so that os.chflags() and
References: <E1QbSUY-00083q-8g@dinsdale.python.org>
	<4E2ED1A3.3050208@netwok.org>
Message-ID: <nad-F6A447.14172426072011@news.gmane.org>

In article <4E2ED1A3.3050208 at netwok.org>,
 Eric Araujo <merwok at netwok.org> wrote:
> > changeset:   71030:abfe28e7e5cd
> > branch:      2.7
> > user:        Ned Deily <nad at acm.org>
> 
> > diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py
> > --- a/Lib/test/test_posix.py
> > +++ b/Lib/test/test_posix.py
> > @@ -11,10 +11,12 @@
> >  import os
> >  import pwd
> >  import shutil
> > +import stat
> >  import sys
> >  import unittest
> >  import warnings
> >  
> > +_DUMMY_SYMLINK = '%s/dummy-symlink' % os.getenv('TMPDIR', '/tmp')
> 
> Why not let the tempfile module do this for you?

I didn't feel it was worthwhile modifying the OP's submitted working 
patch.  But, since you've pointed it out, I've modified the test to use 
tempfile and randomize the symlink file name as well.

-- 
 Ned Deily,
 nad at acm.org


From sdaoden at googlemail.com  Tue Jul 26 22:18:12 2011
From: sdaoden at googlemail.com (Steffen Daode Nurpmeso)
Date: Tue, 26 Jul 2011 22:18:12 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Issue #12102: Merge with 3.2.
In-Reply-To: <4E2EBE81.8010701@netwok.org>
References: <E1QlDWD-0002za-Am@dinsdale.python.org>
	<4E2EBE81.8010701@netwok.org>
Message-ID: <20110726201812.GA49358@sherwood.local>

I was contacted off-list due to anxiety about cleanliness of
python-dev in respect to the following lines:

> But this i tell you - if i would be an italian.. then i.. would..
> Viva la mamma - per que!!!

Sob.

This is **NOT** offensive against just anybody.
(Except maybe that i don't take *myself* too seriously, because
doing so is a bit strange on a planet which disappears in some
billion years, at which time i'll be gone and away.  At latest.)
But especially *NOT* against italians.
I *love* Italy!  I'm a german!!

I'm actually talking about
http://www.youtube.com/watch?v=czjGpFyS-5w here, though i feel
more like http://www.youtube.com/watch?v=lblz8g5mQRk now.
(Aah, Branduardi.  He's a good one.)

Hm.  Maybe "that" Italy is gone now though, i haven't been
there for some years ...  But that is something completely
different.  (Dunno - is it still possible to do sightseeing?
We here in Germany do have some ancient Rome stuff in good
condition, just in case it would be needed?
Hey - you know where you can find it ...)

Yes, yes, i *love* France, too!
My lovely woman and i came together during a short trip to France,
20 years ago.  [Maybe we will spend our old days there, if we can
afford it (and i'm able to learn the language).  But *i* also can
imagine some italian island south-of-Rome for that purpose.]

But first i was there due to a city partnership, in 1985, football
(aeh - this one you play with your feet) tournament.  Small town.
Played the german national anthem.  Played the french national
anthem.  Sorry, maybe i misunderstood "Marchons, marchons".  The
french boys grinned anyway.  I was a german goalkeeper, then.

And my guest-family (little farmer) grilled their major (!!!!)
rooster for me (and a second boy they hosted).  I mean -
i wouldn't have grilled a lion (heraldic animal) if they came to
Hessen in turn!  Isn't that a bit strange??  Luckily i already
knew the meaning of "Je ne regrette rien".  (Aah - Edith Piaf!!)
But the farmers wife stated "NO RIEN!!  NO RIEN!!!" in turn!

Can you imagine that?
Such an aggressive rooster!!!
Even dead.  Even *completely* eaten up!

I can tell you.  The worst thing was that i couldn't go to toilet
for those three days, because - it was *soo* shitty!

Sob.

Just in case i won't get blacklisted: next time i'm contacted
off-list i'll tell the story when our football (with the feet,
with your feet!) team went to Poland due to city partnership in
the year after tchernobyl.  And guess who talked :) to the mayor
of the city?  Hah!!

[Nose blow.]

Oh, and before i forget it: my boys (Football - feet!) were mostly
Turks, one Moroccan, one Italian (!), Andreas from Poland
(fantastic defender on the left side!) and less than 50% germans.
And we were a really good *playing* youth team.  No, not always
the winning team.

Horsy horse says Nighty night.

--Steffen
Ciao, sdaoden(*)(gmail.com)
ASCII ribbon campaign           ( ) More nuclear fission plants
  against HTML e-mail            X    can serve more coloured
    and proprietary attachments / \     and sounding animations

From solipsis at pitrou.net  Wed Jul 27 00:19:12 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 00:19:12 +0200
Subject: [Python-Dev] hard linking executables
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org>
Message-ID: <20110727001912.4cf67481@pitrou.net>


Ok, apparently the decision to make hard links for executables dates at
least back to:

changeset:   16221:588691f806f4
branch:      legacy-trunk
user:        Neil Schemenauer <nascheme at enme.ucalgary.ca>
date:        Wed Jan 24 17:11:43 2001 +0000
files:       Makefile.pre.in
description:
Flat makefile based on toplevel Makefile.in and makefiles in build
subdirectories.  Those other makefiles will go away eventually.

[...]

+# Install the interpreter (by creating a hard link to python$(VERSION))
+bininstall:    altbininstall
+       -if test -f $(BINDIR)/$(PYTHON); \
+       then rm -f $(BINDIR)/$(PYTHON); \
+       else true; \
+       fi
+       (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT))
+


Regards

Antoine.



From solipsis at pitrou.net  Wed Jul 27 00:23:23 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 00:23:23 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Issue #12102: Merge with 3.2.
References: <E1QlDWD-0002za-Am@dinsdale.python.org>
	<4E2EBE81.8010701@netwok.org>
	<20110726201812.GA49358@sherwood.local>
Message-ID: <20110727002323.62cb398e@pitrou.net>


Hi,

> > But this i tell you - if i would be an italian.. then i.. would..
> > Viva la mamma - per que!!!
> 
> Sob.
> 
> This is **NOT** offensive against just anybody.
> (Except maybe that i don't take *myself* too seriously, because
> doing so is a bit strange on a planet which disappears in some
> billion years, at which time i'll be gone and away.  At latest.)
> But especially *NOT* against italians.
> I *love* Italy!  I'm a german!!

Well I don't think anyone would take it as offensive.

However, obscure jokes without any context don't (IMHO) make your
messages very readable either, and I think they would benefit from
being concise and to the point ;)

Hope that helps.

Regards

Antoine.



From solipsis at pitrou.net  Wed Jul 27 00:49:55 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 00:49:55 +0200
Subject: [Python-Dev] Comments of the PEP 3151
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
Message-ID: <20110727004955.1642e815@pitrou.net>

On Mon, 25 Jul 2011 15:28:47 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> There may be some error codes that we choose to map to these generic
> errors, even if we don't give them their own exception types at this
> point (e.g. ECONSHUTDOWN could map directly to ConnectionError).

Ok, I can find neither ECONSHUTDOWN nor ECONNSHUTDOWN on
www.opengroup.org, and it's not mentioned in errnomodule.c.  Is it some
system-specific error code?

Regards

Antoine.



From glyph at twistedmatrix.com  Wed Jul 27 01:32:56 2011
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 26 Jul 2011 19:32:56 -0400
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <20110727004955.1642e815@pitrou.net>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110727004955.1642e815@pitrou.net>
Message-ID: <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com>


On Jul 26, 2011, at 6:49 PM, Antoine Pitrou wrote:

> On Mon, 25 Jul 2011 15:28:47 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> There may be some error codes that we choose to map to these generic
>> errors, even if we don't give them their own exception types at this
>> point (e.g. ECONSHUTDOWN could map directly to ConnectionError).
> 
> Ok, I can find neither ECONSHUTDOWN nor ECONNSHUTDOWN on
> www.opengroup.org, and it's not mentioned in errnomodule.c.  Is it some
> system-specific error code?

I assume that ESHUTDOWN is the errno in question?  (This is also already mentioned in the PEP.)


From solipsis at pitrou.net  Wed Jul 27 01:45:18 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 01:45:18 +0200
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110727004955.1642e815@pitrou.net>
	<51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com>
Message-ID: <20110727014518.6515fd5f@pitrou.net>

On Tue, 26 Jul 2011 19:32:56 -0400
Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> 
> On Jul 26, 2011, at 6:49 PM, Antoine Pitrou wrote:
> 
> > On Mon, 25 Jul 2011 15:28:47 +1000
> > Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> There may be some error codes that we choose to map to these generic
> >> errors, even if we don't give them their own exception types at this
> >> point (e.g. ECONSHUTDOWN could map directly to ConnectionError).
> > 
> > Ok, I can find neither ECONSHUTDOWN nor ECONNSHUTDOWN on
> > www.opengroup.org, and it's not mentioned in errnomodule.c.  Is it some
> > system-specific error code?
> 
> I assume that ESHUTDOWN is the errno in question?  (This is also already mentioned in the PEP.)

Indeed, I mentioned it in the PEP, as it appears in asyncore.py.
But I can't find it on www.opengroup.org, and no man page on my Linux
system (except the "errno" man page) seems to mention it.

The description from errnomodule.c says "Cannot send after transport
endpoint shutdown", but send() actually returns EPIPE, not ESHUTDOWN,
when the socket has been shutdown:

>>> conn = socket.create_connection(("www.python.org", 80))
>>> conn.shutdown(socket.SHUT_WR)
>>> conn.send(b"xxx")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
socket.error: [Errno 32] Broken pipe

From the send() man page:

       EPIPE  The  local  end has been shut down on a connection
       oriented socket.  In this case the process will also receive a
       SIGPIPE unless MSG_NOSIGNAL is set.

Regards

Antoine.

From solipsis at pitrou.net  Wed Jul 27 02:03:19 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 02:03:19 +0200
Subject: [Python-Dev] ESHUTDOWN
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110727004955.1642e815@pitrou.net>
	<51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com>
	<20110727014518.6515fd5f@pitrou.net>
Message-ID: <20110727020319.4e5ca9d3@pitrou.net>


Some more digging indicates that ESHUTDOWN appears in asyncore with the
following commit:

changeset:   10934:c089020a7a1e
branch:      legacy-trunk
user:        Guido van Rossum <guido at python.org>
date:        Tue Jun 08 13:20:05 1999 +0000
files:       Lib/asynchat.py Lib/asyncore.py
description:
Sam's latest versions


while it appears in errnomodule.c with the following commit:

changeset:   3804:48776bf4bd49
branch:      legacy-trunk
user:        Guido van Rossum <guido at python.org>
date:        Wed Jul 24 00:51:51 1996 +0000
files:       Modules/Setup.in Modules/errnomodule.c
description:
Added Sam Rushing's errno module


It also seems that WSAESHUTDOWN can be returned under Windows by
send(), rather than EPIPE:

?WSAESHUTDOWN

The socket has been shut down; it is not possible to send on a socket
after shutdown has been invoked with how set to SD_SEND or SD_BOTH.?

Regards

Antoine.



From ncoghlan at gmail.com  Wed Jul 27 02:41:34 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 27 Jul 2011 10:41:34 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <4E2ECAEA.4090308@netwok.org>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net>
	<4E2ECAEA.4090308@netwok.org>
Message-ID: <CADiSq7e1VqORXrT8NvctBtfJ5RuQjEQv2+b07LGejD+xu041Xw@mail.gmail.com>

On Wed, Jul 27, 2011 at 12:10 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Le 26/07/2011 15:30, Antoine Pitrou a ?crit :
>> Actually, you want %a for non-ASCII messages to be escaped.
>
> Thanks for the reminder, I should use more %a instead of %r. ?In the
> packaging code however, we can?t, given that we want to backport.
>
>> (however, there's hardly any reason to worry about it when it comes to
>> stdlib module names)
>
> I lacked context to see that. ?If the code in question only ever handles
> stdlib modules, then okay.

The other reason to use %r is to get the enclosing quotes in the
displayed message. That reason applies even in cases like this where
the escaping aspect isn't a concern.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From brett at python.org  Wed Jul 27 04:04:20 2011
From: brett at python.org (Brett Cannon)
Date: Tue, 26 Jul 2011 19:04:20 -0700
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <CADiSq7e1VqORXrT8NvctBtfJ5RuQjEQv2+b07LGejD+xu041Xw@mail.gmail.com>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org>
	<20110726153025.66b0de8b@pitrou.net> <4E2ECAEA.4090308@netwok.org>
	<CADiSq7e1VqORXrT8NvctBtfJ5RuQjEQv2+b07LGejD+xu041Xw@mail.gmail.com>
Message-ID: <CAP1=2W5DsTu+kDL=GGqRZ5OzPP2_hG3qxg89Dzz0pJ5Y9P-XPA@mail.gmail.com>

On Tue, Jul 26, 2011 at 17:41, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Wed, Jul 27, 2011 at 12:10 AM, ?ric Araujo <merwok at netwok.org> wrote:
> > Le 26/07/2011 15:30, Antoine Pitrou a ?crit :
> >> Actually, you want %a for non-ASCII messages to be escaped.
> >
> > Thanks for the reminder, I should use more %a instead of %r.  In the
> > packaging code however, we can?t, given that we want to backport.
> >
> >> (however, there's hardly any reason to worry about it when it comes to
> >> stdlib module names)
> >
> > I lacked context to see that.  If the code in question only ever handles
> > stdlib modules, then okay.
>
> The other reason to use %r is to get the enclosing quotes in the
> displayed message. That reason applies even in cases like this where
> the escaping aspect isn't a concern.
>

And then you make it {!r} so you can use str.format and you complete the
tweak of the string formatting! =) Seriously, though, it wouldn't hurt to
update it to use str.format().
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110726/9703a60e/attachment.html>

From neologix at free.fr  Wed Jul 27 09:11:35 2011
From: neologix at free.fr (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Wed, 27 Jul 2011 09:11:35 +0200
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <20110727014518.6515fd5f@pitrou.net>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110727004955.1642e815@pitrou.net>
	<51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com>
	<20110727014518.6515fd5f@pitrou.net>
Message-ID: <CAH_1eM31ojS_gtuhT4RoMQXeOnHb05h8LgsqUEJuUaRRAftBVw@mail.gmail.com>

>> I assume that ESHUTDOWN is the errno in question? ?(This is also already mentioned in the PEP.)
>
> Indeed, I mentioned it in the PEP, as it appears in asyncore.py.
> But I can't find it on www.opengroup.org, and no man page on my Linux
> system (except the "errno" man page) seems to mention it.

It's not POSIX, but it's defined on Linux and FreeBSD (at least):
http://lxr.free-electrons.com/source/include/asm-generic/errno.h#L81
http://fxr.watson.org/fxr/source/sys/errno.h?v=FREEBSD53#L122

> The description from errnomodule.c says "Cannot send after transport
> endpoint shutdown", but send() actually returns EPIPE, not ESHUTDOWN,
> when the socket has been shutdown:

Indeed, as required by POSIX.

But grepping through the Linux kernel source code, it seems to be used
extensively for USB devices, see
http://lxr.free-electrons.com/ident?i=ESHUTDOWN
So the "transport endpoint" doesn't necessarily refer to a socket.
It's also documented in
http://lxr.free-electrons.com/source/Documentation/usb/error-codes.txt

Finally, I found one place in the networking stack where ESHUTDOWN is
used, in the SCTP code:
http://lxr.free-electrons.com/source/net/sctp/outqueue.c#L329

From ncoghlan at gmail.com  Wed Jul 27 09:53:30 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 27 Jul 2011 17:53:30 +1000
Subject: [Python-Dev] Comments of the PEP 3151
In-Reply-To: <CAH_1eM31ojS_gtuhT4RoMQXeOnHb05h8LgsqUEJuUaRRAftBVw@mail.gmail.com>
References: <4E2CB130.7060701@haypocalc.com>
	<CADiSq7ci1947Sn5QicXz1JNuY27kL3JEemhTttqKy-DhrSZv2Q@mail.gmail.com>
	<20110727004955.1642e815@pitrou.net>
	<51E7C85E-8AE2-4E7B-987C-EB6CF5BFDCDB@twistedmatrix.com>
	<20110727014518.6515fd5f@pitrou.net>
	<CAH_1eM31ojS_gtuhT4RoMQXeOnHb05h8LgsqUEJuUaRRAftBVw@mail.gmail.com>
Message-ID: <CADiSq7cpcDHPZK7NKr4XJe9AQUn0cJ0QFpYezBEL9+Y+q1rwPw@mail.gmail.com>

2011/7/27 Charles-Fran?ois Natali <neologix at free.fr>:
>>> I assume that ESHUTDOWN is the errno in question? ?(This is also already mentioned in the PEP.)
>>
>> Indeed, I mentioned it in the PEP, as it appears in asyncore.py.
>> But I can't find it on www.opengroup.org, and no man page on my Linux
>> system (except the "errno" man page) seems to mention it.
>
> It's not POSIX, but it's defined on Linux and FreeBSD (at least):
> http://lxr.free-electrons.com/source/include/asm-generic/errno.h#L81
> http://fxr.watson.org/fxr/source/sys/errno.h?v=FREEBSD53#L122
>
>> The description from errnomodule.c says "Cannot send after transport
>> endpoint shutdown", but send() actually returns EPIPE, not ESHUTDOWN,
>> when the socket has been shutdown:
>
> Indeed, as required by POSIX.
>
> But grepping through the Linux kernel source code, it seems to be used
> extensively for USB devices, see
> http://lxr.free-electrons.com/ident?i=ESHUTDOWN
> So the "transport endpoint" doesn't necessarily refer to a socket.
> It's also documented in
> http://lxr.free-electrons.com/source/Documentation/usb/error-codes.txt
>
> Finally, I found one place in the networking stack where ESHUTDOWN is
> used, in the SCTP code:
> http://lxr.free-electrons.com/source/net/sctp/outqueue.c#L329

Perhaps the right thing to do is to just have a ConnectionBrokenError
that covers EPIPE, ESHUTDOWN and ECONNRESET? The current version of
the PEP has these as two separate exception types (BrokenPipeError for
the first two and ConnectionResetError for the last), but I'm not
seeing a strong reason to avoid combining them.

(ECONNABORTED and ECONNREFUSED are different, since they relate to
failures when *initiating* a connection)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From fuzzyman at voidspace.org.uk  Wed Jul 27 11:50:40 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 27 Jul 2011 10:50:40 +0100
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <CAP1=2W5DsTu+kDL=GGqRZ5OzPP2_hG3qxg89Dzz0pJ5Y9P-XPA@mail.gmail.com>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net>
	<4E2ECAEA.4090308@netwok.org>
	<CADiSq7e1VqORXrT8NvctBtfJ5RuQjEQv2+b07LGejD+xu041Xw@mail.gmail.com>
	<CAP1=2W5DsTu+kDL=GGqRZ5OzPP2_hG3qxg89Dzz0pJ5Y9P-XPA@mail.gmail.com>
Message-ID: <4E2FDF70.3080803@voidspace.org.uk>

On 27/07/2011 03:04, Brett Cannon wrote:
>
>
> On Tue, Jul 26, 2011 at 17:41, Nick Coghlan <ncoghlan at gmail.com 
> <mailto:ncoghlan at gmail.com>> wrote:
>
>     On Wed, Jul 27, 2011 at 12:10 AM, ?ric Araujo <merwok at netwok.org
>     <mailto:merwok at netwok.org>> wrote:
>     > Le 26/07/2011 15:30, Antoine Pitrou a ?crit :
>     >> Actually, you want %a for non-ASCII messages to be escaped.
>     >
>     > Thanks for the reminder, I should use more %a instead of %r.  In the
>     > packaging code however, we can't, given that we want to backport.
>     >
>     >> (however, there's hardly any reason to worry about it when it
>     comes to
>     >> stdlib module names)
>     >
>     > I lacked context to see that.  If the code in question only ever
>     handles
>     > stdlib modules, then okay.
>
>     The other reason to use %r is to get the enclosing quotes in the
>     displayed message. That reason applies even in cases like this where
>     the escaping aspect isn't a concern.
>
>
> And then you make it {!r} so you can use str.format and you complete 
> the tweak of the string formatting! =) Seriously, though, it wouldn't 
> hurt to update it to use str.format().
>

Well, except that {!r} is twice as verbose as %r....

Michael

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/632dbbfd/attachment.html>

From solipsis at pitrou.net  Wed Jul 27 11:52:32 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 11:52:32 +0200
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen
 behavior on sites which do not send (or
References: <E1Qlrwi-0007t4-Js@dinsdale.python.org>
Message-ID: <20110727115232.5bbf7aea@pitrou.net>

On Wed, 27 Jul 2011 02:25:56 +0200
senthil.kumaran <python-checkins at python.org> wrote:
>  
> +    def test_sites_no_connection_close(self):
> +        # Some sites do not send Connection: close header.
> +        # Verify that those work properly. (#issue12576)
> +
> +        try:
> +            with urllib.request.urlopen('http://www.imdb.com') as res:
> +                pass

Can you please at least use support.transient_internet() as in other
tests in this file?

> +        except ValueError as e:
> +            self.fail("urlopen failed for sites not sending Connection:close")
> +        else:
> +            self.assertTrue(res)
> +
> +        req = urllib.request.urlopen('http://www.imdb.com')

Why a second time?

> +        res = req.read()
> +        self.assertTrue(res)

Also, when does "req" get closed? Right now I get resource warnings:

test_sites_no_connection_close (test.test_urllib2net.OtherNetworkTests) ... /home/antoine/cpython/default/Lib/socket.py:342: ResourceWarning: unclosed <socket.socket object, fd=3, family=2, type=1, proto=6>
  self._sock = None
/home/antoine/cpython/default/Lib/socket.py:342: ResourceWarning: unclosed <socket.socket object, fd=3, family=2, type=1, proto=6>
  self._sock = None
ok


Regards

Antoine.



From senthil at uthcode.com  Wed Jul 27 14:16:01 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Wed, 27 Jul 2011 20:16:01 +0800
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen
 behavior on sites which do not send (or
In-Reply-To: <20110727115232.5bbf7aea@pitrou.net>
References: <E1Qlrwi-0007t4-Js@dinsdale.python.org>
	<20110727115232.5bbf7aea@pitrou.net>
Message-ID: <20110727121600.GA3140@mathmagic>

On Wed, Jul 27, 2011 at 11:52:32AM +0200, Antoine Pitrou wrote:
> > +
> > +        try:
> > +            with urllib.request.urlopen('http://www.imdb.com') as res:
> > +                pass
> 
> Can you please at least use support.transient_internet() as in other
> tests in this file?

It was intentional because ValueError was raised from context manager
use case for a bug where the request object was closed prematurely.
support.transient_internet, I believe would not have covered that
case (Usage scenario).

> > +        req = urllib.request.urlopen('http://www.imdb.com')
> 
> Why a second time?

When used outside of context manager, it gave empty response instead
of ValueError and the test case was to check that.

> > +        res = req.read()
> > +        self.assertTrue(res)
> 
> Also, when does "req" get closed? Right now I get resource warnings:

I shall fix this one. I think, attempting to fix the Resource warning
caused the regression wherein the request object closed prematurely. I
shall look at this.

Thanks,
Senthil

From solipsis at pitrou.net  Wed Jul 27 14:21:59 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 14:21:59 +0200
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen
 behavior on sites which do not send (or
In-Reply-To: <20110727121600.GA3140@mathmagic>
References: <E1Qlrwi-0007t4-Js@dinsdale.python.org>
	<20110727115232.5bbf7aea@pitrou.net>
	<20110727121600.GA3140@mathmagic>
Message-ID: <20110727142159.65e25267@pitrou.net>

On Wed, 27 Jul 2011 20:16:01 +0800
Senthil Kumaran <senthil at uthcode.com> wrote:
> On Wed, Jul 27, 2011 at 11:52:32AM +0200, Antoine Pitrou wrote:
> > > +
> > > +        try:
> > > +            with urllib.request.urlopen('http://www.imdb.com') as res:
> > > +                pass
> > 
> > Can you please at least use support.transient_internet() as in other
> > tests in this file?
> 
> It was intentional because ValueError was raised from context manager
> use case for a bug where the request object was closed prematurely.
> support.transient_internet, I believe would not have covered that
> case (Usage scenario).

Unless I'm reading wrongly, transient_internet doesn't silence
ValueError at all.

> > > +        res = req.read()
> > > +        self.assertTrue(res)
> > 
> > Also, when does "req" get closed? Right now I get resource warnings:
> 
> I shall fix this one. I think, attempting to fix the Resource warning
> caused the regression wherein the request object closed prematurely. I
> shall look at this.

Well, the test should simply call close() as is done in other tests.

Regards

Antoine.

From eliben at gmail.com  Wed Jul 27 15:14:40 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 16:14:40 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
Message-ID: <CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>

>
>
>
> The mere fact that these functions exist in a different module suggests
> different semantics from those found in other places in the stdlib. I don't
> think they should be renamed simply because some code imports the functions
> directly instead of the module itself (suggesting they should be doing the
> latter over the former). Now if the functions are redundant that's something
> else entirely and removal should be fine.
>

I will try to clarify my concern.

Documented functions from test.support currently appear in the global index
of the Sphinx-generated documentation. Suppose we document test.support's
"unlink". Now the index entry for "unlink" will contain two "unlink"
references (*), with slightly different semantics - one in os and another in
test.support

Will it take long for newbie code to appear with the test.support version?
Not to mention that grepping code that imports the "unlink" function
directly doesn't reveal which one is being used.

I think this is troublesome. I think at least two separate actions are
required here:

1. In the documentation of test.support mention explicitly that it's code
for CPython's internal use only, and these APIs aren't guaranteed to be
stable.
2. Some functions like unlink and rmtree are obviously redundant, and shadow
frequently used Python stdlib functions, so I would either kill them
completely or at least rename them appropriately.

Eli


(*) Actually, three, since there's also xml.dom.minidom.Node, but that's
obviously unrelated).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/2264777a/attachment.html>

From solipsis at pitrou.net  Wed Jul 27 15:24:10 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 15:24:10 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
Message-ID: <20110727152410.22871784@pitrou.net>

On Wed, 27 Jul 2011 16:14:40 +0300
Eli Bendersky <eliben at gmail.com> wrote:
> 
> Will it take long for newbie code to appear with the test.support version?
> Not to mention that grepping code that imports the "unlink" function
> directly doesn't reveal which one is being used.
> 
> I think this is troublesome. I think at least two separate actions are
> required here:
> 
> 1. In the documentation of test.support mention explicitly that it's code
> for CPython's internal use only, and these APIs aren't guaranteed to be
> stable.

There is a top-level note at
http://docs.python.org/dev/library/test.html, but it won't be visible
by people who arrive at an anchor point.

I think officially documenting test.support is a mistake. There is no
guarantee that APIs are stable or will even continue to exist.
Docstrings are sufficient for own our purposes.

> 2. Some functions like unlink and rmtree are obviously redundant, and shadow
> frequently used Python stdlib functions, so I would either kill them
> completely or at least rename them appropriately.

They are not redundant, since they provide slightly different semantics
(for example, they silence errors that happen when the path doesn't
exist).

Regards

Antoine.



From eliben at gmail.com  Wed Jul 27 15:31:54 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 16:31:54 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <20110726153025.66b0de8b@pitrou.net>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net>
Message-ID: <CAF-Rda9xmZ7kOmGAg7_N9+PfALsvyE==Jq9b92MnY3qSk7v-zA@mail.gmail.com>

> > I would use %r instead of %s for both fields here.  Non-ASCII characters
> > and unseen whitespace are at least two reasons to overuse %r in
> > debug/error messages instead of %s.
>
> Actually, you want %a for non-ASCII messages to be escaped.
> (however, there's hardly any reason to worry about it when it comes to
> stdlib module names)
>

I wasn't aware of '%a' at all? It doesn't appear to be documented, and
Python 2.6 doesn't support it:

    ValueError: unsupported format character 'a' (0x61) at index 1

If it's new, it should at least be documented in library/stdtypes with the
other formatting operations.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/aa8f9fe1/attachment.html>

From eliben at gmail.com  Wed Jul 27 15:35:24 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 16:35:24 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110727152410.22871784@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
Message-ID: <CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>

> > 1. In the documentation of test.support mention explicitly that it's code
> > for CPython's internal use only, and these APIs aren't guaranteed to be
> > stable.
>
> There is a top-level note at
> http://docs.python.org/dev/library/test.html, but it won't be visible
> by people who arrive at an anchor point.
>
> I think officially documenting test.support is a mistake. There is no
> guarantee that APIs are stable or will even continue to exist.
> Docstrings are sufficient for own our purposes.
>

Initially I was *for* documenting, but this thing with showing up in the
index is a compelling counter-point.

> 2. Some functions like unlink and rmtree are obviously redundant, and
shadow
> frequently used Python stdlib functions, so I would either kill them
> completely or at least rename them appropriately.

They are not redundant, since they provide slightly different semantics
> (for example, they silence errors that happen when the path doesn't
> exist).
>

Sure, but I'm still leery of two functions with the same name doing acting
slightly differently.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/83af9a19/attachment.html>

From rdmurray at bitdance.com  Wed Jul 27 15:36:19 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 27 Jul 2011 09:36:19 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
Message-ID: <20110727133619.F0B622506ED@webabinitio.net>

On Wed, 27 Jul 2011 16:14:40 +0300, Eli Bendersky <eliben at gmail.com> wrote:
> 1. In the documentation of test.support mention explicitly that it's code
> for CPython's internal use only, and these APIs aren't guaranteed to be
> stable.

This was already done.

> 2. Some functions like unlink and rmtree are obviously redundant, and shadow
> frequently used Python stdlib functions, so I would either kill them
> completely or at least rename them appropriately.

But they aren't redundant, since the test.support versions ignore
errors.

Perhaps what we could do is move the documentation for test.support to
the devguide, and then vet the test suite so that unlink and friends
are always called as 'support.unlink', etc.

--
R. David Murray           http://www.bitdance.com

From solipsis at pitrou.net  Wed Jul 27 15:44:07 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 15:44:07 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <CAF-Rda9xmZ7kOmGAg7_N9+PfALsvyE==Jq9b92MnY3qSk7v-zA@mail.gmail.com>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net>
	<CAF-Rda9xmZ7kOmGAg7_N9+PfALsvyE==Jq9b92MnY3qSk7v-zA@mail.gmail.com>
Message-ID: <20110727154407.4f3b2a69@pitrou.net>

On Wed, 27 Jul 2011 16:31:54 +0300
Eli Bendersky <eliben at gmail.com> wrote:
> 
> I wasn't aware of '%a' at all? It doesn't appear to be documented, and
> Python 2.6 doesn't support it:
> 
>     ValueError: unsupported format character 'a' (0x61) at index 1
> 
> If it's new, it should at least be documented in library/stdtypes with the
> other formatting operations.

It's new in 3.x and maps to the ascii() builtin:

>>> ascii('h?')
"'h\\xe9'"
>>> '%a' % 'h?'
"'h\\xe9'"

The docstring for ascii():

    ascii(object) -> string
    
    As repr(), return a string containing a printable representation of
    an object, but escape the non-ASCII characters in the string
    returned by repr() using \x, \u or \U escapes.  This generates a
    string similar to that returned by repr() in Python 2.

Regards

Antoine.

From eliben at gmail.com  Wed Jul 27 15:54:34 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 16:54:34 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <20110727154407.4f3b2a69@pitrou.net>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net>
	<CAF-Rda9xmZ7kOmGAg7_N9+PfALsvyE==Jq9b92MnY3qSk7v-zA@mail.gmail.com>
	<20110727154407.4f3b2a69@pitrou.net>
Message-ID: <CAF-Rda-UEUO20+9SgQ1B5MPQZ_MXQU4wn6qcDxgKhDXbx4RLhA@mail.gmail.com>

On Wed, Jul 27, 2011 at 16:44, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Wed, 27 Jul 2011 16:31:54 +0300
> Eli Bendersky <eliben at gmail.com> wrote:
> >
> > I wasn't aware of '%a' at all? It doesn't appear to be documented, and
> > Python 2.6 doesn't support it:
> >
> >     ValueError: unsupported format character 'a' (0x61) at index 1
> >
> > If it's new, it should at least be documented in library/stdtypes with
> the
> > other formatting operations.
>
> It's new in 3.x and maps to the ascii() builtin:
>
> >>> ascii('h?')
> "'h\\xe9'"
> >>> '%a' % 'h?'
> "'h\\xe9'"
>
> The docstring for ascii():
>
>    ascii(object) -> string
>
>    As repr(), return a string containing a printable representation of
>    an object, but escape the non-ASCII characters in the string
>    returned by repr() using \x, \u or \U escapes.  This generates a
>    string similar to that returned by repr() in Python 2.
>
>
Thanks. I also saw this documented in the {!a} formatting documentation.

Was it left out of the library/stdtypes doc on purpose (to encourage the new
str.format), or is this an omission?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/5e694aed/attachment.html>

From eliben at gmail.com  Wed Jul 27 15:58:53 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 16:58:53 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110727133619.F0B622506ED@webabinitio.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
Message-ID: <CAF-Rda-1sRhSVZgLPpO8pVciYadu_5S9J40CX4v0FC-o+FdAYQ@mail.gmail.com>

> > 2. Some functions like unlink and rmtree are obviously redundant, and
> shadow
> > frequently used Python stdlib functions, so I would either kill them
> > completely or at least rename them appropriately.
>
> But they aren't redundant, since the test.support versions ignore
> errors.
>

As I mentioned elsewhere, it's not good practice to have two functions with
the same name doing something slightly different, in different modules in
the code-base.


>
> Perhaps what we could do is move the documentation for test.support to
> the devguide, and then vet the test suite so that unlink and friends
> are always called as 'support.unlink', etc.
>
>
Moving the documentation to the devguide is a good compromise between not
documenting them at all and placing the documentation in a user-visible
location.

What do you mean by vetting the test suite so that unlink is always taken
from test.support? I suppose some tests would specifically want the original
unlink's functionality. In fact, at least a few tests use os.unlink
exlicitly.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/342f28a4/attachment.html>

From solipsis at pitrou.net  Wed Jul 27 15:59:11 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 15:59:11 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <CAF-Rda-UEUO20+9SgQ1B5MPQZ_MXQU4wn6qcDxgKhDXbx4RLhA@mail.gmail.com>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net>
	<CAF-Rda9xmZ7kOmGAg7_N9+PfALsvyE==Jq9b92MnY3qSk7v-zA@mail.gmail.com>
	<20110727154407.4f3b2a69@pitrou.net>
	<CAF-Rda-UEUO20+9SgQ1B5MPQZ_MXQU4wn6qcDxgKhDXbx4RLhA@mail.gmail.com>
Message-ID: <1311775151.3583.0.camel@localhost.localdomain>

Le mercredi 27 juillet 2011 ? 16:54 +0300, Eli Bendersky a ?crit :
> 
> Thanks. I also saw this documented in the {!a} formatting
> documentation.
> 
> Was it left out of the library/stdtypes doc on purpose (to encourage
> the new str.format), or is this an omission?

Certainly an omission.

Regards

Antoine.




From ezio.melotti at gmail.com  Wed Jul 27 16:09:58 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 27 Jul 2011 17:09:58 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
 stdlib functions
In-Reply-To: <CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
Message-ID: <4E301C36.1040103@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/5640c25c/attachment.html>

From eliben at gmail.com  Wed Jul 27 16:27:03 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 17:27:03 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <4E301C36.1040103@gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E301C36.1040103@gmail.com>
Message-ID: <CAF-Rda8BGX8EHD7LA03QzGg9MGHC8R8ae2Y1QtMBVQ_eqF_k2Q@mail.gmail.com>

>  Initially I was *for* documenting, but this thing with showing up in the
> index is a compelling counter-point.
>
>
> "The basic version makes entries in the general index; if no index entry is
> desired, you can give the directive option flag :noindex:." (
> http://docs.python.org/documenting/markup.html#information-units)
>

Ezio, this is also a good idea, but currently I really think placing this
documentation in the devguide is probably the best approach. Now we have a
very nice Devguide, and this documentation simply belongs there, and not in
the user-visible portion of the official Python documentation.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/cfe1c533/attachment.html>

From eliben at gmail.com  Wed Jul 27 16:29:45 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 17:29:45 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding
 some tests to test.support
In-Reply-To: <1311775151.3583.0.camel@localhost.localdomain>
References: <E1QkV7i-0002pL-Av@dinsdale.python.org>
	<4E2EBF37.7020702@netwok.org> <20110726153025.66b0de8b@pitrou.net>
	<CAF-Rda9xmZ7kOmGAg7_N9+PfALsvyE==Jq9b92MnY3qSk7v-zA@mail.gmail.com>
	<20110727154407.4f3b2a69@pitrou.net>
	<CAF-Rda-UEUO20+9SgQ1B5MPQZ_MXQU4wn6qcDxgKhDXbx4RLhA@mail.gmail.com>
	<1311775151.3583.0.camel@localhost.localdomain>
Message-ID: <CAF-Rda_05G2KVe=tEffQC3UBJRfcw+fWOAh-7AxW5_Kb_s1Yqg@mail.gmail.com>

> Thanks. I also saw this documented in the {!a} formatting

> > documentation.
> >
> > Was it left out of the library/stdtypes doc on purpose (to encourage
> > the new str.format), or is this an omission?
>
> Certainly an omission.
>
>
Alright, I created issue 12644 as a reminder to add this.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/c773b9b4/attachment.html>

From rdmurray at bitdance.com  Wed Jul 27 17:11:10 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 27 Jul 2011 11:11:10 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda-1sRhSVZgLPpO8pVciYadu_5S9J40CX4v0FC-o+FdAYQ@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAF-Rda-1sRhSVZgLPpO8pVciYadu_5S9J40CX4v0FC-o+FdAYQ@mail.gmail.com>
Message-ID: <20110727151111.4F3C82506ED@webabinitio.net>

On Wed, 27 Jul 2011 16:58:53 +0300, Eli Bendersky <eliben at gmail.com> wrote:
> R. David Murray wrote:
> > But they aren't redundant, since the test.support versions ignore
> > errors.
> 
> As I mentioned elsewhere, it's not good practice to have two functions with
> the same name doing something slightly different, in different modules in
> the code-base.

Well, that would seem to be a matter of opinion.  I see your point, but
I'm not sure that I agree.  But see below.

> What do you mean by vetting the test suite so that unlink is always taken
> from test.support? I suppose some tests would specifically want the original
> unlink's functionality. In fact, at least a few tests use os.unlink
> exlicitly.

What I mean is that if the test code always did:

    import support

    [...]

    support.unlink('testtempfile')

then there would be no confusion when someone grepped the code for
'unlink' or was reading the code without having noticed the import.
That is, give the functions a unique name by using the 'support'
name space explicitly, rather than by renaming them within the
module.

--
R. David Murray           http://www.bitdance.com

From senthil at uthcode.com  Wed Jul 27 17:18:45 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Wed, 27 Jul 2011 23:18:45 +0800
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen
 behavior on sites which do not send (or
In-Reply-To: <20110727142159.65e25267@pitrou.net>
References: <E1Qlrwi-0007t4-Js@dinsdale.python.org>
	<20110727115232.5bbf7aea@pitrou.net>
	<20110727121600.GA3140@mathmagic>
	<20110727142159.65e25267@pitrou.net>
Message-ID: <20110727151845.GA2206@mathmagic>

On Wed, Jul 27, 2011 at 02:21:59PM +0200, Antoine Pitrou wrote:
> transient_internet doesn't silence ValueError at all.

Yes, that is correct. I missed recollecting it in the first place. I
guess, I did not see using a content manager withing another context
manager block as something nice.. (nothing wrong but just the indented
code with additional wrap of exceptional handler).

But yes, it would be better to wrap it with transient_internet call.
Shall do.

> Well, the test should simply call close() as is done in other tests.

I tried that before responding, it does not silence the Resource
Warning. The fix (at least for these cases where Connection:close
header is not sent) lies in closing request in the urllib.request. It
is tricky as the server is not sending a Connection:close header which
httplib relies on to close the socket.

-- 
Senthil

From tjreedy at udel.edu  Wed Jul 27 19:07:24 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 27 Jul 2011 13:07:24 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110727152410.22871784@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
Message-ID: <j0pgkd$pri$1@dough.gmane.org>

On 7/27/2011 9:24 AM, Antoine Pitrou wrote:
> On Wed, 27 Jul 2011 16:14:40 +0300
> Eli Bendersky<eliben at gmail.com>  wrote:

>> Will it take long for newbie code to appear with the test.support version?
>> Not to mention that grepping code that imports the "unlink" function
>> directly doesn't reveal which one is being used.
>>
>> I think this is troublesome. I think at least two separate actions are
>> required here:
>>
>> 1. In the documentation of test.support mention explicitly that it's code
>> for CPython's internal use only, and these APIs aren't guaranteed to be
>> stable.
>
> There is a top-level note at
> http://docs.python.org/dev/library/test.html, but it won't be visible
> by people who arrive at an anchor point.

I think the 'Note' (gray box), not a 'Warning' (red box) should be 
repeated at the top of the test.support section. Or, or in addition, 
footnote each entry (with same number jumping to same footnote, if this 
can be done): "test.support.verbose(1). Seeing every entry decorated 
with the same footnote number should indicate there there is something 
unusual, and that we really mean it.

> I think officially documenting test.support is a mistake. There is no
> guarantee that APIs are stable or will even continue to exist.
> Docstrings are sufficient for own our purposes.

Maybe sufficient for you, but if I am to learn and use this stuff, I 
need a proper listing. Individual docstrings require that you know the 
object exists and its name. Any help(module) listing is harder for me to 
use than the doc chapter.

>> 2. Some functions like unlink and rmtree are obviously redundant, and shadow
>> frequently used Python stdlib functions, so I would either kill them
>> completely or at least rename them appropriately.
>
> They are not redundant, since they provide slightly different semantics
> (for example, they silence errors that happen when the path doesn't
> exist).

rmtree has an ignore_errors=False parameter "ignore_errors is true, 
errors resulting from failed removals will be ignored; ". If that does 
not ignore enough errors, then perhaps it should be changed.

os.unlink is an alias for os.remove. The doc for the latter says error 
if path is a directory or a file in use on Windows (but not *nix). 
Neither should be case in test uses. Doc does not specify errors if file 
not exist or other conditions (inadequate permission.

Being thin wrappers, a quiet param is not allowed in os.remove/unlink, 
though I can imagine others wanting a quieted version for removal of 
files whose creation might have failed. In anycase, naming the quieted 
version in test.support unlink_quiet or q_remove or something would make 
its reason-for-being and use clearer.

---
Side note: test.support.import_fresh_module typo. /is/if/ in
"This function will raise unittest.SkipTest is the named module cannot 
be imported."

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Wed Jul 27 19:17:59 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 27 Jul 2011 13:17:59 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda8BGX8EHD7LA03QzGg9MGHC8R8ae2Y1QtMBVQ_eqF_k2Q@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E301C36.1040103@gmail.com>
	<CAF-Rda8BGX8EHD7LA03QzGg9MGHC8R8ae2Y1QtMBVQ_eqF_k2Q@mail.gmail.com>
Message-ID: <j0ph88$tuu$1@dough.gmane.org>

On 7/27/2011 10:27 AM, Eli Bendersky wrote:
>
>>     Initially I was *for* documenting, but this thing with showing up
>>     in the index is a compelling counter-point.
>
>     "The basic version makes entries in the general index; if no index
>     entry is desired, you can give the directive option flag :noindex:."
>     (http://docs.python.org/documenting/markup.html#information-units)
>
>
> Ezio, this is also a good idea, but currently I really think placing
> this documentation in the devguide is probably the best approach. Now we
> have a very nice Devguide, and this documentation simply belongs there,
> and not in the user-visible portion of the official Python documentation.

You mean the dev guide only accessible online? It could be included in 
HowTos (Help develop Python). If you are going to un-index the section, 
whereever it is, please put the entries in alpha order instead of the 
current jumble. Or is the current order somehow makes sense, an 
alphabetical index at the top.


-- 
Terry Jan Reedy


From brett at python.org  Wed Jul 27 19:27:16 2011
From: brett at python.org (Brett Cannon)
Date: Wed, 27 Jul 2011 10:27:16 -0700
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110727133619.F0B622506ED@webabinitio.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
Message-ID: <CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>

On Wed, Jul 27, 2011 at 06:36, R. David Murray <rdmurray at bitdance.com>wrote:

> On Wed, 27 Jul 2011 16:14:40 +0300, Eli Bendersky <eliben at gmail.com>
> wrote:
> > 1. In the documentation of test.support mention explicitly that it's code
> > for CPython's internal use only, and these APIs aren't guaranteed to be
> > stable.
>
> This was already done.
>
> > 2. Some functions like unlink and rmtree are obviously redundant, and
> shadow
> > frequently used Python stdlib functions, so I would either kill them
> > completely or at least rename them appropriately.
>
> But they aren't redundant, since the test.support versions ignore
> errors.
>
> Perhaps what we could do is move the documentation for test.support to
> the devguide, and then vet the test suite so that unlink and friends
> are always called as 'support.unlink', etc.
>

I like this solution since this issue of documenting test.support keeps
coming up. Otherwise we can not document test.support, but  then we need to
do a pass through the module to make sure  that the docstrings are properly
updated and we start deprecating some of the stuff in there that is just
pure cruft.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/6ed45cdd/attachment.html>

From eliben at gmail.com  Wed Jul 27 19:27:52 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 20:27:52 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <j0ph88$tuu$1@dough.gmane.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E301C36.1040103@gmail.com>
	<CAF-Rda8BGX8EHD7LA03QzGg9MGHC8R8ae2Y1QtMBVQ_eqF_k2Q@mail.gmail.com>
	<j0ph88$tuu$1@dough.gmane.org>
Message-ID: <CAF-Rda9Xr8gmAuzzgiKzCEMELzoviS==i-=m9=Wozi+efR9AEg@mail.gmail.com>

> Ezio, this is also a good idea, but currently I really think placing
>> this documentation in the devguide is probably the best approach. Now we
>> have a very nice Devguide, and this documentation simply belongs there,
>> and not in the user-visible portion of the official Python documentation.
>>
>
> You mean the dev guide only accessible online?


Yes. You can also pull it from http://hg.python.org/devguide/ for a local
copy (hg.python.org also allows to download a ZIP).

My point being - isn't the official Python documentation targeted at *users*
of Python, and wasn't the devguide specifically created for *developers* of
Python? If so, then test.support clearly being the domain of developers
rather than users, belongs in the devguide.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/976522bb/attachment.html>

From eliben at gmail.com  Wed Jul 27 19:31:58 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 20:31:58 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <j0pgkd$pri$1@dough.gmane.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net> <j0pgkd$pri$1@dough.gmane.org>
Message-ID: <CAF-Rda8JLa9EkgY7wGP_G9vYNE78W_PimppYsXzjpnW5dijkzw@mail.gmail.com>

> ---
> Side note: test.support.import_fresh_**module typo. /is/if/ in
> "This function will raise unittest.SkipTest is the named module cannot be
> imported."
>

Fixed in 8989aa5b357c

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/d0f3503c/attachment.html>

From tjreedy at udel.edu  Wed Jul 27 19:44:23 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 27 Jul 2011 13:44:23 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110727152410.22871784@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
Message-ID: <j0pipo$94a$1@dough.gmane.org>

On 7/27/2011 9:24 AM, Antoine Pitrou wrote:

> Docstrings are sufficient for own our purposes.

 >>> import test.support as t
 >>> help(t.rmtree)
Help on function rmtree in module test.support:

rmtree(path)

;-)

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Wed Jul 27 19:47:13 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 27 Jul 2011 13:47:13 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
Message-ID: <j0piv1$94a$2@dough.gmane.org>

On 7/27/2011 1:27 PM, Brett Cannon wrote:

>     Perhaps what we could do is move the documentation for test.support to
>     the devguide, and then vet the test suite so that unlink and friends
>     are always called as 'support.unlink', etc.
>
>
> I like this solution since this issue of documenting test.support keeps
> coming up. Otherwise we can not document test.support,

We already do.

25.6. test.support ? Utility functions for tests
is about half of the page that also contains
25.5. test ? Regression tests package for Python
The latter contains
25.5.1. Writing Unit Tests for the test package
which should also be moved to the dev guide if 25.6 is.

That would leave 25.5 as a short page explaining what lib/test is and 
how to run the tests, which is something user sometimes need to do.


> but  then we need
> to do a pass through the module to make sure  that the docstrings are
> properly updated and we start deprecating some of the stuff in there
> that is just  pure cruft.

I believe that is what Eli is doing and hence the suggestion to dump 
.rmtree. Agreed that missing doc strings should be 'updated' from ''.


-- 
Terry Jan Reedy



From eliben at gmail.com  Wed Jul 27 19:57:48 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 20:57:48 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <j0piv1$94a$2@dough.gmane.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
	<j0piv1$94a$2@dough.gmane.org>
Message-ID: <CAF-Rda-SmWi-sY99KH35+Soa8nUBuYVJiFPOOOUY20_=jWVaaA@mail.gmail.com>

> I like this solution since this issue of documenting test.support keeps
>> coming up. Otherwise we can not document test.support,
>>
>
> We already do.
>
> 25.6. test.support ? Utility functions for tests
> is about half of the page that also contains
> 25.5. test ? Regression tests package for Python
> The latter contains
> 25.5.1. Writing Unit Tests for the test package
> which should also be moved to the dev guide if 25.6 is.
>
>
+1


> That would leave 25.5 as a short page explaining what lib/test is and how
> to run the tests, which is something user sometimes need to do.
>
>
Out of curiosity, why would a user need to run Python's tests?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/7ca8dce5/attachment.html>

From ezio.melotti at gmail.com  Wed Jul 27 19:46:36 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 27 Jul 2011 20:46:36 +0300
Subject: [Python-Dev] [Python-checkins] cpython: fix doc typo for
	library/test.rst
In-Reply-To: <E1Qm7xD-00020C-3P@dinsdale.python.org>
References: <E1Qm7xD-00020C-3P@dinsdale.python.org>
Message-ID: <4E304EFC.6000309@gmail.com>

Hi,

On 27/07/2011 20.31, eli.bendersky wrote:
> http://hg.python.org/cpython/rev/8989aa5b357c
> changeset:   71532:8989aa5b357c
> user:        Eli Bendersky<eliben at gmail.com>
> date:        Wed Jul 27 20:29:59 2011 +0300
> summary:
>    fix doc typo for library/test.rst
>
> files:
>    Doc/library/test.rst |  2 +-
>    1 files changed, 1 insertions(+), 1 deletions(-)
>
>
> diff --git a/Doc/library/test.rst b/Doc/library/test.rst
> --- a/Doc/library/test.rst
> +++ b/Doc/library/test.rst
> @@ -447,7 +447,7 @@
>      Module and package deprecation messages are suppressed during this import
>      if *deprecated* is ``True``.
>
> -   This function will raise :exc:`unittest.SkipTest` is the named module
> +   This function will raise :exc:`unittest.SkipTest` if the named module

Actually I think this is no longer true.  import_fresh_module raises an 
ImportError if *name* can't be imported, or returns None if the fresh 
module is not found.

Its use case is to enable or block accelerations for modules that 
optionally provide one.  All the modules that currently use 
import_fresh_module are (afaik) always available (json, warnings, 
heapq), so raising SkipTest when the module is missing is not useful now.
It returns None in the case an acceleration is missing, so e.g. in 
"cjson = import_fresh_module('json', fresh=['_json'])" cjson will be 
None and it will be possible to do things like @skipUnless(cjson, 
'requires _json').  Here raising an ImportError will defeat (part of) 
the purpose of the function, i.e. avoiding:
try:
   import _json
except ImportError:
   _json = None

and raising a SkipTest when the accelerations are missing is not an 
option if there are other tests (e.g. the tests for the Python 
implementation).

These changes come from http://hg.python.org/cpython/rev/c1a12a308c5b .  
Before the change import_fresh_module was still returning the module 
(e.g. json) even when the acceleration (fresh=['_json']) was missing, 
and the C tests were run twice using the same pure-python module used 
for the Py ones.

The typo and the wrong doc is also on 2.7.

Best Regards,
Ezio Melotti

>      cannot be imported.
>
>      Example use::
>


From eliben at gmail.com  Wed Jul 27 20:04:10 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 27 Jul 2011 21:04:10 +0300
Subject: [Python-Dev] [Python-checkins] cpython: fix doc typo for
	library/test.rst
In-Reply-To: <4E304EFC.6000309@gmail.com>
References: <E1Qm7xD-00020C-3P@dinsdale.python.org>
	<4E304EFC.6000309@gmail.com>
Message-ID: <CAF-Rda86xGOXLEPu+ixjTXEHSr3kxYAjNDBrQhZf+UAj-9gh+g@mail.gmail.com>

Actually I think this is no longer true.  import_fresh_module raises an
ImportError if *name* can't be imported, or returns None if the fresh module
is not found.

>
> Its use case is to enable or block accelerations for modules that
> optionally provide one.  All the modules that currently use
> import_fresh_module are (afaik) always available (json, warnings, heapq), so
> raising SkipTest when the module is missing is not useful now.
> It returns None in the case an acceleration is missing, so e.g. in "cjson =
> import_fresh_module('json', fresh=['_json'])" cjson will be None and it will
> be possible to do things like @skipUnless(cjson, 'requires _json').  Here
> raising an ImportError will defeat (part of) the purpose of the function,
> i.e. avoiding:
> try:
>  import _json
> except ImportError:
>  _json = None
>
> and raising a SkipTest when the accelerations are missing is not an option
> if there are other tests (e.g. the tests for the Python implementation).
>
> These changes come from http://hg.python.org/cpython/**rev/c1a12a308c5b<http://hg.python.org/cpython/rev/c1a12a308c5b>.  Before the change import_fresh_module was still returning the module
> (e.g. json) even when the acceleration (fresh=['_json']) was missing, and
> the C tests were run twice using the same pure-python module used for the Py
> ones.
>
> The typo and the wrong doc is also on 2.7.
>
>
Ezio, thanks. I opened issue 12645 to track this.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/09c84edd/attachment.html>

From tseaver at palladion.com  Wed Jul 27 20:08:30 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 27 Jul 2011 14:08:30 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda-SmWi-sY99KH35+Soa8nUBuYVJiFPOOOUY20_=jWVaaA@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
	<j0piv1$94a$2@dough.gmane.org>
	<CAF-Rda-SmWi-sY99KH35+Soa8nUBuYVJiFPOOOUY20_=jWVaaA@mail.gmail.com>
Message-ID: <j0pk6u$h4i$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/27/2011 01:57 PM, Eli Bendersky wrote:

> Out of curiosity, why would a user need to run Python's tests?

A couple of cases occur to me:

- - To verify that they got a corrrect build with all expected modules
  included.

- - To test the build after updating an underlying system library.



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4wVB4ACgkQ+gerLs4ltQ5NSACfXg7QoZAFNHWehT81oPTUwzoo
+m4AnRMQFcGod1/3XxEk/T6CDVpE7i7c
=/VoI
-----END PGP SIGNATURE-----


From tjreedy at udel.edu  Wed Jul 27 20:12:45 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 27 Jul 2011 14:12:45 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda-SmWi-sY99KH35+Soa8nUBuYVJiFPOOOUY20_=jWVaaA@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
	<j0piv1$94a$2@dough.gmane.org>
	<CAF-Rda-SmWi-sY99KH35+Soa8nUBuYVJiFPOOOUY20_=jWVaaA@mail.gmail.com>
Message-ID: <j0pkeu$k2k$1@dough.gmane.org>

On 7/27/2011 1:57 PM, Eli Bendersky wrote:

> Out of curiosity, why would a user need to run Python's tests?

If one compiles Python, running the tests is essential.
Some people like to run a test suite to verify an installation.
Sometimes people have problems that might arise from an installation 
getting messed up.

Whatever is left of 25.5 in the public doc, I think *all* the info 
should be in the dev guide.

-- 
Terry Jan Reedy


From ethan at stoneleaf.us  Wed Jul 27 20:40:58 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 27 Jul 2011 11:40:58 -0700
Subject: [Python-Dev] Convention on functions that shadow
 existing	stdlib functions
In-Reply-To: <CAF-Rda-SmWi-sY99KH35+Soa8nUBuYVJiFPOOOUY20_=jWVaaA@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>	<20110727133619.F0B622506ED@webabinitio.net>	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>	<j0piv1$94a$2@dough.gmane.org>
	<CAF-Rda-SmWi-sY99KH35+Soa8nUBuYVJiFPOOOUY20_=jWVaaA@mail.gmail.com>
Message-ID: <4E305BBA.5080804@stoneleaf.us>

Eli Bendersky wrote:
> 
>         I like this solution since this issue of documenting
>         test.support keeps
>         coming up. Otherwise we can not document test.support,
> 
> 
>     We already do.
> 
>     25.6. test.support ? Utility functions for tests
>     is about half of the page that also contains
>     25.5. test ? Regression tests package for Python
>     The latter contains
>     25.5.1. Writing Unit Tests for the test package
>     which should also be moved to the dev guide if 25.6 is.
> 
> 
> +1
>  
> 
>     That would leave 25.5 as a short page explaining what lib/test is
>     and how to run the tests, which is something user sometimes need to do.
> 
> 
> Out of curiosity, why would a user need to run Python's tests?


Curiosity.  ;)

~Ethan~

From barry at python.org  Wed Jul 27 00:32:50 2011
From: barry at python.org (Barry Warsaw)
Date: Tue, 26 Jul 2011 18:32:50 -0400
Subject: [Python-Dev] hard linking executables
In-Reply-To: <20110727001912.4cf67481@pitrou.net>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
Message-ID: <20110726183250.03932089@resist.wooz.org>

On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote:

>Ok, apparently the decision to make hard links for executables dates at
>least back to:

That still doesn't explain *why* hardlinks were originally chosen instead of
symlinks.  In the absence of any other compelling argument against it, I think
they should all consistently be symlinks.  I don't see any Ubuntu or Debian
(where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and
I haven't experienced any issues with it personally.

>changeset:   16221:588691f806f4
>branch:      legacy-trunk
>user:        Neil Schemenauer <nascheme at enme.ucalgary.ca>
>date:        Wed Jan 24 17:11:43 2001 +0000
>files:       Makefile.pre.in
>description:
>Flat makefile based on toplevel Makefile.in and makefiles in build
>subdirectories.  Those other makefiles will go away eventually.
>
>[...]
>
>+# Install the interpreter (by creating a hard link to python$(VERSION))
>+bininstall:    altbininstall
>+       -if test -f $(BINDIR)/$(PYTHON); \
>+       then rm -f $(BINDIR)/$(PYTHON); \
>+       else true; \
>+       fi
>+       (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT))
>+

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110726/dc922eb4/attachment.pgp>

From barry at python.org  Wed Jul 27 00:32:50 2011
From: barry at python.org (Barry Warsaw)
Date: Tue, 26 Jul 2011 18:32:50 -0400
Subject: [Python-Dev] hard linking executables
In-Reply-To: <20110727001912.4cf67481@pitrou.net>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
Message-ID: <20110726183250.03932089@resist.wooz.org>

On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote:

>Ok, apparently the decision to make hard links for executables dates at
>least back to:

That still doesn't explain *why* hardlinks were originally chosen instead of
symlinks.  In the absence of any other compelling argument against it, I think
they should all consistently be symlinks.  I don't see any Ubuntu or Debian
(where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and
I haven't experienced any issues with it personally.

>changeset:   16221:588691f806f4
>branch:      legacy-trunk
>user:        Neil Schemenauer <nascheme at enme.ucalgary.ca>
>date:        Wed Jan 24 17:11:43 2001 +0000
>files:       Makefile.pre.in
>description:
>Flat makefile based on toplevel Makefile.in and makefiles in build
>subdirectories.  Those other makefiles will go away eventually.
>
>[...]
>
>+# Install the interpreter (by creating a hard link to python$(VERSION))
>+bininstall:    altbininstall
>+       -if test -f $(BINDIR)/$(PYTHON); \
>+       then rm -f $(BINDIR)/$(PYTHON); \
>+       else true; \
>+       fi
>+       (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT))
>+

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110726/dc922eb4/attachment-0001.pgp>

From barry at python.org  Wed Jul 20 15:41:02 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 20 Jul 2011 09:41:02 -0400
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
 Partitioning"
In-Reply-To: <20110720131415.2644B3A409B@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CADiSq7c7yCramnj4vbeywfvZDKme7CEedRvvLfwXczMFOx7D+w@mail.gmail.com>
	<20110720131415.2644B3A409B@sparrow.telecommunity.com>
Message-ID: <20110720094102.1f84a733@resist.wooz.org>

First off, kudos to PJE for his work on this PEP.  He really had the key
insight for this new approach, and did a great job of explaining his vision in
a clear way so that I think everybody over on import-sig "got it".

On Jul 20, 2011, at 08:57 AM, P.J. Eby wrote:

>At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote:
>>On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> > So, without further ado, here it is:
>>
>>I pushed this version up to the PEPs repo, so it now has a number
>>(402) and can be read in prettier HTML format:
>>http://www.python.org/dev/peps/pep-0402/
>
>Technically, shouldn't this be a 3XXX series PEP?  Or are we not doing those
>any more now that all PEPs would be 3XXX?

Great question.  I don't know if we want/need to make the distinction any
more.  It does feel a little odd putting Python 3 PEPs (the only kind of new
Standards Track PEPs) in the 0XXX numbers, but now that we're all moving to
Python 3 <wink>, it seems like segregating new PEPs to the 3XXX range is a bit
contrived.

I think filling up 0XXX is probably fine.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110720/c9db43ad/attachment.pgp>

From barry at python.org  Wed Jul 20 00:45:46 2011
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jul 2011 18:45:46 -0400
Subject: [Python-Dev] [Email-SIG] email-6.0.0.a1
In-Reply-To: <20110719212139.D5D732500D5@webabinitio.net>
References: <20110719212139.D5D732500D5@webabinitio.net>
Message-ID: <20110719184546.4eb8f52a@resist.wooz.org>

On Jul 19, 2011, at 05:21 PM, R. David Murray wrote:

>OK, so I've released the first iteration of the email6 package on pypi
>as email-6.0.0a1.  After install you import it as email6.  This will
>allow anyone curious and/or motivated to test it out under Python 3.2.
>I'm especially interested in anyone with a working program that uses
>email in 3.2: it should be completely backward compatible, and if it
>isn't I want to know ASAP.[*]

It'll take some time to digest, but congratulations RDM!  You've accomplished
an impressive milestone.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110719/4e1d2bf5/attachment-0001.pgp>

From drsalists at gmail.com  Wed Jul 27 22:00:05 2011
From: drsalists at gmail.com (Dan Stromberg)
Date: Wed, 27 Jul 2011 13:00:05 -0700
Subject: [Python-Dev] hard linking executables
In-Reply-To: <20110726183250.03932089@resist.wooz.org>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
	<20110726183250.03932089@resist.wooz.org>
Message-ID: <CAGGBd_oR=x0uxWHu8k3_4zbMD8DnqQYt7WqjDRMBZ52bUmYs6w@mail.gmail.com>

It's been suggested that *ix has hardlinks because someone thought up
hardlinks before someone thought up symlinks - IOW, there are those who
suggest that if people had added symlinks first, no one would've bothered
adding hardlinks.

Symlinks are almost always more flexible, and almost always more clear.

The main counterexample seems to be rsync-based backup systems, that will
hardlink identical files of a given pathname.  But that seems to be a bit of
a mess when it comes time to transfer such a backup from one filesystem to
another.

On Tue, Jul 26, 2011 at 3:32 PM, Barry Warsaw <barry at python.org> wrote:

> On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote:
>
> >Ok, apparently the decision to make hard links for executables dates at
> >least back to:
>
> That still doesn't explain *why* hardlinks were originally chosen instead
> of
> symlinks.  In the absence of any other compelling argument against it, I
> think
> they should all consistently be symlinks.  I don't see any Ubuntu or Debian
> (where /usr/bin/python3 -> python3.2) bug reports indicating any problems,
> and
> I haven't experienced any issues with it personally.
>
> >changeset:   16221:588691f806f4
> >branch:      legacy-trunk
> >user:        Neil Schemenauer <nascheme at enme.ucalgary.ca>
> >date:        Wed Jan 24 17:11:43 2001 +0000
> >files:       Makefile.pre.in
> >description:
> >Flat makefile based on toplevel Makefile.in and makefiles in build
> >subdirectories.  Those other makefiles will go away eventually.
> >
> >[...]
> >
> >+# Install the interpreter (by creating a hard link to python$(VERSION))
> >+bininstall:    altbininstall
> >+       -if test -f $(BINDIR)/$(PYTHON); \
> >+       then rm -f $(BINDIR)/$(PYTHON); \
> >+       else true; \
> >+       fi
> >+       (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT))
> >+
>
> -Barry
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/drsalists%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/f0ce72a9/attachment.html>

From guido at python.org  Wed Jul 27 22:06:09 2011
From: guido at python.org (Guido van Rossum)
Date: Wed, 27 Jul 2011 13:06:09 -0700
Subject: [Python-Dev] hard linking executables
In-Reply-To: <20110726183250.03932089@resist.wooz.org>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
	<20110726183250.03932089@resist.wooz.org>
Message-ID: <CAP7+vJJmmyE8O0XdHT653qnzGQ6ftb+e9nob6P3SU9uZXKX=Dw@mail.gmail.com>

On Tue, Jul 26, 2011 at 3:32 PM, Barry Warsaw <barry at python.org> wrote:
> On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote:
>
>>Ok, apparently the decision to make hard links for executables dates at
>>least back to:
>
> That still doesn't explain *why* hardlinks were originally chosen instead of
> symlinks. ?In the absence of any other compelling argument against it, I think
> they should all consistently be symlinks. ?I don't see any Ubuntu or Debian
> (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and
> I haven't experienced any issues with it personally.

I ought to remember why because I remember I was involved. (And I have
a feeling that the change Antoine dug up was just a refactoring,
moving an existing hard-link target to a different file. Because in my
memory the hard link was my idea and I implemented it. But that change
is Neil Schemenauer's.)

But I can't. :-(

The best I can come up with is that when it's a hard link, you can
replace either "python" or "python2.1" with another version without
disturbing the other. And I think it has something to do with
altinstall vs. install. So it would mean that after first installing
2.1 (so python and python2.1 are the same file), and then
alt-installing 2.1.1, the original 2.1 binary is still accessible as
"python". But I don't know why you'd want that.

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Wed Jul 27 22:34:46 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 22:34:46 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
Message-ID: <20110727223446.5b83a48f@pitrou.net>

On Wed, 27 Jul 2011 10:27:16 -0700
Brett Cannon <brett at python.org> wrote:
> >
> > Perhaps what we could do is move the documentation for test.support to
> > the devguide, and then vet the test suite so that unlink and friends
> > are always called as 'support.unlink', etc.
> >
> 
> I like this solution since this issue of documenting test.support keeps
> coming up. Otherwise we can not document test.support, but  then we need to
> do a pass through the module to make sure  that the docstrings are properly
> updated and we start deprecating some of the stuff in there that is just
> pure cruft.

We don't need to deprecate that cruft, we can just remove it (and all
uses of it, of course).

Regards

Antoine.



From solipsis at pitrou.net  Wed Jul 27 22:36:38 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 27 Jul 2011 22:36:38 +0200
Subject: [Python-Dev] cpython (2.7): - Issue #12603: Fix
 pydoc.synopsis() on files with non-negative st_mtime.
References: <E1Qm828-0002PK-Ic@dinsdale.python.org>
Message-ID: <20110727223638.1bafb8e8@pitrou.net>

On Wed, 27 Jul 2011 19:36:36 +0200
charles-francois.natali <python-checkins at python.org> wrote:
>  
> +- Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime.
> +

Surely you mean non-positive? Non-negative st_mtime being the common
case.

Regards

Antoine.



From cf.natali at gmail.com  Wed Jul 27 22:55:14 2011
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Wed, 27 Jul 2011 22:55:14 +0200
Subject: [Python-Dev] cpython (2.7): - Issue #12603: Fix
 pydoc.synopsis() on files with non-negative st_mtime.
In-Reply-To: <20110727223638.1bafb8e8@pitrou.net>
References: <E1Qm828-0002PK-Ic@dinsdale.python.org>
	<20110727223638.1bafb8e8@pitrou.net>
Message-ID: <CAH_1eM3NSS-YaczG+i3HuNN2VZPEWpqfE6Q6jwXQNdnQR-x6tA@mail.gmail.com>

>> +- Issue #12603: Fix pydoc.synopsis() on files with non-negative
>> st_mtime.
>> +
>
> Surely you mean non-positive? Non-negative st_mtime being the common
> case.

Of course (st_mtime <= 0).

From ben+python at benfinney.id.au  Wed Jul 27 23:37:47 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 28 Jul 2011 07:37:47 +1000
Subject: [Python-Dev] hard linking executables
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
	<20110726183250.03932089@resist.wooz.org>
	<CAGGBd_oR=x0uxWHu8k3_4zbMD8DnqQYt7WqjDRMBZ52bUmYs6w@mail.gmail.com>
Message-ID: <87k4b3pfpw.fsf@benfinney.id.au>

Dan Stromberg <drsalists at gmail.com> writes:

> It's been suggested that *ix has hardlinks because someone thought up
> hardlinks before someone thought up symlinks - IOW, there are those who
> suggest that if people had added symlinks first, no one would've bothered
> adding hardlinks.

Well, that suggestion is faulty. It ignores the fact that *all* ordinary
files on Unix are hard links. Any ordinary file entry in a directory is
a hard link to the file's data.

The ?hard links? capability, therefore, isn't something that was added;
it's fundamental to Unix filesystems from their inception.

The ?ln? command adds *additional* hard links to an existing file's
data; but, once added, they're exactly the same as any other ordinary
file entry.

> Symlinks are almost always more flexible, and almost always more
> clear.

Yet many tools don't work as expected with symbolic links which will
work with an ordinary file (a ?hard link?). One can argue that such
tools are to that extent buggy, but symbolic links produce deliberately
different behaviour which is sometimes not what one needs.

-- 
 \     ?Wrinkles should merely indicate where smiles have been.? ?Mark |
  `\                                    Twain, _Following the Equator_ |
_o__)                                                                  |
Ben Finney


From victor.stinner at haypocalc.com  Wed Jul 27 23:53:20 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 27 Jul 2011 23:53:20 +0200
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
Message-ID: <4E3088D0.5040900@haypocalc.com>

Hi,

Three weeks ago, I posted a draft on my PEP on this mailing list. I 
tried to include all remarks you made, and the PEP is now online:

    http://www.python.org/dev/peps/pep-0400/

It's now unclear to me if the PEP will be accepted or rejected. I don't 
know what to do to move forward. I asked Guido in private, but I didn't 
get any answer.

Victor


From victor.stinner at haypocalc.com  Thu Jul 28 00:12:51 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 28 Jul 2011 00:12:51 +0200
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
Message-ID: <4E308D63.9090901@haypocalc.com>

Hi,

Three weeks ago, I posted a draft on my PEP on this mailing list. I 
tried to include all remarks you made, and the PEP is now online:

    http://www.python.org/dev/peps/pep-0400/

It's now unclear to me if the PEP will be accepted or rejected. I don't 
know what to do to move forward.

Victor

From guido at python.org  Thu Jul 28 00:36:52 2011
From: guido at python.org (Guido van Rossum)
Date: Wed, 27 Jul 2011 15:36:52 -0700
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <4E3088D0.5040900@haypocalc.com>
References: <4E3088D0.5040900@haypocalc.com>
Message-ID: <CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>

On Wed, Jul 27, 2011 at 2:53 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Three weeks ago, I posted a draft on my PEP on this mailing list. I tried to
> include all remarks you made, and the PEP is now online:
>
> ? http://www.python.org/dev/peps/pep-0400/
>
> It's now unclear to me if the PEP will be accepted or rejected. I don't know
> what to do to move forward. I asked Guido in private, but I didn't get any
> answer.
>
> Victor

Sorry Victor, I somehow didn't see that message even though I received
it (I probably thought it was a continuation of the python-dev thread
which I've been ignoring).

Unfortunately I don't have time to go back and read the whole thread.
I think I haven't used codecs.StreamReader/Writer myself, and I don't
think I've seen much use of them either (which doesn't mean there
isn't). My gut feeling is that yes, they should eventually go away,
but no, there's no particular hurry, and no, I don't think you should
change codecs.open() to call io.open().

I think the best thing is to campaign (e.g. in docs) for people to
stop using codecs.open/StreamReader/Writer and start deprecating them
formally once we feel that most users have switched. It's possible
that that could happen before 3.3 is released, but I'm kind of
doubtful about that myself.

Sorry again for missing your private emails!

-- 
--Guido van Rossum (python.org/~guido)

From ncoghlan at gmail.com  Thu Jul 28 01:05:11 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 28 Jul 2011 09:05:11 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
Message-ID: <CADiSq7dsAUxBx7tx9f9fegLpdafAqGmORXi4yqPtNiQWGbdMwg@mail.gmail.com>

On Thu, Jul 28, 2011 at 3:27 AM, Brett Cannon <brett at python.org> wrote:
> On Wed, Jul 27, 2011 at 06:36, R. David Murray <rdmurray at bitdance.com>
> wrote:
>> Perhaps what we could do is move the documentation for test.support to
>> the devguide, and then vet the test suite so that unlink and friends
>> are always called as 'support.unlink', etc.
>
> I like this solution since this issue of documenting test.support keeps
> coming up. Otherwise we can not document test.support, but? then we need to
> do a pass through the module to make sure? that the docstrings are properly
> updated and we start deprecating some of the stuff in there that is just
> pure cruft.

+1 for relocating this to the devguide. The only reason this is in the
main docs now is that it used to be the only way to easily get pretty
documentation for it.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Thu Jul 28 01:11:33 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 28 Jul 2011 01:11:33 +0200
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>
References: <4E3088D0.5040900@haypocalc.com>
	<CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>
Message-ID: <4E309B25.7090304@haypocalc.com>

Le 28/07/2011 00:36, Guido van Rossum a ?crit :
> Sorry Victor, I somehow didn't see that message even though I received
> it (I probably thought it was a continuation of the python-dev thread
> which I've been ignoring).

No problem.

> no, there's no particular hurry

That's why it's a deprecation process and the removal is schedule for 
later: "3.4 (or maybe later)". I added "or maybe later" before reopening 
a new thread on this list.

> no, I don't think you should change codecs.open() to call io.open()

The PEP is useless without this change. If we don't deprecate any class 
and don't change codecs.open(), it's better to just reject the PEP.

> start deprecating them formally once we feel that most users have switched

Users of codecs.open() or users of codecs.Stream* classes?

Victor

From guido at python.org  Thu Jul 28 01:38:26 2011
From: guido at python.org (Guido van Rossum)
Date: Wed, 27 Jul 2011 16:38:26 -0700
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <4E309B25.7090304@haypocalc.com>
References: <4E3088D0.5040900@haypocalc.com>
	<CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>
	<4E309B25.7090304@haypocalc.com>
Message-ID: <CAP7+vJJsdK9NdGu2AxtM=W=Ug5jF2twL5OFpQUU-x3uwxfvXPA@mail.gmail.com>

On Wed, Jul 27, 2011 at 4:11 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Le 28/07/2011 00:36, Guido van Rossum a ?crit :
>>
>> Sorry Victor, I somehow didn't see that message even though I received
>> it (I probably thought it was a continuation of the python-dev thread
>> which I've been ignoring).
>
> No problem.
>
>> no, there's no particular hurry
>
> That's why it's a deprecation process and the removal is schedule for later:
> "3.4 (or maybe later)". I added "or maybe later" before reopening a new
> thread on this list.

That still sounds fairly aggressive.

>> no, I don't think you should change codecs.open() to call io.open()
>
> The PEP is useless without this change. If we don't deprecate any class and
> don't change codecs.open(), it's better to just reject the PEP.

Why? (Not that I am against rejecting the PEP. I feel weakly opinioned
in this case, about -0.)

>> start deprecating them formally once we feel that most users have switched
>
> Users of codecs.open() or users of codecs.Stream* classes?

I would think both. Is there any reason to continue using codecs.open()?

-- 
--Guido van Rossum (python.org/~guido)

From steve at pearwood.info  Thu Jul 28 01:53:21 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 28 Jul 2011 09:53:21 +1000
Subject: [Python-Dev] Convention on functions that shadow
 existing	stdlib functions
In-Reply-To: <CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
Message-ID: <4E30A4F1.6050305@pearwood.info>

Eli Bendersky wrote:

> Sure, but I'm still leery of two functions with the same name doing acting
> slightly differently.


and then in a later post:

> As I mentioned elsewhere, it's not good practice to have two functions with
> the same name doing something slightly different, in different modules in
> the code-base.

artist.draw() and gunslinger.draw() do not necessarily need to do the 
same thing, and I don't agree that a (futile) preference for globally 
unique names is good practice: it can lead to unnecessarily long names 
(artist.draw_with_pencil_on_paper) or redundant characters prefixing the 
name (itertools.imap -- what does the "i" in imap tell you that the 
itertools didn't already?). Solving this problem is what namespaces are for.

Renaming test.support.ulink to something else doesn't fix the problem of 
unsupported support functions being used in third-party code. It may 
even *encourage* it, by making the extra functionality more explicit.

However, is there any reason why test.support itself shouldn't be 
renamed test._support, or possibly _test.support, so that the *entire* 
suite is marked as a private implementation detail?



-- 
Steven


From ncoghlan at gmail.com  Thu Jul 28 03:00:09 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 28 Jul 2011 11:00:09 +1000
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <CAP7+vJJsdK9NdGu2AxtM=W=Ug5jF2twL5OFpQUU-x3uwxfvXPA@mail.gmail.com>
References: <4E3088D0.5040900@haypocalc.com>
	<CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>
	<4E309B25.7090304@haypocalc.com>
	<CAP7+vJJsdK9NdGu2AxtM=W=Ug5jF2twL5OFpQUU-x3uwxfvXPA@mail.gmail.com>
Message-ID: <CADiSq7cZ3QCrDvVmc126suLRWjT2wcqYYZoYbBGKDMuScCszTA@mail.gmail.com>

On Thu, Jul 28, 2011 at 9:38 AM, Guido van Rossum <guido at python.org> wrote:
>> Users of codecs.open() or users of codecs.Stream* classes?
>
> I would think both. Is there any reason to continue using codecs.open()?

It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x.

The problem is that naive 2.x code will migrate to the optimised IO
stack automatically on the 2.x -> 3.x transition, while code that
tried to do the right thing has to be changed manually (either in 3.x
only, or by switching to the io module for 2.x as well) in order to
adjust for the differences in argument order.

The idea behind changing codecs.open to be a wrapper around io.open
was to allow such code to switch to the new optimised IO stack as
easily as code that just uses the open builtin. If it's acceptable for
the builtin behaviour to change (far more substantially), why not
change codecs.open as well?

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From guido at python.org  Thu Jul 28 04:05:18 2011
From: guido at python.org (Guido van Rossum)
Date: Wed, 27 Jul 2011 19:05:18 -0700
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <CADiSq7cZ3QCrDvVmc126suLRWjT2wcqYYZoYbBGKDMuScCszTA@mail.gmail.com>
References: <4E3088D0.5040900@haypocalc.com>
	<CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>
	<4E309B25.7090304@haypocalc.com>
	<CAP7+vJJsdK9NdGu2AxtM=W=Ug5jF2twL5OFpQUU-x3uwxfvXPA@mail.gmail.com>
	<CADiSq7cZ3QCrDvVmc126suLRWjT2wcqYYZoYbBGKDMuScCszTA@mail.gmail.com>
Message-ID: <CAP7+vJJTe-eTr2QAzsCGCvX45sKfqGS+jySQPaaNgVt5pUobjw@mail.gmail.com>

On Wed, Jul 27, 2011 at 6:00 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Jul 28, 2011 at 9:38 AM, Guido van Rossum <guido at python.org> wrote:
>>> Users of codecs.open() or users of codecs.Stream* classes?
>>
>> I would think both. Is there any reason to continue using codecs.open()?
>
> It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x.

Even on 2.6, where the io module exists?

> The problem is that naive 2.x code will migrate to the optimised IO
> stack automatically on the 2.x -> 3.x transition, while code that
> tried to do the right thing has to be changed manually (either in 3.x
> only, or by switching to the io module for 2.x as well) in order to
> adjust for the differences in argument order.
>
> The idea behind changing codecs.open to be a wrapper around io.open
> was to allow such code to switch to the new optimised IO stack as
> easily as code that just uses the open builtin. If it's acceptable for
> the builtin behaviour to change (far more substantially), why not
> change codecs.open as well?

Aren't the cases different? Using built-in open() just means you want
to open a file in the default way. Using codecs.open() presumably
means that you've thought about Unicode.

TBH, I said I was only -0 on the PEP, and if the stream returned by
codecs.open() in 3.3 is sufficiently compatible with the stream
returned the same function returns in 3.2, I am okay with it. (Except
I also want you to cut a trillion dollars from the non-military
budget, without raising taxes. :-)

-- 
--Guido van Rossum (python.org/~guido)

From drsalists at gmail.com  Thu Jul 28 04:34:36 2011
From: drsalists at gmail.com (Dan Stromberg)
Date: Wed, 27 Jul 2011 19:34:36 -0700
Subject: [Python-Dev] hard linking executables
In-Reply-To: <87k4b3pfpw.fsf@benfinney.id.au>
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
	<20110726183250.03932089@resist.wooz.org>
	<CAGGBd_oR=x0uxWHu8k3_4zbMD8DnqQYt7WqjDRMBZ52bUmYs6w@mail.gmail.com>
	<87k4b3pfpw.fsf@benfinney.id.au>
Message-ID: <CAGGBd_pBudM-A=WJOSoKCDVgULmgX_NmUDhe2DZWLFXtcRciXg@mail.gmail.com>

On Wed, Jul 27, 2011 at 2:37 PM, Ben Finney <ben+python at benfinney.id.au>wrote:

> Dan Stromberg <drsalists at gmail.com> writes:
>
> > It's been suggested that *ix has hardlinks because someone thought up
> > hardlinks before someone thought up symlinks - IOW, there are those who
> > suggest that if people had added symlinks first, no one would've bothered
> > adding hardlinks.
>
> Well, that suggestion is faulty. It ignores the fact that *all* ordinary
> files on Unix are hard links. Any ordinary file entry in a directory is
> a hard link to the file's data.
>

Not really.  Whether hard links is supported is mostly a matter of what
filesystem you are using - in modern times.  It's true that filesystems with
complete POSIX semantics probably all support hardlinks, but that's far from
every file on any given *ix.  And of course, POSIX doesn't appear to have
been created until the late 1990's.


> The ?hard links? capability, therefore, isn't something that was added;
> it's fundamental to Unix filesystems from their inception.
>

Hard linking was reportedly in Unix Version 1, but I see nothing indicating
it was in the original Unics of 1969.  Then again, I don't see much of
anything on the net about what was and wasn't in Unics.


> The ?ln? command adds *additional* hard links to an existing file's
> data; but, once added, they're exactly the same as any other ordinary
> file entry.
>

Well, if you're in a filesystem that supports hardlinking anyway.
Supporting that isn't inherent.  It requires some sort of on-disk
representation for persistence, and not all filesystems support that.


> > Symlinks are almost always more flexible, and almost always more
> > clear.
>
> Yet many tools don't work as expected with symbolic links which will
> work with an ordinary file (a ?hard link?). One can argue that such
> tools are to that extent buggy, but symbolic links produce deliberately
> different behaviour which is sometimes not what one needs.
>

Please recall that I was paraphrasing someone saying that hardlinks were
seldom better, not never better.  I don't know that there's anything in your
post that addresses that.

It's much easier to imagine a system with no hardlinks, than to imagine a
system with no symlinks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110727/1899e5af/attachment.html>

From eliben at gmail.com  Thu Jul 28 05:43:12 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 28 Jul 2011 06:43:12 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <4E30A4F1.6050305@pearwood.info>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
Message-ID: <CAF-Rda8zkZZ0Ujqmz3F2=EWPyJ=x3SwE1hou0Qr_nXZpuaQDyg@mail.gmail.com>

On Thu, Jul 28, 2011 at 02:53, Steven D'Aprano <steve at pearwood.info> wrote:

> Eli Bendersky wrote:
>
>  Sure, but I'm still leery of two functions with the same name doing acting
>> slightly differently.
>>
>
>
> and then in a later post:
>
>
>  As I mentioned elsewhere, it's not good practice to have two functions
>> with
>> the same name doing something slightly different, in different modules in
>> the code-base.
>>
>
> artist.draw() and gunslinger.draw() do not necessarily need to do the same
> thing,
>

Of course, but why do you ignore the "slightly different". Had
support.rmtree been something completely different from shutil.rmtree, I
wouldn't have a problem with it. But it just calls rmtree and ignores some
errors. This is the part I have the problem with.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110728/7ca14730/attachment.html>

From benjamin at python.org  Thu Jul 28 06:10:28 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 27 Jul 2011 23:10:28 -0500
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <CAP7+vJJTe-eTr2QAzsCGCvX45sKfqGS+jySQPaaNgVt5pUobjw@mail.gmail.com>
References: <4E3088D0.5040900@haypocalc.com>
	<CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>
	<4E309B25.7090304@haypocalc.com>
	<CAP7+vJJsdK9NdGu2AxtM=W=Ug5jF2twL5OFpQUU-x3uwxfvXPA@mail.gmail.com>
	<CADiSq7cZ3QCrDvVmc126suLRWjT2wcqYYZoYbBGKDMuScCszTA@mail.gmail.com>
	<CAP7+vJJTe-eTr2QAzsCGCvX45sKfqGS+jySQPaaNgVt5pUobjw@mail.gmail.com>
Message-ID: <CAPZV6o-H_4TB=nqujhEJBJM7==Cze9OWPXfm-bNebNt7i-U=VA@mail.gmail.com>

2011/7/27 Guido van Rossum <guido at python.org>:
> On Wed, Jul 27, 2011 at 6:00 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Thu, Jul 28, 2011 at 9:38 AM, Guido van Rossum <guido at python.org> wrote:
>>>> Users of codecs.open() or users of codecs.Stream* classes?
>>>
>>> I would think both. Is there any reason to continue using codecs.open()?
>>
>> It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x.
>
> Even on 2.6, where the io module exists?

io on 2.6 is fairly broken and dead slow. The advantage of codecs.open
is it hasn't changed in the very long time. It still has the same
reliable buggy behavior no matter what version you're on.

I don't see the problem with leaving codecs.open() to rot a few more
releases before deprecating it while leaving messaging in the docs
suggesting io.*.

>
>> The problem is that naive 2.x code will migrate to the optimised IO
>> stack automatically on the 2.x -> 3.x transition, while code that
>> tried to do the right thing has to be changed manually (either in 3.x
>> only, or by switching to the io module for 2.x as well) in order to
>> adjust for the differences in argument order.
>>
>> The idea behind changing codecs.open to be a wrapper around io.open
>> was to allow such code to switch to the new optimised IO stack as
>> easily as code that just uses the open builtin. If it's acceptable for
>> the builtin behaviour to change (far more substantially), why not
>> change codecs.open as well?
>
> Aren't the cases different? Using built-in open() just means you want
> to open a file in the default way. Using codecs.open() presumably
> means that you've thought about Unicode.
>
> TBH, I said I was only -0 on the PEP, and if the stream returned by
> codecs.open() in 3.3 is sufficiently compatible with the stream
> returned the same function returns in 3.2, I am okay with it. (Except
> I also want you to cut a trillion dollars from the non-military
> budget, without raising taxes. :-)

May I suggest you include the military budget? :)



-- 
Regards,
Benjamin

From ben+python at benfinney.id.au  Thu Jul 28 06:35:59 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 28 Jul 2011 14:35:59 +1000
Subject: [Python-Dev] hard linking executables
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
	<20110726183250.03932089@resist.wooz.org>
	<CAGGBd_oR=x0uxWHu8k3_4zbMD8DnqQYt7WqjDRMBZ52bUmYs6w@mail.gmail.com>
	<87k4b3pfpw.fsf@benfinney.id.au>
	<CAGGBd_pBudM-A=WJOSoKCDVgULmgX_NmUDhe2DZWLFXtcRciXg@mail.gmail.com>
Message-ID: <871uxbowcw.fsf@benfinney.id.au>

Dan Stromberg <drsalists at gmail.com> writes:

> On Wed, Jul 27, 2011 at 2:37 PM, Ben Finney <ben+python at benfinney.id.au>wrote:
>
> > Dan Stromberg <drsalists at gmail.com> writes:
> >
> > > It's been suggested that [?] if people had added symlinks first,
> > > no one would've bothered adding hardlinks.
> >
> > Well, that suggestion is faulty. It ignores the fact that *all*
> > ordinary files on Unix are hard links. Any ordinary file entry in a
> > directory is a hard link to the file's data.
>
> Not really. Whether hard links is supported is mostly a matter of what
> filesystem you are using - in modern times. It's true that filesystems
> with complete POSIX semantics probably all support hardlinks, but
> that's far from every file on any given *ix. And of course, POSIX
> doesn't appear to have been created until the late 1990's.

POSIX didn't create those semantics, though; it formalised semantics
that were already long-time convention.

My only point was that, unlike symbolic links, hard linking isn't some
added feature, it's a property of the way Unix filesystems represent
normal files. And you're right that I mean ?on filesystems with POSIX
semantics?.

> It's much easier to imagine a system with no hardlinks, than to
> imagine a system with no symlinks.

Why imagine? There have been many of both.

-- 
 \            ?But it is permissible to make a judgment after you have |
  `\    examined the evidence. In some circles it is even encouraged.? |
_o__)                    ?Carl Sagan, _The Burden of Skepticism_, 1987 |
Ben Finney


From nas at arctrix.com  Thu Jul 28 07:00:51 2011
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu, 28 Jul 2011 05:00:51 +0000 (UTC)
Subject: [Python-Dev] hard linking executables
References: <CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com>
	<nad-62B81D.01034918072011@dough.gmane.org>
	<CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com>
	<20110724231655.0b69d36a@pitrou.net>
	<20110724232106.53161528@pitrou.net>
	<CANaWP3ymfgYBL6WzqaziVKb=11HGZXKocoKRWFmoob9Bu4E1ZA@mail.gmail.com>
	<1311696328.3648.0.camel@localhost.localdomain>
	<4E2EE813.1080407@netwok.org> <20110727001912.4cf67481@pitrou.net>
	<20110726183250.03932089@resist.wooz.org>
	<CAP7+vJJmmyE8O0XdHT653qnzGQ6ftb+e9nob6P3SU9uZXKX=Dw@mail.gmail.com>
Message-ID: <j0qqe3$h5$1@dough.gmane.org>

Guido van Rossum <guido at python.org> wrote:
> I ought to remember why because I remember I was involved. (And I have
> a feeling that the change Antoine dug up was just a refactoring,

You are correct.  I checked Python 1.5.2 and it also creates hard
links (prior to Makefile overhaul).

> The best I can come up with is that when it's a hard link, you can
> replace either "python" or "python2.1" with another version without
> disturbing the other.

So, I don't think anyone has a good argument for the status quo.  A
symlink would make more sense to me.

  Neil


From victor.stinner at haypocalc.com  Thu Jul 28 10:49:33 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 28 Jul 2011 10:49:33 +0200
Subject: [Python-Dev] Status of the PEP 400?
	(deprecate	codecs.StreamReader/StreamWriter)
In-Reply-To: <CAPZV6o-H_4TB=nqujhEJBJM7==Cze9OWPXfm-bNebNt7i-U=VA@mail.gmail.com>
References: <4E3088D0.5040900@haypocalc.com>	<CAP7+vJ+QspntHaTfKE98Ud2AVFJag_9zwD5vO5MxTihdbJk7fg@mail.gmail.com>	<4E309B25.7090304@haypocalc.com>	<CAP7+vJJsdK9NdGu2AxtM=W=Ug5jF2twL5OFpQUU-x3uwxfvXPA@mail.gmail.com>	<CADiSq7cZ3QCrDvVmc126suLRWjT2wcqYYZoYbBGKDMuScCszTA@mail.gmail.com>	<CAP7+vJJTe-eTr2QAzsCGCvX45sKfqGS+jySQPaaNgVt5pUobjw@mail.gmail.com>
	<CAPZV6o-H_4TB=nqujhEJBJM7==Cze9OWPXfm-bNebNt7i-U=VA@mail.gmail.com>
Message-ID: <4E31229D.9010207@haypocalc.com>

Le 28/07/2011 06:10, Benjamin Peterson a ?crit :
>>>> there any reason to continue using codecs.open()?
>>>
>>> It's the easiest way to write Unicode friendly code that spans both 2.x and 3.x.
>>
>> Even on 2.6, where the io module exists?
>
> io on 2.6 is fairly broken and dead slow. The advantage of codecs.open
> is it hasn't changed in the very long time. It still has the same
> reliable buggy behavior no matter what version you're on.
>
> I don't see the problem with leaving codecs.open() to rot a few more
> releases before deprecating it while leaving messaging in the docs
> suggesting io.*.

All these points were already discussed before.

There is a subsection in "Backwards Compatibility" section in the PEP 
400 explaining why codecs.open is NOT deprecated:

http://www.python.org/dev/peps/pep-0400/#keep-the-public-api-codecs-open

"codecs.open() can be replaced by the builtin open() function. open() 
has a similar API but has also more options. Both functions return 
file-like objects (same API).

codecs.open() was the only way to open a text file in Unicode mode until 
Python 2.6. Many Python 2 programs uses this function. Removing 
codecs.open() implies more work to port programs from Python 2 to Python 
3, especially projets using the same code base for the two Python 
versions (without using 2to3 program).

codecs.open() is kept for backward compatibility with Python 2."

Victor

From victor.stinner at haypocalc.com  Thu Jul 28 11:28:43 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 28 Jul 2011 11:28:43 +0200
Subject: [Python-Dev] Status of the PEP 400?
	(deprecate	codecs.StreamReader/StreamWriter)
In-Reply-To: <4E3125D7.2030103@egenix.com>
References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com>
Message-ID: <4E312BCB.3080301@haypocalc.com>

Le 28/07/2011 11:03, M.-A. Lemburg a ?crit :
> Victor Stinner wrote:
>> Hi,
>>
>> Three weeks ago, I posted a draft on my PEP on this mailing list. I
>> tried to include all remarks you made, and the PEP is now online:
>>
>>     http://www.python.org/dev/peps/pep-0400/
>>
>> It's now unclear to me if the PEP will be accepted or rejected. I don't
>> know what to do to move forward.
>
> The PEP still compares apples and oranges, issues and features,

I don't know how to write a PEP and this is my first PEP. I think that 
it is possible to compare two classes using a list of issues and 
features. How should I change the PEP to compare comparable things?

> and doesn't cover the fact that it is proposing to not just deprecate
> a feature, but a part of a design concept which will then no longer
> be available in Python.

The "Usage of StreamReader and StreamWriter" section tries to list 
usages of these classes, and "Deprecate StreamReader and StreamWriter" 
section explains that these classes will be removed. I agree that these 
sections are short, but I don't know what to add.

Could you please enhance these sections?

> I'm still -1 on that part of the PEP.

Ok.

> As I mentioned before, having
> codecs.open() changed to be a wrapper around io.open() in Python 3.3,
> should be investigated. If it doesn't cause too much trouble, this
> would be a good idea.

I did already try on the full Python test suite, and all test pass. I 
don't know if it's representative.

> Please do keep the original implementation
> around (e.g. renamed to codecs.open_stream()), though, so that it's
> still possible to get easy-to-use access to codec StreamReader/Writers.

I will add your alternative to the PEP (except if you would like to do 
that yourself?). If I understood correctly, you propose to:

  * rename codecs.open() to codecs.open_stream()
  * change codecs.open() to reuse open() (and so io.TextIOWrapper)

(and don't deprecate anything)

Add a new function to Python 3.3 means that we will have to maintain it 
for later versions. It's just the opposite of my proposition :-)

Victor

From mal at egenix.com  Thu Jul 28 11:03:19 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 28 Jul 2011 11:03:19 +0200
Subject: [Python-Dev] Status of the PEP 400?
	(deprecate	codecs.StreamReader/StreamWriter)
In-Reply-To: <4E308D63.9090901@haypocalc.com>
References: <4E308D63.9090901@haypocalc.com>
Message-ID: <4E3125D7.2030103@egenix.com>

Victor Stinner wrote:
> Hi,
> 
> Three weeks ago, I posted a draft on my PEP on this mailing list. I
> tried to include all remarks you made, and the PEP is now online:
> 
>    http://www.python.org/dev/peps/pep-0400/
> 
> It's now unclear to me if the PEP will be accepted or rejected. I don't
> know what to do to move forward.

The PEP still compares apples and oranges, issues and features,
and doesn't cover the fact that it is proposing to not just deprecate
a feature, but a part of a design concept which will then no longer
be available in Python.

I'm still -1 on that part of the PEP.

As I mentioned before, having
codecs.open() changed to be a wrapper around io.open() in Python 3.3,
should be investigated. If it doesn't cause too much trouble, this
would be a good idea. Please do keep the original implementation
around (e.g. renamed to codecs.open_stream()), though, so that it's
still possible to get easy-to-use access to codec StreamReader/Writers.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 28 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mattbasta at gmail.com  Thu Jul 28 20:25:15 2011
From: mattbasta at gmail.com (Matt)
Date: Thu, 28 Jul 2011 11:25:15 -0700
Subject: [Python-Dev] HTMLParser and HTML5
Message-ID: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>

Hello all,

I wanted to ask a few questions and start a discussion about HTML5
support within the HTMLParser class(es). Over on issue?670664, an
inconsistency with the way browsers and the HTMLParser parse script
and style tags was discovered. Currently, HTMLParser adheres strictly
to the HTML4 standard, which says that these tags should exit CDATA
mode when the start of *any* closing tag is found. No browsers, to my
knowledge, have ever supported this (at least in the 21st century).
Instead, all browsers implement the behavior described in the HTML5
spec, which states that script tags should exit their "raw text mode"
when the full closing tag for that element is encountered.

The repercussions of adhering to the HTML4 standard in HTMLParser are
somewhat serious: a good number of documents will either encounter
exceptions for broken markup (which aren't actually broken). Libraries
like Beautiful Soup (which depend on HTMLParser) are also affected,
requiring the use of hacks just to get the document to parse at all.

Rather than bore you all with another paragraph about how HTML4 is
terrible, feel free to look at the issue
(http://bugs.python.org/issue670664), which quite thoroughly outlines
the pros and cons of this particular change. Any feedback/input  on
the proposed changes is welcome.

So here are my questions:

- What plans, if any, are there to support HTML5 parsing behaviors,
since the HTML5 spec effectively describes current web browser
behavior?
- What policies are in place for keeping parity with other HTML
parsers (such as those in web browsers)?

Given the semi-backward-compatible nature of HTML5's syntax, this
seems like a rather unique problem that could use some more
discussion.


Thanks

Matt Basta

From brett at python.org  Thu Jul 28 23:33:39 2011
From: brett at python.org (Brett Cannon)
Date: Thu, 28 Jul 2011 14:33:39 -0700
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <4E30A4F1.6050305@pearwood.info>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
Message-ID: <CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>

On Wed, Jul 27, 2011 at 16:53, Steven D'Aprano <steve at pearwood.info> wrote:

> Eli Bendersky wrote:
>
>  Sure, but I'm still leery of two functions with the same name doing acting
>> slightly differently.
>>
>
>
> and then in a later post:
>
>
>  As I mentioned elsewhere, it's not good practice to have two functions
>> with
>> the same name doing something slightly different, in different modules in
>> the code-base.
>>
>
> artist.draw() and gunslinger.draw() do not necessarily need to do the same
> thing, and I don't agree that a (futile) preference for globally unique
> names is good practice: it can lead to unnecessarily long names
> (artist.draw_with_pencil_on_**paper) or redundant characters prefixing the
> name (itertools.imap -- what does the "i" in imap tell you that the
> itertools didn't already?). Solving this problem is what namespaces are for.
>
> Renaming test.support.ulink to something else doesn't fix the problem of
> unsupported support functions being used in third-party code. It may even
> *encourage* it, by making the extra functionality more explicit.
>
> However, is there any reason why test.support itself shouldn't be renamed
> test._support, or possibly _test.support, so that the *entire* suite is
> marked as a private implementation detail?
>

Technically no for the _test idea, although it would promote the idea of not
shipping Python with its tests, which would be a shame as it's hard enough
to get people to run them. Renaming test.support is much more acceptable,
just more work considering how many times that module is used in the test
suite.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110728/11e63964/attachment.html>

From brett at python.org  Thu Jul 28 23:49:14 2011
From: brett at python.org (Brett Cannon)
Date: Thu, 28 Jul 2011 14:49:14 -0700
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
Message-ID: <CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>

On Thu, Jul 28, 2011 at 11:25, Matt <mattbasta at gmail.com> wrote:

> Hello all,
>
> I wanted to ask a few questions and start a discussion about HTML5
> support within the HTMLParser class(es). Over on issue 670664, an
> inconsistency with the way browsers and the HTMLParser parse script
> and style tags was discovered. Currently, HTMLParser adheres strictly
> to the HTML4 standard, which says that these tags should exit CDATA
> mode when the start of *any* closing tag is found. No browsers, to my
> knowledge, have ever supported this (at least in the 21st century).
> Instead, all browsers implement the behavior described in the HTML5
> spec, which states that script tags should exit their "raw text mode"
> when the full closing tag for that element is encountered.
>
> The repercussions of adhering to the HTML4 standard in HTMLParser are
> somewhat serious: a good number of documents will either encounter
> exceptions for broken markup (which aren't actually broken). Libraries
> like Beautiful Soup (which depend on HTMLParser) are also affected,
> requiring the use of hacks just to get the document to parse at all.
>
> Rather than bore you all with another paragraph about how HTML4 is
> terrible, feel free to look at the issue
> (http://bugs.python.org/issue670664), which quite thoroughly outlines
> the pros and cons of this particular change. Any feedback/input  on
> the proposed changes is welcome.
>
> So here are my questions:
>
> - What plans, if any, are there to support HTML5 parsing behaviors,
> since the HTML5 spec effectively describes current web browser
> behavior?
>

There are not specific plans that have been publicly brought up (to my
knowledge).


> - What policies are in place for keeping parity with other HTML
> parsers (such as those in web browsers)?
>

There aren't any beyond "it would be nice".


>
> Given the semi-backward-compatible nature of HTML5's syntax, this
> seems like a rather unique problem that could use some more
> discussion.
>

It's more of an issue of someone caring enough to do the coding work to
bring the parser up to spec for HTML5 (or introduce new code to live beside
the HTML4 parsing code). IOW there is no policies specifically about this
topic beyond the general desire to stay up-to-date with stable specs.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110728/c69ecb46/attachment.html>

From ncoghlan at gmail.com  Fri Jul 29 02:39:47 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 29 Jul 2011 10:39:47 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
Message-ID: <CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>

On Fri, Jul 29, 2011 at 7:33 AM, Brett Cannon <brett at python.org> wrote:
>> However, is there any reason why test.support itself shouldn't be renamed
>> test._support, or possibly _test.support, so that the *entire* suite is
>> marked as a private implementation detail?
>
> Technically no for the _test idea, although it would promote the idea of not
> shipping Python with its tests, which would be a shame as it's hard enough
> to get people to run them. Renaming test.support is much more acceptable,
> just more work considering how many times that module is used in the test
> suite.

Moving the docs should deal with the module and name indexing issue,
so -1 one to expending any developer effort on unnecessary name
changes in the test suite.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From stefan_ml at behnel.de  Fri Jul 29 06:37:17 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 29 Jul 2011 06:37:17 +0200
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
Message-ID: <j0tddt$20k$1@dough.gmane.org>

Brett Cannon, 28.07.2011 23:49:
> On Thu, Jul 28, 2011 at 11:25, Matt wrote:
>> - What policies are in place for keeping parity with other HTML
>> parsers (such as those in web browsers)?
>
> There aren't any beyond "it would be nice".
>[...]
> It's more of an issue of someone caring enough to do the coding work to
> bring the parser up to spec for HTML5 (or introduce new code to live beside
> the HTML4 parsing code).

Which, given that html5lib readily exists, would likely be a lot more work 
than anyone who is interested in HTML5 handling would want to invest.

I don't think we need a new HTML5 parsing implementation only to have it in 
the stdlib. That's the old sunny Java way of doing it.

Stefan


From eliben at gmail.com  Fri Jul 29 07:24:59 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 29 Jul 2011 08:24:59 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
Message-ID: <CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>

On Fri, Jul 29, 2011 at 03:39, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Fri, Jul 29, 2011 at 7:33 AM, Brett Cannon <brett at python.org> wrote:
> >> However, is there any reason why test.support itself shouldn't be
> renamed
> >> test._support, or possibly _test.support, so that the *entire* suite is
> >> marked as a private implementation detail?
> >
> > Technically no for the _test idea, although it would promote the idea of
> not
> > shipping Python with its tests, which would be a shame as it's hard
> enough
> > to get people to run them. Renaming test.support is much more acceptable,
> > just more work considering how many times that module is used in the test
> > suite.
>
> Moving the docs should deal with the module and name indexing issue,
> so -1 one to expending any developer effort on unnecessary name
> changes in the test suite.
>
>
Alright, I think there's now a sufficiently wide consensus to move the
documentation of Lib/test and Lib/test/support in particular to the
devguide, which raises a question:

Currently test.support is documented both for Python 3K and 2.7, while the
devguide (AFAIK) comes in a single version.

I propose to just move 3K's docs to the devguide, and make both doc pages
(in 3K and 2.7) point to it. Is this acceptable?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/8a0a2ed1/attachment.html>

From ncoghlan at gmail.com  Fri Jul 29 07:48:07 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 29 Jul 2011 15:48:07 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
Message-ID: <CADiSq7et-oS2UFEsX+uLgVo1eLCfXfe1FWsFp9J5s_3ji5bcgQ@mail.gmail.com>

On Fri, Jul 29, 2011 at 3:24 PM, Eli Bendersky <eliben at gmail.com> wrote:
> I propose to just move 3K's docs to the devguide, and make both doc pages
> (in 3K and 2.7) point to it. Is this acceptable?

Yeah, just include a note in the devguide version saying that anything
added in 3.2 or later may not be available when writing tests for the
2.7 maintenance branch.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From eliben at gmail.com  Fri Jul 29 08:48:15 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 29 Jul 2011 09:48:15 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Issue #10639:
 reindent.py tool now accepts a --newline option to specify the
In-Reply-To: <E1Qlk2Y-0004mq-CY@dinsdale.python.org>
References: <E1Qlk2Y-0004mq-CY@dinsdale.python.org>
Message-ID: <CAF-Rda8eJBh=mU+2RiFuq2YDwYa6GxsfDncOSma1sYuZLiC+wg@mail.gmail.com>

On Tue, Jul 26, 2011 at 18:59, jason.coombs <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/4feb889d3bed
> changeset:   71506:4feb889d3bed
> user:        Jason R. Coombs <jaraco at jaraco.com>
> date:        Tue Jul 26 11:38:04 2011 -0400
> summary:
>  Issue #10639: reindent.py tool now accepts a --newline option to specify
> the newline to be used in the output of converted files.
>

Jason, this breaks "make patchcheck". Please see my comment on the issue.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/e1370a86/attachment.html>

From eliben at gmail.com  Fri Jul 29 08:52:40 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 29 Jul 2011 09:52:40 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CADiSq7et-oS2UFEsX+uLgVo1eLCfXfe1FWsFp9J5s_3ji5bcgQ@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<CADiSq7et-oS2UFEsX+uLgVo1eLCfXfe1FWsFp9J5s_3ji5bcgQ@mail.gmail.com>
Message-ID: <CAF-Rda_pomE95a3a+Y_se6ysJgQ2bpYEViO+BT2Q6rxZ2y_aqw@mail.gmail.com>

On Fri, Jul 29, 2011 at 08:48, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Fri, Jul 29, 2011 at 3:24 PM, Eli Bendersky <eliben at gmail.com> wrote:
> > I propose to just move 3K's docs to the devguide, and make both doc pages
> > (in 3K and 2.7) point to it. Is this acceptable?
>
> Yeah, just include a note in the devguide version saying that anything
> added in 3.2 or later may not be available when writing tests for the
> 2.7 maintenance branch.
>

Alright, and what should stay in the docs then? I guess only 25.5.2 (usage
of regrtest). The rest is either documentation of support, or advice for
people writing new tests, both of which belong to the devguide.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/a3ba0cfa/attachment.html>

From ncoghlan at gmail.com  Fri Jul 29 09:10:42 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 29 Jul 2011 17:10:42 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda_pomE95a3a+Y_se6ysJgQ2bpYEViO+BT2Q6rxZ2y_aqw@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<CADiSq7et-oS2UFEsX+uLgVo1eLCfXfe1FWsFp9J5s_3ji5bcgQ@mail.gmail.com>
	<CAF-Rda_pomE95a3a+Y_se6ysJgQ2bpYEViO+BT2Q6rxZ2y_aqw@mail.gmail.com>
Message-ID: <CADiSq7fx+w+o9Su=ZfOVCGQ9fUouiWQJsKoovCtj6SnmwgK0sg@mail.gmail.com>

On Fri, Jul 29, 2011 at 4:52 PM, Eli Bendersky <eliben at gmail.com> wrote:
> On Fri, Jul 29, 2011 at 08:48, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> On Fri, Jul 29, 2011 at 3:24 PM, Eli Bendersky <eliben at gmail.com> wrote:
>> > I propose to just move 3K's docs to the devguide, and make both doc
>> > pages
>> > (in 3K and 2.7) point to it. Is this acceptable?
>>
>> Yeah, just include a note in the devguide version saying that anything
>> added in 3.2 or later may not be available when writing tests for the
>> 2.7 maintenance branch.
>
> Alright, and what should stay in the docs then? I guess only 25.5.2 (usage
> of regrtest). The rest is either documentation of support, or advice for
> people writing new tests, both of which belong to the devguide.

How to run the test suite should stay, along with a pointer to the
additional information in the devguide.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From eliben at gmail.com  Fri Jul 29 09:51:27 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 29 Jul 2011 10:51:27 +0300
Subject: [Python-Dev] Moving documentation of test.support into the devguide
Message-ID: <CAF-Rda87KZOwnLozMXBLLa3pXJR3t7+-Pd=Y0c1-6HcU62z-ew@mail.gmail.com>

I've opened issue 12652 to track this task.

Proposed workflow:

1. First, the devguide incorporates the documentation
2. Then, I'll clean up the official docs and add pointers to the devguide
instead

The transition of the documentation into the devguide also has a few steps:

1. Move the reference of the utilities test.support provides. This reference
documentation isn't complete (not all functions are documented), but can be
completed later in the devguide.
2. (Optionally) incorporate the test-writing advice given in 25.5.1 into the
devguide (which currently has only a short couple of paragraphs
3. Move any other information from the official docs (chapter 25) into the
devguide, leaving only a section on running the tests (which should be
identical to the relevant section in the devguide) and a pointer to the
devguide.


Issue #12652 now has a patch for step (1) above.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/fa359c26/attachment.html>

From barry at python.org  Fri Jul 29 11:26:51 2011
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Jul 2011 05:26:51 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
Message-ID: <20110729052651.311dbd53@resist.wooz.org>

On Jul 29, 2011, at 08:24 AM, Eli Bendersky wrote:

>Alright, I think there's now a sufficiently wide consensus to move the
>documentation of Lib/test and Lib/test/support in particular to the
>devguide, which raises a question:

I haven't been following this thread, so I caught up on Gmane.

I'm somewhat uncomfortable with this decision.  I understand why you want to
do this, but I still think it might not be the right thing.

For better or worse, test.support is part of the standard library, so moving
its documentation elsewhere sets a bad precedent to me.  Many times I work on
Python while completely off-line.  If I've had the foresight to build the docs
before I go off-line, all the better, but even if not, I have the .rst files
to consult.  Now I'll have to remember to *also* grab the devguide so that I
have the complete documentation of the stdlib.

I also think this underestimates how blurry the line is between users and
developers.  Think about yourself: when did you suddenly graduate from user of
Python to developer of Python?  I know that I often talk to people who would
consider themselves "just" users of Python, but feel comfortable with their
occasional spelunking into the Python tree looking for whatever nugget of code
or documentation will help them understand their problem.

The devguide, as useful and cool as it is, is still immature and hard to
discover.  I think more time will improve its organization, and it's not even
linked to from docs.python.org.

So I'm curious, why is this move better than adding noindexes, or just
trusting users to understand the difference between test.support.unlink() and
os.unlink()?  If I currently search for 'unlink', os.unlink comes up first,
which is good, and that should be preserved.  The presence or not of some
test.support.unlink documentation isn't going to make the search results that
much better or worse (there's already 14 hits).

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/740a53c0/attachment.pgp>

From eliben at gmail.com  Fri Jul 29 13:07:50 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 29 Jul 2011 14:07:50 +0300
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729052651.311dbd53@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
Message-ID: <CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>

> >Alright, I think there's now a sufficiently wide consensus to move the
> >documentation of Lib/test and Lib/test/support in particular to the
> >devguide, which raises a question:
>
> I haven't been following this thread, so I caught up on Gmane.
>
> I'm somewhat uncomfortable with this decision.  I understand why you want
> to
> do this, but I still think it might not be the right thing.
>
> For better or worse, test.support is part of the standard library, so
> moving
>

Why is it part of stdlib though? Isn't the stdlib something that's exposed
to all Python programmers? How should an ordinary programmer (not a core
dev) know some parts of stdlib are out of limits, if they are even
documented and appear in the index - should he read the note at the top of
the documentation page? Imagine a desperate programmer behind schedule who
is looking for some particular functionality in stdlib and finds it in
test.support - will he not use it and later complain we don't keep
backwards-compatibility. Sure, he's at fault, but I think we can make such
cases much rarer by making the distinction between user-exposed modules and
core-dev-exposed modules more explicit.


> its documentation elsewhere sets a bad precedent to me.  Many times I work
> on
> Python while completely off-line.  If I've had the foresight to build the
> docs
> before I go off-line, all the better, but even if not, I have the .rst
> files
> to consult.  Now I'll have to remember to *also* grab the devguide so that
> I
> have the complete documentation of the stdlib.
>
>
I think this is because you're experienced and probably know all the
contents of the devguide anyway. Junior core-devs need the devguide handy
for other reasons as well.


> I also think this underestimates how blurry the line is between users and
> developers.  Think about yourself: when did you suddenly graduate from user
> of
> Python to developer of Python?  I know that I often talk to people who
> would
> consider themselves "just" users of Python, but feel comfortable with their
> occasional spelunking into the Python tree looking for whatever nugget of
> code
> or documentation will help them understand their problem.
>

Isn't this what we're trying to prevent, though? One should never even have
to look at test.support unless he's working *on Python*.


>
> The devguide, as useful and cool as it is, is still immature and hard to
> discover.  I think more time will improve its organization, and it's not
> even
> linked to from docs.python.org.
>
>
Maybe this is the main point here - what makes it immature? What makes it
hard to discover? Maybe these issues can be fixed.


> So I'm curious, why is this move better than adding noindexes, or just
> trusting users to understand the difference between test.support.unlink()
> and
> os.unlink()?  If I currently search for 'unlink', os.unlink comes up first,
> which is good, and that should be preserved.  The presence or not of some
> test.support.unlink documentation isn't going to make the search results
> that
> much better or worse (there's already 14 hits).
>

I think the unlink&rmtree functions are just a symptom. The real issue here
is - what is the devguide for, and how is it different from Python's
existing documentation? What should go into the official docs, and what
should go into the devguide? Once we agree on these question, I think the
test.support dilemma will solve itself.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/26b7cd0d/attachment.html>

From jsbueno at python.org.br  Fri Jul 29 13:22:17 2011
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Fri, 29 Jul 2011 08:22:17 -0300
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <j0tddt$20k$1@dough.gmane.org>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
	<j0tddt$20k$1@dough.gmane.org>
Message-ID: <CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>

On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Brett Cannon, 28.07.2011 23:49:
>>
>> On Thu, Jul 28, 2011 at 11:25, Matt wrote:
>>>
>>> - What policies are in place for keeping parity with other HTML
>>> parsers (such as those in web browsers)?
>>
>> There aren't any beyond "it would be nice".
>> [...]
>> It's more of an issue of someone caring enough to do the coding work to
>> bring the parser up to spec for HTML5 (or introduce new code to live
>> beside
>> the HTML4 parsing code).
>
> Which, given that html5lib readily exists, would likely be a lot more work
> than anyone who is interested in HTML5 handling would want to invest.
>
> I don't think we need a new HTML5 parsing implementation only to have it in
> the stdlib. That's the old sunny Java way of doing it.
>

I disaagree.
Having proper html parsing out of the box is part of the "batteries
included" thing.
And it is not a matter of "having html 5" - as stated on this thread, fixing it
for html5 will fix it for html that exists in the "real world".

Python _has_ to work with quick 30-50 lines scripts deliverable everywhere, not
just has proper 3rd party libraries that can work as part of a huge
project using buildout.


  js
 -><-

> Stefan
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br
>

From stefan_ml at behnel.de  Fri Jul 29 13:46:54 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 29 Jul 2011 13:46:54 +0200
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
Message-ID: <j0u6je$cs1$1@dough.gmane.org>

Joao S. O. Bueno, 29.07.2011 13:22:
> On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel wrote:
>> Brett Cannon, 28.07.2011 23:49:
>>>
>>> On Thu, Jul 28, 2011 at 11:25, Matt wrote:
>>>>
>>>> - What policies are in place for keeping parity with other HTML
>>>> parsers (such as those in web browsers)?
>>>
>>> There aren't any beyond "it would be nice".
>>> [...]
>>> It's more of an issue of someone caring enough to do the coding work to
>>> bring the parser up to spec for HTML5 (or introduce new code to live
>>> beside
>>> the HTML4 parsing code).
>>
>> Which, given that html5lib readily exists, would likely be a lot more work
>> than anyone who is interested in HTML5 handling would want to invest.
>>
>> I don't think we need a new HTML5 parsing implementation only to have it in
>> the stdlib. That's the old sunny Java way of doing it.
>
> I disaagree.
> Having proper html parsing out of the box is part of the "batteries
> included" thing.

Well, you can easily prove me wrong by implementing this.

Stefan


From victor.stinner at haypocalc.com  Fri Jul 29 14:26:06 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 29 Jul 2011 14:26:06 +0200
Subject: [Python-Dev] Status of the PEP
	400?	(deprecate	codecs.StreamReader/StreamWriter)
In-Reply-To: <4E312BCB.3080301@haypocalc.com>
References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com>
	<4E312BCB.3080301@haypocalc.com>
Message-ID: <4E32A6DE.7060001@haypocalc.com>

Le 28/07/2011 11:28, Victor Stinner a ?crit :
>> Please do keep the original implementation
>> around (e.g. renamed to codecs.open_stream()), though, so that it's
>> still possible to get easy-to-use access to codec StreamReader/Writers.
>
> I will add your alternative to the PEP (except if you would like to do
> that yourself?). If I understood correctly, you propose to:
>
> * rename codecs.open() to codecs.open_stream()
> * change codecs.open() to reuse open() (and so io.TextIOWrapper)
>
> (and don't deprecate anything)

I added your proposal to the PEP as an "Alternative Approache".

Victor

From solipsis at pitrou.net  Fri Jul 29 14:46:10 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 14:46:10 +0200
Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS
	entries.
References: <E1QmmHf-0003tS-PN@dinsdale.python.org>
Message-ID: <20110729144610.5e7c7681@pitrou.net>

On Fri, 29 Jul 2011 14:35:19 +0200
eric.araujo <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/97527f3f9829
> changeset:   71555:97527f3f9829
> branch:      3.2
> user:        ?ric Araujo <merwok at netwok.org>
> date:        Tue Jul 26 17:32:50 2011 +0200
> summary:
>   Fix sorting or wording of some NEWS entries.
> 
> I would have put io and ctypes fixes into Extension Modules, but I
> respected the choice of Antoine or Victor and left them in Library.

There's no practical difference (from the user's point of view) between
extension modules and the library, so I think the "Extension Modules"
section should simply die.

Regards

Antoine.



From solipsis at pitrou.net  Fri Jul 29 14:48:43 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 14:48:43 +0200
Subject: [Python-Dev] cpython: Remove indirection in threading (issue
	#10968).
References: <E1QmmHj-0003tm-9m@dinsdale.python.org>
Message-ID: <20110729144843.218e9469@pitrou.net>

On Fri, 29 Jul 2011 14:35:23 +0200
eric.araujo <python-checkins at python.org> wrote:
> 
> It is now possible to inherit from Thread and other classes, without
> having to import the private underscored names like multiprocessing did.

A correction: it was already possible to inherit from Thread (since
it's quite useful to do so).

Regards

Antoine.



From nadeem.vawda at gmail.com  Fri Jul 29 14:59:32 2011
From: nadeem.vawda at gmail.com (Nadeem Vawda)
Date: Fri, 29 Jul 2011 14:59:32 +0200
Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS
	entries.
In-Reply-To: <20110729144610.5e7c7681@pitrou.net>
References: <E1QmmHf-0003tS-PN@dinsdale.python.org>
	<20110729144610.5e7c7681@pitrou.net>
Message-ID: <CANF4RMkhKbA8Q3GjUQROFJyR_JES2hmnxGer=taKcEbtkxuiXA@mail.gmail.com>

On Fri, Jul 29, 2011 at 2:46 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> There's no practical difference (from the user's point of view) between
> extension modules and the library, so I think the "Extension Modules"
> section should simply die.

+1. This has been bugging me for a while now.

From solipsis at pitrou.net  Fri Jul 29 14:50:45 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 14:50:45 +0200
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
Message-ID: <20110729145045.2c252ea0@pitrou.net>

On Fri, 29 Jul 2011 14:35:24 +0200
eric.araujo <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/bdad5bc9a2ed
> changeset:   71562:bdad5bc9a2ed
> branch:      3.2
> user:        ?ric Araujo <merwok at netwok.org>
> date:        Thu Jul 28 22:45:46 2011 +0200
> summary:
>   Stop ignoring Mercurial merge conflits files (#12255).
> 
> R. David Murray and I think that it?s more useful to have these files
> show up in the output of ?hg status?, to let the user know that some
> merged file have to be checked before commit.

Mercurial will prevent you from committing before you have solved
conflicts, so I'm not sure what this brings. "hg res -l" is the command
to remember when you want to list files with conflicts.

The fact that "make clean" doesn't wipe these files is an additional
annoyance.

Regards

Antoine.



From merwok at netwok.org  Fri Jul 29 15:17:12 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 29 Jul 2011 15:17:12 +0200
Subject: [Python-Dev] cpython: Remove indirection in threading (issue
 #10968).
In-Reply-To: <20110729144843.218e9469@pitrou.net>
References: <E1QmmHj-0003tm-9m@dinsdale.python.org>
	<20110729144843.218e9469@pitrou.net>
Message-ID: <4E32B2D8.2040701@netwok.org>

Le 29/07/2011 14:48, Antoine Pitrou a ?crit :
>> It is now possible to inherit from Thread and other classes, without
>> having to import the private underscored names like multiprocessing did.
> A correction: it was already possible to inherit from Thread (since
> it's quite useful to do so).

Indeed, and Timer for example inherits from Thread.  The other classes I
changed however were not subclassable (the original bug report was about
subclassing Timer, not Thread), I just picked the one wrong example when
writing ?X and other classes? :)

Regards

From merwok at netwok.org  Fri Jul 29 15:28:44 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 29 Jul 2011 15:28:44 +0200
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
In-Reply-To: <20110729145045.2c252ea0@pitrou.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net>
Message-ID: <4E32B58C.3070809@netwok.org>

Le 29/07/2011 14:50, Antoine Pitrou a ?crit :
>> changeset:   71562:bdad5bc9a2ed
>> user:        ?ric Araujo <merwok at netwok.org>
>> summary:
>>   Stop ignoring Mercurial merge conflits files (#12255).
>>
>> R. David Murray and I think that it?s more useful to have these files
>> show up in the output of ?hg status?, to let the user know that some
>> merged file have to be checked before commit.
> 
> Mercurial will prevent you from committing before you have solved
> conflicts, so I'm not sure what this brings. "hg res -l" is the command
> to remember when you want to list files with conflicts.

People confused by the merge/resolve system could exit their merge tool
without actually merging the changes (I know it happened to me!), so
these files act as a reminder that not everything is right.

.orig is also created by hg revert; my usage is that I remove this file
after I?ve checked that the revert is okay.

> The fact that "make clean" doesn't wipe these files is an additional
> annoyance.

make clean removes generated files, but *.rej and *.orig are backups,
which you may want to save or re-apply.  I?m not sure it would be right
to lose them.  Maybe distclean?

Regards

From solipsis at pitrou.net  Fri Jul 29 15:38:31 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 15:38:31 +0200
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
Message-ID: <20110729153831.455b5d67@pitrou.net>

On Fri, 29 Jul 2011 15:28:44 +0200
?ric Araujo <merwok at netwok.org> wrote:
> Le 29/07/2011 14:50, Antoine Pitrou a ?crit :
> >> changeset:   71562:bdad5bc9a2ed
> >> user:        ?ric Araujo <merwok at netwok.org>
> >> summary:
> >>   Stop ignoring Mercurial merge conflits files (#12255).
> >>
> >> R. David Murray and I think that it?s more useful to have these files
> >> show up in the output of ?hg status?, to let the user know that some
> >> merged file have to be checked before commit.
> > 
> > Mercurial will prevent you from committing before you have solved
> > conflicts, so I'm not sure what this brings. "hg res -l" is the command
> > to remember when you want to list files with conflicts.
> 
> People confused by the merge/resolve system could exit their merge tool
> without actually merging the changes (I know it happened to me!), so
> these files act as a reminder that not everything is right.

I don't know, I don't use a merge tool. But presumably the merge tool
would only call "hg resolve -m" on those files which have actually been
resolved? (if it calls that command at all)

> > The fact that "make clean" doesn't wipe these files is an additional
> > annoyance.
> 
> make clean removes generated files, but *.rej and *.orig are backups,
> which you may want to save or re-apply.

What use are these backups really? We are using a (D)VCS, you are not
losing anything.

Regards

Antoine.



From merwok at netwok.org  Fri Jul 29 15:22:08 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 29 Jul 2011 15:22:08 +0200
Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS
 entries.
In-Reply-To: <20110729144610.5e7c7681@pitrou.net>
References: <E1QmmHf-0003tS-PN@dinsdale.python.org>
	<20110729144610.5e7c7681@pitrou.net>
Message-ID: <4E32B400.9060206@netwok.org>

Le 29/07/2011 14:46, Antoine Pitrou a ?crit :
>> changeset:   71555:97527f3f9829
>> branch:      3.2
>> user:        ?ric Araujo <merwok at netwok.org>
>> summary:
>>   Fix sorting or wording of some NEWS entries.
>>
>> I would have put io and ctypes fixes into Extension Modules, but I
>> respected the choice of Antoine or Victor and left them in Library.
> 
> There's no practical difference (from the user's point of view) between
> extension modules and the library, so I think the "Extension Modules"
> section should simply die.

+1 to that.  Would built-in modules like imp belong to Library or
Interpreter Core?  (I?d say Core.)

When merging 3.2 into 3.3, I started sorting the Misc/NEWS file there
too but stopped after ~30 changes.  On one hand, I know that developer
time would be better spent on fixing bugs rather than reading a commit
sorting NEWS entries, but on the other hand I thought it best for
readers of this file to have something that makes sense.  Build fixes
were not in a dedicated section, Core and Library are happily mixed, so
I thought that my sorting could help even the What?s New author.  I
wanted to ask python-dev first, though.

Regards

From rdmurray at bitdance.com  Fri Jul 29 16:04:26 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 29 Jul 2011 10:04:26 -0400
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
	conflits files (#12255).
In-Reply-To: <4E32B58C.3070809@netwok.org>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
Message-ID: <20110729140426.A78DC2506F0@webabinitio.net>

On Fri, 29 Jul 2011 15:28:44 +0200, =?UTF-8?B?w4lyaWMgQXJhdWpv?= <merwok at netwok.org> wrote:
> Le 29/07/2011 14:50, Antoine Pitrou a ??crit :
> >> changeset:   71562:bdad5bc9a2ed
> >> user:        ??ric Araujo <merwok at netwok.org>
> >> summary:
> >>   Stop ignoring Mercurial merge conflits files (#12255).
> >>
> >> R. David Murray and I think that it???s more useful to have these files
> >> show up in the output of ???hg status???, to let the user know that some
> >> merged file have to be checked before commit.
> >
> > Mercurial will prevent you from committing before you have solved
> > conflicts, so I'm not sure what this brings. "hg res -l" is the command
> > to remember when you want to list files with conflicts.
> 
> People confused by the merge/resolve system could exit their merge tool
> without actually merging the changes (I know it happened to me!), so
> these files act as a reminder that not everything is right.

Or people who decide the particular merge tool isn't going to help in
a particular case.  I do that reasonably often.

> .orig is also created by hg revert; my usage is that I remove this file
> after I???ve checked that the revert is okay.

And by patch.  This is my major reason for disliking having these files
ignored.  I want to know about the .rej and .orig files patch generates
that I may have forgotten about, and not just to clean them up.  Their
existence is a signal that there's something I haven't finished working
through (or that I at least need to check that I worked through it
before deleting them).

> > The fact that "make clean" doesn't wipe these files is an additional
> > annoyance.
> 
> make clean removes generated files, but *.rej and *.orig are backups,
> which you may want to save or re-apply.  I???m not sure it would be right
> to lose them.  Maybe distclean?

Exactly.  Which is another reason these files should not be ignored:
even distclean should probably not remove the backup files generated
by patch, which means that hg status should show the files because they
aren't part of the distribution.

Which means (assuming there is agreement with my logic :) that make
distclean probably *should* remove the stuff generated by coverage,
given that we are ignoring that stuff now.

--David

From merwok at netwok.org  Fri Jul 29 16:21:42 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 29 Jul 2011 16:21:42 +0200
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
In-Reply-To: <20110729153831.455b5d67@pitrou.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>	<20110729145045.2c252ea0@pitrou.net>
	<4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net>
Message-ID: <4E32C1F6.4050204@netwok.org>

>> People confused by the merge/resolve system could exit their merge tool
>> without actually merging the changes (I know it happened to me!), so
>> these files act as a reminder that not everything is right.
> 
> I don't know, I don't use a merge tool. But presumably the merge tool
> would only call "hg resolve -m" on those files which have actually been
> resolved? (if it calls that command at all)

You have it backwards: it?s hg resolve that runs a merge tool.  (BTW,
how can you not use a merge tool?  Even if you use internal hg tools
like the one that adds merge conflict markers, that?s a merge tool.)

> What use are these backups really? We are using a (D)VCS, you are not
> losing anything.

The .orig files after a revert could contain code that?s not committed
anywhere.  See also RDM?s reply to your message.

Regards

From rdmurray at bitdance.com  Fri Jul 29 16:50:57 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 29 Jul 2011 10:50:57 -0400
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
	conflits files (#12255).
In-Reply-To: <20110729153831.455b5d67@pitrou.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
	<20110729153831.455b5d67@pitrou.net>
Message-ID: <20110729145058.22F732506F0@webabinitio.net>

On Fri, 29 Jul 2011 15:38:31 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 29 Jul 2011 15:28:44 +0200 > ??ric Araujo <merwok at netwok.org> wrote:
> > make clean removes generated files, but *.rej and *.orig are backups,
> > which you may want to save or re-apply.
> 
> What use are these backups really? We are using a (D)VCS, you are not
> losing anything.

As ??ric said, the .orig files may contain uncommitted changes.  The
.rej files are information about what parts of the patch were
*not* applied, which is information that is not retained
anywhere else.

--David

From solipsis at pitrou.net  Fri Jul 29 16:49:35 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 16:49:35 +0200
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
	<20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org>
Message-ID: <20110729164935.2dde6219@pitrou.net>

On Fri, 29 Jul 2011 16:21:42 +0200
?ric Araujo <merwok at netwok.org> wrote:
> > What use are these backups really? We are using a (D)VCS, you are not
> > losing anything.
> 
> The .orig files after a revert could contain code that?s not committed
> anywhere.  See also RDM?s reply to your message.

I would point out that by using "hg revert", you deliberately want to
forget some local changes.

Besides, "hg status" is meant to show untracked files which could
*potentially* be tracked. It's not like anybody wants to track .orig
and .rej files, so having them in the ignore list is still the right
thing to do.

Regards

Antoine.



From solipsis at pitrou.net  Fri Jul 29 16:55:03 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 16:55:03 +0200
Subject: [Python-Dev] cpython (3.2): Fix sorting or wording of some NEWS
	entries.
References: <E1QmmHf-0003tS-PN@dinsdale.python.org>
	<20110729144610.5e7c7681@pitrou.net> <4E32B400.9060206@netwok.org>
Message-ID: <20110729165503.2f25d692@pitrou.net>

On Fri, 29 Jul 2011 15:22:08 +0200
?ric Araujo <merwok at netwok.org> wrote:
> > 
> > There's no practical difference (from the user's point of view) between
> > extension modules and the library, so I think the "Extension Modules"
> > section should simply die.
> 
> +1 to that.  Would built-in modules like imp belong to Library or
> Interpreter Core?  (I?d say Core.)

As soon as you are fixing a library API, it should IMO go to Library.
The fact that some modules are "built-in" is an implementation detail.
For example, "_io" is compiled in the executable (for bootstrap
reasons), yet it's still considered part of the library.

> When merging 3.2 into 3.3, I started sorting the Misc/NEWS file there
> too but stopped after ~30 changes.  On one hand, I know that developer
> time would be better spent on fixing bugs rather than reading a commit
> sorting NEWS entries, but on the other hand I thought it best for
> readers of this file to have something that makes sense.  Build fixes
> were not in a dedicated section, Core and Library are happily mixed, so
> I thought that my sorting could help even the What?s New author.  I
> wanted to ask python-dev first, though.

Well, sorting is fine, if you don't think your developer time could be
used for more valuable tasks :)

Regards

Antoine.



From ncoghlan at gmail.com  Fri Jul 29 17:02:31 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 30 Jul 2011 01:02:31 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729052651.311dbd53@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
Message-ID: <CADiSq7c7zJNB_ARAxtvb4dmFrmP_mJqVEgTQYMSmxFdkRWvi6Q@mail.gmail.com>

On Fri, Jul 29, 2011 at 7:26 PM, Barry Warsaw <barry at python.org> wrote:
> The devguide, as useful and cool as it is, is still immature and hard to
> discover. ?I think more time will improve its organization, and it's not even
> linked to from docs.python.org.
>
> So I'm curious, why is this move better than adding noindexes, or just
> trusting users to understand the difference between test.support.unlink() and
> os.unlink()? ?If I currently search for 'unlink', os.unlink comes up first,
> which is good, and that should be preserved. ?The presence or not of some
> test.support.unlink documentation isn't going to make the search results that
> much better or worse (there's already 14 hits).

It's worthwhile because it is what the devguide is for: documenting
how to *change* Python, rather than just using it as it is delivered
to you. There's a clear transition from user of Python to developer of
Python: you stop treating the standard library (and perhaps even the
interpreter core) as sacrosanct, and will actually change the code in
those files if its wrong (and is affecting your own application).

Agreed that docs.python.org front page should have a link to
docs.python.org/devguide, though.

Cheers.
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From barry at python.org  Fri Jul 29 17:18:37 2011
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Jul 2011 11:18:37 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
Message-ID: <20110729111837.4f5b88cd@resist.wooz.org>

On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote:

>Why is it part of stdlib though? Isn't the stdlib something that's exposed
>to all Python programmers? How should an ordinary programmer (not a core
>dev) know some parts of stdlib are out of limits, if they are even
>documented and appear in the index - should he read the note at the top of
>the documentation page? Imagine a desperate programmer behind schedule who
>is looking for some particular functionality in stdlib and finds it in
>test.support - will he not use it and later complain we don't keep
>backwards-compatibility. Sure, he's at fault, but I think we can make such
>cases much rarer by making the distinction between user-exposed modules and
>core-dev-exposed modules more explicit.

How common are those mistakes right now?  I get way more complaints about
squishy APIs in third party libraries and actual *supported* parts of the
stdlib than I do about test.support APIs.

What's the real problem we're trying to solve anyway?  Is it "protect the
harried user from some future breakage" or "eliminate complaints we Python
developers get from harried users"?

I'd much rather solve this problem by adding markup to functions that
explicitly disclaim our normal backward compatibility guarantees.  Squirreling
away documentation for some parts of the stdlib seems similar to
security-by-obscurity arguments.

test.support *is* part of the stdlib.  If you want to propose moving it out of
the stdlib, that's a different argument to make.  Maybe Python's entire test
suite should be in some Cheeseshop package instead of in the core source
tree.  (I wouldn't agree with that either, but I'm just saying.)

So once again, is this a real actual problem you've witnessed in enough cases
to go through all the trouble of moving the documentation, and making it more
difficult to find?

>> its documentation elsewhere sets a bad precedent to me.  Many times I work
>> on Python while completely off-line.  If I've had the foresight to build
>> the docs before I go off-line, all the better, but even if not, I have the
>> .rst files to consult.  Now I'll have to remember to *also* grab the
>> devguide so that I have the complete documentation of the stdlib.
>>
>I think this is because you're experienced and probably know all the
>contents of the devguide anyway. Junior core-devs need the devguide handy
>for other reasons as well.

I'm not saying the devguide isn't handy - in fact it's incredibly handy!  I
love that it explains the nuances of how we structure our Mercurial branches,
and define our workflow for example.  I readily admit those are not things I
know by heart. (But actually, those instructions are not always easy to find,
which is why I think the organization is immature.  I'm not faulting the
devguide authors, I just think it needs more time to bake.)

>> I also think this underestimates how blurry the line is between users and
>> developers.  Think about yourself: when did you suddenly graduate from user
>> of Python to developer of Python?  I know that I often talk to people who
>> would consider themselves "just" users of Python, but feel comfortable with
>> their occasional spelunking into the Python tree looking for whatever
>> nugget of code or documentation will help them understand their problem.
>
>Isn't this what we're trying to prevent, though? One should never even have
>to look at test.support unless he's working *on Python*.

Again, I think that line is blurred here.  Let's say you're working on some
off-beat or new hardware and you want to know if your basic tool chain works
for it.  You build Python, run the test suite, and something fails.  You
probably don't consider yourself a Python developer, but now you're digging
through the test.support to figure out your problem.  Yes, I've seen this
happen.

>> The devguide, as useful and cool as it is, is still immature and hard to
>> discover.  I think more time will improve its organization, and it's not
>> even linked to from docs.python.org.
>>
>Maybe this is the main point here - what makes it immature? What makes it
>hard to discover? Maybe these issues can be fixed.

Sure, there are issues that can be fixed, but they're unrelated to documenting
test.support.

>> So I'm curious, why is this move better than adding noindexes, or just
>> trusting users to understand the difference between test.support.unlink()
>> and os.unlink()?  If I currently search for 'unlink', os.unlink comes up
>> first, which is good, and that should be preserved.  The presence or not of
>> some test.support.unlink documentation isn't going to make the search
>> results that much better or worse (there's already 14 hits).
>>
>
>I think the unlink&rmtree functions are just a symptom. The real issue here
>is - what is the devguide for, and how is it different from Python's
>existing documentation? What should go into the official docs, and what
>should go into the devguide? Once we agree on these question, I think the
>test.support dilemma will solve itself.

Let's see!

I think the devguide should document things like "how to submit bugs", "how to
use Mercurial", "how to propose changes to Python", "how to ensure code works
across all existing interpreter implementations", "where to find continuous
integration results and how to interpret them", "what's the right forum to
discuss Python", etc.  I.e. processes, workflows, and conventions that are
important cultural artifacts we've built up over 20 years.

I don't think the devguide should document the actual code we ship.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/4178ed5a/attachment.pgp>

From solipsis at pitrou.net  Fri Jul 29 17:17:31 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 17:17:31 +0200
Subject: [Python-Dev] Status of the PEP 400? (deprecate
 codecs.StreamReader/StreamWriter)
References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com>
	<4E312BCB.3080301@haypocalc.com>
Message-ID: <20110729171731.2059cc3e@pitrou.net>

On Thu, 28 Jul 2011 11:28:43 +0200
Victor Stinner <victor.stinner at haypocalc.com> wrote:
> 
> I will add your alternative to the PEP (except if you would like to do 
> that yourself?). If I understood correctly, you propose to:
> 
>   * rename codecs.open() to codecs.open_stream()
>   * change codecs.open() to reuse open() (and so io.TextIOWrapper)
> 
> (and don't deprecate anything)

This may be an interesting approach. In a few years, we can evaluate
whether users are calling open_stream(), and if there aren't any, we
can deprecate the whole thing.

Regards

Antoine.



From solipsis at pitrou.net  Fri Jul 29 17:25:00 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 17:25:00 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
Message-ID: <20110729172500.67844af4@pitrou.net>

On Fri, 29 Jul 2011 11:18:37 -0400
Barry Warsaw <barry at python.org> wrote:
> 
> I'd much rather solve this problem by adding markup to functions that
> explicitly disclaim our normal backward compatibility guarantees.  Squirreling
> away documentation for some parts of the stdlib seems similar to
> security-by-obscurity arguments.

Well, here the whole module should not be called by the user. I'm not
sure we want to flag each function individually.

> test.support *is* part of the stdlib.

We have lots of internal APIs which are not documented, though.
And test.support *is* for internal use.

Regards

Antoine.



From ethan at stoneleaf.us  Fri Jul 29 17:42:45 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 29 Jul 2011 08:42:45 -0700
Subject: [Python-Dev] Convention on functions that shadow
 existing	stdlib functions
In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>	<20110727152410.22871784@pitrou.net>	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>	<4E30A4F1.6050305@pearwood.info>	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>	<20110729052651.311dbd53@resist.wooz.org>	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
Message-ID: <4E32D4F5.505@stoneleaf.us>

Barry Warsaw wrote:
> On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote:
>> I think the unlink&rmtree functions are just a symptom. The real issue here
>> is - what is the devguide for, and how is it different from Python's
>> existing documentation? What should go into the official docs, and what
>> should go into the devguide? Once we agree on these question, I think the
>> test.support dilemma will solve itself.
> 
> Let's see!
> 
> I think the devguide should document things like "how to submit bugs", "how to
> use Mercurial", "how to propose changes to Python", "how to ensure code works
> across all existing interpreter implementations", "where to find continuous
> integration results and how to interpret them", "what's the right forum to
> discuss Python", etc.  I.e. processes, workflows, and conventions that are
> important cultural artifacts we've built up over 20 years.
> 
> I don't think the devguide should document the actual code we ship.

+1

~Ethan~

From ncoghlan at gmail.com  Fri Jul 29 17:32:00 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 30 Jul 2011 01:32:00 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
Message-ID: <CADiSq7dq9zsBeUcjOfn4qRGVe9GhQiqeiY8-nmnXOW8SOsdJ3w@mail.gmail.com>

On Sat, Jul 30, 2011 at 1:18 AM, Barry Warsaw <barry at python.org> wrote:
> On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote:
>>I think the unlink&rmtree functions are just a symptom. The real issue here
>>is - what is the devguide for, and how is it different from Python's
>>existing documentation? What should go into the official docs, and what
>>should go into the devguide? Once we agree on these question, I think the
>>test.support dilemma will solve itself.
>
> Let's see!
>
> I think the devguide should document things like "how to submit bugs", "how to
> use Mercurial", "how to propose changes to Python", "how to ensure code works
> across all existing interpreter implementations", "where to find continuous
> integration results and how to interpret them", "what's the right forum to
> discuss Python", etc. ?I.e. processes, workflows, and conventions that are
> important cultural artifacts we've built up over 20 years.
>
> I don't think the devguide should document the actual code we ship.

I think everything related to *changing* Python should be in the
devguide, not the standard library docs. So the documentation on how
to *run* the test suite belongs in the devguide, but the details of
how the test suite works internally, including the APIs that are used
to write new tests, belong in the dev guide.

Now, perhaps a copy of the dev guide should be bundled with all future
releases rather than relying on the availability of a net connection
to access wart, but finally getting rid of the dodgy test.support
sort-of-docs out of the standard library docs is an excellent change.

As far as improving the arrangement goes, the checkin privileges are
the same as those for the main source tree - patches welcome :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Fri Jul 29 17:33:53 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 30 Jul 2011 01:33:53 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <CADiSq7dq9zsBeUcjOfn4qRGVe9GhQiqeiY8-nmnXOW8SOsdJ3w@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<CADiSq7dq9zsBeUcjOfn4qRGVe9GhQiqeiY8-nmnXOW8SOsdJ3w@mail.gmail.com>
Message-ID: <CADiSq7eitg4ms=9MJv7XZqaM6V_FKYbzNxO0xUDVSm+2gMPa8w@mail.gmail.com>

On Sat, Jul 30, 2011 at 1:32 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> So the documentation on how
> to *run* the test suite belongs in the devguide, but the details of
> how the test suite works internally, including the APIs that are used
> to write new tests, belong in the dev guide.

Gah, that first instance of 'the devguide' should read 'the standard
library docs'.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Fri Jul 29 17:37:46 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 30 Jul 2011 01:37:46 +1000
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <20110729171731.2059cc3e@pitrou.net>
References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com>
	<4E312BCB.3080301@haypocalc.com>
	<20110729171731.2059cc3e@pitrou.net>
Message-ID: <CADiSq7cpLRYHe8JSEC1W-Ff6rzsYk1x1c2MPjcaei-FA1PcqvQ@mail.gmail.com>

On Sat, Jul 30, 2011 at 1:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 28 Jul 2011 11:28:43 +0200
> Victor Stinner <victor.stinner at haypocalc.com> wrote:
>>
>> I will add your alternative to the PEP (except if you would like to do
>> that yourself?). If I understood correctly, you propose to:
>>
>> ? * rename codecs.open() to codecs.open_stream()
>> ? * change codecs.open() to reuse open() (and so io.TextIOWrapper)
>>
>> (and don't deprecate anything)
>
> This may be an interesting approach. In a few years, we can evaluate
> whether users are calling open_stream(), and if there aren't any, we
> can deprecate the whole thing.

Indeed. I'm also heavily influenced by MAL's opinion on this
particular topic, so the fact he's OK with this approach counts for a
lot. It achieves the main benefit I'm interested in (transparently
migrating users of the codecs.open API to the new IO stack), while
paving the way for eliminating the redundancy at some point in the
future.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From v+python at g.nevcal.com  Fri Jul 29 17:37:51 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 29 Jul 2011 08:37:51 -0700
Subject: [Python-Dev] Convention on functions that shadow existing
 stdlib functions
In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
Message-ID: <4E32D3CF.40107@g.nevcal.com>

On 7/29/2011 8:18 AM, Barry Warsaw wrote:
> I think the devguide should document things like
...
>   "how to ensure code works
> across all existing interpreter implementations", "where to find continuous
> integration results and how to interpret them"
...
> I don't think the devguide should document the actual code we ship.

Seems to me that test.support documentation helps explain the above items.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/2adddbbf/attachment.html>

From barry at python.org  Fri Jul 29 17:49:01 2011
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Jul 2011 11:49:01 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
 stdlib functions
In-Reply-To: <CADiSq7c7zJNB_ARAxtvb4dmFrmP_mJqVEgTQYMSmxFdkRWvi6Q@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CADiSq7c7zJNB_ARAxtvb4dmFrmP_mJqVEgTQYMSmxFdkRWvi6Q@mail.gmail.com>
Message-ID: <20110729114901.1fbcbd12@resist.wooz.org>

On Jul 30, 2011, at 01:02 AM, Nick Coghlan wrote:

>It's worthwhile because it is what the devguide is for: documenting
>how to *change* Python, rather than just using it as it is delivered
>to you. There's a clear transition from user of Python to developer of
>Python: you stop treating the standard library (and perhaps even the
>interpreter core) as sacrosanct, and will actually change the code in
>those files if its wrong (and is affecting your own application).

I'm not so sure the line's that bright.  But anyway...

Using the criteria in your first sentence, test.support is clearly "what's
delivered to you" not "changing Python".  IOW, we ship test.support in the
tarball, so *if* it's to be documented, then the stdlib is the right place to
document it, IMO.

If test.support is truly and only an internal implementation detail, then it
should adhere to Pythonic convention for such things, and be renamed
test._support.  Then you won't need to document it at all except in the
module.

Here's another problem with moving the documentation for test.support to the
devguide.  They will invariably get out of sync.  It's hard enough to keep
stdlib code and stdlib docs in sync, but I think it will be even harder to
keep stdlib code in sync with devguide documentation.  It will also be harder
to know what version of the devguide corresponds to what version of the code
repository.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/724d320f/attachment.pgp>

From barry at python.org  Fri Jul 29 17:51:18 2011
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Jul 2011 11:51:18 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
 stdlib functions
In-Reply-To: <20110729172500.67844af4@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
Message-ID: <20110729115118.35a5c640@resist.wooz.org>

On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote:

>> test.support *is* part of the stdlib.
>
>We have lots of internal APIs which are not documented, though.
>And test.support *is* for internal use.

The solution then is to rename test.support to test._support to make it clear
it's an internal implementation detail.  Then you can remove the entire
section from the stdlib docs and just document it in the code.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/51bf2031/attachment.pgp>

From status at bugs.python.org  Fri Jul 29 18:07:27 2011
From: status at bugs.python.org (Python tracker)
Date: Fri, 29 Jul 2011 18:07:27 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110729160727.747221CDCC@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-07-22 - 2011-07-29)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2889 ( +3)
  closed 21547 (+40)
  total  24436 (+43)

Open issues with patches: 1259 


Issues opened (31)
==================

#12612: Valgrind suppressions
http://bugs.python.org/issue12612  opened by Paul.Price

#12613: itertools fixer fails
http://bugs.python.org/issue12613  opened by VPeric

#12614: Allow to explicitly set the method of urllib.request.Request
http://bugs.python.org/issue12614  opened by tebeka

#12616: zip fixer fails on zip()[:-1]
http://bugs.python.org/issue12616  opened by VPeric

#12617: Mutable Sequence Type can work not only with iterable in slice
http://bugs.python.org/issue12617  opened by py.user

#12618: py_compile cannot create files in current directory
http://bugs.python.org/issue12618  opened by sjdv1982

#12619: Automatically regenerate platform-specific modules
http://bugs.python.org/issue12619  opened by Arfrever

#12622: failfast argument to TextTestRunner not documented
http://bugs.python.org/issue12622  opened by pitrou

#12623: "universal newlines" subprocess support broken with select- an
http://bugs.python.org/issue12623  opened by pitrou

#12625: sporadic test_unittest failure
http://bugs.python.org/issue12625  opened by pitrou

#12626: run test cases based on a glob filter
http://bugs.python.org/issue12626  opened by pitrou

#12627: Implement PEP 394: The "python" Command on Unix-Like Systems
http://bugs.python.org/issue12627  opened by Kerrick.Staley

#12629: HTMLParser silently stops parsing with malformed attributes
http://bugs.python.org/issue12629  opened by teoryn

#12631: Mutable Sequence Type in .remove() is consistent only with lis
http://bugs.python.org/issue12631  opened by py.user

#12632: Windows GPF with Code Page 65001
http://bugs.python.org/issue12632  opened by bferris57

#12633: sys.modules doc entry should reflect restrictions
http://bugs.python.org/issue12633  opened by ericsnow

#12634: Random Remarks in class documentation
http://bugs.python.org/issue12634  opened by ats.engg

#12636: IDLE ignores -*- coding -*- with -r option
http://bugs.python.org/issue12636  opened by ledave123

#12638: urllib.URLopener prematurely deletes files on cleanup
http://bugs.python.org/issue12638  opened by carlbook

#12639: msilib Directory.start_component() fails if keyfile is not Non
http://bugs.python.org/issue12639  opened by john.keeping

#12640: test_ctypes seg fault (test_callback_register_double); armv7; 
http://bugs.python.org/issue12640  opened by python272

#12641: Remove -mno-cygwin from distutils
http://bugs.python.org/issue12641  opened by jonforums

#12643: code.InteractiveConsole ignores sys.excepthook
http://bugs.python.org/issue12643  opened by aliles

#12645: test.support. import_fresh_module - incorrect doc
http://bugs.python.org/issue12645  opened by eli.bendersky

#12646: zlib.Decompress.decompress/flush do not raise any exceptions w
http://bugs.python.org/issue12646  opened by chortos

#12648: Wrong import module search order on Windows
http://bugs.python.org/issue12648  opened by kota

#12649: email.Header ignores maxlinelen when wrapping encoded words
http://bugs.python.org/issue12649  opened by dandre

#12650: Subprocess leaks fd upon kill()
http://bugs.python.org/issue12650  opened by gabriele.trombetti

#12652: Move documentation of test.support into the devguide
http://bugs.python.org/issue12652  opened by eli.bendersky

#12653: Provide accelerators for all buttons in Windows installers
http://bugs.python.org/issue12653  opened by cool-RR

#12654: sum() works with bytes objects
http://bugs.python.org/issue12654  opened by pitrou



Most recent 15 issues with no replies (15)
==========================================

#12654: sum() works with bytes objects
http://bugs.python.org/issue12654

#12653: Provide accelerators for all buttons in Windows installers
http://bugs.python.org/issue12653

#12648: Wrong import module search order on Windows
http://bugs.python.org/issue12648

#12645: test.support. import_fresh_module - incorrect doc
http://bugs.python.org/issue12645

#12640: test_ctypes seg fault (test_callback_register_double); armv7; 
http://bugs.python.org/issue12640

#12639: msilib Directory.start_component() fails if keyfile is not Non
http://bugs.python.org/issue12639

#12623: "universal newlines" subprocess support broken with select- an
http://bugs.python.org/issue12623

#12622: failfast argument to TextTestRunner not documented
http://bugs.python.org/issue12622

#12616: zip fixer fails on zip()[:-1]
http://bugs.python.org/issue12616

#12613: itertools fixer fails
http://bugs.python.org/issue12613

#12612: Valgrind suppressions
http://bugs.python.org/issue12612

#12599: Use 'is not None' where appropriate in importlib
http://bugs.python.org/issue12599

#12598: Move sys variable initialization from import.c to sysmodule.c
http://bugs.python.org/issue12598

#12586: Enhanced email API: header objects
http://bugs.python.org/issue12586

#12562: calling mmap twice fails on Windows
http://bugs.python.org/issue12562



Most recent 15 issues waiting for review (15)
=============================================

#12652: Move documentation of test.support into the devguide
http://bugs.python.org/issue12652

#12650: Subprocess leaks fd upon kill()
http://bugs.python.org/issue12650

#12646: zlib.Decompress.decompress/flush do not raise any exceptions w
http://bugs.python.org/issue12646

#12639: msilib Directory.start_component() fails if keyfile is not Non
http://bugs.python.org/issue12639

#12633: sys.modules doc entry should reflect restrictions
http://bugs.python.org/issue12633

#12627: Implement PEP 394: The "python" Command on Unix-Like Systems
http://bugs.python.org/issue12627

#12626: run test cases based on a glob filter
http://bugs.python.org/issue12626

#12625: sporadic test_unittest failure
http://bugs.python.org/issue12625

#12619: Automatically regenerate platform-specific modules
http://bugs.python.org/issue12619

#12618: py_compile cannot create files in current directory
http://bugs.python.org/issue12618

#12614: Allow to explicitly set the method of urllib.request.Request
http://bugs.python.org/issue12614

#12605: Enhancements to gdb 7 debugging hooks
http://bugs.python.org/issue12605

#12604: VTRACE macro in _sre.c should use do {} while (0)
http://bugs.python.org/issue12604

#12598: Move sys variable initialization from import.c to sysmodule.c
http://bugs.python.org/issue12598

#12586: Enhanced email API: header objects
http://bugs.python.org/issue12586



Top 10 most discussed issues (10)
=================================

#12540: "Restart Shell" command leaves pythonw.exe processes running
http://bugs.python.org/issue12540  18 msgs

#12463: Calling SocketServer.shutdown() when server_forever() was not 
http://bugs.python.org/issue12463  15 msgs

#670664: HTMLParser.py - more robust SCRIPT tag parsing
http://bugs.python.org/issue670664  13 msgs

#12464: tempfile.TemporaryDirectory.cleanup follows symbolic links
http://bugs.python.org/issue12464  11 msgs

#12394: packaging: generate scripts from callable (dotted paths)
http://bugs.python.org/issue12394  10 msgs

#12632: Windows GPF with Code Page 65001
http://bugs.python.org/issue12632  10 msgs

#11049: add tests for test.support
http://bugs.python.org/issue11049   9 msgs

#12618: py_compile cannot create files in current directory
http://bugs.python.org/issue12618   8 msgs

#12641: Remove -mno-cygwin from distutils
http://bugs.python.org/issue12641   8 msgs

#12649: email.Header ignores maxlinelen when wrapping encoded words
http://bugs.python.org/issue12649   8 msgs



Issues closed (38)
==================

#1813: Codec lookup failing under turkish locale
http://bugs.python.org/issue1813  closed by pitrou

#4506: 3.0 make test failures on Solaris 10
http://bugs.python.org/issue4506  closed by terry.reedy

#8887: "pydoc str" works but not "pydoc str.translate"
http://bugs.python.org/issue8887  closed by eric.araujo

#9723: Add shlex.quote
http://bugs.python.org/issue9723  closed by eric.araujo

#10883: urllib: socket is not closed explicitly
http://bugs.python.org/issue10883  closed by nadeem.vawda

#11409: pysetup --search should return non-zero when a dist is not ins
http://bugs.python.org/issue11409  closed by eric.araujo

#11439: subversion keyword breakage
http://bugs.python.org/issue11439  closed by python-dev

#11784: multiprocessing.Process.join: timeout argument doesn't specify
http://bugs.python.org/issue11784  closed by neologix

#11871: test_default_timeout() of test_threading.BarrierTests failure:
http://bugs.python.org/issue11871  closed by neologix

#12043: Update shutil documentation
http://bugs.python.org/issue12043  closed by eric.araujo

#12102: mmap requires file to be synced
http://bugs.python.org/issue12102  closed by rosslagerwall

#12255: A few changes to .*ignore
http://bugs.python.org/issue12255  closed by eric.araujo

#12381: refactor slice checks made by methods that take "slice like" a
http://bugs.python.org/issue12381  closed by petri.lehtinen

#12439: BaseHTTPServer's send_reponse adds extra "\r\n" when using HTT
http://bugs.python.org/issue12439  closed by petri.lehtinen

#12514: timeit disables garbage collection if timed code raises an exc
http://bugs.python.org/issue12514  closed by rhettinger

#12542: Remove duplicates of cmp_to_key used in tests
http://bugs.python.org/issue12542  closed by eric.araujo

#12560: libpython.so not built on OpenBSD
http://bugs.python.org/issue12560  closed by neologix

#12576: urlib.request fails to open some sites
http://bugs.python.org/issue12576  closed by python-dev

#12581: Increased test coverage of test_urlparse
http://bugs.python.org/issue12581  closed by python-dev

#12590: First line and cursor not visible when opening files
http://bugs.python.org/issue12590  closed by ned.deily

#12591: TextIOWrapper should fall back on read() if read1() doesn't ex
http://bugs.python.org/issue12591  closed by pitrou

#12592: modify configure.in to detect OpenBSD 5.x
http://bugs.python.org/issue12592  closed by neologix

#12595: python.h redundant redeclarations
http://bugs.python.org/issue12595  closed by petri.lehtinen

#12603: pydoc.synopsis breaks if filesystem returns mtime of 0
http://bugs.python.org/issue12603  closed by neologix

#12607: subprocess(stdout=..., stderr=sys.stdout) breaks stderr for ch
http://bugs.python.org/issue12607  closed by rosslagerwall

#12610: Fatal Python error: non-string found in code slot
http://bugs.python.org/issue12610  closed by benjamin.peterson

#12615: add array.zeroes
http://bugs.python.org/issue12615  closed by rhettinger

#12620: pending calls: make `pendingbusy` flag static to Py_MakePendin
http://bugs.python.org/issue12620  closed by neologix

#12621: Errors in docstrings of find and rfind methods of bytes and by
http://bugs.python.org/issue12621  closed by python-dev

#12624: failfast support for regrtest
http://bugs.python.org/issue12624  closed by pitrou

#12628: urllib.request.urlopen gives empty response bodies for some si
http://bugs.python.org/issue12628  closed by ezio.melotti

#12630: pydoc does not remove '#'-s from doc comments
http://bugs.python.org/issue12630  closed by ncoghlan

#12635: use "as" for block scope support
http://bugs.python.org/issue12635  closed by ezio.melotti

#12637: logging lastResort handler not ignoring messages less than WAR
http://bugs.python.org/issue12637  closed by vinay.sajip

#12642: Python2 documentation of the  open()  built-in function
http://bugs.python.org/issue12642  closed by ezio.melotti

#12644: document the "%a" conversion in the old string formatting oper
http://bugs.python.org/issue12644  closed by eli.bendersky

#12647: Add __bool__ to None
http://bugs.python.org/issue12647  closed by rhettinger

#12651: -O2 or -O3? this brings excitement to computing!
http://bugs.python.org/issue12651  closed by benjamin.peterson

From rdmurray at bitdance.com  Fri Jul 29 18:13:44 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 29 Jul 2011 12:13:44 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
Message-ID: <20110729161345.0EE592506F0@webabinitio.net>

On Fri, 29 Jul 2011 11:18:37 -0400, Barry Warsaw <barry at python.org> wrote:
> On Jul 29, 2011, at 02:07 PM, Eli Bendersky wrote:
> >Isn't this what we're trying to prevent, though? One should never even have
> >to look at test.support unless he's working *on Python*.
> 
> Again, I think that line is blurred here.  Let's say you're working on some
> off-beat or new hardware and you want to know if your basic tool chain works
> for it.  You build Python, run the test suite, and something fails.  You
> probably don't consider yourself a Python developer, but now you're digging
> through the test.support to figure out your problem.  Yes, I've seen this
> happen.

In that case, you are working *on Python*.  Not using Python.
Personally, I'd expect such information to be in a devguide, although
really I'd not expect it to be documented at all, for most projects.
Making the devguide a more prominent part of the documentation would be
good: it would invite more people to cross that line and help us improve
Python.

As someone else pointed out, there is lots of stuff in the stdlib that
is not public API, and therefore not documented.  The problem with
test.support is that it is not a public API, but *is* documented.  So
logically either we should remove the documentation, or we should not
only do as you suggested by marking each function as not-a-stable-API,
but also document as many of the other internal APIs as we care to in
the same way.  That would doubtless help our users, but more care than
just marking functions as unstable would be required, I think.

Personally, I always thought the devguide should be part of Docs anyway.
It isn't because people didn't want it versioned along side Python,
but I still don't see why that would be a problem.

--
R. David Murray           http://www.bitdance.com

From rdmurray at bitdance.com  Fri Jul 29 18:19:45 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 29 Jul 2011 12:19:45 -0400
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
	conflits files (#12255).
In-Reply-To: <20110729164935.2dde6219@pitrou.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
	<20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org>
	<20110729164935.2dde6219@pitrou.net>
Message-ID: <20110729161946.559ED2506F0@webabinitio.net>

On Fri, 29 Jul 2011 16:49:35 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 29 Jul 2011 16:21:42 +0200 <merwok at netwok.org> wrote:
> > > What use are these backups really? We are using a (D)VCS, you are not
> > > losing anything.
> >
> > The .orig files after a revert could contain code that???s not committed
> > anywhere.  See also RDM???s reply to your message.
> 
> I would point out that by using "hg revert", you deliberately want to
> forget some local changes.

But *I'm* talking about 'patch'.  That has nothing to do with either
merge or revert.

> Besides, "hg status" is meant to show untracked files which could
> *potentially* be tracked. It's not like anybody wants to track .orig
> and .rej files, so having them in the ignore list is still the right
> thing to do.

That's one view.  My view is that 'hg status' tells me all the files
that have appeared in my tree that I'm either not currently tracking or
explicitly ignoring (because the project's automated tools will deal
with them).  Nothing in there about limiting it to files I *might*
want to track.  That is how I've always used my version control
systems.

--
R. David Murray           http://www.bitdance.com

From guido at python.org  Fri Jul 29 19:01:06 2011
From: guido at python.org (Guido van Rossum)
Date: Fri, 29 Jul 2011 10:01:06 -0700
Subject: [Python-Dev] Status of the PEP 400? (deprecate
	codecs.StreamReader/StreamWriter)
In-Reply-To: <CADiSq7cpLRYHe8JSEC1W-Ff6rzsYk1x1c2MPjcaei-FA1PcqvQ@mail.gmail.com>
References: <4E308D63.9090901@haypocalc.com> <4E3125D7.2030103@egenix.com>
	<4E312BCB.3080301@haypocalc.com> <20110729171731.2059cc3e@pitrou.net>
	<CADiSq7cpLRYHe8JSEC1W-Ff6rzsYk1x1c2MPjcaei-FA1PcqvQ@mail.gmail.com>
Message-ID: <CAP7+vJJQv+fOP7D8E1z2ftjmwm5iap5wxTwQug8nE-5xTM0H-Q@mail.gmail.com>

On Fri, Jul 29, 2011 at 8:37 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, Jul 30, 2011 at 1:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Thu, 28 Jul 2011 11:28:43 +0200
>> Victor Stinner <victor.stinner at haypocalc.com> wrote:
>>>
>>> I will add your alternative to the PEP (except if you would like to do
>>> that yourself?). If I understood correctly, you propose to:
>>>
>>> ? * rename codecs.open() to codecs.open_stream()
>>> ? * change codecs.open() to reuse open() (and so io.TextIOWrapper)
>>>
>>> (and don't deprecate anything)
>>
>> This may be an interesting approach. In a few years, we can evaluate
>> whether users are calling open_stream(), and if there aren't any, we
>> can deprecate the whole thing.
>
> Indeed. I'm also heavily influenced by MAL's opinion on this
> particular topic, so the fact he's OK with this approach counts for a
> lot. It achieves the main benefit I'm interested in (transparently
> migrating users of the codecs.open API to the new IO stack), while
> paving the way for eliminating the redundancy at some point in the
> future.

+1

-- 
--Guido van Rossum (python.org/~guido)

From mal at egenix.com  Fri Jul 29 19:14:16 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 29 Jul 2011 19:14:16 +0200
Subject: [Python-Dev] Status of the
	PEP	400?	(deprecate	codecs.StreamReader/StreamWriter)
In-Reply-To: <4E32A6DE.7060001@haypocalc.com>
References: <4E308D63.9090901@haypocalc.com>
	<4E3125D7.2030103@egenix.com>	<4E312BCB.3080301@haypocalc.com>
	<4E32A6DE.7060001@haypocalc.com>
Message-ID: <4E32EA68.3010603@egenix.com>

Victor Stinner wrote:
> Le 28/07/2011 11:28, Victor Stinner a ?crit :
>>> Please do keep the original implementation
>>> around (e.g. renamed to codecs.open_stream()), though, so that it's
>>> still possible to get easy-to-use access to codec StreamReader/Writers.
>>
>> I will add your alternative to the PEP (except if you would like to do
>> that yourself?). If I understood correctly, you propose to:
>>
>> * rename codecs.open() to codecs.open_stream()
>> * change codecs.open() to reuse open() (and so io.TextIOWrapper)
>>
>> (and don't deprecate anything)
> 
> I added your proposal to the PEP as an "Alternative Approache".

Thanks.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 29 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From barry at python.org  Fri Jul 29 19:27:31 2011
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Jul 2011 13:27:31 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
 stdlib functions
In-Reply-To: <20110729161345.0EE592506F0@webabinitio.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729161345.0EE592506F0@webabinitio.net>
Message-ID: <20110729132731.51b27bb0@resist.wooz.org>

On Jul 29, 2011, at 12:13 PM, R. David Murray wrote:

>In that case, you are working *on Python*.  Not using Python.

My point was, it's a fine line between the two.

>Personally, I always thought the devguide should be part of Docs anyway.
>It isn't because people didn't want it versioned along side Python,
>but I still don't see why that would be a problem.

+1

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/5d5896be/attachment.pgp>

From glyph at twistedmatrix.com  Fri Jul 29 20:03:02 2011
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 29 Jul 2011 14:03:02 -0400
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <j0u6je$cs1$1@dough.gmane.org>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
	<j0u6je$cs1$1@dough.gmane.org>
Message-ID: <D53CAB33-F932-44C5-BA2F-1E472DED6532@twistedmatrix.com>


On Jul 29, 2011, at 7:46 AM, Stefan Behnel wrote:

> Joao S. O. Bueno, 29.07.2011 13:22:
>> On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel wrote:
>>> Brett Cannon, 28.07.2011 23:49:
>>>> 
>>>> On Thu, Jul 28, 2011 at 11:25, Matt wrote:
>>>>> 
>>>>> - What policies are in place for keeping parity with other HTML
>>>>> parsers (such as those in web browsers)?
>>>> 
>>>> There aren't any beyond "it would be nice".
>>>> [...]
>>>> It's more of an issue of someone caring enough to do the coding work to
>>>> bring the parser up to spec for HTML5 (or introduce new code to live
>>>> beside
>>>> the HTML4 parsing code).
>>> 
>>> Which, given that html5lib readily exists, would likely be a lot more work
>>> than anyone who is interested in HTML5 handling would want to invest.
>>> 
>>> I don't think we need a new HTML5 parsing implementation only to have it in
>>> the stdlib. That's the old sunny Java way of doing it.
>> 
>> I disaagree.
>> Having proper html parsing out of the box is part of the "batteries
>> included" thing.
> 
> Well, you can easily prove me wrong by implementing this.
> 
> Stefan

Please don't implement this just to profe Stefan wrong :).

The thing to do, if you want html parsing in the stdlib, is to _incorporate_ html5lib, which is already a perfectly good, thoroughly tested HTML parser, and simply deprecate HTMLParser and friends.  Implementing a new parser would serve no purpose I can see.

-glyph


From tseaver at palladion.com  Fri Jul 29 20:31:50 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 29 Jul 2011 14:31:50 -0400
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
Message-ID: <j0uuam$6ui$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/29/2011 07:22 AM, Joao S. O. Bueno wrote:

> I disaagree. Having proper html parsing out of the box is part of
> the "batteries included" thing. And it is not a matter of "having
> html 5" - as stated on this thread, fixing it for html5 will fix it
> for html that exists in the "real world".
> 
> Python _has_ to work with quick 30-50 lines scripts deliverable 
> everywhere, not just has proper 3rd party libraries that can work as 
> part of a huge project using buildout.

Assuming it were merged today, that parser would only be available on
Python 3.3 and later:  how is that "everywhere"?  Having scripts that
work against html5lib (which *doesn't* need buildout to install, or even
setuptools) makes them portable to any version of Python supported by
the library (Python 2.3+, AFAICT).


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4y/JYACgkQ+gerLs4ltQ4KKwCgkyOlmb8xxhxg1qWH9RRbEpEw
ne0AoL6NgRElbY61QRqnXJjiKoHq0ToW
=fk3k
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Fri Jul 29 20:38:34 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 29 Jul 2011 14:38:34 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729114901.1fbcbd12@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CADiSq7c7zJNB_ARAxtvb4dmFrmP_mJqVEgTQYMSmxFdkRWvi6Q@mail.gmail.com>
	<20110729114901.1fbcbd12@resist.wooz.org>
Message-ID: <20110729183835.8BD9E2506C2@webabinitio.net>

On Fri, 29 Jul 2011 11:49:01 -0400, Barry Warsaw <barry at python.org> wrote:
> If test.support is truly and only an internal implementation detail, then it
> should adhere to Pythonic convention for such things, and be renamed
> test._support.  Then you won't need to document it at all except in the
> module.

I'd be fine with renaming it and not documenting it, myself.  Other
developers prefer to have as much as practical documented in html form.

> Here's another problem with moving the documentation for test.support to the
> devguide.  They will invariably get out of sync.  It's hard enough to keep
> stdlib code and stdlib docs in sync, but I think it will be even harder to
> keep stdlib code in sync with devguide documentation.  It will also be harder
> to know what version of the devguide corresponds to what version of the code
> repository.

Which wouldn't be any more of a problem than the regular python docs if
the devguide were in the normal doc tree.  Just saying :)

--
R. David Murray           http://www.bitdance.com

From mattbasta at gmail.com  Fri Jul 29 21:00:01 2011
From: mattbasta at gmail.com (Matt)
Date: Fri, 29 Jul 2011 12:00:01 -0700
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <D53CAB33-F932-44C5-BA2F-1E472DED6532@twistedmatrix.com>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
	<j0u6je$cs1$1@dough.gmane.org>
	<D53CAB33-F932-44C5-BA2F-1E472DED6532@twistedmatrix.com>
Message-ID: <CAJVcX10xafQGQ-FY8dzx3e-tjdciWKKZFRtebkmoJxzOhFH53w@mail.gmail.com>

On Fri, Jul 29, 2011 at 11:03 AM, Glyph Lefkowitz
<glyph at twistedmatrix.com>wrote:

>
> On Jul 29, 2011, at 7:46 AM, Stefan Behnel wrote:
>
> > Joao S. O. Bueno, 29.07.2011 13:22:
> >> On Fri, Jul 29, 2011 at 1:37 AM, Stefan Behnel wrote:
> >>> Brett Cannon, 28.07.2011 23:49:
> >>>>
> >>>> On Thu, Jul 28, 2011 at 11:25, Matt wrote:
> >>>>>
> >>>>> - What policies are in place for keeping parity with other HTML
> >>>>> parsers (such as those in web browsers)?
> >>>>
> >>>> There aren't any beyond "it would be nice".
> >>>> [...]
> >>>> It's more of an issue of someone caring enough to do the coding work
> to
> >>>> bring the parser up to spec for HTML5 (or introduce new code to live
> >>>> beside
> >>>> the HTML4 parsing code).
> >>>
> >>> Which, given that html5lib readily exists, would likely be a lot more
> work
> >>> than anyone who is interested in HTML5 handling would want to invest.
> >>>
> >>> I don't think we need a new HTML5 parsing implementation only to have
> it in
> >>> the stdlib. That's the old sunny Java way of doing it.
> >>
> >> I disaagree.
> >> Having proper html parsing out of the box is part of the "batteries
> >> included" thing.
> >
> > Well, you can easily prove me wrong by implementing this.
>

As far as the issue described in my initial message goes, there is a patch
and tests for the patch.


>
> Please don't implement this just to profe Stefan wrong :).
>
> The thing to do, if you want html parsing in the stdlib, is to
> _incorporate_ html5lib, which is already a perfectly good, thoroughly tested
> HTML parser, and simply deprecate HTMLParser and friends.  Implementing a
> new parser would serve no purpose I can see.
>

I don't see any real reason to drop a decent piece of code (HTMLParser, that
is) in favor of a third party library when only relatively minor updates are
needed to bring it up to speed with the latest spec. As far as structure
goes, HTML4 and HTML5 are practically identical. The differences between the
two that are applicable to HTMLParser involve the way the specs deal with
special element types and broken syntax. For what it's worth, the rules
HTML4 does define are (in many cases) ignored in favor of more modern,
Postel's Law-agreeable rules. HTML5 simply standardized what browsers
actually do.

Deprecating HTMLParser in favor of a newer/better/faster HTML library is a
bad thing for everybody that's already using HTMLParser, whether directly or
indirectly. html5lib does not have an interface compatible with HTMLParser,
so code would largely need to be rewritten from scratch to gain the benefits
of HTML5's support for broken code. Developers using HTMLParser would be
permanently stuck using a library that throws exceptions for perfectly valid
HTML. Keep in mind that these are solved problems: all of the thinking on
how to handle broken code has been done for us by the folks at the WHATWG.
It's simply a matter of updating our existing code with these new rules.

While I agree that there are merits to dropping support for the old code, it
does not solve the existing problems that folks are having right now (namely
incorrect parser output or exceptions). It would be more ideal to perhaps
patch the obvious issues stemming from HTML4 support for now, leaving
anything that goes beyond parity with browsers for a later time or
implementing as an opt-in feature (i.e.: enabled by a parameter).

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/f773911d/attachment.html>

From brett at python.org  Fri Jul 29 22:08:30 2011
From: brett at python.org (Brett Cannon)
Date: Fri, 29 Jul 2011 13:08:30 -0700
Subject: [Python-Dev] [Python-checkins] cpython: Issue 12647: Add
 __bool__() method to the None object.
In-Reply-To: <E1QmTrr-00024W-QI@dinsdale.python.org>
References: <E1QmTrr-00024W-QI@dinsdale.python.org>
Message-ID: <CAP1=2W7BncbY_H9x8soH3AK3QKY0MweL7-0YrdLhULYW9SySJA@mail.gmail.com>

On Thu, Jul 28, 2011 at 09:55, raymond.hettinger <python-checkins at python.org
> wrote:

> http://hg.python.org/cpython/rev/ccce01988603
> changeset:   71542:ccce01988603
> user:        Raymond Hettinger <python at rcn.com>
> date:        Thu Jul 28 09:55:13 2011 -0700
> summary:
>  Issue 12647: Add __bool__() method to the None object.
>
> files:
>  Lib/test/test_descr.py |   3 -
>  Misc/NEWS              |   4 ++
>  Objects/object.c       |  46 ++++++++++++++++++++++++++++-
>  3 files changed, 48 insertions(+), 5 deletions(-)
>
>
> diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py
> --- a/Lib/test/test_descr.py
> +++ b/Lib/test/test_descr.py
> @@ -2068,9 +2068,6 @@
>         # Two essentially featureless objects, just inheriting stuff from
>         # object.
>         self.assertEqual(dir(NotImplemented), dir(Ellipsis))
> -        if support.check_impl_detail():
> -            # None differs in PyPy: it has a __nonzero__
> -            self.assertEqual(dir(None), dir(Ellipsis))
>
>
Wasn't this change only in 3.3 where __nonzero__ doesn't exist? So when PyPy
eventually supports Python 3 they will have to update to support __bool__ on
None but this test won't exercise that for them. IOW I think the guard is
wrong and should go.

-Brett




>         # Nasty test case for proxied objects
>         class Wrapper(object):
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -13,6 +13,10 @@
>  - Verify the types of AST strings and identifiers provided by the user
> before
>   compiling them.
>
> +- Issue #12647: The None object now has a __bool__() method that returns
> False.
> +  Formerly, bool(None) returned False only because of special case logic
> +  in PyObject_IsTrue().
> +
>  - Issue #12579: str.format_map() now raises a ValueError if used on a
>   format string that contains positional fields. Initial patch by
>   Julian Berman.
> diff --git a/Objects/object.c b/Objects/object.c
> --- a/Objects/object.c
> +++ b/Objects/object.c
> @@ -1255,7 +1255,7 @@
>  }
>
>  /*
> -None is as a non-NULL undefined value.
> +None is a non-NULL undefined value.
>  There is (and should be!) no way to create other objects of this type,
>  so there is exactly one (which is indestructible, by the way).
>  */
> @@ -1277,6 +1277,48 @@
>     Py_FatalError("deallocating None");
>  }
>
> +static int
> +none_bool(PyObject *v)
> +{
> +    return 0;
> +}
> +
> +static PyNumberMethods none_as_number = {
> +    0,                          /* nb_add */
> +    0,                          /* nb_subtract */
> +    0,                          /* nb_multiply */
> +    0,                          /* nb_remainder */
> +    0,                          /* nb_divmod */
> +    0,                          /* nb_power */
> +    0,                          /* nb_negative */
> +    0,                          /* nb_positive */
> +    0,                          /* nb_absolute */
> +    (inquiry)none_bool,         /* nb_bool */
> +    0,                          /* nb_invert */
> +    0,                          /* nb_lshift */
> +    0,                          /* nb_rshift */
> +    0,                          /* nb_and */
> +    0,                          /* nb_xor */
> +    0,                          /* nb_or */
> +    0,                          /* nb_int */
> +    0,                          /* nb_reserved */
> +    0,                          /* nb_float */
> +    0,                          /* nb_inplace_add */
> +    0,                          /* nb_inplace_subtract */
> +    0,                          /* nb_inplace_multiply */
> +    0,                          /* nb_inplace_remainder */
> +    0,                          /* nb_inplace_power */
> +    0,                          /* nb_inplace_lshift */
> +    0,                          /* nb_inplace_rshift */
> +    0,                          /* nb_inplace_and */
> +    0,                          /* nb_inplace_xor */
> +    0,                          /* nb_inplace_or */
> +    0,                          /* nb_floor_divide */
> +    0,                          /* nb_true_divide */
> +    0,                          /* nb_inplace_floor_divide */
> +    0,                          /* nb_inplace_true_divide */
> +    0,                          /* nb_index */
> +};
>
>  static PyTypeObject PyNone_Type = {
>     PyVarObject_HEAD_INIT(&PyType_Type, 0)
> @@ -1289,7 +1331,7 @@
>     0,                  /*tp_setattr*/
>     0,                  /*tp_reserved*/
>     none_repr,          /*tp_repr*/
> -    0,                  /*tp_as_number*/
> +    &none_as_number,    /*tp_as_number*/
>     0,                  /*tp_as_sequence*/
>     0,                  /*tp_as_mapping*/
>     0,                  /*tp_hash */
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/bedd2567/attachment.html>

From glyph at twistedmatrix.com  Fri Jul 29 22:16:07 2011
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 29 Jul 2011 16:16:07 -0400
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <CAJVcX10xafQGQ-FY8dzx3e-tjdciWKKZFRtebkmoJxzOhFH53w@mail.gmail.com>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
	<j0u6je$cs1$1@dough.gmane.org>
	<D53CAB33-F932-44C5-BA2F-1E472DED6532@twistedmatrix.com>
	<CAJVcX10xafQGQ-FY8dzx3e-tjdciWKKZFRtebkmoJxzOhFH53w@mail.gmail.com>
Message-ID: <BBCE847D-7962-405D-99E9-C4EF741771FF@twistedmatrix.com>

On Jul 29, 2011, at 3:00 PM, Matt wrote:

> I don't see any real reason to drop a decent piece of code (HTMLParser, that is) in favor of a third party library when only relatively minor updates are needed to bring it up to speed with the latest spec.

I am not really one to throw stones here, as Twisted contains a lenient pseudo-XML parser which I still maintain - one which decidedly does not agree with html5's requirements for dealing with invalid data, but just a bunch of ad-hoc guesses of my own.

My impression of HTML5 is that HTMLParser would require significant modifications and possibly a drastic re-architecture in order to really do HTML5 "right"; especially the parts that the html5lib authors claim makes HTML5 streaming-unfriendly, i.e. subtree reordering when encountering certain types of invalid data.

But if I'm wrong about that, and there are just a few spec updates and bugfixes that need to be applied, by all means, ignore my comment.

-glyph


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/c07ae689/attachment-0001.html>

From brett at python.org  Fri Jul 29 22:31:48 2011
From: brett at python.org (Brett Cannon)
Date: Fri, 29 Jul 2011 13:31:48 -0700
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <j0uuam$6ui$1@dough.gmane.org>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
	<j0uuam$6ui$1@dough.gmane.org>
Message-ID: <CAP1=2W6x9CYU=ubNEjaHiXp0jwcMpRWwXBrZMY10ycyqWCi=mA@mail.gmail.com>

On Fri, Jul 29, 2011 at 11:31, Tres Seaver <tseaver at palladion.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 07/29/2011 07:22 AM, Joao S. O. Bueno wrote:
>
> > I disaagree. Having proper html parsing out of the box is part of
> > the "batteries included" thing. And it is not a matter of "having
> > html 5" - as stated on this thread, fixing it for html5 will fix it
> > for html that exists in the "real world".
> >
> > Python _has_ to work with quick 30-50 lines scripts deliverable
> > everywhere, not just has proper 3rd party libraries that can work as
> > part of a huge project using buildout.
>
> Assuming it were merged today, that parser would only be available on
> Python 3.3 and later:  how is that "everywhere"?


Well, "everywhere, eventually". This gets down to the usual philosophical
debate of what should (not) be in the stdlib so that those who have strict
third-party code get access to useful libraries while balancing the desire
of those who want to keep the stdlib lean or prevent stagnating the API of a
module.


>  Having scripts that
> work against html5lib (which *doesn't* need buildout to install, or even
> setuptools) makes them portable to any version of Python supported by
> the library (Python 2.3+, AFAICT).
>

If the library was brought in they could probably continue to be portable
with possibly just the addition of a try/finally on the import line.

-Brett


>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
> Palladion Software   "Excellence by Design"    http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk4y/JYACgkQ+gerLs4ltQ4KKwCgkyOlmb8xxhxg1qWH9RRbEpEw
> ne0AoL6NgRElbY61QRqnXJjiKoHq0ToW
> =fk3k
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/967fcc6d/attachment.html>

From brett at python.org  Fri Jul 29 22:34:13 2011
From: brett at python.org (Brett Cannon)
Date: Fri, 29 Jul 2011 13:34:13 -0700
Subject: [Python-Dev] HTMLParser and HTML5
In-Reply-To: <BBCE847D-7962-405D-99E9-C4EF741771FF@twistedmatrix.com>
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
	<j0u6je$cs1$1@dough.gmane.org>
	<D53CAB33-F932-44C5-BA2F-1E472DED6532@twistedmatrix.com>
	<CAJVcX10xafQGQ-FY8dzx3e-tjdciWKKZFRtebkmoJxzOhFH53w@mail.gmail.com>
	<BBCE847D-7962-405D-99E9-C4EF741771FF@twistedmatrix.com>
Message-ID: <CAP1=2W64HN11buJBwtaKSa=t_39ff0TP=M0eQSa_Kh-HZPP8oQ@mail.gmail.com>

On Fri, Jul 29, 2011 at 13:16, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:

> On Jul 29, 2011, at 3:00 PM, Matt wrote:
>
> I don't see any real reason to drop a decent piece of code (HTMLParser,
> that is) in favor of a third party library when only relatively minor
> updates are needed to bring it up to speed with the latest spec.
>
>
> I am not really one to throw stones here, as Twisted contains a lenient
> pseudo-XML parser which I still maintain - one which decidedly does *not* agree
> with html5's requirements for dealing with invalid data, but just a bunch of
> ad-hoc guesses of my own.
>
> My impression of HTML5 is that HTMLParser would require significant
> modifications and possibly a drastic re-architecture in order to really do
> HTML5 "right"; especially the parts that the html5lib authors claim makes
> HTML5 streaming-unfriendly, i.e. subtree reordering when encountering
> certain types of invalid data.
>

We could also have the code live side-by-side for a while (or indefinitely
if that was really desired) by bringing html5lib in as either a separate
module or having the relevant classes live in htmllib under different names.

But all of this is just hypothetical until someone decides to do the legwork
to actually make a proposal and get the coding done.

-Brett


>
> But if I'm wrong about that, and there are just a few spec updates and
> bugfixes that need to be applied, by all means, ignore my comment.
>
> -glyph
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110729/37a99094/attachment.html>

From solipsis at pitrou.net  Fri Jul 29 23:32:57 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 23:32:57 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
	<20110729115118.35a5c640@resist.wooz.org>
Message-ID: <20110729233257.47f03c0d@pitrou.net>

On Fri, 29 Jul 2011 11:51:18 -0400
Barry Warsaw <barry at python.org> wrote:
> On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote:
> 
> >> test.support *is* part of the stdlib.
> >
> >We have lots of internal APIs which are not documented, though.
> >And test.support *is* for internal use.
> 
> The solution then is to rename test.support to test._support to make it clear
> it's an internal implementation detail.  Then you can remove the entire
> section from the stdlib docs and just document it in the code.

Ideally so. Practically, it's a lot of churn and additional pain
merging 3.2 bugfixes into default. The lack of an underscore doesn't
always mean the API is public, because it hasn't always worked like
this (we have many private APIs without an underscore).

Regards

Antoine.



From solipsis at pitrou.net  Fri Jul 29 23:36:03 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 23:36:03 +0200
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
In-Reply-To: <20110729161946.559ED2506F0@webabinitio.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
	<20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org>
	<20110729164935.2dde6219@pitrou.net>
	<20110729161946.559ED2506F0@webabinitio.net>
Message-ID: <20110729233603.6a0adf1c@pitrou.net>

On Fri, 29 Jul 2011 12:19:45 -0400
"R. David Murray" <rdmurray at bitdance.com> wrote:
> 
> > Besides, "hg status" is meant to show untracked files which could
> > *potentially* be tracked. It's not like anybody wants to track .orig
> > and .rej files, so having them in the ignore list is still the right
> > thing to do.
> 
> That's one view.  My view is that 'hg status' tells me all the files
> that have appeared in my tree that I'm either not currently tracking or
> explicitly ignoring (because the project's automated tools will deal
> with them).  Nothing in there about limiting it to files I *might*
> want to track.  That is how I've always used my version control
> systems.

Ok, I understand. However, it also makes things more tedious for other
people who don't user their VCS in such a way, so it would be nice how
other people feel about this.

Regards

Antoine.

From solipsis at pitrou.net  Fri Jul 29 23:35:07 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 29 Jul 2011 23:35:07 +0200
Subject: [Python-Dev] HTMLParser and HTML5
References: <CAJVcX11DmZem3wbp6=RBjmQRC9oeEaMFS5L1+SjiLLUMQphgwQ@mail.gmail.com>
	<CAP1=2W7D3f-QdP-OKcckaM1-xRNFvO8x1ra3NPDkjK9N7WQ+LA@mail.gmail.com>
	<j0tddt$20k$1@dough.gmane.org>
	<CAH0mxTS3bg6gTC0rnWNTxDrjiGZgVsOMMLYmGJb7_k-zMyHSCQ@mail.gmail.com>
	<j0u6je$cs1$1@dough.gmane.org>
	<D53CAB33-F932-44C5-BA2F-1E472DED6532@twistedmatrix.com>
	<CAJVcX10xafQGQ-FY8dzx3e-tjdciWKKZFRtebkmoJxzOhFH53w@mail.gmail.com>
	<BBCE847D-7962-405D-99E9-C4EF741771FF@twistedmatrix.com>
	<CAP1=2W64HN11buJBwtaKSa=t_39ff0TP=M0eQSa_Kh-HZPP8oQ@mail.gmail.com>
Message-ID: <20110729233507.6b863182@pitrou.net>

On Fri, 29 Jul 2011 13:34:13 -0700
Brett Cannon <brett at python.org> wrote:
> On Fri, Jul 29, 2011 at 13:16, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:
> 
> > On Jul 29, 2011, at 3:00 PM, Matt wrote:
> >
> > I don't see any real reason to drop a decent piece of code (HTMLParser,
> > that is) in favor of a third party library when only relatively minor
> > updates are needed to bring it up to speed with the latest spec.
> >
> >
> > I am not really one to throw stones here, as Twisted contains a lenient
> > pseudo-XML parser which I still maintain - one which decidedly does *not* agree
> > with html5's requirements for dealing with invalid data, but just a bunch of
> > ad-hoc guesses of my own.
> >
> > My impression of HTML5 is that HTMLParser would require significant
> > modifications and possibly a drastic re-architecture in order to really do
> > HTML5 "right"; especially the parts that the html5lib authors claim makes
> > HTML5 streaming-unfriendly, i.e. subtree reordering when encountering
> > certain types of invalid data.
> >
> 
> We could also have the code live side-by-side for a while (or indefinitely
> if that was really desired) by bringing html5lib in as either a separate
> module or having the relevant classes live in htmllib under different names.

Unless html5lib is better in some fundamental ways which are difficult
to fix in htmllib, I'm not sure there's any point in adding it to the
stdlib.

We don't really do users a service if we keep adding alternative APIs
for common functionality.

Regards

Antoine.



From tjreedy at udel.edu  Sat Jul 30 00:42:46 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 29 Jul 2011 18:42:46 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729111837.4f5b88cd@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
Message-ID: <j0vd0p$c0$1@dough.gmane.org>

On 7/29/2011 11:18 AM, Barry Warsaw wrote:

> I'd much rather solve this problem by adding markup to functions that
> explicitly disclaim our normal backward compatibility guarantees.

I suggested adding a footnote marker (1) to each one.

> test.support *is* part of the stdlib.


> So once again, is this a real actual problem you've witnessed in enough cases
> to go through all the trouble of moving the documentation, and making it more
> difficult to find?

I agree with this. When working on tests, I would prefer to have 
test.support doc in the same offine doc set as unittest docs.

I only supported *moving* as an alternative to *deleting*.

Terry J. Reedy



From tjreedy at udel.edu  Sat Jul 30 00:47:07 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 29 Jul 2011 18:47:07 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729172500.67844af4@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
Message-ID: <j0vd8u$1u9$1@dough.gmane.org>

On 7/29/2011 11:25 AM, Antoine Pitrou wrote:
t
> We have lots of internal APIs which are not documented, though.

They are generally used only within the module itself as helper 
functions. So one only needs to even know about them when looking at the 
module code.

> And test.support *is* for internal use.

No, the stuff in there is *not* for internal use within the module but 
for external use is possiby every test module.

Terry J. Reedy



From solipsis at pitrou.net  Sat Jul 30 00:54:22 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Jul 2011 00:54:22 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net> <j0vd8u$1u9$1@dough.gmane.org>
Message-ID: <20110730005422.05487503@pitrou.net>

On Fri, 29 Jul 2011 18:47:07 -0400
Terry Reedy <tjreedy at udel.edu> wrote:
> 
> > And test.support *is* for internal use.
> 
> No, the stuff in there is *not* for internal use within the module but 
> for external use is possiby every test module.

I meant internal use for us. Really, whether or not it's
used cross-module is irrelevant.

Regards

Antoine.



From tjreedy at udel.edu  Sat Jul 30 01:02:32 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 29 Jul 2011 19:02:32 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729233257.47f03c0d@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
	<20110729115118.35a5c640@resist.wooz.org>
	<20110729233257.47f03c0d@pitrou.net>
Message-ID: <j0ve5s$8nk$1@dough.gmane.org>

On 7/29/2011 5:32 PM, Antoine Pitrou wrote:
> On Fri, 29 Jul 2011 11:51:18 -0400
> Barry Warsaw<barry at python.org>  wrote:
>> On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote:
>>
>>>> test.support *is* part of the stdlib.
>>>
>>> We have lots of internal APIs which are not documented, though.
>>> And test.support *is* for internal use.
>>
>> The solution then is to rename test.support to test._support to make it clear
>> it's an internal implementation detail.  Then you can remove the entire
>> section from the stdlib docs and just document it in the code.
>
> Ideally so.

The effect of this will be to discourage new people (including myself in 
the category) from writing or reviewing patches.

Terry Jan Reedy



From solipsis at pitrou.net  Sat Jul 30 01:27:05 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Jul 2011 01:27:05 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
	<20110729115118.35a5c640@resist.wooz.org>
	<20110729233257.47f03c0d@pitrou.net> <j0ve5s$8nk$1@dough.gmane.org>
Message-ID: <20110730012705.647abc9f@pitrou.net>

On Fri, 29 Jul 2011 19:02:32 -0400
Terry Reedy <tjreedy at udel.edu> wrote:
> On 7/29/2011 5:32 PM, Antoine Pitrou wrote:
> > On Fri, 29 Jul 2011 11:51:18 -0400
> > Barry Warsaw<barry at python.org>  wrote:
> >> On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote:
> >>
> >>>> test.support *is* part of the stdlib.
> >>>
> >>> We have lots of internal APIs which are not documented, though.
> >>> And test.support *is* for internal use.
> >>
> >> The solution then is to rename test.support to test._support to make it clear
> >> it's an internal implementation detail.  Then you can remove the entire
> >> section from the stdlib docs and just document it in the code.
> >
> > Ideally so.
> 
> The effect of this will be to discourage new people (including myself in 
> the category) from writing or reviewing patches.

I'm sorry, can you be more precise. The effect of what?
And why would that be so?




From ben+python at benfinney.id.au  Sat Jul 30 04:00:53 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 30 Jul 2011 12:00:53 +1000
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
	conflits files (#12255).
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
	<20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org>
	<20110729164935.2dde6219@pitrou.net>
Message-ID: <87r558msru.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Besides, "hg status" is meant to show untracked files which could
> *potentially* be tracked.

And if you don't want to track them, you need to deal with them somehow.

> It's not like anybody wants to track .orig and .rej files, so having
> them in the ignore list is still the right thing to do.

No, because their appearance in the working tree means something needs
to be dealt with by the programmer.

That's unlike other files which are generated and *don't* need to be
dealt with by the programmer (or can be deterministically regenerated at
will), so can safely be ignored.

-- 
 \     ?When I was a kid I used to pray every night for a new bicycle. |
  `\    Then I realised that the Lord doesn't work that way so I stole |
_o__)                   one and asked Him to forgive me.? ?Emo Philips |
Ben Finney


From rdmurray at bitdance.com  Sat Jul 30 04:22:05 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 29 Jul 2011 22:22:05 -0400
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
	conflits files (#12255).
In-Reply-To: <20110729233603.6a0adf1c@pitrou.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
	<20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org>
	<20110729164935.2dde6219@pitrou.net>
	<20110729161946.559ED2506F0@webabinitio.net>
	<20110729233603.6a0adf1c@pitrou.net>
Message-ID: <20110730022206.B12412505A8@webabinitio.net>

On Fri, 29 Jul 2011 23:36:03 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 29 Jul 2011 12:19:45 -0400
> "R. David Murray" <rdmurray at bitdance.com> wrote:
> > 
> > > Besides, "hg status" is meant to show untracked files which could
> > > *potentially* be tracked. It's not like anybody wants to track .orig
> > > and .rej files, so having them in the ignore list is still the right
> > > thing to do.
> > 
> > That's one view.  My view is that 'hg status' tells me all the files
> > that have appeared in my tree that I'm either not currently tracking or
> > explicitly ignoring (because the project's automated tools will deal
> > with them).  Nothing in there about limiting it to files I *might*
> > want to track.  That is how I've always used my version control
> > systems.
> 
> Ok, I understand. However, it also makes things more tedious for other
> people who don't user their VCS in such a way, so it would be nice how
> other people feel about this.

They can add those files to their personal .hgrc.  I can't *remove*
those ignores via mine.

--
R. David Murray           http://www.bitdance.com

From rdmurray at bitdance.com  Sat Jul 30 04:26:28 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 29 Jul 2011 22:26:28 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729233257.47f03c0d@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
	<20110729115118.35a5c640@resist.wooz.org>
	<20110729233257.47f03c0d@pitrou.net>
Message-ID: <20110730022629.18BE82505A8@webabinitio.net>

On Fri, 29 Jul 2011 23:32:57 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 29 Jul 2011 11:51:18 -0400
> Barry Warsaw <barry at python.org> wrote:
> > On Jul 29, 2011, at 05:25 PM, Antoine Pitrou wrote:
> > 
> > >> test.support *is* part of the stdlib.
> > >
> > >We have lots of internal APIs which are not documented, though.
> > >And test.support *is* for internal use.
> > 
> > The solution then is to rename test.support to test._support to make it clear
> > it's an internal implementation detail.  Then you can remove the entire
> > section from the stdlib docs and just document it in the code.
> 
> Ideally so. Practically, it's a lot of churn and additional pain
> merging 3.2 bugfixes into default. The lack of an underscore doesn't
> always mean the API is public, because it hasn't always worked like
> this (we have many private APIs without an underscore).

I'm not sure it makes merging more difficult.  I haven't had any
problems with email test merges even though I moved (i.e. renamed)
the test directory.

--
R. David Murray           http://www.bitdance.com

From guido at python.org  Sat Jul 30 04:28:30 2011
From: guido at python.org (Guido van Rossum)
Date: Fri, 29 Jul 2011 19:28:30 -0700
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
In-Reply-To: <20110730022206.B12412505A8@webabinitio.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net>
	<4E32B58C.3070809@netwok.org> <20110729153831.455b5d67@pitrou.net>
	<4E32C1F6.4050204@netwok.org> <20110729164935.2dde6219@pitrou.net>
	<20110729161946.559ED2506F0@webabinitio.net>
	<20110729233603.6a0adf1c@pitrou.net>
	<20110730022206.B12412505A8@webabinitio.net>
Message-ID: <CAP7+vJJBRvVDBDQMWGBWOR42wCn=Gzw8TdvW8TjZQgWZz9Z+gQ@mail.gmail.com>

On Fri, Jul 29, 2011 at 7:22 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Fri, 29 Jul 2011 23:36:03 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Fri, 29 Jul 2011 12:19:45 -0400
>> "R. David Murray" <rdmurray at bitdance.com> wrote:
>> >
>> > > Besides, "hg status" is meant to show untracked files which could
>> > > *potentially* be tracked. It's not like anybody wants to track .orig
>> > > and .rej files, so having them in the ignore list is still the right
>> > > thing to do.
>> >
>> > That's one view. ?My view is that 'hg status' tells me all the files
>> > that have appeared in my tree that I'm either not currently tracking or
>> > explicitly ignoring (because the project's automated tools will deal
>> > with them). ?Nothing in there about limiting it to files I *might*
>> > want to track. ?That is how I've always used my version control
>> > systems.
>>
>> Ok, I understand. However, it also makes things more tedious for other
>> people who don't user their VCS in such a way, so it would be nice how
>> other people feel about this.
>
> They can add those files to their personal .hgrc. ?I can't *remove*
> those ignores via mine.

Well, *I* don't ever want to check in .orig or .rej files (because
various tools create them) so I want them in my .hgignore file. Here's
a sample .hgignore file I carry around with me:
http://code.google.com/p/appengine-ndb-experiment/source/browse/.hgignore

-- 
--Guido van Rossum (python.org/~guido)

From eliben at gmail.com  Sat Jul 30 05:30:40 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Sat, 30 Jul 2011 06:30:40 +0300
Subject: [Python-Dev] the history of tests being inside Lib/ in Python
Message-ID: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>

The other thread had some claims (*) that made me wonder - why are the tests
in Python kept in Lib/ at all?

AFAIK, this is rather an unusual project structure. Tests usually have a
top-level directory of their own, in parallel to Lib/, Doc/ and others. Some
effects of this in other projects:

* The tests usually aren't even installed. The user can run them during
installation, but once it goes through, tests are not copied into
/usr/whatever...
* Tests naturally become "developer-domain", removed from the "user-domain".
No sane user would even consider using code from inside the Tests/ directory
and somehow expect it to keep working in later versions. In addition, tests
are then usually documented in special "hacking guides" and "developer docs"
instead of in the official documentation of the project.

This mail can appear as if advocating the transfer of Lib/test into Tests/,
but this is not my intention here. Honest :-) I'm just trying to understand
the history and rationale behind this structure in the CPython project.

Eli

(*) I refer to this reasoning someone raised: "test.support is part of the
tests" + "tests are part of stdlib" --> "test.support must be documented
where the rest of stdlib is"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110730/892efde0/attachment.html>

From benjamin at python.org  Sat Jul 30 05:36:21 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 29 Jul 2011 22:36:21 -0500
Subject: [Python-Dev] the history of tests being inside Lib/ in Python
In-Reply-To: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
References: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
Message-ID: <CAPZV6o-sk-c6hvgE-Q9QQsPxjsy1Tbub-cYyNvJifXV-np8hgQ@mail.gmail.com>

2011/7/29 Eli Bendersky <eliben at gmail.com>:
> The other thread had some claims (*) that made me wonder - why are the tests
> in Python kept in Lib/ at all?
>
> AFAIK, this is rather an unusual project structure.

Not really. It seems to be about half/half to me.



-- 
Regards,
Benjamin

From eliben at gmail.com  Sat Jul 30 05:44:07 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Sat, 30 Jul 2011 06:44:07 +0300
Subject: [Python-Dev] the history of tests being inside Lib/ in Python
In-Reply-To: <CAPZV6o-sk-c6hvgE-Q9QQsPxjsy1Tbub-cYyNvJifXV-np8hgQ@mail.gmail.com>
References: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
	<CAPZV6o-sk-c6hvgE-Q9QQsPxjsy1Tbub-cYyNvJifXV-np8hgQ@mail.gmail.com>
Message-ID: <CAF-Rda8bjc8QMFL1UtLfL0Vpv5kxu7x_ANT4cp5etFe7bMRz+w@mail.gmail.com>

On Sat, Jul 30, 2011 at 06:36, Benjamin Peterson <benjamin at python.org>wrote:

> 2011/7/29 Eli Bendersky <eliben at gmail.com>:
> > The other thread had some claims (*) that made me wonder - why are the
> tests
> > in Python kept in Lib/ at all?
> >
> > AFAIK, this is rather an unusual project structure.
>
> Not really. It seems to be about half/half to me.
>

I guess I'm mis-informed then :) Out of curiosity, could you point to a few
projects that also place tests inside their code?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110730/cf4b25c7/attachment.html>

From nad at acm.org  Sat Jul 30 05:48:44 2011
From: nad at acm.org (Ned Deily)
Date: Fri, 29 Jul 2011 20:48:44 -0700
Subject: [Python-Dev] the history of tests being inside Lib/ in Python
References: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
Message-ID: <nad-2ACE4F.20484429072011@news.gmane.org>

In article 
<CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg at mail.gmail.com>,
 Eli Bendersky <eliben at gmail.com> wrote:
> * The tests usually aren't even installed. The user can run them during
> installation, but once it goes through, tests are not copied into
> /usr/whatever...

That's not true across the board.  For instance, the python.org Mac OS X 
installers do install the tests "permanently" and, with the addition of 
the "-m test" (or "-m test.regrtest" in 2.7) command line options, the 
tests can be easily run at any time on the end user's system.  That's 
one of the reasons why it is important that additions and changes to the 
tests need to ensure that auxiliary directories and files are installed 
by the Makefile install targets and not just resident in the build 
directory.

-- 
 Ned Deily,
 nad at acm.org


From guido at python.org  Sat Jul 30 05:50:09 2011
From: guido at python.org (Guido van Rossum)
Date: Fri, 29 Jul 2011 20:50:09 -0700
Subject: [Python-Dev] the history of tests being inside Lib/ in Python
In-Reply-To: <CAF-Rda8bjc8QMFL1UtLfL0Vpv5kxu7x_ANT4cp5etFe7bMRz+w@mail.gmail.com>
References: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
	<CAPZV6o-sk-c6hvgE-Q9QQsPxjsy1Tbub-cYyNvJifXV-np8hgQ@mail.gmail.com>
	<CAF-Rda8bjc8QMFL1UtLfL0Vpv5kxu7x_ANT4cp5etFe7bMRz+w@mail.gmail.com>
Message-ID: <CAP7+vJLk5+L+kKybzs5DMkzYaYP_ONeopQWeuYFt8NZ3gUJciA@mail.gmail.com>

On Fri, Jul 29, 2011 at 8:44 PM, Eli Bendersky <eliben at gmail.com> wrote:
>
> On Sat, Jul 30, 2011 at 06:36, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> 2011/7/29 Eli Bendersky <eliben at gmail.com>:
>> > The other thread had some claims (*) that made me wonder - why are the
>> > tests
>> > in Python kept in Lib/ at all?
>> >
>> > AFAIK, this is rather an unusual project structure.
>>
>> Not really. It seems to be about half/half to me.
>
> I guess I'm mis-informed then :) Out of curiosity, could you point to a few
> projects that also place tests inside their code?

You're asking too many questions. Please do some research of your own.
Really, this isn't going to change, and asking why it is so or stating
that it is rather unusual is just going to irritate the experienced
developers.

If you want a friendlier answer, please sign up for
core-mentorship at python.org, where asking questions about why or why
not are welcome.

-- 
--Guido van Rossum (python.org/~guido)

From benjamin at python.org  Sat Jul 30 05:55:51 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 29 Jul 2011 22:55:51 -0500
Subject: [Python-Dev] the history of tests being inside Lib/ in Python
In-Reply-To: <CAF-Rda8bjc8QMFL1UtLfL0Vpv5kxu7x_ANT4cp5etFe7bMRz+w@mail.gmail.com>
References: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
	<CAPZV6o-sk-c6hvgE-Q9QQsPxjsy1Tbub-cYyNvJifXV-np8hgQ@mail.gmail.com>
	<CAF-Rda8bjc8QMFL1UtLfL0Vpv5kxu7x_ANT4cp5etFe7bMRz+w@mail.gmail.com>
Message-ID: <CAPZV6o_NLgBTCypk_OVa_=DqRdwHuoaDZ3FGPmwoB3qnNFAZXg@mail.gmail.com>

2011/7/29 Eli Bendersky <eliben at gmail.com>:
>
> On Sat, Jul 30, 2011 at 06:36, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> 2011/7/29 Eli Bendersky <eliben at gmail.com>:
>> > The other thread had some claims (*) that made me wonder - why are the
>> > tests
>> > in Python kept in Lib/ at all?
>> >
>> > AFAIK, this is rather an unusual project structure.
>>
>> Not really. It seems to be about half/half to me.
>
> I guess I'm mis-informed then :) Out of curiosity, could you point to a few
> projects that also place tests inside their code?

Twisted, PyPy, simplejson, Bazaar, buildbot, pyramid, SAGE, Jinja2,
Portage, pylint, and Tahoe-LAFS for starters.



-- 
Regards,
Benjamin

From g.brandl at gmx.net  Sat Jul 30 06:35:35 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 30 Jul 2011 06:35:35 +0200
Subject: [Python-Dev] cpython: make the types of None and Ellipsis
	callable
In-Reply-To: <E1QmwMG-0006cy-Hs@dinsdale.python.org>
References: <E1QmwMG-0006cy-Hs@dinsdale.python.org>
Message-ID: <j101l6$234$1@dough.gmane.org>

Am 30.07.2011 01:20, schrieb benjamin.peterson:
> http://hg.python.org/cpython/rev/84c3be27b4c7
> changeset:   71614:84c3be27b4c7
> parent:      71611:a6afd26caa8a
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Fri Jul 29 18:19:43 2011 -0500
> summary:
>   make the types of None and Ellipsis callable
> 
> files:
>   Lib/test/test_builtin.py |   7 +++++
>   Misc/NEWS                |   3 ++
>   Objects/object.c         |  34 ++++++++++++++++++++++++++++
>   Objects/sliceobject.c    |  29 +++++++++++++++++++++++
>   4 files changed, 73 insertions(+), 0 deletions(-)

Shouldn't there be a matching Doc change somewhere?

Georg


From solipsis at pitrou.net  Sat Jul 30 08:36:41 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Jul 2011 08:36:41 +0200
Subject: [Python-Dev] cpython (3.2): Stop ignoring Mercurial merge
 conflits files (#12255).
In-Reply-To: <20110730022206.B12412505A8@webabinitio.net>
References: <E1QmmHk-0003tv-LU@dinsdale.python.org>
	<20110729145045.2c252ea0@pitrou.net> <4E32B58C.3070809@netwok.org>
	<20110729153831.455b5d67@pitrou.net> <4E32C1F6.4050204@netwok.org>
	<20110729164935.2dde6219@pitrou.net>
	<20110729161946.559ED2506F0@webabinitio.net>
	<20110729233603.6a0adf1c@pitrou.net>
	<20110730022206.B12412505A8@webabinitio.net>
Message-ID: <20110730083641.47bb3c73@pitrou.net>

On Fri, 29 Jul 2011 22:22:05 -0400
"R. David Murray" <rdmurray at bitdance.com> wrote:

> On Fri, 29 Jul 2011 23:36:03 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Fri, 29 Jul 2011 12:19:45 -0400
> > "R. David Murray" <rdmurray at bitdance.com> wrote:
> > > 
> > > > Besides, "hg status" is meant to show untracked files which could
> > > > *potentially* be tracked. It's not like anybody wants to track .orig
> > > > and .rej files, so having them in the ignore list is still the right
> > > > thing to do.
> > > 
> > > That's one view.  My view is that 'hg status' tells me all the files
> > > that have appeared in my tree that I'm either not currently tracking or
> > > explicitly ignoring (because the project's automated tools will deal
> > > with them).  Nothing in there about limiting it to files I *might*
> > > want to track.  That is how I've always used my version control
> > > systems.
> > 
> > Ok, I understand. However, it also makes things more tedious for other
> > people who don't user their VCS in such a way, so it would be nice how
> > other people feel about this.
> 
> They can add those files to their personal .hgrc.  I can't *remove*
> those ignores via mine.

That's a good point. I hadn't thought about that.

Regards

Antoine.

From solipsis at pitrou.net  Sat Jul 30 08:46:00 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Jul 2011 08:46:00 +0200
Subject: [Python-Dev] cpython: Fix closes Issue11281 - smtplib.STMP gets
 source_address parameter, which adds
References: <E1QmzlC-0005al-32@dinsdale.python.org>
Message-ID: <20110730084600.33036cb3@pitrou.net>


Hi Senthil,

> +        if source_address: self.source_address = source_address

Could you try to follow PEP 8?
(I know PEP 8 is not always followed in old code, but there's no reason
not to follow it in code that we add to the stdlib)

> +        SMTP.__init__(self, host, port, local_hostname = local_hostname,
> +                source_address = source_address)

Ditto here (and other similar occurrences).

> +    def testSourceAddress(self):
> +        # connect
> +        smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3,
> +                            source_address=('127.0.0.1', 19876))
> +        self.assertEqual(smtp.source_address, ('127.0.0.1', 19876))
> +        self.assertEqual(smtp.local_hostname, 'localhost')

Unless this test is also using some kind of mock socket (it doesn't
seem to), this can break as soon as port 19876 is already in use.
There are utilities in test.support to help with this, such as
bind_port() or (less reliable) find_unused_port().

Actually, this can be triggered quite easily by running the test a
couple of times in parallel:

$ ./python -m test -m testSourceAddress -Fv -j3 test_smtplib

[...]

[ 14/4] test_smtplib
testSourceAddress (test.test_smtplib.GeneralTests) ... ok
testSourceAddress (test.test_smtplib.DebuggingServerTests) ... ERROR

======================================================================
ERROR: testSourceAddress (test.test_smtplib.DebuggingServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_smtplib.py", line
220, in testSourceAddress source_address=('127.0.0.1', 19876))
  File "/home/antoine/cpython/default/Lib/smtplib.py", line 238, in
__init__ (code, msg) = self.connect(host, port)
  File "/home/antoine/cpython/default/Lib/smtplib.py", line 313, in
connect self.sock = self._get_socket(host, port, self.timeout)
  File "/home/antoine/cpython/default/Lib/smtplib.py", line 287, in
_get_socket self.source_address)
  File "/home/antoine/cpython/default/Lib/socket.py", line 407, in
create_connection raise err
  File "/home/antoine/cpython/default/Lib/socket.py", line 397, in
create_connection sock.bind(source_address)
socket.error: [Errno 98] Address already in use

----------------------------------------------------------------------


> +        print(dir(smtp))

Usually, we avoid printing anything in tests, except when
support.verbose is True.

Regards

Antoine.



From g.brandl at gmx.net  Sat Jul 30 08:55:09 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 30 Jul 2011 08:55:09 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729052651.311dbd53@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
Message-ID: <j109qr$1ng$1@dough.gmane.org>

Am 29.07.2011 11:26, schrieb Barry Warsaw:

> So I'm curious, why is this move better than adding noindexes, or just
> trusting users to understand the difference between test.support.unlink() and
> os.unlink()?  If I currently search for 'unlink', os.unlink comes up first,
> which is good, and that should be preserved.  The presence or not of some
> test.support.unlink documentation isn't going to make the search results that
> much better or worse (there's already 14 hits).

Also don't forget that this is open-source: we shouldn't do questionable
decisions (-1 from me on moving test.support docs to the devguide) just
because the tool is lacking.

I can probably hack up a small addition to Doc/tools/sphinxext in five minutes
that ignores the whole of test.* when building the index.

Georg


From g.brandl at gmx.net  Sat Jul 30 08:55:59 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 30 Jul 2011 08:55:59 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <j0pipo$94a$1@dough.gmane.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net> <j0pipo$94a$1@dough.gmane.org>
Message-ID: <j109sd$1ng$2@dough.gmane.org>

Am 27.07.2011 19:44, schrieb Terry Reedy:
> On 7/27/2011 9:24 AM, Antoine Pitrou wrote:
> 
>> Docstrings are sufficient for own our purposes.
> 
>  >>> import test.support as t
>  >>> help(t.rmtree)
> Help on function rmtree in module test.support:
> 
> rmtree(path)

Well, what are you waiting for... just add the docstring!

Georg


From g.brandl at gmx.net  Sat Jul 30 09:02:12 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 30 Jul 2011 09:02:12 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <j0piv1$94a$2@dough.gmane.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727133619.F0B622506ED@webabinitio.net>
	<CAP1=2W7O6p0yv4NzJ=xQ8Wif7+0bonoAqNOhkgjDJT11hOG7fA@mail.gmail.com>
	<j0piv1$94a$2@dough.gmane.org>
Message-ID: <j10a83$6e8$1@dough.gmane.org>

Am 27.07.2011 19:47, schrieb Terry Reedy:
> On 7/27/2011 1:27 PM, Brett Cannon wrote:
> 
>>     Perhaps what we could do is move the documentation for test.support to
>>     the devguide, and then vet the test suite so that unlink and friends
>>     are always called as 'support.unlink', etc.
>>
>>
>> I like this solution since this issue of documenting test.support keeps
>> coming up. Otherwise we can not document test.support,
> 
> We already do.
> 
> 25.6. test.support ? Utility functions for tests

FWIW, this heading could give wrong impressions.  I just changed it to

:mod:`test.support` --- Utilities for the Python test suite

and also added another note under it that the API is subject to change
at any time.

> is about half of the page that also contains
> 25.5. test ? Regression tests package for Python
> The latter contains
> 25.5.1. Writing Unit Tests for the test package
> which should also be moved to the dev guide if 25.6 is.

Georg


From sandro.tosi at gmail.com  Sat Jul 30 11:40:52 2011
From: sandro.tosi at gmail.com (Sandro Tosi)
Date: Sat, 30 Jul 2011 11:40:52 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <20110720040505.400E23A4116@sparrow.telecommunity.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
Message-ID: <CAPdtAj2pHOZBtsDLKhvqXPkib5Cd+zG3E86P_V0_3N21HKt-3g@mail.gmail.com>

Hi,
sorry for nitpicking, but...

On Wed, Jul 20, 2011 at 05:58, P.J. Eby <pje at telecommunity.com> wrote:
...
> For those implementing PEP \302 importer objects:

the '\' should be removed, right?

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

From merwok at netwok.org  Sat Jul 30 14:57:47 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sat, 30 Jul 2011 14:57:47 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout
	and	Partitioning"
In-Reply-To: <CAPdtAj2pHOZBtsDLKhvqXPkib5Cd+zG3E86P_V0_3N21HKt-3g@mail.gmail.com>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAPdtAj2pHOZBtsDLKhvqXPkib5Cd+zG3E86P_V0_3N21HKt-3g@mail.gmail.com>
Message-ID: <4E33FFCB.6040507@netwok.org>

Hi Sandro,

> On Wed, Jul 20, 2011 at 05:58, P.J. Eby <pje at telecommunity.com> wrote:
>> For those implementing PEP \302 importer objects:
> 
> the '\' should be removed, right?

No.  Philip used backslashes to prevent the HTML conversion to transform
each and every instance of ?PEP \d+? to a link, which gets annoying
after the few first hundred times.  (It was discussed a few months ago
probably on web-sig or python-dev for PEP 333 or 3333, if memory serves.)

Cheers

From tjreedy at udel.edu  Sat Jul 30 15:07:19 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 30 Jul 2011 09:07:19 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110730005422.05487503@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net> <j0vd8u$1u9$1@dough.gmane.org>
	<20110730005422.05487503@pitrou.net>
Message-ID: <j10vlq$gc6$1@dough.gmane.org>

On 7/29/2011 6:54 PM, Antoine Pitrou wrote:
> On Fri, 29 Jul 2011 18:47:07 -0400
> Terry Reedy<tjreedy at udel.edu>  wrote:
>>
>>> And test.support *is* for internal use.
>>
>> No, the stuff in there is *not* for internal use within the module but
>> for external use is possiby every test module.
>
> I meant internal use for us. Really, whether or not it's
> used cross-module is irrelevant.

It is not at all irrelevant to me as a newer developer. I see uses of 
test.support.x quite often in checkins and I apparently should know 
about the contents of test.support for writing tests so I can use things 
as appropriate. Ditto for everyone else. On the other hand, the private 
support functions in trace.py are irrelevant to everyone except for the 
occasional person doing occasion maintenance work on that module.

Terry Jan Reedy



From tjreedy at udel.edu  Sat Jul 30 15:25:27 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 30 Jul 2011 09:25:27 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110730012705.647abc9f@pitrou.net>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
	<20110729115118.35a5c640@resist.wooz.org>
	<20110729233257.47f03c0d@pitrou.net> <j0ve5s$8nk$1@dough.gmane.org>
	<20110730012705.647abc9f@pitrou.net>
Message-ID: <j110nt$mag$1@dough.gmane.org>

On 7/29/2011 7:27 PM, Antoine Pitrou wrote:
> On Fri, 29 Jul 2011 19:02:32 -0400
> Terry Reedy<tjreedy at udel.edu>  wrote:
>> On 7/29/2011 5:32 PM, Antoine Pitrou wrote:
>>> On Fri, 29 Jul 2011 11:51:18 -0400
>>> Barry Warsaw<barry at python.org>   wrote:

>>>> The solution then is to rename test.support to test._support to make it clear
>>>> it's an internal implementation detail.  Then you can remove the entire
>>>> section from the stdlib docs and just document it in the code.
>>>
>>> Ideally so.
>>
>> The effect of this will be to discourage new people (including myself in
>> the category) from writing or reviewing patches.
>
> I'm sorry, can you be more precise. The effect of what?

Your proposal to remove the current formatted documentation of 
test.support instead of completing it and force all developers to only 
have reference to the docstrings scattered through the code.

> And why would that be so?

Because not everyone is like you. Because the condensed formatted docs 
make learning and using a module significantly easier for some people.

Terry Jan Reedy


From rdmurray at bitdance.com  Sat Jul 30 15:31:55 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 30 Jul 2011 09:31:55 -0400
Subject: [Python-Dev] the history of tests being inside Lib/ in Python
In-Reply-To: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
References: <CAF-Rda-M5QL3bhzAcf2d=0U94Qdr_-u_KVPRai9HWZkHTKhUCg@mail.gmail.com>
Message-ID: <20110730133155.DDF6E2506C2@webabinitio.net>

On Sat, 30 Jul 2011 06:30:40 +0300, Eli Bendersky <eliben at gmail.com> wrote:
> This mail can appear as if advocating the transfer of Lib/test into Tests/,
> but this is not my intention here. Honest :-) I'm just trying to understand
> the history and rationale behind this structure in the CPython project.

My understanding (I could well be wrong) is that one reason is that we
prefer that the tests be installed.  That is also the reason that a number
of tests that use large data files fetch them from the network (if and
only if the relevant resource is enabled).

--
R. David Murray           http://www.bitdance.com

From solipsis at pitrou.net  Sat Jul 30 15:38:00 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Jul 2011 15:38:00 +0200
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CAF-Rda9EK8=ZqG-oP18yBCEYS_2JeAvQqRRqvtHMs-TouKJBww@mail.gmail.com>
	<20110729111837.4f5b88cd@resist.wooz.org>
	<20110729172500.67844af4@pitrou.net>
	<20110729115118.35a5c640@resist.wooz.org>
	<20110729233257.47f03c0d@pitrou.net> <j0ve5s$8nk$1@dough.gmane.org>
	<20110730012705.647abc9f@pitrou.net> <j110nt$mag$1@dough.gmane.org>
Message-ID: <20110730153800.4cb9a540@pitrou.net>

On Sat, 30 Jul 2011 09:25:27 -0400
Terry Reedy <tjreedy at udel.edu> wrote:
> >
> > I'm sorry, can you be more precise. The effect of what?
> 
> Your proposal to remove the current formatted documentation of 
> test.support instead of completing it and force all developers to only 
> have reference to the docstrings scattered through the code.
> 
> > And why would that be so?
> 
> Because not everyone is like you. Because the condensed formatted docs 
> make learning and using a module significantly easier for some people.

Ok, I understand. I just hope the notice at the top of the module doc
is prominent enough that nobody will think there's any API guarantee
there.

Regards

Antoine.



From senthil at uthcode.com  Sat Jul 30 16:07:58 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Sat, 30 Jul 2011 22:07:58 +0800
Subject: [Python-Dev] cpython: Fix closes Issue11281 - smtplib.STMP gets
 source_address parameter, which adds
In-Reply-To: <20110730084600.33036cb3@pitrou.net>
References: <E1QmzlC-0005al-32@dinsdale.python.org>
	<20110730084600.33036cb3@pitrou.net>
Message-ID: <20110730140758.GA2262@mathmagic>

Hello Antoine,

On Sat, Jul 30, 2011 at 08:46:00AM +0200, Antoine Pitrou wrote:

> (I know PEP 8 is not always followed in old code, but there's no reason
> not to follow it in code that we add to the stdlib)
> 

Thanks for pointing out. I somehow overlooked it. I shall refactor
that lib.


> Unless this test is also using some kind of mock socket (it doesn't
> seem to), this can break as soon as port 19876 is already in use.

Yes, there is one test which does not follow the mock socket and had
not realized this it may break when run in parallel (and 19876 is
use). Shall use the test facilities which are provided to resolve
(/synchronize) that condition.

> > +        print(dir(smtp))

:-) That's was definitely my unintentional mistake. Funny that when I
ran the individual tests a couple of times, I did not see that
remaining and hard to see it when run the entire suite. Should
be removed.

-- 
Senthil

From benjamin at python.org  Sat Jul 30 16:12:19 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 30 Jul 2011 09:12:19 -0500
Subject: [Python-Dev] cpython: make the types of None and Ellipsis
	callable
In-Reply-To: <j101l6$234$1@dough.gmane.org>
References: <E1QmwMG-0006cy-Hs@dinsdale.python.org>
	<j101l6$234$1@dough.gmane.org>
Message-ID: <CAPZV6o8yyd6hN_yMKWNRUBug+6mKsk2ntMcdOd7+KE_qVrck2g@mail.gmail.com>

2011/7/29 Georg Brandl <g.brandl at gmx.net>:
> Am 30.07.2011 01:20, schrieb benjamin.peterson:
>> http://hg.python.org/cpython/rev/84c3be27b4c7
>> changeset: ? 71614:84c3be27b4c7
>> parent: ? ? ?71611:a6afd26caa8a
>> user: ? ? ? ?Benjamin Peterson <benjamin at python.org>
>> date: ? ? ? ?Fri Jul 29 18:19:43 2011 -0500
>> summary:
>> ? make the types of None and Ellipsis callable
>>
>> files:
>> ? Lib/test/test_builtin.py | ? 7 +++++
>> ? Misc/NEWS ? ? ? ? ? ? ? ?| ? 3 ++
>> ? Objects/object.c ? ? ? ? | ?34 ++++++++++++++++++++++++++++
>> ? Objects/sliceobject.c ? ?| ?29 +++++++++++++++++++++++
>> ? 4 files changed, 73 insertions(+), 0 deletions(-)
>
> Shouldn't there be a matching Doc change somewhere?

Somewhere is the question!


-- 
Regards,
Benjamin

From sandro.tosi at gmail.com  Sat Jul 30 16:43:50 2011
From: sandro.tosi at gmail.com (Sandro Tosi)
Date: Sat, 30 Jul 2011 16:43:50 +0200
Subject: [Python-Dev] Draft PEP: "Simplified Package Layout and
	Partitioning"
In-Reply-To: <4E33FFCB.6040507@netwok.org>
References: <20110720040505.400E23A4116@sparrow.telecommunity.com>
	<CAPdtAj2pHOZBtsDLKhvqXPkib5Cd+zG3E86P_V0_3N21HKt-3g@mail.gmail.com>
	<4E33FFCB.6040507@netwok.org>
Message-ID: <CAPdtAj1zH591H7ttd01WrLuZVLRAAEDNR0hNMxN8cLQ0C=qT9w@mail.gmail.com>

On Sat, Jul 30, 2011 at 14:57, ?ric Araujo <merwok at netwok.org> wrote:
>> On Wed, Jul 20, 2011 at 05:58, P.J. Eby <pje at telecommunity.com> wrote:
>>> For those implementing PEP \302 importer objects:
>>
>> the '\' should be removed, right?
>
> No. ?Philip used backslashes to prevent the HTML conversion to transform
> each and every instance of ?PEP \d+? to a link, which gets annoying
> after the few first hundred times. ?(It was discussed a few months ago
> probably on web-sig or python-dev for PEP 333 or 3333, if memory serves.)

Gaah, sorry for the noise then! (but at least I learnt a new thing!)

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

From ncoghlan at gmail.com  Sat Jul 30 17:23:40 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Jul 2011 01:23:40 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110729114901.1fbcbd12@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CADiSq7c7zJNB_ARAxtvb4dmFrmP_mJqVEgTQYMSmxFdkRWvi6Q@mail.gmail.com>
	<20110729114901.1fbcbd12@resist.wooz.org>
Message-ID: <CADiSq7czfG0z1wZ7f0nsz7K13JOzxsr=Xo4O+DDRMhxiFKE+yQ@mail.gmail.com>

On Sat, Jul 30, 2011 at 1:49 AM, Barry Warsaw <barry at python.org> wrote:
> On Jul 30, 2011, at 01:02 AM, Nick Coghlan wrote:
> If test.support is truly and only an internal implementation detail, then it
> should adhere to Pythonic convention for such things, and be renamed
> test._support. ?Then you won't need to document it at all except in the
> module.

Aside from test.regrtest and test.support, the entirety of the rest of
the test package is completely undocumented. test.support is unique,
as it is the only component other than the main executable that is
documented *at all*, solely for the benefits of people writing new
tests. We don't even document the resources that can be enabled from
the command line, instead telling people to look at the command line
help output.

The entire test package is an implementation detail, underscore or no
underscore - we will never, ever, offer any kind of backwards
compatibility guarantee for code in that part of the package
hierarchy.

> Here's another problem with moving the documentation for test.support to the
> devguide. ?They will invariably get out of sync. ?It's hard enough to keep
> stdlib code and stdlib docs in sync, but I think it will be even harder to
> keep stdlib code in sync with devguide documentation. ?It will also be harder
> to know what version of the devguide corresponds to what version of the code
> repository.

By that reasoning, we should just give up and delete the test.support
docs entirely  (since they're *already* treated with disrespect by
contrast to the real stdlib docs).

It sounds to me like you're really objecting to the devguide living in
a separate clone. This doesn't bode well for the prospects of ever
splitting the stdlib out from the CPython interpreter core...

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Jul 30 17:30:41 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Jul 2011 01:30:41 +1000
Subject: [Python-Dev] cpython: make the types of None and Ellipsis
	callable
In-Reply-To: <CAPZV6o8yyd6hN_yMKWNRUBug+6mKsk2ntMcdOd7+KE_qVrck2g@mail.gmail.com>
References: <E1QmwMG-0006cy-Hs@dinsdale.python.org>
	<j101l6$234$1@dough.gmane.org>
	<CAPZV6o8yyd6hN_yMKWNRUBug+6mKsk2ntMcdOd7+KE_qVrck2g@mail.gmail.com>
Message-ID: <CADiSq7dZOD_9oXjiC89BT8DAfKXscM4UeDXtd0VcDVHnMcahww@mail.gmail.com>

On Sun, Jul 31, 2011 at 12:12 AM, Benjamin Peterson <benjamin at python.org> wrote:
> 2011/7/29 Georg Brandl <g.brandl at gmx.net>:
>> Shouldn't there be a matching Doc change somewhere?
>
> Somewhere is the question!

While None/NotImplemented/Ellipsis are all documented as singletons,
the behaviour of calling their types is not specified anywhere. If
this was to be detailed anywhere, then the sections on
None/NotImplemented/Ellipsis in section 3.2 of the language reference
would be the place.

And on that note... was there any particular reason for leaving
NotImplemented out for this change?

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From barry at python.org  Sat Jul 30 17:38:22 2011
From: barry at python.org (Barry Warsaw)
Date: Sat, 30 Jul 2011 11:38:22 -0400
Subject: [Python-Dev] Convention on functions that shadow existing
 stdlib functions
In-Reply-To: <CADiSq7czfG0z1wZ7f0nsz7K13JOzxsr=Xo4O+DDRMhxiFKE+yQ@mail.gmail.com>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CADiSq7c7zJNB_ARAxtvb4dmFrmP_mJqVEgTQYMSmxFdkRWvi6Q@mail.gmail.com>
	<20110729114901.1fbcbd12@resist.wooz.org>
	<CADiSq7czfG0z1wZ7f0nsz7K13JOzxsr=Xo4O+DDRMhxiFKE+yQ@mail.gmail.com>
Message-ID: <20110730113822.469a0d6b@resist.wooz.org>

On Jul 31, 2011, at 01:23 AM, Nick Coghlan wrote:

>It sounds to me like you're really objecting to the devguide living in
>a separate clone. This doesn't bode well for the prospects of ever
>splitting the stdlib out from the CPython interpreter core...

Actually, no.  I'm objecting to moving documentation for code to a different
repo than the code.  If/when the stdlib is split out (which I support), then
the documentation should go with it.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110730/591ef772/attachment.pgp>

From ncoghlan at gmail.com  Sat Jul 30 18:00:45 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Jul 2011 02:00:45 +1000
Subject: [Python-Dev] Convention on functions that shadow existing
	stdlib functions
In-Reply-To: <20110730113822.469a0d6b@resist.wooz.org>
References: <CAF-Rda90Ktd-uDgYLhRZT_dBygaMHvPLZLDDyhcA1FuAyuXoww@mail.gmail.com>
	<CAP1=2W4q7dxpT8CtA3LF04aADv8us7p0BUbRFTqPtxRLT=KRKQ@mail.gmail.com>
	<CAF-Rda_sF9eQsJonz8c0evkr4Dze23zS1A8wnC10u-xOYO3T8w@mail.gmail.com>
	<20110727152410.22871784@pitrou.net>
	<CAF-Rda_wJUsS9BZNSx_CvEuGU+2dR=1BXvWVXd-OgzknQgHv_A@mail.gmail.com>
	<4E30A4F1.6050305@pearwood.info>
	<CAP1=2W6VKh+mK88GZX6gJYaVEPPcKR2anEGAUacpY1dMM_LAhQ@mail.gmail.com>
	<CADiSq7em6oTfEp7bC0bNvnhOWd9BGpLfNfPkhrnKdC1VLRXUgA@mail.gmail.com>
	<CAF-Rda92_V2Q-V2j7f0Ee9DWyaTQOZ0XheO742Hr=fi08sQGEQ@mail.gmail.com>
	<20110729052651.311dbd53@resist.wooz.org>
	<CADiSq7c7zJNB_ARAxtvb4dmFrmP_mJqVEgTQYMSmxFdkRWvi6Q@mail.gmail.com>
	<20110729114901.1fbcbd12@resist.wooz.org>
	<CADiSq7czfG0z1wZ7f0nsz7K13JOzxsr=Xo4O+DDRMhxiFKE+yQ@mail.gmail.com>
	<20110730113822.469a0d6b@resist.wooz.org>
Message-ID: <CADiSq7chi+dyo1zi5sBMQ2RPkLpkvYnGR-2Q0Q=QN0nXdUVzQA@mail.gmail.com>

On Sun, Jul 31, 2011 at 1:38 AM, Barry Warsaw <barry at python.org> wrote:
> On Jul 31, 2011, at 01:23 AM, Nick Coghlan wrote:
>
>>It sounds to me like you're really objecting to the devguide living in
>>a separate clone. This doesn't bode well for the prospects of ever
>>splitting the stdlib out from the CPython interpreter core...
>
> Actually, no. ?I'm objecting to moving documentation for code to a different
> repo than the code. ?If/when the stdlib is split out (which I support), then
> the documentation should go with it.

That's a rather valid point that I can definitely agree with.

OK, you've convinced me that the right thing to do is leave the
test.support docs where they are and just apply the appropriate markup
to keep them out of the various indices.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Jul 30 18:18:15 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Jul 2011 02:18:15 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Issue 12647: Add
 __bool__() method to the None object.
In-Reply-To: <CAP1=2W7BncbY_H9x8soH3AK3QKY0MweL7-0YrdLhULYW9SySJA@mail.gmail.com>
References: <E1QmTrr-00024W-QI@dinsdale.python.org>
	<CAP1=2W7BncbY_H9x8soH3AK3QKY0MweL7-0YrdLhULYW9SySJA@mail.gmail.com>
Message-ID: <CADiSq7e+Gr_nXf5r_du-+zPP0uTiHiGJkpPJbm=DYrmU3YhQ+g@mail.gmail.com>

On Sat, Jul 30, 2011 at 6:08 AM, Brett Cannon <brett at python.org> wrote:
> Wasn't this change only in 3.3 where __nonzero__ doesn't exist? So when PyPy
> eventually supports Python 3 they will have to update to support __bool__ on
> None but this test won't exercise that for them. IOW I think the guard is
> wrong and should go.

The entire assertion was removed by Raymond's checkin, as the addition
of None.__bool__ made it false on CPython as well.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Jul 30 18:19:23 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Jul 2011 02:19:23 +1000
Subject: [Python-Dev] cpython: make the types of None and Ellipsis
	callable
In-Reply-To: <CADiSq7dZOD_9oXjiC89BT8DAfKXscM4UeDXtd0VcDVHnMcahww@mail.gmail.com>
References: <E1QmwMG-0006cy-Hs@dinsdale.python.org>
	<j101l6$234$1@dough.gmane.org>
	<CAPZV6o8yyd6hN_yMKWNRUBug+6mKsk2ntMcdOd7+KE_qVrck2g@mail.gmail.com>
	<CADiSq7dZOD_9oXjiC89BT8DAfKXscM4UeDXtd0VcDVHnMcahww@mail.gmail.com>
Message-ID: <CADiSq7eoik1_vLuBW2QOH3UROz8Kf44Tvs-MK-DZn=Qzahdz8A@mail.gmail.com>

On Sun, Jul 31, 2011 at 1:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> And on that note... was there any particular reason for leaving
> NotImplemented out for this change?

Never mind, just noticed the subsequest checkin.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From g.brandl at gmx.net  Sat Jul 30 19:12:02 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 30 Jul 2011 19:12:02 +0200
Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax
In-Reply-To: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
References: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
Message-ID: <j11e0p$v9e$1@dough.gmane.org>

On 07/30/11 17:00, benjamin.peterson wrote:
> http://hg.python.org/cpython/rev/402f94edf11b
> changeset:   71637:402f94edf11b
> branch:      2.7
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Sat Jul 30 09:59:12 2011 -0500
> summary:
>   note Ellipsis syntax
> 
> files:
>   Doc/library/stdtypes.rst |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
> --- a/Doc/library/stdtypes.rst
> +++ b/Doc/library/stdtypes.rst
> @@ -2930,7 +2930,7 @@
>  supports no special operations.  There is exactly one ellipsis object, named
>  :const:`Ellipsis` (a built-in name).
>  
> -It is written as ``Ellipsis``.
> +It is written as ``Ellipsis`` or ``...``.

In 2.7, this is not true; ``...`` only works in slices there.

Georg


From g.brandl at gmx.net  Sat Jul 30 19:13:20 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 30 Jul 2011 19:13:20 +0200
Subject: [Python-Dev] cpython: we can call singleton types now
In-Reply-To: <E1QnB4Q-0007hN-LX@dinsdale.python.org>
References: <E1QnB4Q-0007hN-LX@dinsdale.python.org>
Message-ID: <j11e36$v9e$2@dough.gmane.org>

On 07/30/11 17:03, benjamin.peterson wrote:
> http://hg.python.org/cpython/rev/4a07b772f0e0
> changeset:   71639:4a07b772f0e0
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Sat Jul 30 10:03:09 2011 -0500
> summary:
>   we can call singleton types now
> 
> files:
>   Doc/library/stdtypes.rst |  8 +++++---
>   1 files changed, 5 insertions(+), 3 deletions(-)
> 
> 
> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
> --- a/Doc/library/stdtypes.rst
> +++ b/Doc/library/stdtypes.rst
> @@ -2706,7 +2706,7 @@
>  
>  This object is returned by functions that don't explicitly return a value.  It
>  supports no special operations.  There is exactly one null object, named
> -``None`` (a built-in name).
> +``None`` (a built-in name).  Calling ``type(None)`` produces the same singleton.

I know this is technically correct, but it will look like you mean "calling
type with None as argument" (same for the other singletons).

Probably better to say something like "``type(None)()`` produces ...".

Georg


From benjamin at python.org  Sat Jul 30 19:25:23 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 30 Jul 2011 12:25:23 -0500
Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax
In-Reply-To: <j11e0p$v9e$1@dough.gmane.org>
References: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
	<j11e0p$v9e$1@dough.gmane.org>
Message-ID: <CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>

2011/7/30 Georg Brandl <g.brandl at gmx.net>:
> On 07/30/11 17:00, benjamin.peterson wrote:
>> http://hg.python.org/cpython/rev/402f94edf11b
>> changeset: ? 71637:402f94edf11b
>> branch: ? ? ?2.7
>> user: ? ? ? ?Benjamin Peterson <benjamin at python.org>
>> date: ? ? ? ?Sat Jul 30 09:59:12 2011 -0500
>> summary:
>> ? note Ellipsis syntax
>>
>> files:
>> ? Doc/library/stdtypes.rst | ?2 +-
>> ? 1 files changed, 1 insertions(+), 1 deletions(-)
>>
>>
>> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
>> --- a/Doc/library/stdtypes.rst
>> +++ b/Doc/library/stdtypes.rst
>> @@ -2930,7 +2930,7 @@
>> ?supports no special operations. ?There is exactly one ellipsis object, named
>> ?:const:`Ellipsis` (a built-in name).
>>
>> -It is written as ``Ellipsis``.
>> +It is written as ``Ellipsis`` or ``...``.
>
> In 2.7, this is not true; ``...`` only works in slices there.

I know, but why would you use Ellipsis outside of slices?



-- 
Regards,
Benjamin

From ezio.melotti at gmail.com  Sat Jul 30 21:05:20 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sat, 30 Jul 2011 22:05:20 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Modernize modulefinder
 module and tests a bit.
In-Reply-To: <E1QmmHp-0003uO-5X@dinsdale.python.org>
References: <E1QmmHp-0003uO-5X@dinsdale.python.org>
Message-ID: <4E3455F0.5010507@gmail.com>

Hi,

On 29/07/2011 15.35, eric.araujo wrote:
> http://hg.python.org/cpython/rev/1521d9837d16
> changeset:   71569:1521d9837d16
> user:        ?ric Araujo<merwok at netwok.org>
> date:        Thu Jul 28 23:35:29 2011 +0200
> summary:
>    Modernize modulefinder module and tests a bit.
>
> The tests don?t use an internal distutils function anymore, and use
> regular assertEqual with sorted lists instead of a convoluted manual
> diff.
>
> files:
>    Lib/modulefinder.py           |  15 ++----
>    Lib/test/test_modulefinder.py |  48 +++++++++++-----------
>    2 files changed, 31 insertions(+), 32 deletions(-)
>
>
> diff --git a/Lib/modulefinder.py b/Lib/modulefinder.py
> --- a/Lib/modulefinder.py
> +++ b/Lib/modulefinder.py
> @@ -1,6 +1,5 @@
>   """Find modules used by a script, using introspection."""
>
> -from __future__ import generators
>   import dis
>   import imp
>   import marshal
> @@ -9,8 +8,6 @@
>   import types
>   import struct
>
> -READ_MODE = "rU"
> -
>   # XXX Clean up once str8's cstor matches bytes.
>   LOAD_CONST = bytes([dis.opname.index('LOAD_CONST')])
>   IMPORT_NAME = bytes([dis.opname.index('IMPORT_NAME')])
> @@ -29,8 +26,7 @@
>
>   # A Public interface
>   def AddPackagePath(packagename, path):
> -    paths = packagePathMap.get(packagename, [])
> -    paths.append(path)
> +    paths = packagePathMap.setdefault(packagename, []).append(path)

I'm assuming that packagePathMap is a dict that might contain or not a 
*packagename* key that maps to a list of paths.
Now, unless I'm missing something, the old code assigned to *paths* the 
list of paths or [] if it wasn't there, and then appended *path* to it.
AFAICS, the new code introduced two changes:
   1) the packagename key is added to the dict if it was missing -- and 
this seems reasonable;
   2) append is now on the same line, it returns None, and None is 
assigned to *paths* -- and this seems wrong;

>       packagePathMap[packagename] = paths

Also this is not necessary anymore if you use setdefault.

>   replacePackageMap = {}
> @@ -106,14 +102,14 @@
>
>   [...]

Best Regards,
Ezio Melotti

From ezio.melotti at gmail.com  Sat Jul 30 22:11:08 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sat, 30 Jul 2011 23:11:08 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 -
 smtplib.STMP gets source_address parameter, which adds
In-Reply-To: <E1QmzlC-0005al-32@dinsdale.python.org>
References: <E1QmzlC-0005al-32@dinsdale.python.org>
Message-ID: <4E34655C.1030201@gmail.com>

Hi,

On 30/07/2011 5.58, senthil.kumaran wrote:
> http://hg.python.org/cpython/rev/26839edf3cc1
> changeset:   71617:26839edf3cc1
> parent:      71613:018e14a46454
> user:        Senthil Kumaran<senthil at uthcode.com>
> date:        Sat Jul 30 10:56:50 2011 +0800
> summary:
>    Fix closes Issue11281 - smtplib.STMP gets source_address parameter, which adds the ability to bind to specific source address on a machine with multiple interfaces. Patch by Paulo Scardine.
>
> files:
>    Doc/library/smtplib.rst  |  33 +++++++++++++++++-----
>    Lib/smtplib.py           |  40 ++++++++++++++++++---------
>    Lib/test/mock_socket.py  |   3 +-
>    Lib/test/test_smtplib.py |  17 +++++++++++
>    Misc/NEWS                |   4 ++
>    5 files changed, 75 insertions(+), 22 deletions(-)
>
>
> diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
> --- a/Doc/library/smtplib.rst
> +++ b/Doc/library/smtplib.rst
> @@ -20,7 +20,7 @@
>   Protocol) and :rfc:`1869` (SMTP Service Extensions).
>
>
> -.. class:: SMTP(host='', port=0, local_hostname=None[, timeout])
> +.. class:: SMTP(host='', port=0, local_hostname=None[, timeout], source_address=None)

The "[, timeout]" now looks weird there, and it would be better to 
convert it to ", timeout=..." to match the other args.
However I don't know what the value should be, since the real value is 
socket._GLOBAL_DEFAULT_TIMEOUT (i.e. object()) and I don't think it's a 
good idea to expose that.  Maybe "None" can be used instead?

>
>      A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
>      that support a full repertoire of SMTP and ESMTP operations. If the optional
> @@ -29,7 +29,12 @@
>      raised if the specified host doesn't respond correctly. The optional
>      *timeout* parameter specifies a timeout in seconds for blocking operations
>      like the connection attempt (if not specified, the global default timeout
> -   setting will be used).
> +   setting will be used). The optional source_address parameter allows to bind to some
> +   specific source address in a machine with multiple network interfaces,
> +   and/or to some specific source tcp port. It takes a 2-tuple (host, port),

I think TCP should be uppercase.

> +   for the socket to bind to as its source address before connecting. If
> +   ommited (or if host or port are '' and/or 0 respectively) the OS default

s/ommited/omitted/ and s/''/``''``/

> +   behavior will be used.
>
>      For normal use, you should only require the initialization/connect,
>      :meth:`sendmail`, and :meth:`quit` methods.  An example is included below.
> @@ -48,8 +53,10 @@
>      .. versionchanged:: 3.3
>         Support for the :keyword:`with` statement was added.
>
> +   .. versionadded:: 3.3
> +      source_address parameter.

I think the convention is to use "versionadded" when the 
function/method/class/etc has been added, and "versionchanged" for all 
the changes, including new arguments.

> -.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout], context=None)
> +.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout], context=None, source_address=None)

Ditto for "[, timeout]" and the typos/markup below.

>      A :class:`SMTP_SSL` instance behaves exactly the same as instances of
>      :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is
> @@ -62,18 +69,28 @@
>      keyfile and certfile must be None.  The optional *timeout*
>      parameter specifies a timeout in seconds for blocking operations like the
>      connection attempt (if not specified, the global default timeout setting
> -   will be used).
> +   will be used). The optional source_address parameter allows to bind to some
> +   specific source address in a machine with multiple network interfaces,
> +   and/or to some specific source tcp port. It takes a 2-tuple (host, port),
> +   for the socket to bind to as its source address before connecting. If
> +   ommited (or if host or port are '' and/or 0 respectively) the OS default
> +   behavior will be used.
>
>      .. versionchanged:: 3.3
>         *context* was added.
>
> +   .. versionadded:: 3.3
> +      source_address parameter.
>
> -.. class:: LMTP(host='', port=LMTP_PORT, local_hostname=None)
> +
> +.. class:: LMTP(host='', port=LMTP_PORT, local_hostname=None, source_address=None)
>
>      The LMTP protocol, which is very similar to ESMTP, is heavily based on the
> -   standard SMTP client. It's common to use Unix sockets for LMTP, so our :meth:`connect`
> -   method must support that as well as a regular host:port server. To specify a
> -   Unix socket, you must use an absolute path for *host*, starting with a '/'.
> +   standard SMTP client. It's common to use Unix sockets for LMTP, so our
> +   :meth:`connect` method must support that as well as a regular host:port
> +   server. The optional parameters local_hostname and source_address has the

s/has/have/?
Also I prefer 'arguments' rather than 'parameters', the smtplib doc uses 
both, but 'arguments' seems to be used more.

> +   same meaning as that of SMTP client.To specify a Unix socket, you must use

Missing space after the '.' (there should be two spaces, but here a 
single space is used consistently so it's fine).

> +   an absolute path for *host*, starting with a '/'.
>
>      Authentication is supported, using the regular SMTP mechanism. When using a Unix
>      socket, LMTP generally don't support or require any authentication, but your
> diff --git a/Lib/smtplib.py b/Lib/smtplib.py
> --- a/Lib/smtplib.py
> +++ b/Lib/smtplib.py
> @@ -215,7 +215,8 @@
>   [...]

Best Regards,
Ezio Melotti

From greg.ewing at canterbury.ac.nz  Sun Jul 31 01:44:30 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 31 Jul 2011 11:44:30 +1200
Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax
In-Reply-To: <CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>
References: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
	<j11e0p$v9e$1@dough.gmane.org>
	<CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>
Message-ID: <4E34975E.9060004@canterbury.ac.nz>

Benjamin Peterson wrote:

> why would you use Ellipsis outside of slices?

I could imagine someone wanting to use it as part of a
function API. For example,

    print(a, b, c, ...)

would have been a nice way to tell print() not to put
a newline on the end.

-- 
Greg

From senthil at uthcode.com  Sun Jul 31 03:26:33 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Sun, 31 Jul 2011 09:26:33 +0800
Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 -
 smtplib.STMP gets source_address parameter, which adds
In-Reply-To: <4E34655C.1030201@gmail.com>
References: <E1QmzlC-0005al-32@dinsdale.python.org>
	<4E34655C.1030201@gmail.com>
Message-ID: <20110731012633.GA2394@mathmagic>

On Sat, Jul 30, 2011 at 11:11:08PM +0300, Ezio Melotti wrote:
> >-.. class:: SMTP(host='', port=0, local_hostname=None[, timeout])
> >+.. class:: SMTP(host='', port=0, local_hostname=None[, timeout], source_address=None)
> 
> The "[, timeout]" now looks weird there, and it would be better to
> convert it to ", timeout=..." to match the other args.
> However I don't know what the value should be, since the real value
> is socket._GLOBAL_DEFAULT_TIMEOUT (i.e. object()) and I don't think
> it's a good idea to expose that.  Maybe "None" can be used instead?

I found that [, timeout] bit odd too. But is not mentioning that as
timeout=None when it is timeout=socket._GLOBAL_DEFAULT_TIME
technically inaccurate?

FWIW, I see similar style (...,[,timeout], kw=val) adopted elsewhere
in the library docs too.  urllib, httplib, nntplib etc. Though it does
not look okay, it is better than giving inaccurate information.

While ftplib and poplib, has them as timeout=None, while they default
to socket._GLOBAL_DEFAULT_TIMEOUT object.

If we decide upon something, it can be made consistent. So, the
question is, when the timeout argument refers to
socket._GLOBAL_DEFAULT_TIME, how should we write the docs.

1. def somesocketmethod(arg1,arg2, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, ...)

- I don't see anything is wrong with this.

2. def somesocketmethod(arg1,arg2, timeout=None, ...)

- And explain that it actually points to a socket default timeout
object, which is odd and can lead to user errors.

3. def somesocketmethod(arg1,arg2[,timeout], kwarg=value)

- That's how it is currently explained at some places. If this style
  is okay, we can change the places were it refers to None  to be
  above.

Thanks for your review comments, I have address the remaining ones.

-- 
Senthil

From g.brandl at gmx.net  Sun Jul 31 08:28:24 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 31 Jul 2011 08:28:24 +0200
Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax
In-Reply-To: <CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>
References: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
	<j11e0p$v9e$1@dough.gmane.org>
	<CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>
Message-ID: <j12slu$pcc$1@dough.gmane.org>

On 07/30/11 19:25, Benjamin Peterson wrote:
> 2011/7/30 Georg Brandl <g.brandl at gmx.net>:
>> On 07/30/11 17:00, benjamin.peterson wrote:
>>> http://hg.python.org/cpython/rev/402f94edf11b
>>> changeset:   71637:402f94edf11b
>>> branch:      2.7
>>> user:        Benjamin Peterson <benjamin at python.org>
>>> date:        Sat Jul 30 09:59:12 2011 -0500
>>> summary:
>>>   note Ellipsis syntax
>>>
>>> files:
>>>   Doc/library/stdtypes.rst |  2 +-
>>>   1 files changed, 1 insertions(+), 1 deletions(-)
>>>
>>>
>>> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
>>> --- a/Doc/library/stdtypes.rst
>>> +++ b/Doc/library/stdtypes.rst
>>> @@ -2930,7 +2930,7 @@
>>>  supports no special operations.  There is exactly one ellipsis object, named
>>>  :const:`Ellipsis` (a built-in name).
>>>
>>> -It is written as ``Ellipsis``.
>>> +It is written as ``Ellipsis`` or ``...``.
>>
>> In 2.7, this is not true; ``...`` only works in slices there.
> 
> I know, but why would you use Ellipsis outside of slices?

I wouldn't, but that's not the point: the wording as it is now will lead readers
to think that they can use the Ellipsis singleton as in Python 3, and they will
complain and report bugs about this.

(Also, there must have been some reason to make "..." available everywhere
for Python 3.)

Georg


From raymond.hettinger at gmail.com  Sun Jul 31 08:47:36 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sat, 30 Jul 2011 23:47:36 -0700
Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax
In-Reply-To: <j12slu$pcc$1@dough.gmane.org>
References: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
	<j11e0p$v9e$1@dough.gmane.org>
	<CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>
	<j12slu$pcc$1@dough.gmane.org>
Message-ID: <C8B25BD0-D450-467F-900C-693A3F243F1F@gmail.com>


On Jul 30, 2011, at 11:28 PM, Georg Brandl wrote:

> 
> (Also, there must have been some reason to make "..." available everywhere
> for Python 3.)
> 

It's really nice for stub functions:

def foo(x):
    ...


Raymond


From ncoghlan at gmail.com  Sun Jul 31 10:15:25 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Jul 2011 18:15:25 +1000
Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax
In-Reply-To: <j12slu$pcc$1@dough.gmane.org>
References: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
	<j11e0p$v9e$1@dough.gmane.org>
	<CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>
	<j12slu$pcc$1@dough.gmane.org>
Message-ID: <CADiSq7eLgGOagkz2CHuu9GD8p8XLky8babHVLOcjPkq-PwY5bg@mail.gmail.com>

On Sun, Jul 31, 2011 at 4:28 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> (Also, there must have been some reason to make "..." available everywhere
> for Python 3.)

Not really - it just let us ditch some special casing in the
compilation toolchain that *restricted* it to being used in subscripts
(i.e. we were looking at the question from the "is there a good
rationale for keeping this arbitrary restriction?" angle).

Functionality wise, you could already write 'Ellipsis' everywhere you
would otherwise have written '...' and you still have to write ':' as
'slice(None)' outside the context of a subscript operation.

Although, as Raymond notes, it can make a nice substitute for 'pass'
as a placeholder statement, and can also be used as a placeholder
expression.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sun Jul 31 10:26:42 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Jul 2011 18:26:42 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve
	Makefile test targets.
In-Reply-To: <E1QnIej-0005ds-SR@dinsdale.python.org>
References: <E1QnIej-0005ds-SR@dinsdale.python.org>
Message-ID: <CADiSq7e=pddsYO_cEGO29XJocrGfg8SBKiy1jbf+qeP80NSLzA@mail.gmail.com>

On Sun, Jul 31, 2011 at 9:09 AM, nadeem.vawda
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/b76684d62a8d
> changeset: ? 71645:b76684d62a8d
> user: ? ? ? ?Nadeem Vawda <nadeem.vawda at gmail.com>
> date: ? ? ? ?Sun Jul 31 01:09:04 2011 +0200
> summary:
> ?Issue #11651: Improve Makefile test targets.
>
> - Use -j0 option by default
> - Remove duplicate test run in "make test" and "make testuniversal"

That seems very questionable - the rationale for running the test
suite twice by default to ensure PYC generation is working correctly
still holds.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From nadeem.vawda at gmail.com  Sun Jul 31 11:52:32 2011
From: nadeem.vawda at gmail.com (Nadeem Vawda)
Date: Sun, 31 Jul 2011 11:52:32 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve
	Makefile test targets.
In-Reply-To: <CADiSq7e=pddsYO_cEGO29XJocrGfg8SBKiy1jbf+qeP80NSLzA@mail.gmail.com>
References: <E1QnIej-0005ds-SR@dinsdale.python.org>
	<CADiSq7e=pddsYO_cEGO29XJocrGfg8SBKiy1jbf+qeP80NSLzA@mail.gmail.com>
Message-ID: <CANF4RMnnmAMQJ6AyDnVP36wmPGL9ZbXmNumAYV3kWSD4=y1T6w@mail.gmail.com>

On Sun, Jul 31, 2011 at 10:26 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Jul 31, 2011 at 9:09 AM, nadeem.vawda
> <python-checkins at python.org> wrote:
>> - Remove duplicate test run in "make test" and "make testuniversal"
>
> That seems very questionable - the rationale for running the test
> suite twice by default to ensure PYC generation is working correctly
> still holds.

The consensus on the tracker was that it isn't worth doubling the time taken to
run "make test" to check for a class of bug that seems to be relatively rare.
For changes that touch anything as far-reaching as .pyc generation, you should
be using "make testall" anyway (and that does still use two passes).

Cheers,
Nadeem

From barry at python.org  Sun Jul 31 12:55:30 2011
From: barry at python.org (Barry Warsaw)
Date: Sun, 31 Jul 2011 06:55:30 -0400
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve
 Makefile test targets.
In-Reply-To: <CADiSq7e=pddsYO_cEGO29XJocrGfg8SBKiy1jbf+qeP80NSLzA@mail.gmail.com>
References: <E1QnIej-0005ds-SR@dinsdale.python.org>
	<CADiSq7e=pddsYO_cEGO29XJocrGfg8SBKiy1jbf+qeP80NSLzA@mail.gmail.com>
Message-ID: <20110731065530.71962597@resist.wooz.org>

On Jul 31, 2011, at 06:26 PM, Nick Coghlan wrote:

>That seems very questionable - the rationale for running the test
>suite twice by default to ensure PYC generation is working correctly
>still holds.

Agreed.  I'd at least like to have seen discussion on python-dev instead of
just in the tracker.  FWIW, I wasn't even aware of this issue.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110731/246803a2/attachment.pgp>

From barry at python.org  Sun Jul 31 12:58:19 2011
From: barry at python.org (Barry Warsaw)
Date: Sun, 31 Jul 2011 06:58:19 -0400
Subject: [Python-Dev] [Python-checkins] cpython: Issue #11651: Improve
 Makefile test targets.
In-Reply-To: <20110731065530.71962597@resist.wooz.org>
References: <E1QnIej-0005ds-SR@dinsdale.python.org>
	<CADiSq7e=pddsYO_cEGO29XJocrGfg8SBKiy1jbf+qeP80NSLzA@mail.gmail.com>
	<20110731065530.71962597@resist.wooz.org>
Message-ID: <20110731065819.1fd0cb2e@resist.wooz.org>

On Jul 31, 2011, at 06:55 AM, Barry Warsaw wrote:

>On Jul 31, 2011, at 06:26 PM, Nick Coghlan wrote:
>
>>That seems very questionable - the rationale for running the test
>>suite twice by default to ensure PYC generation is working correctly
>>still holds.
>
>Agreed.  I'd at least like to have seen discussion on python-dev instead of
>just in the tracker.  FWIW, I wasn't even aware of this issue.

Er, nm.  I need coffee.

-B
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110731/034ab7d2/attachment.pgp>

From solipsis at pitrou.net  Sun Jul 31 15:07:06 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 31 Jul 2011 15:07:06 +0200
Subject: [Python-Dev] cpython (2.7): note Ellipsis syntax
References: <E1QnB1X-0007a7-Ae@dinsdale.python.org>
	<j11e0p$v9e$1@dough.gmane.org>
	<CAPZV6o_sFBEptadqdP738Qs5xbzxCraXLRJissC9zbrQ2x9KTg@mail.gmail.com>
	<j12slu$pcc$1@dough.gmane.org>
	<C8B25BD0-D450-467F-900C-693A3F243F1F@gmail.com>
Message-ID: <20110731150706.29ffd099@pitrou.net>

On Sat, 30 Jul 2011 23:47:36 -0700
Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> > 
> > (Also, there must have been some reason to make "..." available everywhere
> > for Python 3.)
> > 
> 
> It's really nice for stub functions:
> 
> def foo(x):
>     ...

Using a docstring looks a lot less hackish (and it encourages you to
write a doc!):

def foo(x):
    """Some stub function."""





From solipsis at pitrou.net  Sun Jul 31 15:10:43 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 31 Jul 2011 15:10:43 +0200
Subject: [Python-Dev] cpython: Issue #11651: Improve Makefile test
	targets.
References: <E1QnIej-0005ds-SR@dinsdale.python.org>
	<CADiSq7e=pddsYO_cEGO29XJocrGfg8SBKiy1jbf+qeP80NSLzA@mail.gmail.com>
Message-ID: <20110731151043.042638e6@pitrou.net>

On Sun, 31 Jul 2011 18:26:42 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sun, Jul 31, 2011 at 9:09 AM, nadeem.vawda
> <python-checkins at python.org> wrote:
> > http://hg.python.org/cpython/rev/b76684d62a8d
> > changeset: ? 71645:b76684d62a8d
> > user: ? ? ? ?Nadeem Vawda <nadeem.vawda at gmail.com>
> > date: ? ? ? ?Sun Jul 31 01:09:04 2011 +0200
> > summary:
> > ?Issue #11651: Improve Makefile test targets.
> >
> > - Use -j0 option by default
> > - Remove duplicate test run in "make test" and "make testuniversal"
> 
> That seems very questionable - the rationale for running the test
> suite twice by default to ensure PYC generation is working correctly
> still holds.

But nobody did it anyway, even the buildbots.

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Sun Jul 31 18:17:00 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 31 Jul 2011 17:17:00 +0100
Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 -
	smtplib.STMP gets source_address parameter, which adds
In-Reply-To: <20110731012633.GA2394@mathmagic>
References: <E1QmzlC-0005al-32@dinsdale.python.org>
	<4E34655C.1030201@gmail.com> <20110731012633.GA2394@mathmagic>
Message-ID: <C45D955C-2188-419D-8EEB-26FF6EF5E9E7@voidspace.org.uk>

On 31 Jul 2011, at 02:26, Senthil Kumaran wrote:
> On Sat, Jul 30, 2011 at 11:11:08PM +0300, Ezio Melotti wrote:
>>> -.. class:: SMTP(host='', port=0, local_hostname=None[, timeout])
>>> +.. class:: SMTP(host='', port=0, local_hostname=None[, timeout], source_address=None)
>> 
>> The "[, timeout]" now looks weird there, and it would be better to
>> convert it to ", timeout=..." to match the other args.
>> However I don't know what the value should be, since the real value
>> is socket._GLOBAL_DEFAULT_TIMEOUT (i.e. object()) and I don't think
>> it's a good idea to expose that.  Maybe "None" can be used instead?
> 
> I found that [, timeout] bit odd too. But is not mentioning that as
> timeout=None when it is timeout=socket._GLOBAL_DEFAULT_TIME
> technically inaccurate?
> 

It does mean that users will expect to be able to call with an explicit timeout=None and get the default behaviour. Just use the numeric value of the global timeout perhaps?

MIchael Foord

> FWIW, I see similar style (...,[,timeout], kw=val) adopted elsewhere
> in the library docs too.  urllib, httplib, nntplib etc. Though it does
> not look okay, it is better than giving inaccurate information.
> 
> While ftplib and poplib, has them as timeout=None, while they default
> to socket._GLOBAL_DEFAULT_TIMEOUT object.
> 
> If we decide upon something, it can be made consistent. So, the
> question is, when the timeout argument refers to
> socket._GLOBAL_DEFAULT_TIME, how should we write the docs.
> 
> 1. def somesocketmethod(arg1,arg2, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, ...)
> 
> - I don't see anything is wrong with this.
> 
> 2. def somesocketmethod(arg1,arg2, timeout=None, ...)
> 
> - And explain that it actually points to a socket default timeout
> object, which is odd and can lead to user errors.
> 
> 3. def somesocketmethod(arg1,arg2[,timeout], kwarg=value)
> 
> - That's how it is currently explained at some places. If this style
>  is okay, we can change the places were it refers to None  to be
>  above.
> 
> Thanks for your review comments, I have address the remaining ones.




--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html







From solipsis at pitrou.net  Sun Jul 31 18:41:49 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 31 Jul 2011 18:41:49 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Fix closes Issue11281 -
 smtplib.STMP gets source_address parameter, which adds
References: <E1QmzlC-0005al-32@dinsdale.python.org>
	<4E34655C.1030201@gmail.com> <20110731012633.GA2394@mathmagic>
	<C45D955C-2188-419D-8EEB-26FF6EF5E9E7@voidspace.org.uk>
Message-ID: <20110731184149.0a3c7266@pitrou.net>

On Sun, 31 Jul 2011 17:17:00 +0100
Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> > I found that [, timeout] bit odd too. But is not mentioning that as
> > timeout=None when it is timeout=socket._GLOBAL_DEFAULT_TIME
> > technically inaccurate?
> > 
> 
> It does mean that users will expect to be able to call with an explicit timeout=None and get the default behaviour. Just use the numeric value of the global timeout perhaps?

The global timeout is controllable at runtime through
socket.setdefaulttimeout(). That's the whole point of using an opaque
sentinel.

Regards

Antoine.